[GitHub] [pulsar-client-go] sunlc210 opened a new issue #114: The pulsar go client missing data when transfer

[GitBox <gi...@apache.org>]

sunlc210 opened a new issue #114:  The pulsar go client missing data when transfer
URL: https://github.com/apache/pulsar-client-go/issues/114
 
 
   Now I found one problem relate to pulsar go client 
   
   The whole process of mine is   csv file  filebeatpulsar flume hive
   
   At past time I verified the data count ,found that the data by this path missing about 0.46% compare with the path of logstashkafka .
   
   No idea the reason why ,so I tried below steps 
   
   1 Reduce the amount of data from 100% to 50%, excluding data loss caused by too much pressure in the pulsar cluster, the data loss is still 0.46%
   2 Reduce the amount of data from 50% to 10%, make the pulsar cluster stress-free, and the data loss is still 0.46%
   3 Use bin / pulsar-perf to quantitatively send test test data to the pulsar cluster, which is 100% acceptable, eliminating the problems of the pulsar cluster
   4 Quantitatively send test data to the pulsar cluster with filebeat. 99.6% can receive it (send 36077, receive 35091)
    
   
   I put the data file in the attachment. Please take a look at this problem. Now the loss rate is about five thousandths, which is a little too large.
   
   Another thing to say, I use the asynchronous sending of the go client, which has been confirmed with @tuteng
   
   Hopefully your response 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services