You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/02/13 14:11:32 UTC

[GitHub] kty1965 commented on issue #23772: [SPARK-26866][SS] Support checkpoint with SubSequenceNumber to external/kinesis-asl

kty1965 commented on issue #23772: [SPARK-26866][SS] Support checkpoint with SubSequenceNumber to external/kinesis-asl
URL: https://github.com/apache/spark/pull/23772#issuecomment-463212943
 
 
   - I will re-open PR against master.
   - ok, change the title to [DSTREAM].
   - we would like to use KPL(Kinesis Producer Library) which is optimize record size. But this library use called UserRecord which exetended Recrod has SubsequenceNumber.
   
   Kinesis-asl's checkpointer does not cotrol the subsequenceNumber, only sequence number. When re-submit spark stream application, processed duplicate record.
   
   We want to reduce this duplicate record.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org