You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/05/15 17:04:07 UTC
[GitHub] [spark] seglo commented on issue #24613: [SPARK-27549][SS] Add
support for committing kafka offsets per batch for supporting external
tooling
seglo commented on issue #24613: [SPARK-27549][SS] Add support for committing kafka offsets per batch for supporting external tooling
URL: https://github.com/apache/spark/pull/24613#issuecomment-492739340
> The source offsets are available in the `StreamingQueryProgress` in the listener so it may be another place where the offsets could be committed without touching the source
@arunmahadevan I was considering open sourcing a project that does just this. It works well enough, but I asked @skonto about proposing it upstream in Spark's Kafka integration instead. Once this is merged there won't be much reason to open source the project. We call it `spark-committer` :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org