You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "kaisun2000 (via GitHub)" <gi...@apache.org> on 2023/03/28 21:24:11 UTC
[GitHub] [druid] kaisun2000 opened a new issue, #13994: Realtime consistent query with Kafka offsets as a hint
kaisun2000 opened a new issue, #13994:
URL: https://github.com/apache/druid/issues/13994
See the discussion in this slack thread https://apachedruidworkspace.slack.com/archives/C0303FDCZEZ/p1680036508569219?thread_ts=1679951040.402899&cid=C0303FDCZEZ.
Overall, if we have replicated ingestion task, due to various reasons, there is a possibility that the same real time query may see result back in time if the query goes to the slower ingestion task the 2nd time.
One proposal is that we use the Kafka offset as a hint to the query via query context. The query can be configured to return the Kafka offsets corresponding to all the Kafka partitions. These offsets can be also used as a hint via query context to the peons serving the query. The peon should only serve the query once its ingestion passed all the offsets of the partitions the peon is responsible for.
This would eliminate the potential inconsistent result for realtime query.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Realtime consistent query with Kafka offsets as a hint (druid)
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #13994:
URL: https://github.com/apache/druid/issues/13994#issuecomment-1936762899
This issue has been marked as stale due to 280 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If this issue is still
relevant, please simply write any comment. Even if closed, you can still revive the
issue at any time or discuss it on the dev@druid.apache.org list.
Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Realtime consistent query with Kafka offsets as a hint (druid)
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #13994:
URL: https://github.com/apache/druid/issues/13994#issuecomment-1987020834
This issue has been closed due to lack of activity. If you think that
is incorrect, or the issue requires additional review, you can revive the issue at
any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Realtime consistent query with Kafka offsets as a hint (druid)
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #13994: Realtime consistent query with Kafka offsets as a hint
URL: https://github.com/apache/druid/issues/13994
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org