You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/03/16 00:33:32 UTC
[GitHub] [druid] techdocsmith opened a new pull request #10998: remove experimental from Kinesis with caveats
techdocsmith opened a new pull request #10998:
URL: https://github.com/apache/druid/pull/10998
Removes experimental note from Kinesis extension. Adds *known issues* for those who would implement the extension in production.
This PR has:
- [x] been self-reviewed.
cc @2bethere , @jon-wei
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s merged pull request #10998: remove experimental from Kinesis with caveats
Posted by GitBox <gi...@apache.org>.
suneet-s merged pull request #10998:
URL: https://github.com/apache/druid/pull/10998
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #10998: remove experimental from Kinesis with caveats
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 commented on a change in pull request #10998:
URL: https://github.com/apache/druid/pull/10998#discussion_r596716274
##########
File path: docs/development/extensions-core/kinesis-ingestion.md
##########
@@ -471,3 +470,13 @@ with an assignment of closed shards that have been fully read and to ensure a ba
This window with early task shutdowns and possible task failures will conclude when:
- All closed shards have been fully read and the Kinesis ingestion tasks have published the data from those shards, committing the "closed" state to metadata storage
- Any remaining tasks that had inactive shards in the assignment have been shutdown (these tasks would have been created before the closed shards were completely drained)
+
+## Kinesis known issues
+
+Before you deploy the Kinesis extension to production, consider the following known issues:
Review comment:
Another known issue -
Kinesis supervisor can sometimes check if the checkpointed offset has fallen behind the retention window of the stream. These checks fetch the earliest sequence number of kinesis which can result in `IteratorAgeMilliseconds` becoming very high in AWS cloudwatch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org