[I] [Feature Request]: Kafka consumer needs a maximum consumation duration (hop)

["mattcasters (via GitHub)" <gi...@apache.org>]

mattcasters opened a new issue, #8508:
URL: https://github.com/apache/hop/issues/8508

   ### What would you like to happen?
   
   While it's possible to specify a maximum idle time in the batch settings of the `Kafka Consumer` transform, it would be nice to also have an option to poll for a maximum duration. This is useful when the topic we read from continuously receives messages at a low pace.  If you then want to process all new messages on a topic for 15 minutes at high pace, once per hour, this is not possible.
   
   Deliverables:
   * A new option to consume messages for a maximum fixed duration specified in ms.
   * When this duration is reached (since the start of the transform execution) the transform is done and finishes gracefully.
   * If the "stop when idle" condition is reach a graceful stop is done beforehand.
   * Documentation update
   * Integration test
   
   
   ### Issue Priority
   
   Priority: 2
   
   ### Issue Component
   
   Component: Transforms


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Feature Request]: Kafka consumer needs a maximum consumation duration (hop)

["hansva (via GitHub)" <gi...@apache.org>]

hansva closed issue #8508: [Feature Request]: Kafka consumer needs a maximum consumation duration
URL: https://github.com/apache/hop/issues/8508


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org