You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "EhsanMaani (via GitHub)" <gi...@apache.org> on 2024/06/12 16:17:43 UTC

[I] Pinot consumer group in kafka ui [pinot]

EhsanMaani opened a new issue, #13378:
URL: https://github.com/apache/pinot/issues/13378

   First of all, thank you for your invaluable contributions.
   
   I am writing to seek assistance with an issue we are experiencing with the Pinot consumer in our Kafka UI (image:provectuslabs/kafka-ui).
   Our Kafka cluster is running in KRaft mode with Apache Kafka version 2.13-3.7.0. The consumer group configuration for Pinot is as follows:
   
   > streamConfigs": {
       "streamType": "kafka",
       "stream.kafka.topic.name": "test",
       "stream.kafka.broker.list": "broker1:9092,broker2:9092,broker3:9093",
       "stream.kafka.consumer.type": "simple",
       "stream.kafka.consumer.prop.auto.offset.reset": "smallest",
       "stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
       "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
       "realtime.segment.flush.threshold.rows": "0",
       "stream.kafka.consumer.group.id": "pinot-consumer-group",
       "realtime.segment.flush.threshold.time": "24h",
       "realtime.segment.flush.threshold.segment.size": "100M"
   }
   
   Pinot version is 1.1.0 and consuming messages from Kafka is ok but we can't see Pinot consumer in Kafka ui or in Kafka brokers doesn't respond to this command
   `bin/kafka-consumer-groups.sh --bootstrap-server broker:9092 --list`
   
   Thank you for your assistance.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] Pinot consumer group in kafka ui [pinot]

Posted by "Conaire (via GitHub)" <gi...@apache.org>.
Conaire commented on issue #13378:
URL: https://github.com/apache/pinot/issues/13378#issuecomment-2266788248

   same


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] Pinot consumer group in kafka ui [pinot]

Posted by "Conaire (via GitHub)" <gi...@apache.org>.
Conaire commented on issue #13378:
URL: https://github.com/apache/pinot/issues/13378#issuecomment-2266794029

   Pinot doesn't use kafka consumer groups: [Explanation](https://dev.startree.ai/docs/pinot/concepts/real-time-data-ingestion#why-does-pinot-not-use-kafka-consumer-groups)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] Pinot consumer group in kafka ui [pinot]

Posted by "EhsanMaani (via GitHub)" <gi...@apache.org>.
EhsanMaani closed issue #13378: Pinot consumer group in kafka ui 
URL: https://github.com/apache/pinot/issues/13378


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org