You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "EhsanMaani (via GitHub)" <gi...@apache.org> on 2024/06/12 16:17:43 UTC
[I] Pinot consumer group in kafka ui [pinot]
EhsanMaani opened a new issue, #13378:
URL: https://github.com/apache/pinot/issues/13378
First of all, thank you for your invaluable contributions.
I am writing to seek assistance with an issue we are experiencing with the Pinot consumer in our Kafka UI (image:provectuslabs/kafka-ui).
Our Kafka cluster is running in KRaft mode with Apache Kafka version 2.13-3.7.0. The consumer group configuration for Pinot is as follows:
> streamConfigs": {
"streamType": "kafka",
"stream.kafka.topic.name": "test",
"stream.kafka.broker.list": "broker1:9092,broker2:9092,broker3:9093",
"stream.kafka.consumer.type": "simple",
"stream.kafka.consumer.prop.auto.offset.reset": "smallest",
"stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
"realtime.segment.flush.threshold.rows": "0",
"stream.kafka.consumer.group.id": "pinot-consumer-group",
"realtime.segment.flush.threshold.time": "24h",
"realtime.segment.flush.threshold.segment.size": "100M"
}
Pinot version is 1.1.0 and consuming messages from Kafka is ok but we can't see Pinot consumer in Kafka ui or in Kafka brokers doesn't respond to this command
`bin/kafka-consumer-groups.sh --bootstrap-server broker:9092 --list`
Thank you for your assistance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Pinot consumer group in kafka ui [pinot]
Posted by "Conaire (via GitHub)" <gi...@apache.org>.
Conaire commented on issue #13378:
URL: https://github.com/apache/pinot/issues/13378#issuecomment-2266788248
same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Pinot consumer group in kafka ui [pinot]
Posted by "Conaire (via GitHub)" <gi...@apache.org>.
Conaire commented on issue #13378:
URL: https://github.com/apache/pinot/issues/13378#issuecomment-2266794029
Pinot doesn't use kafka consumer groups: [Explanation](https://dev.startree.ai/docs/pinot/concepts/real-time-data-ingestion#why-does-pinot-not-use-kafka-consumer-groups)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Pinot consumer group in kafka ui [pinot]
Posted by "EhsanMaani (via GitHub)" <gi...@apache.org>.
EhsanMaani closed issue #13378: Pinot consumer group in kafka ui
URL: https://github.com/apache/pinot/issues/13378
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org