You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/08/02 14:05:02 UTC
[GitHub] [pinot] pjpringle opened a new issue #7239: Low level kafka consumer assumes string key
pjpringle opened a new issue #7239:
URL: https://github.com/apache/pinot/issues/7239
Low level Kafka consumer assumes string deserializer for the key.
This causes exceptions if the kafka data is not keyed by a string.
Proposal to make the key deserializer class configurable.
Code link
https://www.github.com/apache/pinot/tree/master/pinot-plugins%2Fpinot-stream-ingestion%2Fpinot-kafka-2.0%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fplugin%2Fstream%2Fkafka20%2FKafkaPartitionLevelConnectionHandler.java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] npawar commented on issue #7239: Low level kafka consumer assumes string key
Posted by GitBox <gi...@apache.org>.
npawar commented on issue #7239:
URL: https://github.com/apache/pinot/issues/7239#issuecomment-892984275
What kind of exception are you seeing? Is there a stack trace/log? I'm wondering why it is not getting read as a string automatically. I'm sure this is not the first time someone has tried to ingest a topic that used a non-string column as a key.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] pjpringle commented on issue #7239: Low level kafka consumer assumes string key
Posted by GitBox <gi...@apache.org>.
pjpringle commented on issue #7239:
URL: https://github.com/apache/pinot/issues/7239#issuecomment-893069403
Potentially a different issue with confluent Kafka schema registry
properties which we have resolved.
The kafka serialization exception caused by 401 unauthorized by schema
registry went to the console log and is not present in the on disk log
files, this created some confusion.
Although still think it's worth changing the 2.0 kafka consumer code to use
byte[] key / serializer like 0.9 and to document the fact the the key is
not used and also the location for kafka logs.
On Thu, 5 Aug 2021, 05:23 Neha Pawar, ***@***.***> wrote:
> What kind of exception are you seeing? Is there a stack trace/log? I'm
> wondering why it is not getting read as a string automatically. I'm sure
> this is not the first time someone has tried to ingest a topic that used a
> non-string column as a key.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/pinot/issues/7239#issuecomment-892984275>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AFPNQ4WY7BKW3OFZKTLIZRTT3GVULANCNFSM5BMZ5GRQ>
> .
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] npawar commented on issue #7239: Low level kafka consumer assumes string key
Posted by GitBox <gi...@apache.org>.
npawar commented on issue #7239:
URL: https://github.com/apache/pinot/issues/7239#issuecomment-892984275
What kind of exception are you seeing? Is there a stack trace/log? I'm wondering why it is not getting read as a string automatically. I'm sure this is not the first time someone has tried to ingest a topic that used a non-string column as a key.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] pjpringle commented on issue #7239: Low level kafka consumer assumes string key
Posted by GitBox <gi...@apache.org>.
pjpringle commented on issue #7239:
URL: https://github.com/apache/pinot/issues/7239#issuecomment-893069403
Potentially a different issue with confluent Kafka schema registry
properties which we have resolved.
The kafka serialization exception caused by 401 unauthorized by schema
registry went to the console log and is not present in the on disk log
files, this created some confusion.
Although still think it's worth changing the 2.0 kafka consumer code to use
byte[] key / serializer like 0.9 and to document the fact the the key is
not used and also the location for kafka logs.
On Thu, 5 Aug 2021, 05:23 Neha Pawar, ***@***.***> wrote:
> What kind of exception are you seeing? Is there a stack trace/log? I'm
> wondering why it is not getting read as a string automatically. I'm sure
> this is not the first time someone has tried to ingest a topic that used a
> non-string column as a key.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/pinot/issues/7239#issuecomment-892984275>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AFPNQ4WY7BKW3OFZKTLIZRTT3GVULANCNFSM5BMZ5GRQ>
> .
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] shreelakshmijoshi commented on issue #7239: Low level kafka consumer assumes string key
Posted by GitBox <gi...@apache.org>.
shreelakshmijoshi commented on issue #7239:
URL: https://github.com/apache/pinot/issues/7239#issuecomment-903329857
Hi @pjpringle !
I am a new open source contributor and would like to solve this issue, could you help me understand the issue better by providing some extra references ?
Is this is issue open to contribute ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org