You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "yyfhust (via GitHub)" <gi...@apache.org> on 2025/04/02 01:19:36 UTC
[I] [Bug]: ReadFromKafkaViaSDF does not set coder when using withValueDeserializerAndCoder [beam]
yyfhust opened a new issue, #34506:
URL: https://github.com/apache/beam/issues/34506
### What happened?
<img width="424" alt="Screenshot 2025-04-01 at 9 05 09 PM" src="https://github.com/user-attachments/assets/d931b887-988a-4d3b-94c8-5a48cc58ab91" />
TL,DR is we need to also set corder in SDF. no coder is explicitly set as of now, it means beam will infer the type from the Deserializer (this is not a problem if we use beam built-in Deserializer).
however, if we use customized Deserializer , i,.e foo_bar_Deserializer Deserializer<Row> , it will raise error as beam is unable to infer coder from foo_bar_Deserializer. In this case, we need to provide both Deserializer and coder.
In legacy kafkaIO, we set both Deserializer and coder based on input, but in SDF kafkaIO, coder is not set,
to solve this issue, we need to set the coder based on input
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Infrastructure
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: ReadFromKafkaViaSDF does not set coder when using withValueDeserializerAndCoder [beam]
Posted by "yyfhust (via GitHub)" <gi...@apache.org>.
yyfhust commented on issue #34506:
URL: https://github.com/apache/beam/issues/34506#issuecomment-2771231455
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: ReadFromKafkaViaSDF does not set coder when using withValueDeserializerAndCoder [beam]
Posted by "yyfhust (via GitHub)" <gi...@apache.org>.
yyfhust commented on issue #34506:
URL: https://github.com/apache/beam/issues/34506#issuecomment-2771023259
WIP https://github.com/apache/beam/pull/34505
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: ReadFromKafkaViaSDF does not set coder when using withValueDeserializerAndCoder [beam]
Posted by "yyfhust (via GitHub)" <gi...@apache.org>.
yyfhust closed issue #34506: [Bug]: ReadFromKafkaViaSDF does not set coder when using withValueDeserializerAndCoder
URL: https://github.com/apache/beam/issues/34506
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org