[GitHub] [pulsar] MarvinCai edited a comment on issue #12284: Pulsar SQL BufferUnderflowException after upgrading from 2.7.0 to 2.8.1

[GitBox <gi...@apache.org>]

MarvinCai edited a comment on issue #12284:
URL: https://github.com/apache/pulsar/issues/12284#issuecomment-957368879


   sorry for delay, had some discussion with @gaoran10 offline
   The Websocket endpoint doesn't support schema, the internal implementation simply use byte schema([ref](https://github.com/apache/pulsar/blob/master/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/ProducerHandler.java#L73))
   And when the Pulsar topic has schema, Pulsar SQL try to ready the message and will expect a schema, but there'll be no schema, causing NPE.
   But when we remove the schema, Pulsar SQL will know that and just use a Byte schema by default([ref](https://github.com/apache/pulsar/blob/master/pulsar-sql/presto-pulsar/src/main/java/org/apache/pulsar/sql/presto/PulsarSplitManager.java#L117-L118)), that should explain why it's working after you remove the schema
   If you try produce with a client which support passing schema, Pulsar SQL should work as expected.
   Actually here's a handy cli tool to ingest NYC taxi data to a Pulsar cluster with Schema which make it easy to test Pulsar SQL: https://github.com/streamnative/examples/tree/master/nyctaxi/taxidata , the schema: https://github.com/streamnative/examples/blob/master/nyctaxi/taxidata/pkg/types/yellow.go


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org