You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/06 14:55:47 UTC
[GitHub] [pulsar-client-go] r-funke commented on issue #546: go client producer + schema breaks Pulsar SQL
r-funke commented on issue #546:
URL: https://github.com/apache/pulsar-client-go/issues/546#issuecomment-894317565
I did check with Pulsar 2.7.2 now. It doesn't work either, but the behavior is a little bit different. The sql-worker doesn't crash, but there is `NULL` values returned by the queries instead of the actual data.
```
presto> select * from pulsar."public/default".xgojson;
id | name | __partition__ | __event_time__ | __publish_time__ | __message_id__ | __sequence_id__ | __producer_name__ | __key__ | __properties__
------+------+---------------+-------------------------+-------------------------+----------------+-----------------+-------------------+---------+----------------
NULL | NULL | -1 | 2339-03-21 22:18:14.838 | 2021-07-28 19:03:32.459 | (13,0,0) | 0 | standalone-0-2 | NULL | {}
NULL | NULL | -1 | 2339-03-21 22:18:14.838 | 2021-07-28 19:03:53.141 | (13,1,0) | 0 | standalone-0-3 | NULL | {}
NULL | NULL | -1 | 2339-03-21 22:18:14.838 | 2021-07-28 19:03:54.242 | (13,2,0) | 0 | standalone-0-4 | NULL | {}
NULL | NULL | -1 | 2339-03-21 22:18:14.838 | 2021-07-28 19:03:55.621 | (13,3,0) | 0 | standalone-0-5 | NULL | {}
(4 rows)
Query 20210728_190621_00020_ghidm, FINISHED, 1 node
Splits: 18 total, 18 done (100.00%)
0:00 [4 rows, 474B] [9 rows/s, 1.13KB/s]
```
sql-worker log output:
```
2021-07-28T19:07:04.337Z INFO Query-20210728_190704_00021_ghidm-406 org.apache.bookkeeper.mledger.impl.ReadOnlyCursorImpl [public/default/persistent/xgojson] Skipping 2 entries on read-only cursor read-only-cursor
2021-07-28T19:07:04.338Z INFO Query-20210728_190704_00021_ghidm-406 org.apache.bookkeeper.mledger.impl.ReadOnlyCursorImpl [public/default/persistent/xgojson] Skipping 2 entries on read-only cursor read-only-cursor
2021-07-28T19:07:04.405Z INFO 20210728_190704_00021_ghidm.1.0-0-111 org.apache.pulsar.sql.presto.PulsarRecordCursor Initializing split with parameters: PulsarSplit{splitId=0, connectorId='pulsar', originSchemaName='xgojson', schemaName='public/default', tableName='xgojson', splitSize=2, schema='{"type":"record","name":"Example","namespace":"test","fields":[{"name":"ID","type":"int"},{"name":"Name","type":"string"}]}', schemaType=JSON, startPositionEntryId=0, endPositionEntryId=2, startPositionLedgerId=13, endPositionLedgerId=13, schemaInfoProperties={"pulsar":"hello"}}
2021-07-28T19:07:04.411Z INFO 20210728_190704_00021_ghidm.1.0-1-113 org.apache.pulsar.sql.presto.PulsarRecordCursor Initializing split with parameters: PulsarSplit{splitId=1, connectorId='pulsar', originSchemaName='xgojson', schemaName='public/default', tableName='xgojson', splitSize=2, schema='{"type":"record","name":"Example","namespace":"test","fields":[{"name":"ID","type":"int"},{"name":"Name","type":"string"}]}', schemaType=JSON, startPositionEntryId=2, endPositionEntryId=4, startPositionLedgerId=13, endPositionLedgerId=13, schemaInfoProperties={"pulsar":"hello"}}
2021-07-28T19:07:04.430Z INFO 20210728_190704_00021_ghidm.1.0-0-111 org.apache.pulsar.sql.presto.PulsarRecordCursor Closing cursor record
2021-07-28T19:07:04.444Z INFO 20210728_190704_00021_ghidm.1.0-1-113 org.apache.pulsar.sql.presto.PulsarRecordCursor Closing cursor record
2021-07-28T19:07:04.508Z INFO dispatcher-query-12 io.prestosql.event.QueryMonitor TIMELINE: Query 20210728_190704_00021_ghidm :: Transaction:[fcc136aa-e6ff-4ddd-a57a-4a244651f57e] :: elapsed 317ms :: planning 126ms :: waiting 69ms :: scheduling 95ms :: running 59ms :: finishing 37ms :: begin 2021-07-28T19:07:04.183Z :: end 2021-07-28T19:07:04.500Z
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org