You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/08/23 08:49:46 UTC
[GitHub] [pulsar] Nintorac created a discussion: Presto connector schema issue
GitHub user Nintorac created a discussion: Presto connector schema issue
Hey :)
I am trying to query my datastream and I get the following
```
presto> select * from pulsar."public/default"."avro-schema-topic" limit 10;
Query 20220823_084000_00014_2f95a failed: Failed to get pulsar topic schema information for topic persistent://public/default/avro-schema-topic: null
```
my schema is defined like this
```
class SubmissionID(Record):
submission_id = String(required=True)
subreddit = String(required=True)
client = Client('pulsar://localhost:6650')
producer = client.create_producer(
'avro-schema-topic',
schema=AvroSchema(SubmissionID)
)
```
and when I get the schema from the schema endpoint it's fine
```
curl http://localhost:8080/admin/v2/schemas/public/default/avro-schema-topic/schema
{"version":0,"type":"AVRO","timestamp":0,"data":"{\n \"name\": \"SubmissionID\",\n \"type\": \"record\",\n \"fields\": [\n {\n \"name\": \"submission_id\",\n \"type\": \"string\",\n \"default\": null\n },\n {\n \"name\": \"subreddit\",\n \"type\": \"string\",\n \"default\": null\n }\n ]\n}","properties":{}}root@pulsar:/pulsar#
```
GitHub link: https://github.com/apache/pulsar/discussions/17234
----
This is an automatically sent email for dev@pulsar.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@pulsar.apache.org
[GitHub] [pulsar] gaoran10 added a comment to the discussion: Presto connector schema issue
Posted by GitBox <gi...@apache.org>.
GitHub user gaoran10 added a comment to the discussion: Presto connector schema issue
Could you try to reproduce this problem in standalone mode? You should find log file `$PULSAR_HOME/lib/presto/var/log/server.log`.
If there are reproduce steps, it's better.
GitHub link: https://github.com/apache/pulsar/discussions/17234#discussioncomment-3471125
----
This is an automatically sent email for dev@pulsar.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@pulsar.apache.org
[GitHub] [pulsar] Nintorac edited a comment on the discussion: Presto connector schema issue
Posted by GitBox <gi...@apache.org>.
GitHub user Nintorac edited a comment on the discussion: Presto connector schema issue
And for logs in that folder I only see `logs/pulsar-standalone.log` and `logs/gc.log`
GitHub link: https://github.com/apache/pulsar/discussions/17234#discussioncomment-3471296
----
This is an automatically sent email for dev@pulsar.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@pulsar.apache.org
[GitHub] [pulsar] codelipenghui added a comment to the discussion: Presto connector schema issue
Posted by GitBox <gi...@apache.org>.
GitHub user codelipenghui added a comment to the discussion: Presto connector schema issue
@Nintorac Could you please share the logs of the presto? I think there are some error logs which can help us to understand the problem
GitHub link: https://github.com/apache/pulsar/discussions/17234#discussioncomment-3455236
----
This is an automatically sent email for dev@pulsar.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@pulsar.apache.org
[GitHub] [pulsar] Nintorac added a comment to the discussion: Presto connector schema issue
Posted by GitBox <gi...@apache.org>.
GitHub user Nintorac added a comment to the discussion: Presto connector schema issue
And for logs I only see `logs/pulsar-standalone.log` and `logs/gc.log`
GitHub link: https://github.com/apache/pulsar/discussions/17234#discussioncomment-3471296
----
This is an automatically sent email for dev@pulsar.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@pulsar.apache.org
[GitHub] [pulsar] Nintorac added a comment to the discussion: Presto connector schema issue
Posted by GitBox <gi...@apache.org>.
GitHub user Nintorac added a comment to the discussion: Presto connector schema issue
[pulsar-trino.tar.gz](https://github.com/apache/pulsar/files/9421624/pulsar-trino.tar.gz)
Here is the setup -- all in docker compose. It is missing the pulsar presto connctor plugin folder found in the release for version `apache-pulsar-2.10.1`.
I also changed the pulsar docker image to match `2.10.1` (it is `2.9.1` in the archive) and tried again but same error
GitHub link: https://github.com/apache/pulsar/discussions/17234#discussioncomment-3471265
----
This is an automatically sent email for dev@pulsar.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@pulsar.apache.org
[GitHub] [pulsar] Nintorac added a comment to the discussion: Presto connector schema issue
Posted by GitBox <gi...@apache.org>.
GitHub user Nintorac added a comment to the discussion: Presto connector schema issue
Thanks, can't find anything, container logs show nothing after loading connectors. Also tried setting `/usr/lib/presto/etc/log.properties` with `com.facebook.presto=DEBUG`
nothing in `/data/var/` has anything
GitHub link: https://github.com/apache/pulsar/discussions/17234#discussioncomment-3455424
----
This is an automatically sent email for dev@pulsar.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@pulsar.apache.org
[GitHub] [pulsar] Nintorac added a comment to the discussion: Presto connector schema issue
Posted by GitBox <gi...@apache.org>.
GitHub user Nintorac added a comment to the discussion: Presto connector schema issue
Got the logs to work now something weird about the config I was using was killing the logs I think?
eg. `# pulsar.nar-extraction-directory=System.getProperty("java.io.tmpdir")`
anyway logs like this
```
2022-08-25T09:02:01.071Z INFO Query-20220825_090200_00000_7ef2s-221 org.apache.pulsar.sql.presto.PulsarMetadata matched topic persistent://public/default/avro-schema-topic for table public/default.avro-schema-topic
2022-08-25T09:03:01.102Z INFO dispatcher-query-1 io.prestosql.event.QueryMonitor TIMELINE: Query 20220825_090200_00000_7ef2s :: Transaction:[94dba901-e337-4db9-8889-20e778ea7931] :: elapsed 60581ms :: planning 60581ms :: waiting 0ms :: scheduling 0ms :: running 0ms :: finishing 0ms :: begin 2022-08-25T09:02:00.497Z :: end 2022-08-25T09:03:01.078Z
2022-08-25T09:03:51.778Z INFO Query-20220825_090351_00001_7ef2s-221 org.apache.pulsar.sql.presto.PulsarMetadata matched topic persistent://public/default/avro-schema-topic for table public/default.avro-schema-topic
```
Also in the UI it stays in `waiting for resources` until the 60s timeout and then it says `Internal error`
GitHub link: https://github.com/apache/pulsar/discussions/17234#discussioncomment-3472692
----
This is an automatically sent email for dev@pulsar.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@pulsar.apache.org