[GitHub] [pulsar] rfuerst87 opened a new issue #4715: [pulsar-sql] BKNoSuchLedgerExistsException when using custom ZooKeeper path for pulsar

[GitBox <gi...@apache.org>]

rfuerst87 opened a new issue #4715: [pulsar-sql] BKNoSuchLedgerExistsException when using custom ZooKeeper path for pulsar
URL: https://github.com/apache/pulsar/issues/4715
 
 
   **Describe the bug**
   I have an existing ZooKeeper cluster where I store any Pulsar related stuff under `/pulsar` setting `zkServers` to `zk-url:2181/pulsar`. This works fine for brokers and bookies, but not for pulsar-sql. When querying a table I always get a `BKNoSuchLedgerExistsException`:
   
   ```
   // pulsar-sql logfile
   2019-07-12T08:44:13.062Z	ERROR	Query-20190712_084408_00001_k8t7u-140	org.apache.pulsar.sql.presto.PulsarSplitManager	Failed to get splits
   org.apache.bookkeeper.mledger.ManagedLedgerException: java.util.concurrent.CompletionException: org.apache.bookkeeper.client.BKException$BKNoSuchLedgerExistsException: No such ledger exists
   Caused by: java.util.concurrent.CompletionException: org.apache.bookkeeper.client.BKException$BKNoSuchLedgerExistsException: No such ledger exists
   	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
   	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
   	at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:647)
   	at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632)
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
   	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
   	at org.apache.bookkeeper.client.SyncCallbackUtils.finish(SyncCallbackUtils.java:83)
   	at org.apache.bookkeeper.client.SyncCallbackUtils$SyncOpenCallback.openComplete(SyncCallbackUtils.java:157)
   	at org.apache.bookkeeper.client.LedgerOpenOp.openComplete(LedgerOpenOp.java:232)
   	at org.apache.bookkeeper.client.LedgerOpenOp.lambda$initiate$0(LedgerOpenOp.java:117)
   	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
   	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
   	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
   	at org.apache.bookkeeper.meta.AbstractZkLedgerManager$3.processResult(AbstractZkLedgerManager.java:396)
   	at org.apache.bookkeeper.zookeeper.ZooKeeperClient$19$1.processResult(ZooKeeperClient.java:994)
   	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:575)
   	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:508)
   Caused by: org.apache.bookkeeper.client.BKException$BKNoSuchLedgerExistsException: No such ledger exists
   	... 12 more
   ```
   
   Indeed I did set `pulsar.zookeeper-uri` to `zk-url:2181/pulsar` in the pulsar-sql config (see below) but this seems to be ignored when a query is executed.
   
   ```
   // pulsar-sql logfile
   2019-07-12T08:42:23.754Z	INFO	main	Bootstrap	pulsar.zookeeper-uri                       localhost:2181                                 master.mesos:2181/pulsar
   ```
   
   **To Reproduce**
   Steps to reproduce the behavior:
   1. Set up a pulsar cluster in an existing ZK environment under `/pulsar`
   2. Create some test data in a topic
   3. Use presto to query data --> `BKNoSuchLedgerExistsException`
   
   **Expected behavior**
   Seeing query results as expected.
   
   **Desktop (please complete the following information):**
    - Issue observed in docker image `apachepulsar/pulsar-all:2.4.0`
   
   **Additional context**
   As said above I suspect paths in ZooKeeper connection strings are simply ignored in pulsar-sql. If you point me to the relevant bits in the code I maybe could contribute a patch here. Let me know when I can help.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services