You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/05/20 07:08:56 UTC
[GitHub] [pulsar] wolfstudy opened a new issue #7000: Offload data from bk to aws-s3 error on 2.5.2
wolfstudy opened a new issue #7000:
URL: https://github.com/apache/pulsar/issues/7000
**Describe the bug**
conf/standalone.conf:
```
managedLedgerOffloadDriver=aws-s3
s3ManagedLedgerOffloadBucket=test-pulsar-offload
s3ManagedLedgerOffloadRegion=us-west-2
s3ManagedLedgerOffloadServiceEndpoint=http://s3.amazonaws.com
```
the broker error as follows:
```
NarClassLoader@3a8dc651] info register aspect org.apache.pulsar.broker.zookeeper.aspectj.ClientCnxnAspect
13:36:59.190 [pulsar-ordered-OrderedExecutor-6-0] INFO org.apache.bookkeeper.mledger.offload.OffloaderUtils - Found and loaded 1 offloaders
13:36:59.196 [pulsar-ordered-OrderedExecutor-6-0] ERROR org.apache.pulsar.broker.PulsarService - create ledgerOffloader failed for namespace public/functions
org.apache.pulsar.broker.PulsarServerException: org.apache.pulsar.broker.PulsarServerException: Either s3ManagedLedgerOffloadRegion or s3ManagedLedgerOffloadServiceEndpoint must be set if s3 offload enabled
at org.apache.pulsar.broker.PulsarService.createManagedLedgerOffloader(PulsarService.java:836) ~[org.apache.pulsar-pulsar-broker-2.5.2.jar:2.5.2]
at org.apache.pulsar.broker.PulsarService.lambda$getManagedLedgerOffloader$4(PulsarService.java:800) ~[org.apache.pulsar-pulsar-broker-2.5.2.jar:2.5.2]
at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1877) ~[?:1.8.0_201]
at org.apache.pulsar.broker.PulsarService.getManagedLedgerOffloader(PulsarService.java:792) ~[org.apache.pulsar-pulsar-broker-2.5.2.jar:2.5.2]
at org.apache.pulsar.broker.service.BrokerService.lambda$getManagedLedgerConfig$33(BrokerService.java:1065) ~[org.apache.pulsar-pulsar-broker-2.5.2.jar:2.5.2]
at org.apache.bookkeeper.mledger.util.SafeRun$2.safeRun(SafeRun.java:49) [org.apache.pulsar-managed-ledger-2.5.2.jar:2.5.2]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.10.0.jar:4.10.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.48.Final.jar:4.1.48.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
Caused by: org.apache.pulsar.broker.PulsarServerException: Either s3ManagedLedgerOffloadRegion or s3ManagedLedgerOffloadServiceEndpoint must be set if s3 offload enabled
at org.apache.pulsar.broker.PulsarService.createManagedLedgerOffloader(PulsarService.java:829) ~[org.apache.pulsar-pulsar-broker-2.5.2.jar:2.5.2]
... 10 more
```
**To Reproduce**
Steps to reproduce the behavior:
1. connfig conf/standalone.conf
2. start pulsar 2.5.2 standalone
**Expected behavior**
The broker can start normally
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lukestephenson-zendesk commented on issue #7000: Offload data from bk to aws-s3 error on 2.5.2
Posted by GitBox <gi...@apache.org>.
lukestephenson-zendesk commented on issue #7000:
URL: https://github.com/apache/pulsar/issues/7000#issuecomment-642658601
I've also encountered the same issue with pulsar 2.5.2.
From reviewing commits, it looks like it was introduced here https://github.com/apache/pulsar/commit/c3fa923e46ae37c7bab5503dbd40c3a0c54fe154#diff-bb90edad9f69a5d791cb43b04d9a4c82R1036-R1045
Note that the `ManagedLedgerOffloadDriver` is copied across, but not other fields on the configuration.
That code has since been removed in https://github.com/apache/pulsar/pull/7011/files#diff-bb90edad9f69a5d791cb43b04d9a4c82L1066, so fingers crossed this is fixed in pulsar 2.6.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on issue #7000: Offload data from bk to aws-s3 error on 2.5.2
Posted by GitBox <gi...@apache.org>.
jiazhai commented on issue #7000:
URL: https://github.com/apache/pulsar/issues/7000#issuecomment-646423190
since 2.6.0 is released, would like to close this issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on issue #7000: Offload data from bk to aws-s3 error on 2.5.2
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on issue #7000:
URL: https://github.com/apache/pulsar/issues/7000#issuecomment-645397616
@lukestephenson-zendesk Yes, this issue will be fixed at 2.6.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wolfstudy commented on issue #7000: Offload data from bk to aws-s3 error on 2.5.2
Posted by GitBox <gi...@apache.org>.
wolfstudy commented on issue #7000:
URL: https://github.com/apache/pulsar/issues/7000#issuecomment-631289759
On `2.5.1`, everything is ok. It seems that we made corresponding changes in 2.5.2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai closed issue #7000: Offload data from bk to aws-s3 error on 2.5.2
Posted by GitBox <gi...@apache.org>.
jiazhai closed issue #7000:
URL: https://github.com/apache/pulsar/issues/7000
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on issue #7000: Offload data from bk to aws-s3 error on 2.5.2
Posted by GitBox <gi...@apache.org>.
jiazhai commented on issue #7000:
URL: https://github.com/apache/pulsar/issues/7000#issuecomment-631853378
Seems this is Root Caused by @codelipenghui and @gaoran10 . Would you please help paste the founding?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org