You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/01/19 23:41:46 UTC
[GitHub] [pulsar] joncashe opened a new issue #9242: Issue with offload on 2.7.0 migrating from 2.6.1
joncashe opened a new issue #9242:
URL: https://github.com/apache/pulsar/issues/9242
Hello
After migrate to 2.7.0 from 2.6.1 in aws eks we start to noticed an error since files were not offloaded automatically anymore after checking further i found this error from the offload-status
```
root@pulsar-broker-0:/pulsar# bin/pulsar-admin topics offload-status -w persistent://public/default/test3
Error in offload
null
Reason: Error offloading: org.apache.bookkeeper.mledger.ManagedLedgerException: java.util.concurrent.CompletionException: org.jclouds.http.HttpResponseException: read from input stream error connecting to PUT https://s3.amazonaws.com/bucket/39a5c277-9b17-497f-93c6-f3f515c46ef5-ledger-691?partNumber=1&uploadId=seTTegFBZaSZ9rw67f6nlLGdtuDZs3lpy1w3vIPh75gu5mhEwyr_.mmhPL_SaMfkczv832x0oigyxFL9LHczs0NYRF0O8MNnH7mgIlSpKovCKC0vRFOprKMZW1NXSsRaduSxvVBYRNpZ_LVgQRZ6SJji1Ardd_ORoHL_qmqrdvs- HTTP/1.1
```
if i rollback it works so i guess its something new related to the version, we have the offload policy set at namespace level since that was the way was available on 2.6.1
any feedback will be appreciated
thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9242: Issue with offload on 2.7.0 migrating from 2.6.1
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9242:
URL: https://github.com/apache/pulsar/issues/9242#issuecomment-1058894067
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on issue #9242: Issue with offload on 2.7.0 migrating from 2.6.1
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on issue #9242:
URL: https://github.com/apache/pulsar/issues/9242#issuecomment-780971894
The IAM role authentication may cause the problem, I'll have a try.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on issue #9242: Issue with offload on 2.7.0 migrating from 2.6.1
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on issue #9242:
URL: https://github.com/apache/pulsar/issues/9242#issuecomment-775111384
Yes, the broker logs could help us to find more Infomation.
And I have some questions, please take a look.
1. If the topic didn't set the namespace offload policies, the offload operation could be executed successfully?
2. What authentication way do you use? refer to http://pulsar.apache.org/docs/zh-CN/tiered-storage-aws/#authentication-required
3. Do you upgrade the offloader nar package to `2.7.0`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Addyvan commented on issue #9242: Issue with offload on 2.7.0 migrating from 2.6.1
Posted by GitBox <gi...@apache.org>.
Addyvan commented on issue #9242:
URL: https://github.com/apache/pulsar/issues/9242#issuecomment-786184305
I am also seeing this error with `2.7.0` using a ceph s3 store.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] joncashe commented on issue #9242: Issue with offload on 2.7.0 migrating from 2.6.1
Posted by GitBox <gi...@apache.org>.
joncashe commented on issue #9242:
URL: https://github.com/apache/pulsar/issues/9242#issuecomment-775326486
> Yes, the broker logs could help us to find more Infomation.
>
> And I have some questions, please take a look.
>
> 1. If the topic didn't set the namespace offload policies, the offload operation could be executed successfully?
> 2. What authentication way do you use? refer to http://pulsar.apache.org/docs/zh-CN/tiered-storage-aws/#authentication-required
> 3. Do you upgrade the offloader nar package to `2.7.0`?
hello,
since we rollbacked this and we are close to deliver will need to get the broker logs again but i remember there was nothing out of normal
related to your questions
1. havent tried that just with the offload policy
2. we are using iam role so im passing s3ManagedLedgerOffloadRole which works in 2.6.1
3. not sure about this but since we are using apachepulsar/pulsar-all/2.7.0 image, will that include it right? i didnt see it mentioned in the helm https://github.com/apache/pulsar-helm-chart
thanks,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9242: Issue with offload on 2.7.0 migrating from 2.6.1
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9242:
URL: https://github.com/apache/pulsar/issues/9242#issuecomment-770229973
@joncashe Sorry for the late response, could you please provide the broker logs? This will help us to locate the problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org