You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by John Calcote <jo...@gmail.com> on 2022/08/04 21:48:38 UTC
Re: getBucketLocation failing
Additional info:
We tried running the aws command line tool on the same host, calling
getBucketLocation against the same bucket - it works there just fine - and
returns in a timely fashion. But the internal call to getBucketLocation is
failing in the jclouds aws S3 provider.
Question: Is there any way to configure the endpoint such that
getBucketLocation is not called by getBlob?
I didn't mention it above, but we're running jclouds 2.5.0 - the latest,
afaik.
On Thu, Jul 28, 2022 at 6:29 PM John Calcote <jo...@gmail.com> wrote:
> Hi all,
>
> We have an AWS scenario where jclouds attempt to getBucketLocation is
> failing (internally) on 5 retries, and then backing off (presumably) to the
> default bucket (us-east-1?). This, of course, is making the download take a
> lot longer than it should.
>
> Does anyone know why you might see this error in jclouds:
>
> 2022-07-14 21:37:29.263 +0000,3124098302712886 {} ERROR
> o.j.h.h.BackoffLimitedRetryHandler [clrd-highpri-1] Cannot retry after
> server error, command has exceeded retry limit 5:
> [method=org.jclouds.aws.s3.AWSS3Client.public abstract java.lang.String
> org.jclouds.s3.S3Client.getBucketLocation(java.lang.String)[hammerspace-data-bucket-us-west-2],
> request=GET
> https://s3.amazonaws.com/hammerspace-data-bucket-us-west-2?location
> HTTP/1.1]
>
> Note - we are not calling jclouds' getBucketLocation. We're simply calling
> getBlob and (we believe) getBlob is attempting to call getBucketLocation
> under the covers.
>
> Thanks in advance,
> John
>
Re: getBucketLocation failing
Posted by John Calcote <jo...@gmail.com>.
Thank you!
On Sat, Oct 1, 2022, 4:41 AM Geoff Macartney <ge...@apache.org> wrote:
> Hi John,
>
> It's great you managed to get to the bottom of this problem. That's an
> exemplary answer on Stackoverflow!
>
> I'm not a jclouds dev, but to any who are reading this, it looks to me as
> if there is useful code and suggestions in John's SO answer that could
> relatively easily be adapted into a useful update to the jclouds AWS
> adaptor.
>
> Regards,
> Geoff
>
>
>
> On Fri, 30 Sep 2022, 17:32 John Calcote, <jo...@gmail.com> wrote:
>
> > Hi all - I found a solution for this problem and thought others might
> find
> > it useful. I wrote up the issue in this stack overflow issue:
> >
> >
> https://stackoverflow.com/questions/73169813/jclouds-getbucketlocation-timeout-on-getblob/73902608#73902608
> >
> >
> > On Thu, Aug 4, 2022 at 3:48 PM John Calcote <jo...@gmail.com>
> > wrote:
> >
> > > Additional info:
> > >
> > > We tried running the aws command line tool on the same host, calling
> > > getBucketLocation against the same bucket - it works there just fine -
> > and
> > > returns in a timely fashion. But the internal call to getBucketLocation
> > is
> > > failing in the jclouds aws S3 provider.
> > >
> > > Question: Is there any way to configure the endpoint such that
> > > getBucketLocation is not called by getBlob?
> > >
> > > I didn't mention it above, but we're running jclouds 2.5.0 - the
> latest,
> > > afaik.
> > >
> > > On Thu, Jul 28, 2022 at 6:29 PM John Calcote <jo...@gmail.com>
> > > wrote:
> > >
> > >> Hi all,
> > >>
> > >> We have an AWS scenario where jclouds attempt to getBucketLocation is
> > >> failing (internally) on 5 retries, and then backing off (presumably)
> to
> > the
> > >> default bucket (us-east-1?). This, of course, is making the download
> > take a
> > >> lot longer than it should.
> > >>
> > >> Does anyone know why you might see this error in jclouds:
> > >>
> > >> 2022-07-14 21:37:29.263 +0000,3124098302712886 {} ERROR
> > >> o.j.h.h.BackoffLimitedRetryHandler [clrd-highpri-1] Cannot retry after
> > >> server error, command has exceeded retry limit 5:
> > >> [method=org.jclouds.aws.s3.AWSS3Client.public abstract
> java.lang.String
> > >>
> >
> org.jclouds.s3.S3Client.getBucketLocation(java.lang.String)[hammerspace-data-bucket-us-west-2],
> > >> request=GET
> > >> https://s3.amazonaws.com/hammerspace-data-bucket-us-west-2?location
> > >> HTTP/1.1]
> > >>
> > >> Note - we are not calling jclouds' getBucketLocation. We're simply
> > >> calling getBlob and (we believe) getBlob is attempting to call
> > >> getBucketLocation under the covers.
> > >>
> > >> Thanks in advance,
> > >> John
> > >>
> > >
> >
>
Re: getBucketLocation failing
Posted by Geoff Macartney <ge...@apache.org>.
Hi John,
It's great you managed to get to the bottom of this problem. That's an
exemplary answer on Stackoverflow!
I'm not a jclouds dev, but to any who are reading this, it looks to me as
if there is useful code and suggestions in John's SO answer that could
relatively easily be adapted into a useful update to the jclouds AWS
adaptor.
Regards,
Geoff
On Fri, 30 Sep 2022, 17:32 John Calcote, <jo...@gmail.com> wrote:
> Hi all - I found a solution for this problem and thought others might find
> it useful. I wrote up the issue in this stack overflow issue:
>
> https://stackoverflow.com/questions/73169813/jclouds-getbucketlocation-timeout-on-getblob/73902608#73902608
>
>
> On Thu, Aug 4, 2022 at 3:48 PM John Calcote <jo...@gmail.com>
> wrote:
>
> > Additional info:
> >
> > We tried running the aws command line tool on the same host, calling
> > getBucketLocation against the same bucket - it works there just fine -
> and
> > returns in a timely fashion. But the internal call to getBucketLocation
> is
> > failing in the jclouds aws S3 provider.
> >
> > Question: Is there any way to configure the endpoint such that
> > getBucketLocation is not called by getBlob?
> >
> > I didn't mention it above, but we're running jclouds 2.5.0 - the latest,
> > afaik.
> >
> > On Thu, Jul 28, 2022 at 6:29 PM John Calcote <jo...@gmail.com>
> > wrote:
> >
> >> Hi all,
> >>
> >> We have an AWS scenario where jclouds attempt to getBucketLocation is
> >> failing (internally) on 5 retries, and then backing off (presumably) to
> the
> >> default bucket (us-east-1?). This, of course, is making the download
> take a
> >> lot longer than it should.
> >>
> >> Does anyone know why you might see this error in jclouds:
> >>
> >> 2022-07-14 21:37:29.263 +0000,3124098302712886 {} ERROR
> >> o.j.h.h.BackoffLimitedRetryHandler [clrd-highpri-1] Cannot retry after
> >> server error, command has exceeded retry limit 5:
> >> [method=org.jclouds.aws.s3.AWSS3Client.public abstract java.lang.String
> >>
> org.jclouds.s3.S3Client.getBucketLocation(java.lang.String)[hammerspace-data-bucket-us-west-2],
> >> request=GET
> >> https://s3.amazonaws.com/hammerspace-data-bucket-us-west-2?location
> >> HTTP/1.1]
> >>
> >> Note - we are not calling jclouds' getBucketLocation. We're simply
> >> calling getBlob and (we believe) getBlob is attempting to call
> >> getBucketLocation under the covers.
> >>
> >> Thanks in advance,
> >> John
> >>
> >
>
Re: getBucketLocation failing
Posted by John Calcote <jo...@gmail.com>.
Hi all - I found a solution for this problem and thought others might find
it useful. I wrote up the issue in this stack overflow issue:
https://stackoverflow.com/questions/73169813/jclouds-getbucketlocation-timeout-on-getblob/73902608#73902608
On Thu, Aug 4, 2022 at 3:48 PM John Calcote <jo...@gmail.com> wrote:
> Additional info:
>
> We tried running the aws command line tool on the same host, calling
> getBucketLocation against the same bucket - it works there just fine - and
> returns in a timely fashion. But the internal call to getBucketLocation is
> failing in the jclouds aws S3 provider.
>
> Question: Is there any way to configure the endpoint such that
> getBucketLocation is not called by getBlob?
>
> I didn't mention it above, but we're running jclouds 2.5.0 - the latest,
> afaik.
>
> On Thu, Jul 28, 2022 at 6:29 PM John Calcote <jo...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> We have an AWS scenario where jclouds attempt to getBucketLocation is
>> failing (internally) on 5 retries, and then backing off (presumably) to the
>> default bucket (us-east-1?). This, of course, is making the download take a
>> lot longer than it should.
>>
>> Does anyone know why you might see this error in jclouds:
>>
>> 2022-07-14 21:37:29.263 +0000,3124098302712886 {} ERROR
>> o.j.h.h.BackoffLimitedRetryHandler [clrd-highpri-1] Cannot retry after
>> server error, command has exceeded retry limit 5:
>> [method=org.jclouds.aws.s3.AWSS3Client.public abstract java.lang.String
>> org.jclouds.s3.S3Client.getBucketLocation(java.lang.String)[hammerspace-data-bucket-us-west-2],
>> request=GET
>> https://s3.amazonaws.com/hammerspace-data-bucket-us-west-2?location
>> HTTP/1.1]
>>
>> Note - we are not calling jclouds' getBucketLocation. We're simply
>> calling getBlob and (we believe) getBlob is attempting to call
>> getBucketLocation under the covers.
>>
>> Thanks in advance,
>> John
>>
>