You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@whirr.apache.org by Karel Vervaeke <ka...@outerthought.org> on 2011/08/02 08:37:10 UTC
Re: Problem using blobstore
Hi there.
Yesterday I have digging into this problem. Before the end of the day
my uploads to eu-west-1 started working intermittently, and by the end
of the day I couldn't reproduce the problem anymore (without any
changes to the code).
I guess I don't have enough cloud-karma.
For what it's worth I have sent a pull request with a partial request
for WHIRR-349
What I've done is just remove the prefix 'file://' from the url. I
don't know if that's correct way to do it for Windows-style paths.
It also includes a testcase for uploading slightly larger blobs (a 1MB
random alphanumeric string).
I haven't gone into the multiparts uploads.
Regards,
Karel
On Fri, Jul 29, 2011 at 11:53 AM, Karel Vervaeke <ka...@outerthought.org> wrote:
> I just got around to testing all 4 combinations:
>
> jclouds 1.0.0 + eu-west-1: doesn't work
> jclouds 1.0.0 + us-east-1: works
> jclouds 1.1.0-SNAPSHOT + eu-west-1: doesn't work
> jclouds 1.1.0-SNAPSHOT + us-east-1: works
>
> Would you be willing to try using the s3 blobstore on eu-west-1 so I
> know if it's just me or not?
>
> Regards,
> Karel
>
> On Fri, Jul 29, 2011 at 10:49 AM, Karel Vervaeke <ka...@outerthought.org> wrote:
>> with jclouds 1.0.0 it failed with eu-west-1 (several attempts)
>> with jclouds 1.0.0 it worked with us-east-1 on the first try.
>>
>> I still have to try with jclouds 1.1.0-SNAPSHOT on us-east-1, but if that works,
>> the problem is related to eu-west-1.
>>
>> Regards,
>> Karel
>>
>> On Fri, Jul 29, 2011 at 9:49 AM, Karel Vervaeke <ka...@outerthought.org> wrote:
>>> Ah, Bruno just alerted me that I need to mention that I am using his
>>> (github) whirr fork, which is using a jclouds 1.1 snapshot.
>>>
>>> I can fall back to using jclouds 1.0 for a second (with the limitation
>>> that I won't be able to use eu-west-1. Just give me a second while I
>>> try that. (And in the mean time I'll open the upload-retry issue)
>>>
>>> Regards,
>>> Karel
>>>
>>> On Fri, Jul 29, 2011 at 9:28 AM, Andrei Savu <sa...@gmail.com> wrote:
>>>> I've just took a quick look at the AWS service status dashboard and
>>>> everything seems to be working there.
>>>>
>>>> The error you are seeing when trying to upload a larger file is
>>>> something that I believe we should handle by retrying the upload from
>>>> scratch. It would be great if you could open an issue with this
>>>> suggestion.
>>>>
>>>> I have no idea why the small upload is failing because we are doing
>>>> the same thing in the integration test and it works. I will spend some
>>>> more time on this later tomorrow. Adrian any ideas? Have you seen
>>>> something similar?
>>>>
>>>> -- Andrei Savu / andreisavu.ro
>>>>
>>>> On Fri, Jul 29, 2011 at 12:06 AM, Karel Vervaeke <ka...@outerthought.org> wrote:
>>>>> It is pretty consistent.
>>>>>
>>>>> Today there's something new on the menu...
>>>>> http://pastebin.com/MSKDn6rt
>>>>> (strange, because that's using the same file I was yesterday, so
>>>>> nothing really changed except today's date)
>>>>>
>>>>> The number of bytes already written (mentioned in the stack trace) is
>>>>> always different.
>>>>>
>>>>> When I replace the file with something very small (184 bytes, to be exact),
>>>>> I get the original error again.
>>>>>
>>>>> Regards,
>>>>> Karel
>>>>>
>>>>> On Thu, Jul 28, 2011 at 7:17 PM, Andrei Savu <sa...@gmail.com> wrote:
>>>>>> I've wrote that code but this is the first time I see this error.
>>>>>>
>>>>>> I tend to believe it's somehow related to jclouds. Does it happen all
>>>>>> the time? How about with different files?
>>>>>>
>>>>>> -- Andrei Savu / andreisavu.ro
>>>>>>
>>>>>> On Thu, Jul 28, 2011 at 8:59 AM, Karel Vervaeke <ka...@outerthought.org> wrote:
>>>>>>> I'm trying out the local file upload from WHIRR-220, but I'm running
>>>>>>> in to a problem:
>>>>>>>
>>>>>>> While calling prepareRemoteFileUrl(...) (during the install phase), I
>>>>>>> get this as the last bit of output from whirr on the console:
>>>>>>> http://pastebin.com/LZ92AUhQ
>>>>>>>
>>>>>>> After that there's no more output on the console for several minutes
>>>>>>> (I just killed the process after that)
>>>>>>>
>>>>>>> I have tried the BlobCacheTest, which seems to work fine for my AWS settings.
>>>>>>> Before I start digging into jcloud's code, does anyone have any
>>>>>>> thought as to what might cause this?
>>>>>>>
>>>>>>> --
>>>>>>> Karel Vervaeke
>>>>>>> http://outerthought.org/
>>>>>>> Open Source Content Applications
>>>>>>> Makers of Kauri, Daisy CMS and Lily
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Karel Vervaeke
>>>>> http://outerthought.org/
>>>>> Open Source Content Applications
>>>>> Makers of Kauri, Daisy CMS and Lily
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Karel Vervaeke
>>> http://outerthought.org/
>>> Open Source Content Applications
>>> Makers of Kauri, Daisy CMS and Lily
>>>
>>
>>
>>
>> --
>> Karel Vervaeke
>> http://outerthought.org/
>> Open Source Content Applications
>> Makers of Kauri, Daisy CMS and Lily
>>
>
>
>
> --
> Karel Vervaeke
> http://outerthought.org/
> Open Source Content Applications
> Makers of Kauri, Daisy CMS and Lily
>
--
Karel Vervaeke
http://outerthought.org/
Open Source Content Applications
Makers of Kauri, Daisy CMS and Lily
Re: Problem using blobstore
Posted by Karel Vervaeke <ka...@outerthought.org>.
I've had a couple more failures yesterday. Today, I hit another one
(after 2 succesful runs).
I'm attaching the console output, the surefire-report and the
jclouds.wire/header logs found under target/test-data to the issue.
On Tue, Aug 2, 2011 at 10:17 PM, Andrei Savu <sa...@gmail.com> wrote:
> Looks good to me but I'm not convinced that removing the file:// prefix is
> the right way of handling paths on Windows.
>
> I was also suspecting an intermittent AWS failure. I'm glad they managed to
> fix it.
>
> Could you create a patch from that pull request and attach to WHIRR-349 so
> that we can also handle the multipart uploads before committing?
>
> Thanks,
>
> -- Andrei Savu / andreisavu.ro
>
> On Mon, Aug 1, 2011 at 11:37 PM, Karel Vervaeke <ka...@outerthought.org>
> wrote:
>>
>> Hi there.
>>
>> Yesterday I have digging into this problem. Before the end of the day
>> my uploads to eu-west-1 started working intermittently, and by the end
>> of the day I couldn't reproduce the problem anymore (without any
>> changes to the code).
>>
>> I guess I don't have enough cloud-karma.
>>
>> For what it's worth I have sent a pull request with a partial request
>> for WHIRR-349
>> What I've done is just remove the prefix 'file://' from the url. I
>> don't know if that's correct way to do it for Windows-style paths.
>> It also includes a testcase for uploading slightly larger blobs (a 1MB
>> random alphanumeric string).
>> I haven't gone into the multiparts uploads.
>>
>> Regards,
>> Karel
>>
>> On Fri, Jul 29, 2011 at 11:53 AM, Karel Vervaeke <ka...@outerthought.org>
>> wrote:
>> > I just got around to testing all 4 combinations:
>> >
>> > jclouds 1.0.0 + eu-west-1: doesn't work
>> > jclouds 1.0.0 + us-east-1: works
>> > jclouds 1.1.0-SNAPSHOT + eu-west-1: doesn't work
>> > jclouds 1.1.0-SNAPSHOT + us-east-1: works
>> >
>> > Would you be willing to try using the s3 blobstore on eu-west-1 so I
>> > know if it's just me or not?
>> >
>> > Regards,
>> > Karel
>> >
>> > On Fri, Jul 29, 2011 at 10:49 AM, Karel Vervaeke
>> > <ka...@outerthought.org> wrote:
>> >> with jclouds 1.0.0 it failed with eu-west-1 (several attempts)
>> >> with jclouds 1.0.0 it worked with us-east-1 on the first try.
>> >>
>> >> I still have to try with jclouds 1.1.0-SNAPSHOT on us-east-1, but if
>> >> that works,
>> >> the problem is related to eu-west-1.
>> >>
>> >> Regards,
>> >> Karel
>> >>
>> >> On Fri, Jul 29, 2011 at 9:49 AM, Karel Vervaeke
>> >> <ka...@outerthought.org> wrote:
>> >>> Ah, Bruno just alerted me that I need to mention that I am using his
>> >>> (github) whirr fork, which is using a jclouds 1.1 snapshot.
>> >>>
>> >>> I can fall back to using jclouds 1.0 for a second (with the limitation
>> >>> that I won't be able to use eu-west-1. Just give me a second while I
>> >>> try that. (And in the mean time I'll open the upload-retry issue)
>> >>>
>> >>> Regards,
>> >>> Karel
>> >>>
>> >>> On Fri, Jul 29, 2011 at 9:28 AM, Andrei Savu <sa...@gmail.com>
>> >>> wrote:
>> >>>> I've just took a quick look at the AWS service status dashboard and
>> >>>> everything seems to be working there.
>> >>>>
>> >>>> The error you are seeing when trying to upload a larger file is
>> >>>> something that I believe we should handle by retrying the upload from
>> >>>> scratch. It would be great if you could open an issue with this
>> >>>> suggestion.
>> >>>>
>> >>>> I have no idea why the small upload is failing because we are doing
>> >>>> the same thing in the integration test and it works. I will spend
>> >>>> some
>> >>>> more time on this later tomorrow. Adrian any ideas? Have you seen
>> >>>> something similar?
>> >>>>
>> >>>> -- Andrei Savu / andreisavu.ro
>> >>>>
>> >>>> On Fri, Jul 29, 2011 at 12:06 AM, Karel Vervaeke
>> >>>> <ka...@outerthought.org> wrote:
>> >>>>> It is pretty consistent.
>> >>>>>
>> >>>>> Today there's something new on the menu...
>> >>>>> http://pastebin.com/MSKDn6rt
>> >>>>> (strange, because that's using the same file I was yesterday, so
>> >>>>> nothing really changed except today's date)
>> >>>>>
>> >>>>> The number of bytes already written (mentioned in the stack trace)
>> >>>>> is
>> >>>>> always different.
>> >>>>>
>> >>>>> When I replace the file with something very small (184 bytes, to be
>> >>>>> exact),
>> >>>>> I get the original error again.
>> >>>>>
>> >>>>> Regards,
>> >>>>> Karel
>> >>>>>
>> >>>>> On Thu, Jul 28, 2011 at 7:17 PM, Andrei Savu <sa...@gmail.com>
>> >>>>> wrote:
>> >>>>>> I've wrote that code but this is the first time I see this error.
>> >>>>>>
>> >>>>>> I tend to believe it's somehow related to jclouds. Does it happen
>> >>>>>> all
>> >>>>>> the time? How about with different files?
>> >>>>>>
>> >>>>>> -- Andrei Savu / andreisavu.ro
>> >>>>>>
>> >>>>>> On Thu, Jul 28, 2011 at 8:59 AM, Karel Vervaeke
>> >>>>>> <ka...@outerthought.org> wrote:
>> >>>>>>> I'm trying out the local file upload from WHIRR-220, but I'm
>> >>>>>>> running
>> >>>>>>> in to a problem:
>> >>>>>>>
>> >>>>>>> While calling prepareRemoteFileUrl(...) (during the install
>> >>>>>>> phase), I
>> >>>>>>> get this as the last bit of output from whirr on the console:
>> >>>>>>> http://pastebin.com/LZ92AUhQ
>> >>>>>>>
>> >>>>>>> After that there's no more output on the console for several
>> >>>>>>> minutes
>> >>>>>>> (I just killed the process after that)
>> >>>>>>>
>> >>>>>>> I have tried the BlobCacheTest, which seems to work fine for my
>> >>>>>>> AWS settings.
>> >>>>>>> Before I start digging into jcloud's code, does anyone have any
>> >>>>>>> thought as to what might cause this?
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Karel Vervaeke
>> >>>>>>> http://outerthought.org/
>> >>>>>>> Open Source Content Applications
>> >>>>>>> Makers of Kauri, Daisy CMS and Lily
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Karel Vervaeke
>> >>>>> http://outerthought.org/
>> >>>>> Open Source Content Applications
>> >>>>> Makers of Kauri, Daisy CMS and Lily
>> >>>>>
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Karel Vervaeke
>> >>> http://outerthought.org/
>> >>> Open Source Content Applications
>> >>> Makers of Kauri, Daisy CMS and Lily
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Karel Vervaeke
>> >> http://outerthought.org/
>> >> Open Source Content Applications
>> >> Makers of Kauri, Daisy CMS and Lily
>> >>
>> >
>> >
>> >
>> > --
>> > Karel Vervaeke
>> > http://outerthought.org/
>> > Open Source Content Applications
>> > Makers of Kauri, Daisy CMS and Lily
>> >
>>
>>
>>
>> --
>> Karel Vervaeke
>> http://outerthought.org/
>> Open Source Content Applications
>> Makers of Kauri, Daisy CMS and Lily
>
>
--
Karel Vervaeke
http://outerthought.org/
Open Source Content Applications
Makers of Kauri, Daisy CMS and Lily
Re: Problem using blobstore
Posted by Andrei Savu <sa...@gmail.com>.
Looks good to me but I'm not convinced that removing the file:// prefix is
the right way of handling paths on Windows.
I was also suspecting an intermittent AWS failure. I'm glad they managed to
fix it.
Could you create a patch from that pull request and attach to WHIRR-349 so
that we can also handle the multipart uploads before committing?
Thanks,
-- Andrei Savu / andreisavu.ro
On Mon, Aug 1, 2011 at 11:37 PM, Karel Vervaeke <ka...@outerthought.org>wrote:
> Hi there.
>
> Yesterday I have digging into this problem. Before the end of the day
> my uploads to eu-west-1 started working intermittently, and by the end
> of the day I couldn't reproduce the problem anymore (without any
> changes to the code).
>
> I guess I don't have enough cloud-karma.
>
> For what it's worth I have sent a pull request with a partial request
> for WHIRR-349
> What I've done is just remove the prefix 'file://' from the url. I
> don't know if that's correct way to do it for Windows-style paths.
> It also includes a testcase for uploading slightly larger blobs (a 1MB
> random alphanumeric string).
> I haven't gone into the multiparts uploads.
>
> Regards,
> Karel
>
> On Fri, Jul 29, 2011 at 11:53 AM, Karel Vervaeke <ka...@outerthought.org>
> wrote:
> > I just got around to testing all 4 combinations:
> >
> > jclouds 1.0.0 + eu-west-1: doesn't work
> > jclouds 1.0.0 + us-east-1: works
> > jclouds 1.1.0-SNAPSHOT + eu-west-1: doesn't work
> > jclouds 1.1.0-SNAPSHOT + us-east-1: works
> >
> > Would you be willing to try using the s3 blobstore on eu-west-1 so I
> > know if it's just me or not?
> >
> > Regards,
> > Karel
> >
> > On Fri, Jul 29, 2011 at 10:49 AM, Karel Vervaeke <ka...@outerthought.org>
> wrote:
> >> with jclouds 1.0.0 it failed with eu-west-1 (several attempts)
> >> with jclouds 1.0.0 it worked with us-east-1 on the first try.
> >>
> >> I still have to try with jclouds 1.1.0-SNAPSHOT on us-east-1, but if
> that works,
> >> the problem is related to eu-west-1.
> >>
> >> Regards,
> >> Karel
> >>
> >> On Fri, Jul 29, 2011 at 9:49 AM, Karel Vervaeke <ka...@outerthought.org>
> wrote:
> >>> Ah, Bruno just alerted me that I need to mention that I am using his
> >>> (github) whirr fork, which is using a jclouds 1.1 snapshot.
> >>>
> >>> I can fall back to using jclouds 1.0 for a second (with the limitation
> >>> that I won't be able to use eu-west-1. Just give me a second while I
> >>> try that. (And in the mean time I'll open the upload-retry issue)
> >>>
> >>> Regards,
> >>> Karel
> >>>
> >>> On Fri, Jul 29, 2011 at 9:28 AM, Andrei Savu <sa...@gmail.com>
> wrote:
> >>>> I've just took a quick look at the AWS service status dashboard and
> >>>> everything seems to be working there.
> >>>>
> >>>> The error you are seeing when trying to upload a larger file is
> >>>> something that I believe we should handle by retrying the upload from
> >>>> scratch. It would be great if you could open an issue with this
> >>>> suggestion.
> >>>>
> >>>> I have no idea why the small upload is failing because we are doing
> >>>> the same thing in the integration test and it works. I will spend some
> >>>> more time on this later tomorrow. Adrian any ideas? Have you seen
> >>>> something similar?
> >>>>
> >>>> -- Andrei Savu / andreisavu.ro
> >>>>
> >>>> On Fri, Jul 29, 2011 at 12:06 AM, Karel Vervaeke <
> karel@outerthought.org> wrote:
> >>>>> It is pretty consistent.
> >>>>>
> >>>>> Today there's something new on the menu...
> >>>>> http://pastebin.com/MSKDn6rt
> >>>>> (strange, because that's using the same file I was yesterday, so
> >>>>> nothing really changed except today's date)
> >>>>>
> >>>>> The number of bytes already written (mentioned in the stack trace) is
> >>>>> always different.
> >>>>>
> >>>>> When I replace the file with something very small (184 bytes, to be
> exact),
> >>>>> I get the original error again.
> >>>>>
> >>>>> Regards,
> >>>>> Karel
> >>>>>
> >>>>> On Thu, Jul 28, 2011 at 7:17 PM, Andrei Savu <sa...@gmail.com>
> wrote:
> >>>>>> I've wrote that code but this is the first time I see this error.
> >>>>>>
> >>>>>> I tend to believe it's somehow related to jclouds. Does it happen
> all
> >>>>>> the time? How about with different files?
> >>>>>>
> >>>>>> -- Andrei Savu / andreisavu.ro
> >>>>>>
> >>>>>> On Thu, Jul 28, 2011 at 8:59 AM, Karel Vervaeke <
> karel@outerthought.org> wrote:
> >>>>>>> I'm trying out the local file upload from WHIRR-220, but I'm
> running
> >>>>>>> in to a problem:
> >>>>>>>
> >>>>>>> While calling prepareRemoteFileUrl(...) (during the install phase),
> I
> >>>>>>> get this as the last bit of output from whirr on the console:
> >>>>>>> http://pastebin.com/LZ92AUhQ
> >>>>>>>
> >>>>>>> After that there's no more output on the console for several
> minutes
> >>>>>>> (I just killed the process after that)
> >>>>>>>
> >>>>>>> I have tried the BlobCacheTest, which seems to work fine for my AWS
> settings.
> >>>>>>> Before I start digging into jcloud's code, does anyone have any
> >>>>>>> thought as to what might cause this?
> >>>>>>>
> >>>>>>> --
> >>>>>>> Karel Vervaeke
> >>>>>>> http://outerthought.org/
> >>>>>>> Open Source Content Applications
> >>>>>>> Makers of Kauri, Daisy CMS and Lily
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Karel Vervaeke
> >>>>> http://outerthought.org/
> >>>>> Open Source Content Applications
> >>>>> Makers of Kauri, Daisy CMS and Lily
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Karel Vervaeke
> >>> http://outerthought.org/
> >>> Open Source Content Applications
> >>> Makers of Kauri, Daisy CMS and Lily
> >>>
> >>
> >>
> >>
> >> --
> >> Karel Vervaeke
> >> http://outerthought.org/
> >> Open Source Content Applications
> >> Makers of Kauri, Daisy CMS and Lily
> >>
> >
> >
> >
> > --
> > Karel Vervaeke
> > http://outerthought.org/
> > Open Source Content Applications
> > Makers of Kauri, Daisy CMS and Lily
> >
>
>
>
> --
> Karel Vervaeke
> http://outerthought.org/
> Open Source Content Applications
> Makers of Kauri, Daisy CMS and Lily
>