You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by OrielResearch Eila Arich-Landkof <ei...@orielresearch.org> on 2018/04/26 16:02:05 UTC
saving image object on GCS from DoFn
Hello all,
I am running the following simplified code from DoFn (ParDo)
from PIL import Image
img = Image.fromarray(array)img.save('testrgb.png')
img.save() with gs:// drive does not work.
What would be the recommended way to save the img object on google drive as
.png file
any advice is appreciated
Many thanks,
--
Eila
www.orielresearch.org
https://www.meetup.com/Deep-Learning-In-Production/
Re: saving image object on GCS from DoFn
Posted by OrielResearch Eila Arich-Landkof <ei...@orielresearch.org>.
👍 thank you!
On Fri, Apr 27, 2018 at 11:03 AM, Eugene Kirpichov <ki...@google.com>
wrote:
> You need to close the file object, I think that should fix the issue.
>
>
> On Fri, Apr 27, 2018, 7:47 AM OrielResearch Eila Arich-Landkof <
> eila@orielresearch.org> wrote:
>
>> I tried with the following code:
>> No error was fired but the file was not created as well
>>
>> import apache_beam.io.filesystems as filesystems
>> fileHandle = filesystems.FileSystems.create(path="gs://bucket/try.png")
>> im.save(fileHandle, "PNG")
>>
>>
>> Could I save it locally on the working machine and copy to gs://
>> is there any unseen data risk involved with that?
>> the code will be the following:
>>
>> im.save("try.png", "PNG")
>> !gsutil cp try.png gs://bucket/try.png
>>
>> Please let me know what your thoughts are
>>
>> Best,
>> Eila
>>
>> On Thu, Apr 26, 2018 at 2:00 PM, Eugene Kirpichov <ki...@google.com>
>> wrote:
>>
>>> You can use FileSystems.create()
>>> <https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/filesystems.py>
>>> to create a file on gs:// and you can pass the result of that method to
>>> img.save().
>>>
>>>
>>> On Thu, Apr 26, 2018, 9:02 AM OrielResearch Eila Arich-Landkof <
>>> eila@orielresearch.org> wrote:
>>>
>>>> Hello all,
>>>>
>>>> I am running the following simplified code from DoFn (ParDo)
>>>>
>>>> from PIL import Image
>>>> img = Image.fromarray(array)img.save('testrgb.png')
>>>>
>>>>
>>>> img.save() with gs:// drive does not work.
>>>> What would be the recommended way to save the img object on google
>>>> drive as .png file
>>>> any advice is appreciated
>>>>
>>>> Many thanks,
>>>> --
>>>> Eila
>>>> www.orielresearch.org
>>>> https://www.meetup.com/Deep-Learning-In-Production/
>>>>
>>>
>>
>>
>> --
>> Eila
>> www.orielresearch.org
>> https://www.meetup.com/Deep-Learning-In-Production/
>>
>
--
Eila
www.orielresearch.org
https://www.meetup.com/Deep-Learning-In-Production/
Re: saving image object on GCS from DoFn
Posted by Eugene Kirpichov <ki...@google.com>.
You need to close the file object, I think that should fix the issue.
On Fri, Apr 27, 2018, 7:47 AM OrielResearch Eila Arich-Landkof <
eila@orielresearch.org> wrote:
> I tried with the following code:
> No error was fired but the file was not created as well
>
> import apache_beam.io.filesystems as filesystems
> fileHandle = filesystems.FileSystems.create(path="gs://bucket/try.png")
> im.save(fileHandle, "PNG")
>
>
> Could I save it locally on the working machine and copy to gs://
> is there any unseen data risk involved with that?
> the code will be the following:
>
> im.save("try.png", "PNG")
> !gsutil cp try.png gs://bucket/try.png
>
> Please let me know what your thoughts are
>
> Best,
> Eila
>
> On Thu, Apr 26, 2018 at 2:00 PM, Eugene Kirpichov <ki...@google.com>
> wrote:
>
>> You can use FileSystems.create()
>> <https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/filesystems.py>
>> to create a file on gs:// and you can pass the result of that method to
>> img.save().
>>
>>
>> On Thu, Apr 26, 2018, 9:02 AM OrielResearch Eila Arich-Landkof <
>> eila@orielresearch.org> wrote:
>>
>>> Hello all,
>>>
>>> I am running the following simplified code from DoFn (ParDo)
>>>
>>> from PIL import Image
>>> img = Image.fromarray(array)img.save('testrgb.png')
>>>
>>>
>>> img.save() with gs:// drive does not work.
>>> What would be the recommended way to save the img object on google drive
>>> as .png file
>>> any advice is appreciated
>>>
>>> Many thanks,
>>> --
>>> Eila
>>> www.orielresearch.org
>>> https://www.meetup.com/Deep-Learning-In-Production/
>>>
>>
>
>
> --
> Eila
> www.orielresearch.org
> https://www.meetup.com/Deep-Learning-In-Production/
>
Re: saving image object on GCS from DoFn
Posted by OrielResearch Eila Arich-Landkof <ei...@orielresearch.org>.
I tried with the following code:
No error was fired but the file was not created as well
import apache_beam.io.filesystems as filesystems
fileHandle = filesystems.FileSystems.create(path="gs://bucket/try.png")
im.save(fileHandle, "PNG")
Could I save it locally on the working machine and copy to gs://
is there any unseen data risk involved with that?
the code will be the following:
im.save("try.png", "PNG")
!gsutil cp try.png gs://bucket/try.png
Please let me know what your thoughts are
Best,
Eila
On Thu, Apr 26, 2018 at 2:00 PM, Eugene Kirpichov <ki...@google.com>
wrote:
> You can use FileSystems.create()
> <https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/filesystems.py>
> to create a file on gs:// and you can pass the result of that method to
> img.save().
>
>
> On Thu, Apr 26, 2018, 9:02 AM OrielResearch Eila Arich-Landkof <
> eila@orielresearch.org> wrote:
>
>> Hello all,
>>
>> I am running the following simplified code from DoFn (ParDo)
>>
>> from PIL import Image
>> img = Image.fromarray(array)img.save('testrgb.png')
>>
>>
>> img.save() with gs:// drive does not work.
>> What would be the recommended way to save the img object on google drive
>> as .png file
>> any advice is appreciated
>>
>> Many thanks,
>> --
>> Eila
>> www.orielresearch.org
>> https://www.meetup.com/Deep-Learning-In-Production/
>>
>
--
Eila
www.orielresearch.org
https://www.meetup.com/Deep-Learning-In-Production/
Re: saving image object on GCS from DoFn
Posted by Eugene Kirpichov <ki...@google.com>.
You can use FileSystems.create()
<https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/filesystems.py>
to create a file on gs:// and you can pass the result of that method to
img.save().
On Thu, Apr 26, 2018, 9:02 AM OrielResearch Eila Arich-Landkof <
eila@orielresearch.org> wrote:
> Hello all,
>
> I am running the following simplified code from DoFn (ParDo)
>
> from PIL import Image
> img = Image.fromarray(array)img.save('testrgb.png')
>
>
> img.save() with gs:// drive does not work.
> What would be the recommended way to save the img object on google drive
> as .png file
> any advice is appreciated
>
> Many thanks,
> --
> Eila
> www.orielresearch.org
> https://www.meetup.com/Deep-Learning-In-Production/
>