You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Hilmi Yildirim <hi...@neofonie.de> on 2015/05/22 15:52:33 UTC
Write File on specific machine
Hi,
I want to write the result of a batch process into a file. I want that
the result is written on a specific machine. For example, I have 4
machines (m1.example.com, m2.example.com, m3.example.com,
m4.example.com). Now, I want that the results go to m1 and that m1
writes the result in a file in its local filesystem.
Does anyone knows how I can do that?
Best Regards,
Hilmi
--
--
Hilmi Yildirim
Software Developer R&D
http://www.neofonie.de
Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/
Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh
Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko
Re: Write File on specific machine
Posted by Fabian Hueske <fh...@gmail.com>.
A work around might be to set the parallelism to 1 as Till suggested and
write into a network-shared directory.
Nonetheless, it sounds like a good feature to support strict assignments of
data sink tasks to machines.
Fabian
2015-05-22 16:10 GMT+02:00 Hilmi Yildirim <hi...@neofonie.de>:
> Hi Till,
> thank you very much.
>
> BR,
> Hilmi
>
>
> Am 22.05.2015 um 16:09 schrieb Till Rohrmann:
>
> Hi Hilmi,
>
> in Flink you cannot control on which machines the individual tasks are
> scheduled. Therefore, you cannot control on which machine the data is
> written. However, you can control that only on one machine a file is
> created by setting the degree of parallelism of the DataSink to *1*.
>
> Cheers,
> Till
>
>
> On Fri, May 22, 2015 at 3:52 PM, Hilmi Yildirim <
> hilmi.yildirim@neofonie.de> wrote:
>
>> Hi,
>> I want to write the result of a batch process into a file. I want that
>> the result is written on a specific machine. For example, I have 4 machines
>> (m1.example.com, m2.example.com, m3.example.com, m4.example.com). Now, I
>> want that the results go to m1 and that m1 writes the result in a file in
>> its local filesystem.
>>
>> Does anyone knows how I can do that?
>>
>> Best Regards,
>> Hilmi
>>
>> --
>> --
>> Hilmi Yildirim
>> Software Developer R&D
>>
>> http://www.neofonie.de
>>
>> Besuchen Sie den Neo Tech Blog für Anwender:
>> http://blog.neofonie.de/
>>
>> Folgen Sie uns:
>> https://plus.google.com/+neofonie
>> http://www.linkedin.com/company/neofonie-gmbh
>> https://www.xing.com/companies/neofoniegmbh
>>
>> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
>> Handelsregister Berlin-Charlottenburg: HRB 67460
>> Geschäftsführung: Thomas Kitlitschko
>>
>>
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
> T: +49 30 24627-281Hilmi.Yildirim@neofonie.de
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:http://blog.neofonie.de/
>
> Folgen Sie uns:https://plus.google.com/+neofoniehttp://www.linkedin.com/company/neofonie-gmbhhttps://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
>
>
Re: Write File on specific machine
Posted by Hilmi Yildirim <hi...@neofonie.de>.
Hi Till,
thank you very much.
BR,
Hilmi
Am 22.05.2015 um 16:09 schrieb Till Rohrmann:
>
> Hi Hilmi,
>
> in Flink you cannot control on which machines the individual tasks are
> scheduled. Therefore, you cannot control on which machine the data is
> written. However, you can control that only on one machine a file is
> created by setting the degree of parallelism of the |DataSink| to *1*.
>
> Cheers,
> Till
>
>
>
> On Fri, May 22, 2015 at 3:52 PM, Hilmi Yildirim
> <hilmi.yildirim@neofonie.de <ma...@neofonie.de>> wrote:
>
> Hi,
> I want to write the result of a batch process into a file. I want
> that the result is written on a specific machine. For example, I
> have 4 machines (m1.example.com <http://m1.example.com>,
> m2.example.com <http://m2.example.com>, m3.example.com
> <http://m3.example.com>, m4.example.com <http://m4.example.com>).
> Now, I want that the results go to m1 and that m1 writes the
> result in a file in its local filesystem.
>
> Does anyone knows how I can do that?
>
> Best Regards,
> Hilmi
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:
> http://blog.neofonie.de/
>
> Folgen Sie uns:
> https://plus.google.com/+neofonie
> http://www.linkedin.com/company/neofonie-gmbh
> https://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
>
>
--
--
Hilmi Yildirim
Software Developer R&D
T: +49 30 24627-281
Hilmi.Yildirim@neofonie.de
http://www.neofonie.de
Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/
Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh
Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko
Re: Write File on specific machine
Posted by Till Rohrmann <tr...@apache.org>.
Hi Hilmi,
in Flink you cannot control on which machines the individual tasks are
scheduled. Therefore, you cannot control on which machine the data is
written. However, you can control that only on one machine a file is
created by setting the degree of parallelism of the DataSink to *1*.
Cheers,
Till
On Fri, May 22, 2015 at 3:52 PM, Hilmi Yildirim <hi...@neofonie.de>
wrote:
> Hi,
> I want to write the result of a batch process into a file. I want that the
> result is written on a specific machine. For example, I have 4 machines (
> m1.example.com, m2.example.com, m3.example.com, m4.example.com). Now, I
> want that the results go to m1 and that m1 writes the result in a file in
> its local filesystem.
>
> Does anyone knows how I can do that?
>
> Best Regards,
> Hilmi
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:
> http://blog.neofonie.de/
>
> Folgen Sie uns:
> https://plus.google.com/+neofonie
> http://www.linkedin.com/company/neofonie-gmbh
> https://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
>
>