You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Łukasz Gajowy <lu...@gmail.com> on 2017/10/25 15:08:49 UTC
[Proposal] Add performance tests for commonly used file-based I/O PTransforms
Hello Beam Community!
During the last year many of Beam developers has put much effort in
developing and discussing means of testing beam transforms. We would like
to benefit from that and implement performance tests for file-based I/O
Transforms.
This proposal is strictly related to BEAM-3060 issue. Here’s the link to
the doc:
https://docs.google.com/document/d/1dA-5s6OHiP_cz-NRAbwapoKF5MEC1wKps4A5tFbIPKE/edit
All comments are deeply appreciated.
Thanks!
ŁG
Re: [Proposal] Add performance tests for commonly used file-based I/O
PTransforms
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks for the update. I will take a look.
Regards
JB
On 11/10/2017 11:43 AM, Kamil Szewczyk wrote:
> We updated Step #2 in our proposal.
> Comments and suggestions are highly appreciated.
>
> Thanks
>
> 2017-10-31 15:42 GMT+01:00 Łukasz Gajowy <lu...@gmail.com>:
>
>> We edited the "Roadmap" section a little bit to reflect our state of
>> knowledge. As before, all comments are welcome.
>>
>> Thank you in advance!
>>
>> 2017-10-27 5:10 GMT+02:00 Kenneth Knowles <kl...@google.com.invalid>:
>>
>>> I am really excited about this development. Glad to have such a detailed
>>> document! Thanks for taking the time to write it up thoughtfully.
>>>
>>> On Wed, Oct 25, 2017 at 10:00 AM, Chamikara Jayalath <
>>> chamikara@google.com.invalid> wrote:
>>>
>>>> Thanks Łukasz and the team for the proposal. I think fixing this JIRA
>>> will
>>>> allow us to keep track of the performance of widely used
>>>> source/sink/runner/file-system combinations of Beam SDK. As Łukasz
>>>> mentioned, all comments are welcome.
>>>>
>>>> Thanks,
>>>> Cham
>>>>
>>>> On Wed, Oct 25, 2017 at 8:08 AM Łukasz Gajowy <lukasz.gajowy@gmail.com
>>>
>>>> wrote:
>>>>
>>>>> Hello Beam Community!
>>>>>
>>>>>
>>>>> During the last year many of Beam developers has put much effort in
>>>>> developing and discussing means of testing beam transforms. We would
>>> like
>>>>> to benefit from that and implement performance tests for file-based
>> I/O
>>>>> Transforms.
>>>>>
>>>>>
>>>>> This proposal is strictly related to BEAM-3060 issue. Here’s the link
>>> to
>>>>> the doc:
>>>>>
>>>>> https://docs.google.com/document/d/1dA-5s6OHiP_cz-
>>>> NRAbwapoKF5MEC1wKps4A5tFbIPKE/edit
>>>>>
>>>>>
>>>>> All comments are deeply appreciated.
>>>>>
>>>>>
>>>>> Thanks!
>>>>>
>>>>> ŁG
>>>>>
>>>>
>>>
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [Proposal] Add performance tests for commonly used file-based I/O PTransforms
Posted by Kamil Szewczyk <sz...@gmail.com>.
We updated Step #2 in our proposal.
Comments and suggestions are highly appreciated.
Thanks
2017-10-31 15:42 GMT+01:00 Łukasz Gajowy <lu...@gmail.com>:
> We edited the "Roadmap" section a little bit to reflect our state of
> knowledge. As before, all comments are welcome.
>
> Thank you in advance!
>
> 2017-10-27 5:10 GMT+02:00 Kenneth Knowles <kl...@google.com.invalid>:
>
> > I am really excited about this development. Glad to have such a detailed
> > document! Thanks for taking the time to write it up thoughtfully.
> >
> > On Wed, Oct 25, 2017 at 10:00 AM, Chamikara Jayalath <
> > chamikara@google.com.invalid> wrote:
> >
> > > Thanks Łukasz and the team for the proposal. I think fixing this JIRA
> > will
> > > allow us to keep track of the performance of widely used
> > > source/sink/runner/file-system combinations of Beam SDK. As Łukasz
> > > mentioned, all comments are welcome.
> > >
> > > Thanks,
> > > Cham
> > >
> > > On Wed, Oct 25, 2017 at 8:08 AM Łukasz Gajowy <lukasz.gajowy@gmail.com
> >
> > > wrote:
> > >
> > > > Hello Beam Community!
> > > >
> > > >
> > > > During the last year many of Beam developers has put much effort in
> > > > developing and discussing means of testing beam transforms. We would
> > like
> > > > to benefit from that and implement performance tests for file-based
> I/O
> > > > Transforms.
> > > >
> > > >
> > > > This proposal is strictly related to BEAM-3060 issue. Here’s the link
> > to
> > > > the doc:
> > > >
> > > > https://docs.google.com/document/d/1dA-5s6OHiP_cz-
> > > NRAbwapoKF5MEC1wKps4A5tFbIPKE/edit
> > > >
> > > >
> > > > All comments are deeply appreciated.
> > > >
> > > >
> > > > Thanks!
> > > >
> > > > ŁG
> > > >
> > >
> >
>
Re: [Proposal] Add performance tests for commonly used file-based I/O PTransforms
Posted by Łukasz Gajowy <lu...@gmail.com>.
We edited the "Roadmap" section a little bit to reflect our state of
knowledge. As before, all comments are welcome.
Thank you in advance!
2017-10-27 5:10 GMT+02:00 Kenneth Knowles <kl...@google.com.invalid>:
> I am really excited about this development. Glad to have such a detailed
> document! Thanks for taking the time to write it up thoughtfully.
>
> On Wed, Oct 25, 2017 at 10:00 AM, Chamikara Jayalath <
> chamikara@google.com.invalid> wrote:
>
> > Thanks Łukasz and the team for the proposal. I think fixing this JIRA
> will
> > allow us to keep track of the performance of widely used
> > source/sink/runner/file-system combinations of Beam SDK. As Łukasz
> > mentioned, all comments are welcome.
> >
> > Thanks,
> > Cham
> >
> > On Wed, Oct 25, 2017 at 8:08 AM Łukasz Gajowy <lu...@gmail.com>
> > wrote:
> >
> > > Hello Beam Community!
> > >
> > >
> > > During the last year many of Beam developers has put much effort in
> > > developing and discussing means of testing beam transforms. We would
> like
> > > to benefit from that and implement performance tests for file-based I/O
> > > Transforms.
> > >
> > >
> > > This proposal is strictly related to BEAM-3060 issue. Here’s the link
> to
> > > the doc:
> > >
> > > https://docs.google.com/document/d/1dA-5s6OHiP_cz-
> > NRAbwapoKF5MEC1wKps4A5tFbIPKE/edit
> > >
> > >
> > > All comments are deeply appreciated.
> > >
> > >
> > > Thanks!
> > >
> > > ŁG
> > >
> >
>
Re: [Proposal] Add performance tests for commonly used file-based I/O PTransforms
Posted by Kenneth Knowles <kl...@google.com.INVALID>.
I am really excited about this development. Glad to have such a detailed
document! Thanks for taking the time to write it up thoughtfully.
On Wed, Oct 25, 2017 at 10:00 AM, Chamikara Jayalath <
chamikara@google.com.invalid> wrote:
> Thanks Łukasz and the team for the proposal. I think fixing this JIRA will
> allow us to keep track of the performance of widely used
> source/sink/runner/file-system combinations of Beam SDK. As Łukasz
> mentioned, all comments are welcome.
>
> Thanks,
> Cham
>
> On Wed, Oct 25, 2017 at 8:08 AM Łukasz Gajowy <lu...@gmail.com>
> wrote:
>
> > Hello Beam Community!
> >
> >
> > During the last year many of Beam developers has put much effort in
> > developing and discussing means of testing beam transforms. We would like
> > to benefit from that and implement performance tests for file-based I/O
> > Transforms.
> >
> >
> > This proposal is strictly related to BEAM-3060 issue. Here’s the link to
> > the doc:
> >
> > https://docs.google.com/document/d/1dA-5s6OHiP_cz-
> NRAbwapoKF5MEC1wKps4A5tFbIPKE/edit
> >
> >
> > All comments are deeply appreciated.
> >
> >
> > Thanks!
> >
> > ŁG
> >
>
Re: [Proposal] Add performance tests for commonly used file-based I/O PTransforms
Posted by Chamikara Jayalath <ch...@google.com.INVALID>.
Thanks Łukasz and the team for the proposal. I think fixing this JIRA will
allow us to keep track of the performance of widely used
source/sink/runner/file-system combinations of Beam SDK. As Łukasz
mentioned, all comments are welcome.
Thanks,
Cham
On Wed, Oct 25, 2017 at 8:08 AM Łukasz Gajowy <lu...@gmail.com>
wrote:
> Hello Beam Community!
>
>
> During the last year many of Beam developers has put much effort in
> developing and discussing means of testing beam transforms. We would like
> to benefit from that and implement performance tests for file-based I/O
> Transforms.
>
>
> This proposal is strictly related to BEAM-3060 issue. Here’s the link to
> the doc:
>
> https://docs.google.com/document/d/1dA-5s6OHiP_cz-NRAbwapoKF5MEC1wKps4A5tFbIPKE/edit
>
>
> All comments are deeply appreciated.
>
>
> Thanks!
>
> ŁG
>