You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Maximilian Michels <mx...@apache.org> on 2020/05/12 17:21:32 UTC
Re: Running NexMark Tests
A heads-up if anybody else sees this, we have removed the flag:
https://jira.apache.org/jira/browse/BEAM-9900
Further contributions are very welcome :)
-Max
On 11.05.20 17:05, Sruthi Sree Kumar wrote:
> I have opened a PR with the documentation change.
> https://github.com/apache/beam/pull/11662
>
> Regards,
> Sruthi
>
> On 2020/04/21 20:22:17, Ismaël Mejía <ie...@gmail.com> wrote:
>> You need to instruct the Flink runner to shutdown the the source
>> otherwise it will stay waiting.
>> You can this by adding the extra
>> argument`--shutdownSourcesOnFinalWatermark=true`
>> And if that works and you want to open a PR to update our
>> documentation that would be greatly appreciated.
>>
>> Regards,
>> Ismaël
>>
>>
>> On Tue, Apr 21, 2020 at 10:04 PM Sruthi Sree Kumar
>> <sr...@gmail.com> wrote:
>>>
>>> Hello,
>>>
>>> I am trying to run nexmark queries using flink runner streaming. Followed the documentation and used the command
>>> ./gradlew :sdks:java:testing:nexmark:run \
>>>
>>> -Pnexmark.runner=":runners:flink:1.10" \
>>> -Pnexmark.args="
>>> --runner=FlinkRunner
>>> --suite=SMOKE
>>> --streamTimeout=60
>>> --streaming=true
>>> --manageResources=false
>>> --monitorJobs=true
>>> --flinkMaster=[local]"
>>>
>>>
>>> But after the events are read from the source, there is no further progress and the job is always stuck at 99%. Is there any configuration that I am missing?
>>>
>>> Regards,
>>> Sruthi
>>
Re: Running NexMark Tests
Posted by Sruthi Sree Kumar <sr...@gmail.com>.
PR for the update: https://github.com/apache/beam/pull/11751
Regards,
Sruthi
On Tue, May 19, 2020 at 3:51 PM Maximilian Michels <mx...@apache.org> wrote:
> Looks like an accidental change to me. Running with either version, 1.9
> or 1.10 works, but this should be changed back to using the latest version.
>
> Do you mind creating a PR?
>
> Thanks,
> Max
>
> On 19.05.20 13:02, Sruthi Sree Kumar wrote:
> > On the documentation, the version of Flink runner is changed to 1.9
> > which was 1.10(latest)
> > before
> https://github.com/apache/beam/commit/1d2700818474c008eaa324ac1b5c49c9d2857298#diff-0e75160f4b09a1a300671557930589d9
> .
> >
> > Is this an accidental change or is there any particular reason for this
> > downgrade of version?
> >
> > Regards,
> > Sruthi
> >
> > On Tue, May 12, 2020 at 7:21 PM Maximilian Michels <mxm@apache.org
> > <ma...@apache.org>> wrote:
> >
> > A heads-up if anybody else sees this, we have removed the flag:
> > https://jira.apache.org/jira/browse/BEAM-9900
> >
> > Further contributions are very welcome :)
> >
> > -Max
> >
> > On 11.05.20 17:05, Sruthi Sree Kumar wrote:
> > > I have opened a PR with the documentation change.
> > > https://github.com/apache/beam/pull/11662
> > >
> > > Regards,
> > > Sruthi
> > >
> > > On 2020/04/21 20:22:17, Ismaël Mejía <iemejia@gmail.com
> > <ma...@gmail.com>> wrote:
> > >> You need to instruct the Flink runner to shutdown the the source
> > >> otherwise it will stay waiting.
> > >> You can this by adding the extra
> > >> argument`--shutdownSourcesOnFinalWatermark=true`
> > >> And if that works and you want to open a PR to update our
> > >> documentation that would be greatly appreciated.
> > >>
> > >> Regards,
> > >> Ismaël
> > >>
> > >>
> > >> On Tue, Apr 21, 2020 at 10:04 PM Sruthi Sree Kumar
> > >> <sruthisreekumar2712@gmail.com
> > <ma...@gmail.com>> wrote:
> > >>>
> > >>> Hello,
> > >>>
> > >>> I am trying to run nexmark queries using flink runner streaming.
> > Followed the documentation and used the command
> > >>> ./gradlew :sdks:java:testing:nexmark:run \
> > >>>
> > >>> -Pnexmark.runner=":runners:flink:1.10" \
> > >>> -Pnexmark.args="
> > >>> --runner=FlinkRunner
> > >>> --suite=SMOKE
> > >>> --streamTimeout=60
> > >>> --streaming=true
> > >>> --manageResources=false
> > >>> --monitorJobs=true
> > >>> --flinkMaster=[local]"
> > >>>
> > >>>
> > >>> But after the events are read from the source, there is no
> > further progress and the job is always stuck at 99%. Is there any
> > configuration that I am missing?
> > >>>
> > >>> Regards,
> > >>> Sruthi
> > >>
> >
>
Re: Running NexMark Tests
Posted by Sruthi Sree Kumar <sr...@gmail.com>.
PR for the update: https://github.com/apache/beam/pull/11751
Regards,
Sruthi
On Tue, May 19, 2020 at 3:51 PM Maximilian Michels <mx...@apache.org> wrote:
> Looks like an accidental change to me. Running with either version, 1.9
> or 1.10 works, but this should be changed back to using the latest version.
>
> Do you mind creating a PR?
>
> Thanks,
> Max
>
> On 19.05.20 13:02, Sruthi Sree Kumar wrote:
> > On the documentation, the version of Flink runner is changed to 1.9
> > which was 1.10(latest)
> > before
> https://github.com/apache/beam/commit/1d2700818474c008eaa324ac1b5c49c9d2857298#diff-0e75160f4b09a1a300671557930589d9
> .
> >
> > Is this an accidental change or is there any particular reason for this
> > downgrade of version?
> >
> > Regards,
> > Sruthi
> >
> > On Tue, May 12, 2020 at 7:21 PM Maximilian Michels <mxm@apache.org
> > <ma...@apache.org>> wrote:
> >
> > A heads-up if anybody else sees this, we have removed the flag:
> > https://jira.apache.org/jira/browse/BEAM-9900
> >
> > Further contributions are very welcome :)
> >
> > -Max
> >
> > On 11.05.20 17:05, Sruthi Sree Kumar wrote:
> > > I have opened a PR with the documentation change.
> > > https://github.com/apache/beam/pull/11662
> > >
> > > Regards,
> > > Sruthi
> > >
> > > On 2020/04/21 20:22:17, Ismaël Mejía <iemejia@gmail.com
> > <ma...@gmail.com>> wrote:
> > >> You need to instruct the Flink runner to shutdown the the source
> > >> otherwise it will stay waiting.
> > >> You can this by adding the extra
> > >> argument`--shutdownSourcesOnFinalWatermark=true`
> > >> And if that works and you want to open a PR to update our
> > >> documentation that would be greatly appreciated.
> > >>
> > >> Regards,
> > >> Ismaël
> > >>
> > >>
> > >> On Tue, Apr 21, 2020 at 10:04 PM Sruthi Sree Kumar
> > >> <sruthisreekumar2712@gmail.com
> > <ma...@gmail.com>> wrote:
> > >>>
> > >>> Hello,
> > >>>
> > >>> I am trying to run nexmark queries using flink runner streaming.
> > Followed the documentation and used the command
> > >>> ./gradlew :sdks:java:testing:nexmark:run \
> > >>>
> > >>> -Pnexmark.runner=":runners:flink:1.10" \
> > >>> -Pnexmark.args="
> > >>> --runner=FlinkRunner
> > >>> --suite=SMOKE
> > >>> --streamTimeout=60
> > >>> --streaming=true
> > >>> --manageResources=false
> > >>> --monitorJobs=true
> > >>> --flinkMaster=[local]"
> > >>>
> > >>>
> > >>> But after the events are read from the source, there is no
> > further progress and the job is always stuck at 99%. Is there any
> > configuration that I am missing?
> > >>>
> > >>> Regards,
> > >>> Sruthi
> > >>
> >
>
Re: Running NexMark Tests
Posted by Maximilian Michels <mx...@apache.org>.
Looks like an accidental change to me. Running with either version, 1.9
or 1.10 works, but this should be changed back to using the latest version.
Do you mind creating a PR?
Thanks,
Max
On 19.05.20 13:02, Sruthi Sree Kumar wrote:
> On the documentation, the version of Flink runner is changed to 1.9
> which was 1.10(latest)
> before https://github.com/apache/beam/commit/1d2700818474c008eaa324ac1b5c49c9d2857298#diff-0e75160f4b09a1a300671557930589d9.
>
> Is this an accidental change or is there any particular reason for this
> downgrade of version?
>
> Regards,
> Sruthi
>
> On Tue, May 12, 2020 at 7:21 PM Maximilian Michels <mxm@apache.org
> <ma...@apache.org>> wrote:
>
> A heads-up if anybody else sees this, we have removed the flag:
> https://jira.apache.org/jira/browse/BEAM-9900
>
> Further contributions are very welcome :)
>
> -Max
>
> On 11.05.20 17:05, Sruthi Sree Kumar wrote:
> > I have opened a PR with the documentation change.
> > https://github.com/apache/beam/pull/11662
> >
> > Regards,
> > Sruthi
> >
> > On 2020/04/21 20:22:17, Ismaël Mejía <iemejia@gmail.com
> <ma...@gmail.com>> wrote:
> >> You need to instruct the Flink runner to shutdown the the source
> >> otherwise it will stay waiting.
> >> You can this by adding the extra
> >> argument`--shutdownSourcesOnFinalWatermark=true`
> >> And if that works and you want to open a PR to update our
> >> documentation that would be greatly appreciated.
> >>
> >> Regards,
> >> Ismaël
> >>
> >>
> >> On Tue, Apr 21, 2020 at 10:04 PM Sruthi Sree Kumar
> >> <sruthisreekumar2712@gmail.com
> <ma...@gmail.com>> wrote:
> >>>
> >>> Hello,
> >>>
> >>> I am trying to run nexmark queries using flink runner streaming.
> Followed the documentation and used the command
> >>> ./gradlew :sdks:java:testing:nexmark:run \
> >>>
> >>> -Pnexmark.runner=":runners:flink:1.10" \
> >>> -Pnexmark.args="
> >>> --runner=FlinkRunner
> >>> --suite=SMOKE
> >>> --streamTimeout=60
> >>> --streaming=true
> >>> --manageResources=false
> >>> --monitorJobs=true
> >>> --flinkMaster=[local]"
> >>>
> >>>
> >>> But after the events are read from the source, there is no
> further progress and the job is always stuck at 99%. Is there any
> configuration that I am missing?
> >>>
> >>> Regards,
> >>> Sruthi
> >>
>
Re: Running NexMark Tests
Posted by Maximilian Michels <mx...@apache.org>.
Looks like an accidental change to me. Running with either version, 1.9
or 1.10 works, but this should be changed back to using the latest version.
Do you mind creating a PR?
Thanks,
Max
On 19.05.20 13:02, Sruthi Sree Kumar wrote:
> On the documentation, the version of Flink runner is changed to 1.9
> which was 1.10(latest)
> before https://github.com/apache/beam/commit/1d2700818474c008eaa324ac1b5c49c9d2857298#diff-0e75160f4b09a1a300671557930589d9.
>
> Is this an accidental change or is there any particular reason for this
> downgrade of version?
>
> Regards,
> Sruthi
>
> On Tue, May 12, 2020 at 7:21 PM Maximilian Michels <mxm@apache.org
> <ma...@apache.org>> wrote:
>
> A heads-up if anybody else sees this, we have removed the flag:
> https://jira.apache.org/jira/browse/BEAM-9900
>
> Further contributions are very welcome :)
>
> -Max
>
> On 11.05.20 17:05, Sruthi Sree Kumar wrote:
> > I have opened a PR with the documentation change.
> > https://github.com/apache/beam/pull/11662
> >
> > Regards,
> > Sruthi
> >
> > On 2020/04/21 20:22:17, Ismaël Mejía <iemejia@gmail.com
> <ma...@gmail.com>> wrote:
> >> You need to instruct the Flink runner to shutdown the the source
> >> otherwise it will stay waiting.
> >> You can this by adding the extra
> >> argument`--shutdownSourcesOnFinalWatermark=true`
> >> And if that works and you want to open a PR to update our
> >> documentation that would be greatly appreciated.
> >>
> >> Regards,
> >> Ismaël
> >>
> >>
> >> On Tue, Apr 21, 2020 at 10:04 PM Sruthi Sree Kumar
> >> <sruthisreekumar2712@gmail.com
> <ma...@gmail.com>> wrote:
> >>>
> >>> Hello,
> >>>
> >>> I am trying to run nexmark queries using flink runner streaming.
> Followed the documentation and used the command
> >>> ./gradlew :sdks:java:testing:nexmark:run \
> >>>
> >>> -Pnexmark.runner=":runners:flink:1.10" \
> >>> -Pnexmark.args="
> >>> --runner=FlinkRunner
> >>> --suite=SMOKE
> >>> --streamTimeout=60
> >>> --streaming=true
> >>> --manageResources=false
> >>> --monitorJobs=true
> >>> --flinkMaster=[local]"
> >>>
> >>>
> >>> But after the events are read from the source, there is no
> further progress and the job is always stuck at 99%. Is there any
> configuration that I am missing?
> >>>
> >>> Regards,
> >>> Sruthi
> >>
>
Re: Running NexMark Tests
Posted by Sruthi Sree Kumar <sr...@gmail.com>.
On the documentation, the version of Flink runner is changed to 1.9 which
was 1.10(latest) before
https://github.com/apache/beam/commit/1d2700818474c008eaa324ac1b5c49c9d2857298#diff-0e75160f4b09a1a300671557930589d9
.
Is this an accidental change or is there any particular reason for this
downgrade of version?
Regards,
Sruthi
On Tue, May 12, 2020 at 7:21 PM Maximilian Michels <mx...@apache.org> wrote:
> A heads-up if anybody else sees this, we have removed the flag:
> https://jira.apache.org/jira/browse/BEAM-9900
>
> Further contributions are very welcome :)
>
> -Max
>
> On 11.05.20 17:05, Sruthi Sree Kumar wrote:
> > I have opened a PR with the documentation change.
> > https://github.com/apache/beam/pull/11662
> >
> > Regards,
> > Sruthi
> >
> > On 2020/04/21 20:22:17, Ismaël Mejía <ie...@gmail.com> wrote:
> >> You need to instruct the Flink runner to shutdown the the source
> >> otherwise it will stay waiting.
> >> You can this by adding the extra
> >> argument`--shutdownSourcesOnFinalWatermark=true`
> >> And if that works and you want to open a PR to update our
> >> documentation that would be greatly appreciated.
> >>
> >> Regards,
> >> Ismaël
> >>
> >>
> >> On Tue, Apr 21, 2020 at 10:04 PM Sruthi Sree Kumar
> >> <sr...@gmail.com> wrote:
> >>>
> >>> Hello,
> >>>
> >>> I am trying to run nexmark queries using flink runner streaming.
> Followed the documentation and used the command
> >>> ./gradlew :sdks:java:testing:nexmark:run \
> >>>
> >>> -Pnexmark.runner=":runners:flink:1.10" \
> >>> -Pnexmark.args="
> >>> --runner=FlinkRunner
> >>> --suite=SMOKE
> >>> --streamTimeout=60
> >>> --streaming=true
> >>> --manageResources=false
> >>> --monitorJobs=true
> >>> --flinkMaster=[local]"
> >>>
> >>>
> >>> But after the events are read from the source, there is no further
> progress and the job is always stuck at 99%. Is there any configuration
> that I am missing?
> >>>
> >>> Regards,
> >>> Sruthi
> >>
>
Re: Running NexMark Tests
Posted by Sruthi Sree Kumar <sr...@gmail.com>.
On the documentation, the version of Flink runner is changed to 1.9 which
was 1.10(latest) before
https://github.com/apache/beam/commit/1d2700818474c008eaa324ac1b5c49c9d2857298#diff-0e75160f4b09a1a300671557930589d9
.
Is this an accidental change or is there any particular reason for this
downgrade of version?
Regards,
Sruthi
On Tue, May 12, 2020 at 7:21 PM Maximilian Michels <mx...@apache.org> wrote:
> A heads-up if anybody else sees this, we have removed the flag:
> https://jira.apache.org/jira/browse/BEAM-9900
>
> Further contributions are very welcome :)
>
> -Max
>
> On 11.05.20 17:05, Sruthi Sree Kumar wrote:
> > I have opened a PR with the documentation change.
> > https://github.com/apache/beam/pull/11662
> >
> > Regards,
> > Sruthi
> >
> > On 2020/04/21 20:22:17, Ismaël Mejía <ie...@gmail.com> wrote:
> >> You need to instruct the Flink runner to shutdown the the source
> >> otherwise it will stay waiting.
> >> You can this by adding the extra
> >> argument`--shutdownSourcesOnFinalWatermark=true`
> >> And if that works and you want to open a PR to update our
> >> documentation that would be greatly appreciated.
> >>
> >> Regards,
> >> Ismaël
> >>
> >>
> >> On Tue, Apr 21, 2020 at 10:04 PM Sruthi Sree Kumar
> >> <sr...@gmail.com> wrote:
> >>>
> >>> Hello,
> >>>
> >>> I am trying to run nexmark queries using flink runner streaming.
> Followed the documentation and used the command
> >>> ./gradlew :sdks:java:testing:nexmark:run \
> >>>
> >>> -Pnexmark.runner=":runners:flink:1.10" \
> >>> -Pnexmark.args="
> >>> --runner=FlinkRunner
> >>> --suite=SMOKE
> >>> --streamTimeout=60
> >>> --streaming=true
> >>> --manageResources=false
> >>> --monitorJobs=true
> >>> --flinkMaster=[local]"
> >>>
> >>>
> >>> But after the events are read from the source, there is no further
> progress and the job is always stuck at 99%. Is there any configuration
> that I am missing?
> >>>
> >>> Regards,
> >>> Sruthi
> >>
>