You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Lukasz Cwik <lc...@google.com> on 2018/02/01 02:14:21 UTC
Re: TextIO operation not supported by Flink for Beam master
Thanks for the feedback, we are currently in the middle of releasing 2.3.0
from pretty close to what is on Apache Beam master so your issue should be
investigated before release.
On Wed, Jan 31, 2018 at 5:36 PM, Thomas Pelletier <tp...@zendesk.com>
wrote:
> Hi,
>
> I'm trying to run a pipeline containing just a TextIO.read() step on a
> Flink cluster, using the latest Beam git revision (ff37337
> <https://github.com/apache/beam/commit/ff37337d85aa5af23418f3be4611b913395ccc88>).
> The job fails to start with the Exception:
>
> java.lang.UnsupportedOperationException: The transform is currently
> not supported.
>
> It does work with Beam 2.2.0 though. All code, logs, and reproduction
> steps on this repository <https://github.com/pelletier/beam-flink-example>
> .
>
> Any idea what might be going on?
>
> Thanks,
> Thomas
>
Re: TextIO operation not supported by Flink for Beam master
Posted by Kenneth Knowles <kl...@google.com>.
Filed https://issues.apache.org/jira/browse/BEAM-3587
On Wed, Jan 31, 2018 at 6:31 PM, Kenneth Knowles <kl...@google.com> wrote:
> Yes, a Jira ticket will help to keep track of things.
>
> I have a guess that this has to do with switching to running from a
> portable pipeline representation, and it looks like there's a non-composite
> transform with an empty URN and it threw a bad error message. We can try to
> root cause but may also mitigate short-term by removing the round-trip
> through pipeline proto for now.
>
> What is curious is that the ValidatesRunner and WordCountIT are working -
> they only run on a local Flink, yet this seems to be a translation issue
> that would occur for local or distributed runs.
>
> I'll copy all this onto the Jira ticket but I wanted to communicate it
> immediately.
>
> Kenn
>
> On Wed, Jan 31, 2018 at 6:25 PM, Thomas Pelletier <tp...@zendesk.com>
> wrote:
>
>> Let me know if there is anything I can do to help! Should I file a jira
>> ticket?
>>
>> On Wed 31 Jan 2018 at 18:14, Lukasz Cwik <lc...@google.com> wrote:
>>
>>> Thanks for the feedback, we are currently in the middle of releasing
>>> 2.3.0 from pretty close to what is on Apache Beam master so your issue
>>> should be investigated before release.
>>>
>>> On Wed, Jan 31, 2018 at 5:36 PM, Thomas Pelletier <
>>> tpelletier@zendesk.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm trying to run a pipeline containing just a TextIO.read() step on a
>>>> Flink cluster, using the latest Beam git revision (ff37337
>>>> <https://github.com/apache/beam/commit/ff37337d85aa5af23418f3be4611b913395ccc88>).
>>>> The job fails to start with the Exception:
>>>>
>>>> java.lang.UnsupportedOperationException: The transform is currently
>>>> not supported.
>>>>
>>>> It does work with Beam 2.2.0 though. All code, logs, and reproduction
>>>> steps on this repository
>>>> <https://github.com/pelletier/beam-flink-example>.
>>>>
>>>> Any idea what might be going on?
>>>>
>>>> Thanks,
>>>> Thomas
>>>>
>>>
>>>
>
Re: TextIO operation not supported by Flink for Beam master
Posted by Kenneth Knowles <kl...@google.com>.
Yes, a Jira ticket will help to keep track of things.
I have a guess that this has to do with switching to running from a
portable pipeline representation, and it looks like there's a non-composite
transform with an empty URN and it threw a bad error message. We can try to
root cause but may also mitigate short-term by removing the round-trip
through pipeline proto for now.
What is curious is that the ValidatesRunner and WordCountIT are working -
they only run on a local Flink, yet this seems to be a translation issue
that would occur for local or distributed runs.
I'll copy all this onto the Jira ticket but I wanted to communicate it
immediately.
Kenn
On Wed, Jan 31, 2018 at 6:25 PM, Thomas Pelletier <tp...@zendesk.com>
wrote:
> Let me know if there is anything I can do to help! Should I file a jira
> ticket?
>
> On Wed 31 Jan 2018 at 18:14, Lukasz Cwik <lc...@google.com> wrote:
>
>> Thanks for the feedback, we are currently in the middle of releasing
>> 2.3.0 from pretty close to what is on Apache Beam master so your issue
>> should be investigated before release.
>>
>> On Wed, Jan 31, 2018 at 5:36 PM, Thomas Pelletier <tpelletier@zendesk.com
>> > wrote:
>>
>>> Hi,
>>>
>>> I'm trying to run a pipeline containing just a TextIO.read() step on a
>>> Flink cluster, using the latest Beam git revision (ff37337
>>> <https://github.com/apache/beam/commit/ff37337d85aa5af23418f3be4611b913395ccc88>).
>>> The job fails to start with the Exception:
>>>
>>> java.lang.UnsupportedOperationException: The transform is currently
>>> not supported.
>>>
>>> It does work with Beam 2.2.0 though. All code, logs, and reproduction
>>> steps on this repository
>>> <https://github.com/pelletier/beam-flink-example>.
>>>
>>> Any idea what might be going on?
>>>
>>> Thanks,
>>> Thomas
>>>
>>
>>
Re: TextIO operation not supported by Flink for Beam master
Posted by Thomas Pelletier <tp...@zendesk.com>.
Let me know if there is anything I can do to help! Should I file a jira
ticket?
On Wed 31 Jan 2018 at 18:14, Lukasz Cwik <lc...@google.com> wrote:
> Thanks for the feedback, we are currently in the middle of releasing 2.3.0
> from pretty close to what is on Apache Beam master so your issue should be
> investigated before release.
>
> On Wed, Jan 31, 2018 at 5:36 PM, Thomas Pelletier <tp...@zendesk.com>
> wrote:
>
>> Hi,
>>
>> I'm trying to run a pipeline containing just a TextIO.read() step on a
>> Flink cluster, using the latest Beam git revision (ff37337
>> <https://github.com/apache/beam/commit/ff37337d85aa5af23418f3be4611b913395ccc88>).
>> The job fails to start with the Exception:
>>
>> java.lang.UnsupportedOperationException: The transform is currently
>> not supported.
>>
>> It does work with Beam 2.2.0 though. All code, logs, and reproduction
>> steps on this repository
>> <https://github.com/pelletier/beam-flink-example>.
>>
>> Any idea what might be going on?
>>
>> Thanks,
>> Thomas
>>
>
>