You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Sandeep Sharat <sa...@gmail.com> on 2022/04/04 15:37:00 UTC
Error Using datastream API
Dear Users,
I am using pyflink 1.14.0 and I have been trying to play around with the
datastream api. What I am trying to do is to read data in the form of json
coming in from a kafka queue. This is where I am faced with a very weird
error, i.e. whenever I try to do a *"map" *operation to the datastream.
Same error happens when I try to do the *flatmap* as well. If I take the
*"map"* operation out of the equation I am able to print out the data
coming in from the kafka. Below is a snippet of the error. I have also
attached the complete error trace along with this email. The only
assumption I can make out from the error is that it is caused by some py4j
incompatibility or that it has something to do with the version
incompatibility of flink, I may be wrong also. Any help would be
appreciated.
[image: image.png]
--
Thanks & Regards
Sandeep Sharat Kumar
Re: Error Using datastream API
Posted by Sandeep Sharat <sa...@gmail.com>.
Thank you Dian and Robert for responding. I will try out those and let you
know if something fruitful comes out of it.
On Wed, 6 Apr, 2022, 8:38 am Dian Fu, <di...@gmail.com> wrote:
> Hi Sandeep,
>
> Loopback mode is a newly introduced execution mode in 1.14.0 which allows
> debugging the Python user-defined functions such as map, flatmap easily in
> IDE. It's enabled by default when executing the job locally. However, there
> is a known issue[1] before 1.14.5 which may cause the actual exception
> message which occurred during startup to be hidden. You could try to set
> the environment variable **_python_worker_execution_mode** to **process**
> to force it to run in process mode instead of loopback mode to see the root
> cause of the failure.
>
> Regards,
> Dian
>
> [1] https://issues.apache.org/jira/browse/FLINK-26543
>
> On Tue, Apr 5, 2022 at 9:25 PM Robert Metzger <rm...@apache.org> wrote:
>
>> Hi Sandeep,
>>
>> the relevant message of the exception is actually much further down: "IllegalStateException:
>> Process died with exit code 0". Can you try to figure out why the
>> process died?
>>
>>
>> On Mon, Apr 4, 2022 at 5:37 PM Sandeep Sharat <sa...@gmail.com>
>> wrote:
>>
>>> Dear Users,
>>>
>>> I am using pyflink 1.14.0 and I have been trying to play around with the
>>> datastream api. What I am trying to do is to read data in the form of json
>>> coming in from a kafka queue. This is where I am faced with a very weird
>>> error, i.e. whenever I try to do a *"map" *operation to the datastream.
>>> Same error happens when I try to do the *flatmap* as well. If I take
>>> the *"map"* operation out of the equation I am able to print out the
>>> data coming in from the kafka. Below is a snippet of the error. I have also
>>> attached the complete error trace along with this email. The only
>>> assumption I can make out from the error is that it is caused by some py4j
>>> incompatibility or that it has something to do with the version
>>> incompatibility of flink, I may be wrong also. Any help would be
>>> appreciated.
>>>
>>>
>>> [image: image.png]
>>>
>>>
>>>
>>>
>>> --
>>> Thanks & Regards
>>> Sandeep Sharat Kumar
>>>
>>
Re: Error Using datastream API
Posted by Dian Fu <di...@gmail.com>.
Hi Sandeep,
Loopback mode is a newly introduced execution mode in 1.14.0 which allows
debugging the Python user-defined functions such as map, flatmap easily in
IDE. It's enabled by default when executing the job locally. However, there
is a known issue[1] before 1.14.5 which may cause the actual exception
message which occurred during startup to be hidden. You could try to set
the environment variable **_python_worker_execution_mode** to **process**
to force it to run in process mode instead of loopback mode to see the root
cause of the failure.
Regards,
Dian
[1] https://issues.apache.org/jira/browse/FLINK-26543
On Tue, Apr 5, 2022 at 9:25 PM Robert Metzger <rm...@apache.org> wrote:
> Hi Sandeep,
>
> the relevant message of the exception is actually much further down: "IllegalStateException:
> Process died with exit code 0". Can you try to figure out why the process
> died?
>
>
> On Mon, Apr 4, 2022 at 5:37 PM Sandeep Sharat <sa...@gmail.com>
> wrote:
>
>> Dear Users,
>>
>> I am using pyflink 1.14.0 and I have been trying to play around with the
>> datastream api. What I am trying to do is to read data in the form of json
>> coming in from a kafka queue. This is where I am faced with a very weird
>> error, i.e. whenever I try to do a *"map" *operation to the datastream.
>> Same error happens when I try to do the *flatmap* as well. If I take the
>> *"map"* operation out of the equation I am able to print out the data
>> coming in from the kafka. Below is a snippet of the error. I have also
>> attached the complete error trace along with this email. The only
>> assumption I can make out from the error is that it is caused by some py4j
>> incompatibility or that it has something to do with the version
>> incompatibility of flink, I may be wrong also. Any help would be
>> appreciated.
>>
>>
>> [image: image.png]
>>
>>
>>
>>
>> --
>> Thanks & Regards
>> Sandeep Sharat Kumar
>>
>
Re: Error Using datastream API
Posted by Robert Metzger <rm...@apache.org>.
Hi Sandeep,
the relevant message of the exception is actually much further down:
"IllegalStateException:
Process died with exit code 0". Can you try to figure out why the process
died?
On Mon, Apr 4, 2022 at 5:37 PM Sandeep Sharat <sa...@gmail.com>
wrote:
> Dear Users,
>
> I am using pyflink 1.14.0 and I have been trying to play around with the
> datastream api. What I am trying to do is to read data in the form of json
> coming in from a kafka queue. This is where I am faced with a very weird
> error, i.e. whenever I try to do a *"map" *operation to the datastream.
> Same error happens when I try to do the *flatmap* as well. If I take the
> *"map"* operation out of the equation I am able to print out the data
> coming in from the kafka. Below is a snippet of the error. I have also
> attached the complete error trace along with this email. The only
> assumption I can make out from the error is that it is caused by some py4j
> incompatibility or that it has something to do with the version
> incompatibility of flink, I may be wrong also. Any help would be
> appreciated.
>
>
> [image: image.png]
>
>
>
>
> --
> Thanks & Regards
> Sandeep Sharat Kumar
>