You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by flyisland <fl...@gmail.com> on 2018/09/25 08:21:08 UTC
Bug of the MqttIO.java
Hi
There is a bug of the built-in MqttIO, please check the <
https://github.com/apache/beam/blob/master/sdks/java/io/mqtt/src/main/java/org/apache/beam/sdk/io/mqtt/MqttIO.java#L336>,
this readObject() method forget to invoke the "stream.defaultReadObject()"
method.
// set an empty list to messages when deserialize
private void readObject(java.io.ObjectInputStream stream)
throws IOException, ClassNotFoundException {
messages = new ArrayList<>();
}
}
So there is an exception while the runner tried to deserialize the
checkpoint object.
java.lang.RuntimeException: org.apache.beam.sdk.coders.CoderException: 95
unexpected extra bytes after decoding
org.apache.beam.sdk.io.mqtt.MqttIO$MqttCheckpointMark@6764e219 at
org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:340)
...
Re: Bug of the MqttIO.java
Posted by flyisland <fl...@gmail.com>.
Hi, I've created PR for this bug, https://github.com/apache/beam/pull/6701,
please check it.
The PR page said the "java precommit" failed, but I'm not sure where is the
failure and all tests of MqttIO are passed. (
https://builds.apache.org/job/beam_PreCommit_Java_Commit/1816/testReport/org.apache.beam.sdk.io.mqtt/MqttIOTest/
)
On Fri, Sep 28, 2018 at 7:20 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Thanks Luke
>
> Regards
> JB
> Le 27 sept. 2018, à 18:41, Lukasz Cwik <lc...@google.com> a écrit:
>>
>> Thanks, I assigned BEAM-5496. Feel free to ask questions on dev@ as
>> needed.
>>
>> On Wed, Sep 26, 2018 at 7:11 PM flyisland <fl...@gmail.com> wrote:
>>
>>> Hi, My jira id is "flyisland", thanks!
>>>
>>> On Thu, Sep 27, 2018 at 3:25 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Yes please. Create a jira, I will tackle that. Thanks.
>>>>
>>>> Regards
>>>> JB
>>>> Le 26 sept. 2018, à 15:12, Lukasz Cwik <lc...@google.com> a écrit:
>>>>>
>>>>> Yes, please create a JIRA account on issues.apache.org
>>>>>
>>>>> Once you have one, please tell my the JIRA id and I'll add you as a
>>>>> contributor to Apache Beam and assign BEAM-5496 to you.
>>>>>
>>>>> Also this guide https://beam.apache.org/contribute/ helps
>>>>> people learn how to contribute. It has useful information about how to
>>>>> build, test, open PRs, find reviewers.
>>>>>
>>>>> On Tue, Sep 25, 2018 at 9:37 PM flyisland <fl...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Cool, I'd like to.
>>>>>>
>>>>>> Is there anything I should've done first, like create an account etc.
>>>>>>
>>>>>> On Tue, Sep 25, 2018 at 11:46 PM Lukasz Cwik <lc...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks, I filed https://issues.apache.org/jira/browse/BEAM-5496
>>>>>>> with the details of your report.
>>>>>>>
>>>>>>> Would you be interested in submitting a patch with a test that
>>>>>>> exercises the bug?
>>>>>>>
>>>>>>> On Tue, Sep 25, 2018 at 1:21 AM flyisland <fl...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> There is a bug of the built-in MqttIO, please check the <
>>>>>>>> https://github.com/apache/beam/blob/master/sdks/java/io/mqtt/src/main/java/org/apache/beam/sdk/io/mqtt/MqttIO.java#L336>,
>>>>>>>> this readObject() method forget to invoke the "stream.defaultReadObject()"
>>>>>>>> method.
>>>>>>>>
>>>>>>>> // set an empty list to messages when deserialize
>>>>>>>> private void readObject(java.io.ObjectInputStream stream)
>>>>>>>> throws IOException, ClassNotFoundException {
>>>>>>>> messages = new ArrayList<>();
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> So there is an exception while the runner tried to deserialize the
>>>>>>>> checkpoint object.
>>>>>>>> java.lang.RuntimeException:
>>>>>>>> org.apache.beam.sdk.coders.CoderException: 95 unexpected extra bytes after
>>>>>>>> decoding org.apache.beam.sdk.io.mqtt.MqttIO$MqttCheckpointMark@6764e219
>>>>>>>> at
>>>>>>>> org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:340)
>>>>>>>> ...
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
Re: Bug of the MqttIO.java
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks Luke
Regards
JB
Le 27 sept. 2018 à 18:41, à 18:41, Lukasz Cwik <lc...@google.com> a écrit:
>Thanks, I assigned BEAM-5496. Feel free to ask questions on dev@ as
>needed.
>
>On Wed, Sep 26, 2018 at 7:11 PM flyisland <fl...@gmail.com> wrote:
>
>> Hi, My jira id is "flyisland", thanks!
>>
>> On Thu, Sep 27, 2018 at 3:25 AM Jean-Baptiste Onofré
><jb...@nanthrax.net>
>> wrote:
>>
>>> Yes please. Create a jira, I will tackle that. Thanks.
>>>
>>> Regards
>>> JB
>>> Le 26 sept. 2018, à 15:12, Lukasz Cwik <lc...@google.com> a écrit:
>>>>
>>>> Yes, please create a JIRA account on issues.apache.org
>>>>
>>>> Once you have one, please tell my the JIRA id and I'll add you as a
>>>> contributor to Apache Beam and assign BEAM-5496 to you.
>>>>
>>>> Also this guide https://beam.apache.org/contribute/ helps people
>learn
>>>> how to contribute. It has useful information about how to build,
>test, open
>>>> PRs, find reviewers.
>>>>
>>>> On Tue, Sep 25, 2018 at 9:37 PM flyisland <fl...@gmail.com>
>wrote:
>>>>
>>>>> Cool, I'd like to.
>>>>>
>>>>> Is there anything I should've done first, like create an account
>etc.
>>>>>
>>>>> On Tue, Sep 25, 2018 at 11:46 PM Lukasz Cwik <lc...@google.com>
>wrote:
>>>>>
>>>>>> Thanks, I filed https://issues.apache.org/jira/browse/BEAM-5496
>with
>>>>>> the details of your report.
>>>>>>
>>>>>> Would you be interested in submitting a patch with a test that
>>>>>> exercises the bug?
>>>>>>
>>>>>> On Tue, Sep 25, 2018 at 1:21 AM flyisland <fl...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> There is a bug of the built-in MqttIO, please check the <
>>>>>>>
>https://github.com/apache/beam/blob/master/sdks/java/io/mqtt/src/main/java/org/apache/beam/sdk/io/mqtt/MqttIO.java#L336>,
>>>>>>> this readObject() method forget to invoke the
>"stream.defaultReadObject()"
>>>>>>> method.
>>>>>>>
>>>>>>> // set an empty list to messages when deserialize
>>>>>>> private void readObject(java.io.ObjectInputStream stream)
>>>>>>> throws IOException, ClassNotFoundException {
>>>>>>> messages = new ArrayList<>();
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> So there is an exception while the runner tried to deserialize
>the
>>>>>>> checkpoint object.
>>>>>>> java.lang.RuntimeException:
>>>>>>> org.apache.beam.sdk.coders.CoderException: 95 unexpected extra
>bytes after
>>>>>>> decoding
>org.apache.beam.sdk.io.mqtt.MqttIO$MqttCheckpointMark@6764e219
>>>>>>> at
>>>>>>>
>org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:340)
>>>>>>> ...
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
Re: Bug of the MqttIO.java
Posted by Lukasz Cwik <lc...@google.com>.
Thanks, I assigned BEAM-5496. Feel free to ask questions on dev@ as needed.
On Wed, Sep 26, 2018 at 7:11 PM flyisland <fl...@gmail.com> wrote:
> Hi, My jira id is "flyisland", thanks!
>
> On Thu, Sep 27, 2018 at 3:25 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Yes please. Create a jira, I will tackle that. Thanks.
>>
>> Regards
>> JB
>> Le 26 sept. 2018, à 15:12, Lukasz Cwik <lc...@google.com> a écrit:
>>>
>>> Yes, please create a JIRA account on issues.apache.org
>>>
>>> Once you have one, please tell my the JIRA id and I'll add you as a
>>> contributor to Apache Beam and assign BEAM-5496 to you.
>>>
>>> Also this guide https://beam.apache.org/contribute/ helps people learn
>>> how to contribute. It has useful information about how to build, test, open
>>> PRs, find reviewers.
>>>
>>> On Tue, Sep 25, 2018 at 9:37 PM flyisland <fl...@gmail.com> wrote:
>>>
>>>> Cool, I'd like to.
>>>>
>>>> Is there anything I should've done first, like create an account etc.
>>>>
>>>> On Tue, Sep 25, 2018 at 11:46 PM Lukasz Cwik <lc...@google.com> wrote:
>>>>
>>>>> Thanks, I filed https://issues.apache.org/jira/browse/BEAM-5496 with
>>>>> the details of your report.
>>>>>
>>>>> Would you be interested in submitting a patch with a test that
>>>>> exercises the bug?
>>>>>
>>>>> On Tue, Sep 25, 2018 at 1:21 AM flyisland <fl...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> There is a bug of the built-in MqttIO, please check the <
>>>>>> https://github.com/apache/beam/blob/master/sdks/java/io/mqtt/src/main/java/org/apache/beam/sdk/io/mqtt/MqttIO.java#L336>,
>>>>>> this readObject() method forget to invoke the "stream.defaultReadObject()"
>>>>>> method.
>>>>>>
>>>>>> // set an empty list to messages when deserialize
>>>>>> private void readObject(java.io.ObjectInputStream stream)
>>>>>> throws IOException, ClassNotFoundException {
>>>>>> messages = new ArrayList<>();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> So there is an exception while the runner tried to deserialize the
>>>>>> checkpoint object.
>>>>>> java.lang.RuntimeException:
>>>>>> org.apache.beam.sdk.coders.CoderException: 95 unexpected extra bytes after
>>>>>> decoding org.apache.beam.sdk.io.mqtt.MqttIO$MqttCheckpointMark@6764e219
>>>>>> at
>>>>>> org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:340)
>>>>>> ...
>>>>>>
>>>>>>
>>>>>>
>>>>>>
Re: Bug of the MqttIO.java
Posted by flyisland <fl...@gmail.com>.
Hi, My jira id is "flyisland", thanks!
On Thu, Sep 27, 2018 at 3:25 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Yes please. Create a jira, I will tackle that. Thanks.
>
> Regards
> JB
> Le 26 sept. 2018, à 15:12, Lukasz Cwik <lc...@google.com> a écrit:
>>
>> Yes, please create a JIRA account on issues.apache.org
>>
>> Once you have one, please tell my the JIRA id and I'll add you as a
>> contributor to Apache Beam and assign BEAM-5496 to you.
>>
>> Also this guide https://beam.apache.org/contribute/ helps people learn
>> how to contribute. It has useful information about how to build, test, open
>> PRs, find reviewers.
>>
>> On Tue, Sep 25, 2018 at 9:37 PM flyisland <fl...@gmail.com> wrote:
>>
>>> Cool, I'd like to.
>>>
>>> Is there anything I should've done first, like create an account etc.
>>>
>>> On Tue, Sep 25, 2018 at 11:46 PM Lukasz Cwik <lc...@google.com> wrote:
>>>
>>>> Thanks, I filed https://issues.apache.org/jira/browse/BEAM-5496 with
>>>> the details of your report.
>>>>
>>>> Would you be interested in submitting a patch with a test that
>>>> exercises the bug?
>>>>
>>>> On Tue, Sep 25, 2018 at 1:21 AM flyisland <fl...@gmail.com> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> There is a bug of the built-in MqttIO, please check the <
>>>>> https://github.com/apache/beam/blob/master/sdks/java/io/mqtt/src/main/java/org/apache/beam/sdk/io/mqtt/MqttIO.java#L336>,
>>>>> this readObject() method forget to invoke the "stream.defaultReadObject()"
>>>>> method.
>>>>>
>>>>> // set an empty list to messages when deserialize
>>>>> private void readObject(java.io.ObjectInputStream stream)
>>>>> throws IOException, ClassNotFoundException {
>>>>> messages = new ArrayList<>();
>>>>> }
>>>>> }
>>>>>
>>>>> So there is an exception while the runner tried to deserialize the
>>>>> checkpoint object.
>>>>> java.lang.RuntimeException: org.apache.beam.sdk.coders.CoderException:
>>>>> 95 unexpected extra bytes after decoding
>>>>> org.apache.beam.sdk.io.mqtt.MqttIO$MqttCheckpointMark@6764e219 at
>>>>> org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:340)
>>>>> ...
>>>>>
>>>>>
>>>>>
>>>>>
Re: Bug of the MqttIO.java
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Yes please. Create a jira, I will tackle that. Thanks.
Regards
JB
Le 26 sept. 2018 à 15:12, à 15:12, Lukasz Cwik <lc...@google.com> a écrit:
>Yes, please create a JIRA account on issues.apache.org
>
>Once you have one, please tell my the JIRA id and I'll add you as a
>contributor to Apache Beam and assign BEAM-5496 to you.
>
>Also this guide https://beam.apache.org/contribute/ helps people learn
>how
>to contribute. It has useful information about how to build, test, open
>PRs, find reviewers.
>
>On Tue, Sep 25, 2018 at 9:37 PM flyisland <fl...@gmail.com> wrote:
>
>> Cool, I'd like to.
>>
>> Is there anything I should've done first, like create an account etc.
>>
>> On Tue, Sep 25, 2018 at 11:46 PM Lukasz Cwik <lc...@google.com>
>wrote:
>>
>>> Thanks, I filed https://issues.apache.org/jira/browse/BEAM-5496 with
>the
>>> details of your report.
>>>
>>> Would you be interested in submitting a patch with a test that
>exercises
>>> the bug?
>>>
>>> On Tue, Sep 25, 2018 at 1:21 AM flyisland <fl...@gmail.com>
>wrote:
>>>
>>>> Hi
>>>>
>>>> There is a bug of the built-in MqttIO, please check the <
>>>>
>https://github.com/apache/beam/blob/master/sdks/java/io/mqtt/src/main/java/org/apache/beam/sdk/io/mqtt/MqttIO.java#L336>,
>>>> this readObject() method forget to invoke the
>"stream.defaultReadObject()"
>>>> method.
>>>>
>>>> // set an empty list to messages when deserialize
>>>> private void readObject(java.io.ObjectInputStream stream)
>>>> throws IOException, ClassNotFoundException {
>>>> messages = new ArrayList<>();
>>>> }
>>>> }
>>>>
>>>> So there is an exception while the runner tried to deserialize the
>>>> checkpoint object.
>>>> java.lang.RuntimeException:
>org.apache.beam.sdk.coders.CoderException:
>>>> 95 unexpected extra bytes after decoding
>>>> org.apache.beam.sdk.io.mqtt.MqttIO$MqttCheckpointMark@6764e219 at
>>>>
>org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:340)
>>>> ...
>>>>
>>>>
>>>>
>>>>
Re: Bug of the MqttIO.java
Posted by Lukasz Cwik <lc...@google.com>.
Yes, please create a JIRA account on issues.apache.org
Once you have one, please tell my the JIRA id and I'll add you as a
contributor to Apache Beam and assign BEAM-5496 to you.
Also this guide https://beam.apache.org/contribute/ helps people learn how
to contribute. It has useful information about how to build, test, open
PRs, find reviewers.
On Tue, Sep 25, 2018 at 9:37 PM flyisland <fl...@gmail.com> wrote:
> Cool, I'd like to.
>
> Is there anything I should've done first, like create an account etc.
>
> On Tue, Sep 25, 2018 at 11:46 PM Lukasz Cwik <lc...@google.com> wrote:
>
>> Thanks, I filed https://issues.apache.org/jira/browse/BEAM-5496 with the
>> details of your report.
>>
>> Would you be interested in submitting a patch with a test that exercises
>> the bug?
>>
>> On Tue, Sep 25, 2018 at 1:21 AM flyisland <fl...@gmail.com> wrote:
>>
>>> Hi
>>>
>>> There is a bug of the built-in MqttIO, please check the <
>>> https://github.com/apache/beam/blob/master/sdks/java/io/mqtt/src/main/java/org/apache/beam/sdk/io/mqtt/MqttIO.java#L336>,
>>> this readObject() method forget to invoke the "stream.defaultReadObject()"
>>> method.
>>>
>>> // set an empty list to messages when deserialize
>>> private void readObject(java.io.ObjectInputStream stream)
>>> throws IOException, ClassNotFoundException {
>>> messages = new ArrayList<>();
>>> }
>>> }
>>>
>>> So there is an exception while the runner tried to deserialize the
>>> checkpoint object.
>>> java.lang.RuntimeException: org.apache.beam.sdk.coders.CoderException:
>>> 95 unexpected extra bytes after decoding
>>> org.apache.beam.sdk.io.mqtt.MqttIO$MqttCheckpointMark@6764e219 at
>>> org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:340)
>>> ...
>>>
>>>
>>>
>>>
Re: Bug of the MqttIO.java
Posted by flyisland <fl...@gmail.com>.
Cool, I'd like to.
Is there anything I should've done first, like create an account etc.
On Tue, Sep 25, 2018 at 11:46 PM Lukasz Cwik <lc...@google.com> wrote:
> Thanks, I filed https://issues.apache.org/jira/browse/BEAM-5496 with the
> details of your report.
>
> Would you be interested in submitting a patch with a test that exercises
> the bug?
>
> On Tue, Sep 25, 2018 at 1:21 AM flyisland <fl...@gmail.com> wrote:
>
>> Hi
>>
>> There is a bug of the built-in MqttIO, please check the <
>> https://github.com/apache/beam/blob/master/sdks/java/io/mqtt/src/main/java/org/apache/beam/sdk/io/mqtt/MqttIO.java#L336>,
>> this readObject() method forget to invoke the "stream.defaultReadObject()"
>> method.
>>
>> // set an empty list to messages when deserialize
>> private void readObject(java.io.ObjectInputStream stream)
>> throws IOException, ClassNotFoundException {
>> messages = new ArrayList<>();
>> }
>> }
>>
>> So there is an exception while the runner tried to deserialize the
>> checkpoint object.
>> java.lang.RuntimeException: org.apache.beam.sdk.coders.CoderException: 95
>> unexpected extra bytes after decoding
>> org.apache.beam.sdk.io.mqtt.MqttIO$MqttCheckpointMark@6764e219 at
>> org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:340)
>> ...
>>
>>
>>
>>
Re: Bug of the MqttIO.java
Posted by Lukasz Cwik <lc...@google.com>.
Thanks, I filed https://issues.apache.org/jira/browse/BEAM-5496 with the
details of your report.
Would you be interested in submitting a patch with a test that exercises
the bug?
On Tue, Sep 25, 2018 at 1:21 AM flyisland <fl...@gmail.com> wrote:
> Hi
>
> There is a bug of the built-in MqttIO, please check the <
> https://github.com/apache/beam/blob/master/sdks/java/io/mqtt/src/main/java/org/apache/beam/sdk/io/mqtt/MqttIO.java#L336>,
> this readObject() method forget to invoke the "stream.defaultReadObject()"
> method.
>
> // set an empty list to messages when deserialize
> private void readObject(java.io.ObjectInputStream stream)
> throws IOException, ClassNotFoundException {
> messages = new ArrayList<>();
> }
> }
>
> So there is an exception while the runner tried to deserialize the
> checkpoint object.
> java.lang.RuntimeException: org.apache.beam.sdk.coders.CoderException: 95
> unexpected extra bytes after decoding
> org.apache.beam.sdk.io.mqtt.MqttIO$MqttCheckpointMark@6764e219 at
> org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:340)
> ...
>
>
>
>