You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Vishwas Bm <bm...@gmail.com> on 2019/08/02 15:14:56 UTC
Re: Query about JdbcIO.readRows()
Hi Kishor,
+ dev (dev@beam.apache.org)
This looks like a bug. The attribute statementPreparator is nullable
It should have been handled in the same way as in the expand method of
Read<T> class.
*Thanks & Regards,*
*Vishwas *
On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi <jo...@yahoo.com> wrote:
> Hi,
>
> I am using the just released 2.14 version for JdbcIO with the newly added
> "readRows" functionality.
>
> I want to read table data with a query without parameters (select * from
> table_name).
> As per my understanding, this should not require "StatementPreperator".
> However, if I use the newly added "readRows" function, I get an exception
> that seems to force me to use the "StatementPreperator".
> Stacktrace below.
>
> java.lang.IllegalArgumentException: statementPreparator can not be null
> at
> org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
> at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
> at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
> at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
> at
> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
> at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>
>
>
> The test added in JdbcIOTest.java for this functionality only tests for
> queries with parameters.
> Is this new function supported only in the above case and not for normal
> "withQuery" (without parameters) ?
>
>
> Thanks & regards,
> Kishor
>
Re: Query about JdbcIO.readRows()
Posted by Kenneth Knowles <ke...@apache.org>.
Hi Kishor,
If you could not find a Jira, would you file one? Your contribution would
be very appreciated.
Kenn
On Tue, Aug 20, 2019 at 10:04 PM Kishor Joshi <jo...@yahoo.com> wrote:
> Hi,
>
> This fix is still not available in the Beam 2.15.0. Is there any Jira that
> has been created for this issue ? I am interested to contribute in that.
>
> Thanks & Regards,
> Kishor
>
> On Friday, August 2, 2019, 10:19:17 PM GMT+5:30, Jean-Baptiste Onofré <
> jb@nanthrax.net> wrote:
>
>
> Agree. I will fix that.
>
> Regards
> JB
> Le 2 août 2019, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
>
> Hi Kishor,
>
> + dev ( dev@beam.apache.org)
>
> This looks like a bug. The attribute statementPreparator is nullable
> It should have been handled in the same way as in the expand method of
> Read<T> class.
>
>
> *Thanks & Regards,*
>
> *Vishwas *
>
>
> On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi < joshik24@yahoo.com> wrote:
>
> Hi,
>
> I am using the just released 2.14 version for JdbcIO with the newly added
> "readRows" functionality.
>
> I want to read table data with a query without parameters (select * from
> table_name).
> As per my understanding, this should not require "StatementPreperator".
> However, if I use the newly added "readRows" function, I get an exception
> that seems to force me to use the "StatementPreperator".
> Stacktrace below.
>
> java.lang.IllegalArgumentException: statementPreparator can not be null
> at
> org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
>
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
>
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
> at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
>
> at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
>
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
> at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
> at
> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
> at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>
>
>
> The test added in JdbcIOTest.java for this functionality only tests for
> queries with parameters.
> Is this new function supported only in the above case and not for normal
> "withQuery" (without parameters) ?
>
>
> Thanks & regards,
> Kishor
>
>
Re: Query about JdbcIO.readRows()
Posted by Kenneth Knowles <ke...@apache.org>.
Hi Kishor,
If you could not find a Jira, would you file one? Your contribution would
be very appreciated.
Kenn
On Tue, Aug 20, 2019 at 10:04 PM Kishor Joshi <jo...@yahoo.com> wrote:
> Hi,
>
> This fix is still not available in the Beam 2.15.0. Is there any Jira that
> has been created for this issue ? I am interested to contribute in that.
>
> Thanks & Regards,
> Kishor
>
> On Friday, August 2, 2019, 10:19:17 PM GMT+5:30, Jean-Baptiste Onofré <
> jb@nanthrax.net> wrote:
>
>
> Agree. I will fix that.
>
> Regards
> JB
> Le 2 août 2019, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
>
> Hi Kishor,
>
> + dev ( dev@beam.apache.org)
>
> This looks like a bug. The attribute statementPreparator is nullable
> It should have been handled in the same way as in the expand method of
> Read<T> class.
>
>
> *Thanks & Regards,*
>
> *Vishwas *
>
>
> On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi < joshik24@yahoo.com> wrote:
>
> Hi,
>
> I am using the just released 2.14 version for JdbcIO with the newly added
> "readRows" functionality.
>
> I want to read table data with a query without parameters (select * from
> table_name).
> As per my understanding, this should not require "StatementPreperator".
> However, if I use the newly added "readRows" function, I get an exception
> that seems to force me to use the "StatementPreperator".
> Stacktrace below.
>
> java.lang.IllegalArgumentException: statementPreparator can not be null
> at
> org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
>
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
>
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
> at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
> at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
>
> at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
>
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
> at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
> at
> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
> at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>
>
>
> The test added in JdbcIOTest.java for this functionality only tests for
> queries with parameters.
> Is this new function supported only in the above case and not for normal
> "withQuery" (without parameters) ?
>
>
> Thanks & regards,
> Kishor
>
>
Re: Query about JdbcIO.readRows()
Posted by Kishor Joshi <jo...@yahoo.com>.
Hi,
This fix is still not available in the Beam 2.15.0. Is there any Jira that has been created for this issue ? I am interested to contribute in that.
Thanks & Regards,Kishor
On Friday, August 2, 2019, 10:19:17 PM GMT+5:30, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
Agree. I will fix that.
Regards
JBLe 2 août 2019, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
Hi Kishor,
+ dev ( dev@beam.apache.org)
This looks like a bug. The attribute statementPreparator is nullable
It should have been handled in the same way as in the expand method of Read<T> class.
Thanks & Regards, Vishwas
On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi < joshik24@yahoo.com> wrote:
Hi,
I am using the just released 2.14 version for JdbcIO with the newly added "readRows" functionality.
I want to read table data with a query without parameters (select * from table_name). As per my understanding, this should not require "StatementPreperator". However, if I use the newly added "readRows" function, I get an exception that seems to force me to use the "StatementPreperator". Stacktrace below.
java.lang.IllegalArgumentException: statementPreparator can not be null at org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122) at org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600) at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499) at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410) at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537) at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471) at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44) at com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34) at com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10) at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537) at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488) at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56) at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182) at com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49) at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
The test added in JdbcIOTest.java for this functionality only tests for queries with parameters. Is this new function supported only in the above case and not for normal "withQuery" (without parameters) ?
Thanks & regards, Kishor
Re: Query about JdbcIO.readRows()
Posted by Kishor Joshi <jo...@yahoo.com>.
Hi,
This fix is still not available in the Beam 2.15.0. Is there any Jira that has been created for this issue ? I am interested to contribute in that.
Thanks & Regards,Kishor
On Friday, August 2, 2019, 10:19:17 PM GMT+5:30, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
Agree. I will fix that.
Regards
JBLe 2 août 2019, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
Hi Kishor,
+ dev ( dev@beam.apache.org)
This looks like a bug. The attribute statementPreparator is nullable
It should have been handled in the same way as in the expand method of Read<T> class.
Thanks & Regards, Vishwas
On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi < joshik24@yahoo.com> wrote:
Hi,
I am using the just released 2.14 version for JdbcIO with the newly added "readRows" functionality.
I want to read table data with a query without parameters (select * from table_name). As per my understanding, this should not require "StatementPreperator". However, if I use the newly added "readRows" function, I get an exception that seems to force me to use the "StatementPreperator". Stacktrace below.
java.lang.IllegalArgumentException: statementPreparator can not be null at org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122) at org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600) at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499) at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410) at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537) at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471) at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44) at com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34) at com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10) at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537) at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488) at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56) at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182) at com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49) at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
The test added in JdbcIOTest.java for this functionality only tests for queries with parameters. Is this new function supported only in the above case and not for normal "withQuery" (without parameters) ?
Thanks & regards, Kishor
Re: Query about JdbcIO.readRows()
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Agree. I will fix that.
Regards
JB
Le 2 août 2019 à 17:15, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
>Hi Kishor,
>
>+ dev (dev@beam.apache.org)
>
>This looks like a bug. The attribute statementPreparator is nullable
>It should have been handled in the same way as in the expand method of
>Read<T> class.
>
>
>*Thanks & Regards,*
>
>*Vishwas *
>
>
>On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi <jo...@yahoo.com> wrote:
>
>> Hi,
>>
>> I am using the just released 2.14 version for JdbcIO with the newly
>added
>> "readRows" functionality.
>>
>> I want to read table data with a query without parameters (select *
>from
>> table_name).
>> As per my understanding, this should not require
>"StatementPreperator".
>> However, if I use the newly added "readRows" function, I get an
>exception
>> that seems to force me to use the "StatementPreperator".
>> Stacktrace below.
>>
>> java.lang.IllegalArgumentException: statementPreparator can not be
>null
>> at
>>
>org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
>> at
>>
>org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
>> at
>> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
>> at
>> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
>> at
>org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>> at
>org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
>> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
>> at
>>
>com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
>> at
>>
>com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
>> at
>org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>> at
>org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
>> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
>> at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
>> at
>> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
>> at
>com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>>
>>
>>
>> The test added in JdbcIOTest.java for this functionality only tests
>for
>> queries with parameters.
>> Is this new function supported only in the above case and not for
>normal
>> "withQuery" (without parameters) ?
>>
>>
>> Thanks & regards,
>> Kishor
>>
Re: Query about JdbcIO.readRows()
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Agree. I will fix that.
Regards
JB
Le 2 août 2019 à 17:15, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
>Hi Kishor,
>
>+ dev (dev@beam.apache.org)
>
>This looks like a bug. The attribute statementPreparator is nullable
>It should have been handled in the same way as in the expand method of
>Read<T> class.
>
>
>*Thanks & Regards,*
>
>*Vishwas *
>
>
>On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi <jo...@yahoo.com> wrote:
>
>> Hi,
>>
>> I am using the just released 2.14 version for JdbcIO with the newly
>added
>> "readRows" functionality.
>>
>> I want to read table data with a query without parameters (select *
>from
>> table_name).
>> As per my understanding, this should not require
>"StatementPreperator".
>> However, if I use the newly added "readRows" function, I get an
>exception
>> that seems to force me to use the "StatementPreperator".
>> Stacktrace below.
>>
>> java.lang.IllegalArgumentException: statementPreparator can not be
>null
>> at
>>
>org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
>> at
>>
>org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
>> at
>> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
>> at
>> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
>> at
>org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>> at
>org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
>> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
>> at
>>
>com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
>> at
>>
>com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
>> at
>org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>> at
>org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
>> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
>> at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
>> at
>> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
>> at
>com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>>
>>
>>
>> The test added in JdbcIOTest.java for this functionality only tests
>for
>> queries with parameters.
>> Is this new function supported only in the above case and not for
>normal
>> "withQuery" (without parameters) ?
>>
>>
>> Thanks & regards,
>> Kishor
>>