You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Kishor Joshi <jo...@yahoo.com> on 2019/08/02 09:18:04 UTC

Query about JdbcIO.readRows()

Hi,
I am using the just released 2.14 version for JdbcIO with the newly added "readRows" functionality. 
I want to read table data with a query without parameters (select * from table_name). As per my understanding, this should not require "StatementPreperator". However, if I use the newly added "readRows" function, I get an exception that seems to force me to use the "StatementPreperator". Stacktrace below.  
java.lang.IllegalArgumentException: statementPreparator can not be null        at org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)        at org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)        at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)        at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)        at com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)        at com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)        at com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)        at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)


The test added in JdbcIOTest.java for this functionality only tests for queries with parameters. Is this new function supported only in the above case and not for normal "withQuery" (without parameters) ? 

Thanks & regards,Kishor

Re: Query about JdbcIO.readRows()

Posted by Kenneth Knowles <ke...@apache.org>.
Hi Kishor,

If you could not find a Jira, would you file one? Your contribution would
be very appreciated.

Kenn

On Tue, Aug 20, 2019 at 10:04 PM Kishor Joshi <jo...@yahoo.com> wrote:

> Hi,
>
> This fix is still not available in the Beam 2.15.0. Is there any Jira that
> has been created for this issue ? I am interested to contribute in that.
>
> Thanks & Regards,
> Kishor
>
> On Friday, August 2, 2019, 10:19:17 PM GMT+5:30, Jean-Baptiste Onofré <
> jb@nanthrax.net> wrote:
>
>
> Agree. I will fix that.
>
> Regards
> JB
> Le 2 août 2019, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
>
> Hi Kishor,
>
> + dev ( dev@beam.apache.org)
>
> This looks like a bug.  The attribute statementPreparator is nullable
> It should have been handled in the same way as in the expand method of
> Read<T> class.
>
>
> *Thanks & Regards,*
>
> *Vishwas *
>
>
> On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi < joshik24@yahoo.com> wrote:
>
> Hi,
>
> I am using the just released 2.14 version for JdbcIO with the newly added
> "readRows" functionality.
>
> I want to read table data with a query without parameters (select * from
> table_name).
> As per my understanding, this should not require "StatementPreperator".
> However, if I use the newly added "readRows" function, I get an exception
> that seems to force me to use the "StatementPreperator".
> Stacktrace below.
>
> java.lang.IllegalArgumentException: statementPreparator can not be null
>         at
> org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
>
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
>
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
>         at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>         at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
>         at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
>
>         at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
>
>         at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>         at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
>         at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
>         at
> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
>         at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>
>
>
> The test added in JdbcIOTest.java for this functionality only tests for
> queries with parameters.
> Is this new function supported only in the above case and not for normal
> "withQuery" (without parameters) ?
>
>
> Thanks & regards,
> Kishor
>
>

Re: Query about JdbcIO.readRows()

Posted by Kenneth Knowles <ke...@apache.org>.
Hi Kishor,

If you could not find a Jira, would you file one? Your contribution would
be very appreciated.

Kenn

On Tue, Aug 20, 2019 at 10:04 PM Kishor Joshi <jo...@yahoo.com> wrote:

> Hi,
>
> This fix is still not available in the Beam 2.15.0. Is there any Jira that
> has been created for this issue ? I am interested to contribute in that.
>
> Thanks & Regards,
> Kishor
>
> On Friday, August 2, 2019, 10:19:17 PM GMT+5:30, Jean-Baptiste Onofré <
> jb@nanthrax.net> wrote:
>
>
> Agree. I will fix that.
>
> Regards
> JB
> Le 2 août 2019, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
>
> Hi Kishor,
>
> + dev ( dev@beam.apache.org)
>
> This looks like a bug.  The attribute statementPreparator is nullable
> It should have been handled in the same way as in the expand method of
> Read<T> class.
>
>
> *Thanks & Regards,*
>
> *Vishwas *
>
>
> On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi < joshik24@yahoo.com> wrote:
>
> Hi,
>
> I am using the just released 2.14 version for JdbcIO with the newly added
> "readRows" functionality.
>
> I want to read table data with a query without parameters (select * from
> table_name).
> As per my understanding, this should not require "StatementPreperator".
> However, if I use the newly added "readRows" function, I get an exception
> that seems to force me to use the "StatementPreperator".
> Stacktrace below.
>
> java.lang.IllegalArgumentException: statementPreparator can not be null
>         at
> org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
>
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
>
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
>         at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>         at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
>         at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
>
>         at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
>
>         at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>         at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
>         at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
>         at
> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
>         at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>
>
>
> The test added in JdbcIOTest.java for this functionality only tests for
> queries with parameters.
> Is this new function supported only in the above case and not for normal
> "withQuery" (without parameters) ?
>
>
> Thanks & regards,
> Kishor
>
>

Re: Query about JdbcIO.readRows()

Posted by Kishor Joshi <jo...@yahoo.com>.
 Hi,
This fix is still not available in the Beam 2.15.0. Is there any Jira that has been created for this issue ? I am interested to contribute in that.
Thanks & Regards,Kishor
    On Friday, August 2, 2019, 10:19:17 PM GMT+5:30, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:  
 
 Agree. I will fix that. 

Regards
JBLe 2 août 2019, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
  Hi Kishor,   
   + dev ( dev@beam.apache.org)   
   This looks like a bug.  The attribute statementPreparator is nullable 
   It should have been handled in the same way as in the expand method of Read<T> class.
   
   
       Thanks & Regards,  Vishwas 
  
     
  On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi < joshik24@yahoo.com> wrote: 
  
    Hi,   
   I am using the just released 2.14 version for JdbcIO with the newly added "readRows" functionality.    
   I want to read table data with a query without parameters (select * from table_name).    As per my understanding, this should not require "StatementPreperator".    However, if I use the newly added "readRows" function, I get an exception that seems to force me to use the "StatementPreperator".    Stacktrace below.     
     java.lang.IllegalArgumentException: statementPreparator can not be null           at org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)           at org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)           at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)           at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)           at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)           at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)           at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)           at com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)           at com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)           at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)           at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)           at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)           at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)           at com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)           at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)   
   
   
   The test added in JdbcIOTest.java for this functionality only tests for queries with parameters.    Is this new function supported only in the above case and not for normal "withQuery" (without parameters) ?    
   
   Thanks & regards,   Kishor    

  

Re: Query about JdbcIO.readRows()

Posted by Kishor Joshi <jo...@yahoo.com>.
 Hi,
This fix is still not available in the Beam 2.15.0. Is there any Jira that has been created for this issue ? I am interested to contribute in that.
Thanks & Regards,Kishor
    On Friday, August 2, 2019, 10:19:17 PM GMT+5:30, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:  
 
 Agree. I will fix that. 

Regards
JBLe 2 août 2019, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
  Hi Kishor,   
   + dev ( dev@beam.apache.org)   
   This looks like a bug.  The attribute statementPreparator is nullable 
   It should have been handled in the same way as in the expand method of Read<T> class.
   
   
       Thanks & Regards,  Vishwas 
  
     
  On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi < joshik24@yahoo.com> wrote: 
  
    Hi,   
   I am using the just released 2.14 version for JdbcIO with the newly added "readRows" functionality.    
   I want to read table data with a query without parameters (select * from table_name).    As per my understanding, this should not require "StatementPreperator".    However, if I use the newly added "readRows" function, I get an exception that seems to force me to use the "StatementPreperator".    Stacktrace below.     
     java.lang.IllegalArgumentException: statementPreparator can not be null           at org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)           at org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)           at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)           at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)           at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)           at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)           at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)           at com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)           at com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)           at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)           at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)           at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)           at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)           at com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)           at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)   
   
   
   The test added in JdbcIOTest.java for this functionality only tests for queries with parameters.    Is this new function supported only in the above case and not for normal "withQuery" (without parameters) ?    
   
   Thanks & regards,   Kishor    

  

Re: Query about JdbcIO.readRows()

Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Agree. I will fix that. 

Regards
JB

Le 2 août 2019 à 17:15, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
>Hi Kishor,
>
>+ dev (dev@beam.apache.org)
>
>This looks like a bug.  The attribute statementPreparator is nullable
>It should have been handled in the same way as in the expand method of
>Read<T> class.
>
>
>*Thanks & Regards,*
>
>*Vishwas *
>
>
>On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi <jo...@yahoo.com> wrote:
>
>> Hi,
>>
>> I am using the just released 2.14 version for JdbcIO with the newly
>added
>> "readRows" functionality.
>>
>> I want to read table data with a query without parameters (select *
>from
>> table_name).
>> As per my understanding, this should not require
>"StatementPreperator".
>> However, if I use the newly added "readRows" function, I get an
>exception
>> that seems to force me to use the "StatementPreperator".
>> Stacktrace below.
>>
>> java.lang.IllegalArgumentException: statementPreparator can not be
>null
>>         at
>>
>org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
>>         at
>>
>org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
>>         at
>> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
>>         at
>> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
>>         at
>org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>>         at
>org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
>>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
>>         at
>>
>com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
>>         at
>>
>com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
>>         at
>org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>>         at
>org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
>>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
>>         at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
>>         at
>> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
>>         at
>com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>>
>>
>>
>> The test added in JdbcIOTest.java for this functionality only tests
>for
>> queries with parameters.
>> Is this new function supported only in the above case and not for
>normal
>> "withQuery" (without parameters) ?
>>
>>
>> Thanks & regards,
>> Kishor
>>

Re: Query about JdbcIO.readRows()

Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Agree. I will fix that. 

Regards
JB

Le 2 août 2019 à 17:15, à 17:15, Vishwas Bm <bm...@gmail.com> a écrit:
>Hi Kishor,
>
>+ dev (dev@beam.apache.org)
>
>This looks like a bug.  The attribute statementPreparator is nullable
>It should have been handled in the same way as in the expand method of
>Read<T> class.
>
>
>*Thanks & Regards,*
>
>*Vishwas *
>
>
>On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi <jo...@yahoo.com> wrote:
>
>> Hi,
>>
>> I am using the just released 2.14 version for JdbcIO with the newly
>added
>> "readRows" functionality.
>>
>> I want to read table data with a query without parameters (select *
>from
>> table_name).
>> As per my understanding, this should not require
>"StatementPreperator".
>> However, if I use the newly added "readRows" function, I get an
>exception
>> that seems to force me to use the "StatementPreperator".
>> Stacktrace below.
>>
>> java.lang.IllegalArgumentException: statementPreparator can not be
>null
>>         at
>>
>org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
>>         at
>>
>org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
>>         at
>> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
>>         at
>> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
>>         at
>org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>>         at
>org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
>>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
>>         at
>>
>com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
>>         at
>>
>com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
>>         at
>org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>>         at
>org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
>>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
>>         at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
>>         at
>> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
>>         at
>com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>>
>>
>>
>> The test added in JdbcIOTest.java for this functionality only tests
>for
>> queries with parameters.
>> Is this new function supported only in the above case and not for
>normal
>> "withQuery" (without parameters) ?
>>
>>
>> Thanks & regards,
>> Kishor
>>

Re: Query about JdbcIO.readRows()

Posted by Vishwas Bm <bm...@gmail.com>.
Hi Kishor,

+ dev (dev@beam.apache.org)

This looks like a bug.  The attribute statementPreparator is nullable
It should have been handled in the same way as in the expand method of
Read<T> class.


*Thanks & Regards,*

*Vishwas *


On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi <jo...@yahoo.com> wrote:

> Hi,
>
> I am using the just released 2.14 version for JdbcIO with the newly added
> "readRows" functionality.
>
> I want to read table data with a query without parameters (select * from
> table_name).
> As per my understanding, this should not require "StatementPreperator".
> However, if I use the newly added "readRows" function, I get an exception
> that seems to force me to use the "StatementPreperator".
> Stacktrace below.
>
> java.lang.IllegalArgumentException: statementPreparator can not be null
>         at
> org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
>         at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>         at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
>         at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
>         at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
>         at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>         at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
>         at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
>         at
> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
>         at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>
>
>
> The test added in JdbcIOTest.java for this functionality only tests for
> queries with parameters.
> Is this new function supported only in the above case and not for normal
> "withQuery" (without parameters) ?
>
>
> Thanks & regards,
> Kishor
>

Re: Query about JdbcIO.readRows()

Posted by Vishwas Bm <bm...@gmail.com>.
Hi Kishor,

+ dev (dev@beam.apache.org)

This looks like a bug.  The attribute statementPreparator is nullable
It should have been handled in the same way as in the expand method of
Read<T> class.


*Thanks & Regards,*

*Vishwas *


On Fri, Aug 2, 2019 at 2:48 PM Kishor Joshi <jo...@yahoo.com> wrote:

> Hi,
>
> I am using the just released 2.14 version for JdbcIO with the newly added
> "readRows" functionality.
>
> I want to read table data with a query without parameters (select * from
> table_name).
> As per my understanding, this should not require "StatementPreperator".
> However, if I use the newly added "readRows" function, I get an exception
> that seems to force me to use the "StatementPreperator".
> Stacktrace below.
>
> java.lang.IllegalArgumentException: statementPreparator can not be null
>         at
> org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$Read.withStatementPreparator(JdbcIO.java:600)
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:499)
>         at
> org.apache.beam.sdk.io.jdbc.JdbcIO$ReadRows.expand(JdbcIO.java:410)
>         at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>         at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
>         at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:34)
>         at
> com.nokia.csf.dfle.transforms.DfleRdbmsSource.expand(DfleRdbmsSource.java:10)
>         at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
>         at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
>         at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
>         at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
>         at
> com.nokia.csf.dfle.dsl.DFLEBeamMain.dagWireUp(DFLEBeamMain.java:49)
>         at com.nokia.csf.dfle.dsl.DFLEBeamMain.main(DFLEBeamMain.java:120)
>
>
>
> The test added in JdbcIOTest.java for this functionality only tests for
> queries with parameters.
> Is this new function supported only in the above case and not for normal
> "withQuery" (without parameters) ?
>
>
> Thanks & regards,
> Kishor
>