You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by john desmond <jo...@gmail.com> on 2019/07/15 22:25:56 UTC

Using the BigQuery Storage API

Hello,

I am trying to follow the tutorial for reading data from a Big Query table
using java, specifically using the BigQuery Storage API. I am using the
tutorial found here:
https://beam.apache.org/documentation/io/built-in/google-bigquery/

The tutorial suggests using the 2.13.0 version of beam, but many of the
suggested import statements from the source code:
https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/cookbook/BigQueryTornadoes.java

are not supported with this version of beam. For example I am getting
errors in my IDE by trying to use import statements like,

import org.apache.beam.sdk.Pipeline;

I'm not sure how to resolve these statements because the only documentation
I can find is from Beam 2.0.0, e.g.
https://beam.apache.org/releases/javadoc/2.0.0/org/apache/beam/sdk/Pipeline.html


Any suggestions on how to resolve these kinds of issues?

Thank you!

John M. Desmond
(631) 833-2836
https://www.linkedin.com/in/yohn-dezmon/
https://github.com/yohn-dezmon

Re: Using the BigQuery Storage API

Posted by sridhar inuog <sr...@gmail.com>.

Hi John,
           I encountered similar errors while going through the following
guide

https://cwiki.apache.org/confluence/display/BEAM/Using+IntelliJ+IDE

and I missed this step

Open *File | Settings...* In left pane, navigate to *Build, Execution,
Deployment | Build Tools | Gradle | Runner*. Select *Delegate IDE build/run
actions to gradle*, and *Run tests using: Gradle Test Runner*. Then select
*Apply* and *OK*.

Cheers,
Sridhar

On Wed, Jul 17, 2019 at 3:52 AM Ismaël Mejía <ie...@gmail.com> wrote:

> This document and other developer related guides are now maintained in
> the Beam wiki:
> https://cwiki.apache.org/confluence/display/BEAM/Using+IntelliJ+IDE
>
>
> On Wed, Jul 17, 2019 at 12:40 AM Rui Wang <ru...@google.com> wrote:
> >
> > Hi,
> >
> > I have a fresh repo cloned and switch to release-2.13.0. I tried to add
> "import org.apache.beam.sdk.Pipeline" to BigQueryTornadoes.java  and
> succeeded(I am using intellij)
> >
> > I am thinking you might not setup your IDE right.
> >
> >
> > If you are using intellij (and BTW intellij has a community version
> which is free and good enough for Beam). There was a doc developed by
> people in Beam community on how to setup intellij [1].
> >
> >
> > [1]:
> https://docs.google.com/document/d/18eXrO9IYll4oOnFb53EBhOtIfx-JLOinTWZSIBFkLk4/edit
> >
> >
> > -Rui
> >
> > On Mon, Jul 15, 2019 at 9:01 PM john desmond <jo...@gmail.com>
> wrote:
> >>
> >> Hello,
> >>
> >> I am trying to follow the tutorial for reading data from a Big Query
> table using java, specifically using the BigQuery Storage API. I am using
> the  tutorial found here:
> https://beam.apache.org/documentation/io/built-in/google-bigquery/
> >>
> >> The tutorial suggests using the 2.13.0 version of beam, but many of the
> suggested import statements from the source code:
> https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/cookbook/BigQueryTornadoes.java
> >> are not supported with this version of beam. For example I am getting
> errors in my IDE by trying to use import statements like,
> >>
> >> import org.apache.beam.sdk.Pipeline;
> >>
> >> I'm not sure how to resolve these statements because the only
> documentation I can find is from Beam 2.0.0, e.g.
> https://beam.apache.org/releases/javadoc/2.0.0/org/apache/beam/sdk/Pipeline.html
> >>
> >> Any suggestions on how to resolve these kinds of issues?
> >>
> >> Thank you!
> >>
> >> John M. Desmond
> >> (631) 833-2836
> >> https://www.linkedin.com/in/yohn-dezmon/
> >> https://github.com/yohn-dezmon
> >>
>

Re: Using the BigQuery Storage API

Posted by Ismaël Mejía <ie...@gmail.com>.

This document and other developer related guides are now maintained in
the Beam wiki:
https://cwiki.apache.org/confluence/display/BEAM/Using+IntelliJ+IDE


On Wed, Jul 17, 2019 at 12:40 AM Rui Wang <ru...@google.com> wrote:
>
> Hi,
>
> I have a fresh repo cloned and switch to release-2.13.0. I tried to add "import org.apache.beam.sdk.Pipeline" to BigQueryTornadoes.java  and succeeded(I am using intellij)
>
> I am thinking you might not setup your IDE right.
>
>
> If you are using intellij (and BTW intellij has a community version which is free and good enough for Beam). There was a doc developed by people in Beam community on how to setup intellij [1].
>
>
> [1]: https://docs.google.com/document/d/18eXrO9IYll4oOnFb53EBhOtIfx-JLOinTWZSIBFkLk4/edit
>
>
> -Rui
>
> On Mon, Jul 15, 2019 at 9:01 PM john desmond <jo...@gmail.com> wrote:
>>
>> Hello,
>>
>> I am trying to follow the tutorial for reading data from a Big Query table using java, specifically using the BigQuery Storage API. I am using the  tutorial found here: https://beam.apache.org/documentation/io/built-in/google-bigquery/
>>
>> The tutorial suggests using the 2.13.0 version of beam, but many of the suggested import statements from the source code: https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/cookbook/BigQueryTornadoes.java
>> are not supported with this version of beam. For example I am getting errors in my IDE by trying to use import statements like,
>>
>> import org.apache.beam.sdk.Pipeline;
>>
>> I'm not sure how to resolve these statements because the only documentation I can find is from Beam 2.0.0, e.g. https://beam.apache.org/releases/javadoc/2.0.0/org/apache/beam/sdk/Pipeline.html
>>
>> Any suggestions on how to resolve these kinds of issues?
>>
>> Thank you!
>>
>> John M. Desmond
>> (631) 833-2836
>> https://www.linkedin.com/in/yohn-dezmon/
>> https://github.com/yohn-dezmon
>>

Re: Using the BigQuery Storage API

Posted by Rui Wang <ru...@google.com>.

Hi,

I have a fresh repo cloned and switch to release-2.13.0. I tried to add
"import org.apache.beam.sdk.Pipeline" to BigQueryTornadoes.java
<https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/cookbook/BigQueryTornadoes.java>
and succeeded(I am using intellij)

I am thinking you might not setup your IDE right.

If you are using intellij (and BTW intellij has a community version which
is free and good enough for Beam). There was a doc developed by people in
Beam community on how to setup intellij [1].

[1]:
https://docs.google.com/document/d/18eXrO9IYll4oOnFb53EBhOtIfx-JLOinTWZSIBFkLk4/edit

-Rui

On Mon, Jul 15, 2019 at 9:01 PM john desmond <jo...@gmail.com>
wrote:

> Hello,
>
> I am trying to follow the tutorial for reading data from a Big Query table
> using java, specifically using the BigQuery Storage API. I am using the
> tutorial found here:
> https://beam.apache.org/documentation/io/built-in/google-bigquery/
>
> The tutorial suggests using the 2.13.0 version of beam, but many of the
> suggested import statements from the source code:
> https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/cookbook/BigQueryTornadoes.java
>
> are not supported with this version of beam. For example I am getting
> errors in my IDE by trying to use import statements like,
>
> import org.apache.beam.sdk.Pipeline;
>
> I'm not sure how to resolve these statements because the only
> documentation I can find is from Beam 2.0.0, e.g.
> https://beam.apache.org/releases/javadoc/2.0.0/org/apache/beam/sdk/Pipeline.html
>
>
> Any suggestions on how to resolve these kinds of issues?
>
> Thank you!
>
> John M. Desmond
> (631) 833-2836
> https://www.linkedin.com/in/yohn-dezmon/
> https://github.com/yohn-dezmon
>
>