[jira] [Work logged] (BEAM-2817) Bigquery queries should allow options to run in batch mode or not

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-2817?focusedWorklogId=80563&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80563 ]

ASF GitHub Bot logged work on BEAM-2817:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 14/Mar/18 21:13
            Start Date: 14/Mar/18 21:13
    Worklog Time Spent: 10m 
      Work Description: jkff commented on a change in pull request #4606: [BEAM-2817] BigQuery queries are allowed to run in either BATCH or IN…
URL: https://github.com/apache/beam/pull/4606#discussion_r174611949
 
 

 ##########
 File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySource.java
 ##########
 @@ -74,12 +77,19 @@ private BigQueryQuerySource(
       Boolean useLegacySql,
       BigQueryServices bqServices,
       Coder<T> coder,
-      SerializableFunction<SchemaAndRecord, T> parseFn) {
+      SerializableFunction<SchemaAndRecord, T> parseFn,
+      Priority priority) {
     super(stepUuid, bqServices, coder, parseFn);
     this.query = checkNotNull(query, "query");
     this.flattenResults = checkNotNull(flattenResults, "flattenResults");
     this.useLegacySql = checkNotNull(useLegacySql, "useLegacySql");
     this.dryRunJobStats = new AtomicReference<>();
+    if (priority != BigQueryIO.TypedRead.Priority.BATCH
 
 Review comment:
   This condition is always true - "priority != a || priority != b" is true by definition, because it can't equal a and b at the same time. I suppose you probably meant "&&", but then again I'm not sure why this condition is needed at all - why not just "this.priority = priority"?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 80563)
    Time Spent: 2h 10m  (was: 2h)

> Bigquery queries should allow options to run in batch mode or not
> -----------------------------------------------------------------
>
>                 Key: BEAM-2817
>                 URL: https://issues.apache.org/jira/browse/BEAM-2817
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-gcp
>    Affects Versions: 2.0.0
>            Reporter: Lara Schmidt
>            Assignee: Justin Tumale
>            Priority: Major
>              Labels: newbie, starter
>             Fix For: 2.4.0
>
>          Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> When bigquery read does a query it sets the mode to batch. A batch query can be very slow to schedule as it batches it with other queries. However it doesn't use batch quota which is better for some cases. However, in some cases a fast query is better (especially in timed tests). It would be a good idea to have a configuration to the bigquery source to set this per-read.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)