[jira] [Commented] (BEAM-5966) When working "LiteralGqlQuery", Error: Query cannot have any sort orders.

["Chamikara Jayalath (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739585#comment-16739585 ] 

Chamikara Jayalath commented on BEAM-5966:
------------------------------------------

Was your question resolved or is there something more to address here ?

> When working "LiteralGqlQuery", Error: Query cannot have any sort orders.
> -------------------------------------------------------------------------
>
>                 Key: BEAM-5966
>                 URL: https://issues.apache.org/jira/browse/BEAM-5966
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-gcp
>    Affects Versions: 2.8.0
>            Reporter: Murat ODUNC
>            Assignee: Chamikara Jayalath
>            Priority: Major
>
> Hello..
> I have used Apache Beam pipelines on my project. And I have a problem to read data from GCP DataStore.
>  
> My First question is "How to create "Query" object" . I'am sharing a sample of regarding JavaDoc.
> [https://beam.apache.org/releases/javadoc/2.8.0/]
> {code:java}
>  Query query = ...; // I dont know any idea how configure the object
>  String projectId = "...";
>  Pipeline p = Pipeline.create(options);
>  PCollection<Entity> entities = p.apply(
>      DatastoreIO.v1().read()
>          .withProjectId(projectId)
>          .withQuery(query));
> {code}
> My second question is how to set "sort oder" of the query?
> I tried to 'LiteralGqlQuery' to read data but DataStore IO Reader  failed by follow error trace..
> {noformat}
> java.lang.IllegalArgumentException: Query cannot have any sort orders. at com.google.datastore.v1.client.QuerySplitterImpl.validateQuery(QuerySplitterImpl.java:128) ~[datastore-v1-proto-client-1.6.0.jar:na] at com.google.datastore.v1.client.QuerySplitterImpl.getSplits(QuerySplitterImpl.java:69) ~[datastore-v1-proto-client-1.6.0.jar:na] at org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read.splitQuery(DatastoreV1.java:454) ~[beam-sdks-java-io-google-cloud-platform-2.8.0.jar:na] at org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read.access$100(DatastoreV1.java:264) ~[beam-sdks-java-io-google-cloud-platform-2.8.0.jar:na] at org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read$SplitQueryFn.processElement(DatastoreV1.java:813) ~[beam-sdks-java-io-google-cloud-platform-2.8.0.jar:na] at org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read$SplitQueryFn$DoFnInvoker.invokeProcessElement(Unknown Source) [na:na] at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:275) [beam-runners-direct-java-2.8.0.jar:na] at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:240) [beam-runners-direct-java-2.8.0.jar:na] at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimplePushbackSideInputDoFnRunner.processElementInReadyWindows(SimplePushbackSideInputDoFnRunner.java:78) [beam-runners-direct-java-2.8.0.jar:na]
> at org.apache.beam.runners.direct.ParDoEvaluator.processElement(ParDoEvaluator.java:207) [beam-runners-direct-java-2.8.0.jar:na] at org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.processElement(DoFnLifecycleManagerRemovingTransformEvaluator.java:55) [beam-runners-direct-java-2.8.0.jar:na] at org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160) [beam-runners-direct-java-2.8.0.jar:na] at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:124) [beam-runners-direct-java-2.8.0.jar:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_162] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_162] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_162] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_162] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
> {noformat}
> My code is  here
>  
> {code:java}
> PipelineOptions options = PipelineOptionsFactory.create(); 
> options.setRunner(DirectRunner.class); String gqlQuery = String.format("SELECT * FROM task " + "WHERE createdAt > DATETIME('%s') "+ "AND createdAt < DATETIME('%s') ORDER BY createdAt ASC", "2018-11-03T00:00:00Z", "2018-11-03T23:59:59Z" ); log.info("GQL: " + gqlQuery); String projectId = "foo-bar"; String ns = "prod-01ce205a-22ff-4bab-b133-926bdc54c8b3"; 
> Pipeline p = Pipeline.create(options); 
> PCollection<String> entities = p.apply( DatastoreIO.v1().read()   .withProjectId(projectId).withLiteralGqlQuery(gqlQuery) .withQuery(query) .withNamespace(ns)) .apply("Debug", ParDo.of(new DoFn<Entity, String>(){ 
>               @ProcessElement 
>               public void processElement(ProcessContext c) {
>                     log.info(c.element().getProperties().get("foo").getStringValue());  
>               }
>    })); 
> p.run().waitUntilFinish();
> {code}
> ```
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)