You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 21:42:22 UTC

[GitHub] [beam] damccorm opened a new issue, #21155: Call the stored procedure in dataflow pipeline

damccorm opened a new issue, #21155:
URL: https://github.com/apache/beam/issues/21155

   https://stackoverflow.com/questions/69863008/call-the-bigquery-stored-procedure-in-dataflow-pipeline
   
   I have written a stored procedure in Bigquery and trying to call it within a dataflow pipeline. This works for the `SELECT` queries but not for the stored procedure:
   
   ```
   
   pipeLine = beam.Pipeline(options=options)
   rawdata = ( pipeLine
               | beam.io.ReadFromBigQuery(
   
                 query="CALL my_dataset.create_customer()", use_standard_sql=True)
             )
        
       pipeLine.run().wait_until_finish()
   
   ```
   
   
   Stored procedure:
   
   ```
   
   CREATE OR REPLACE PROCEDURE my_dataset.create_customer()
   BEGIN
       SELECT * 
       FROM `project_name.my_dataset.my_table`
   
       WHERE customer_name LIKE "%John%"
       ORDER BY created_time
       LIMIT 5;
   END;
   
   ```
   
   
   I am able to create the stored procedure and call it within the Bigquery console. But, in the dataflow pipeline, it throws an error:
   > "code": 400,
   > essage": "configuration.query.destinationEncryptionConfiguration cannot be set for scripts",
   > "message": "configuration.query.destinationEncryptionConfiguration cannot be set for scripts", "domain": "global",
   > eason": "invalid"
   > "status": "INVALID_ARGUMENT"
   >  
   
    
   
   Imported from Jira [BEAM-13194](https://issues.apache.org/jira/browse/BEAM-13194). Original Jira may contain additional context.
   Reported by: ahalya_h.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Call the stored procedure in dataflow pipeline [beam]

Posted by "leotrs (via GitHub)" <gi...@apache.org>.
leotrs commented on issue #21155:
URL: https://github.com/apache/beam/issues/21155#issuecomment-1973933921

   Same error here.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] shotarok commented on issue #21155: Call the stored procedure in dataflow pipeline

Posted by "shotarok (via GitHub)" <gi...@apache.org>.
shotarok commented on issue #21155:
URL: https://github.com/apache/beam/issues/21155#issuecomment-1485898819

   The same error happened to my dataflow pieline that used variables with `DECLARE `: [link](https://cloud.google.com/bigquery/docs/reference/standard-sql/procedural-language#declare)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org