You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/11/16 10:06:08 UTC

[GitHub] [arrow-datafusion] Jefffrey opened a new issue, #4239: Cannot query s3 data from datafusion-cli

Jefffrey opened a new issue, #4239:
URL: https://github.com/apache/arrow-datafusion/issues/4239

   **Describe the bug**
   
   When following steps from user guide for cli:
   
   https://github.com/apache/arrow-datafusion/blob/fc669d5892954cbd2612a272314785758a7cb176/docs/source/user-guide/cli.md?plain=1#L154-L187
   
   Cannot create a table as it states.
   
   **To Reproduce**
   
   ```bash
   jeffrey:~/Code/arrow-datafusion/datafusion-cli$ export AWS_ACCESS_KEY_ID=****************
   jeffrey:~/Code/arrow-datafusion/datafusion-cli$ export AWS_SECRET_ACCESS_KEY=****************
   jeffrey:~/Code/arrow-datafusion/datafusion-cli$ export AWS_REGION=ap-southeast-2
   jeffrey:~/Code/arrow-datafusion/datafusion-cli$ cargo run
       Finished dev [unoptimized + debuginfo] target(s) in 0.10s
        Running `/media/jeffrey/1tb_860evo_ssd/.cargo_target_cache/debug/datafusion-cli`
   DataFusion CLI v14.0.0
   ❯ create external table test stored as parquet location 's3://tremendous-orange-fork/customer/part-0.parquet';
   Execution("Generic S3 error: Missing region")
   ❯
   ```
   
   **Expected behavior**
   
   Be able to create a table from s3 and query it, per instructions in the user doc.
   
   **Additional context**
   
   User doc also states that `AWS_REGION` must be set as per:
   
   https://github.com/apache/arrow-datafusion/blob/fc669d5892954cbd2612a272314785758a7cb176/docs/source/user-guide/cli.md?plain=1#L162-L164
   
   However it seems this issue https://github.com/apache/arrow-rs/issues/2795 has been resolved and the fix merged a while back, and attempting the above steps without an `AWS_REGION` env var set still results in same error. Unsure whether user doc is wrong or there is another bug relating to `AWS_REGION`?
   
   ```bash
   jeffrey:~/Code/arrow-datafusion/datafusion-cli$ export AWS_ACCESS_KEY_ID=****************
   jeffrey:~/Code/arrow-datafusion/datafusion-cli$ export AWS_SECRET_ACCESS_KEY=****************
   jeffrey:~/Code/arrow-datafusion/datafusion-cli$ export AWS_REGION=
   jeffrey:~/Code/arrow-datafusion/datafusion-cli$ cargo run
       Finished dev [unoptimized + debuginfo] target(s) in 0.10s
        Running `/media/jeffrey/1tb_860evo_ssd/.cargo_target_cache/debug/datafusion-cli`
   DataFusion CLI v14.0.0
   ❯ create external table test stored as parquet location 's3://tremendous-orange-fork/customer/part-0.parquet';
   Execution("Generic S3 error: Missing region")
   ❯
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] psvri commented on issue #4239: Cannot query s3 data from datafusion-cli

Posted by GitBox <gi...@apache.org>.
psvri commented on issue #4239:
URL: https://github.com/apache/arrow-datafusion/issues/4239#issuecomment-1323969277

   The region is specified via `AWS_DEFAULT_REGION`  environment variable.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] alamb closed issue #4239: Cannot query s3 data from datafusion-cli

Posted by GitBox <gi...@apache.org>.
alamb closed issue #4239: Cannot query s3 data from datafusion-cli
URL: https://github.com/apache/arrow-datafusion/issues/4239


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org