[GitHub] [superset] Mageswaran1989 commented on issue #20708: Unable to read S3 parquet files using DuckDB as Database Connector

[GitBox <gi...@apache.org>]

Mageswaran1989 commented on issue #20708:
URL: https://github.com/apache/superset/issues/20708#issuecomment-1187735730

   Update: By patching the `conenct` method of the DuckDB engine [here](https://github.com/Mause/duckdb_engine/blob/master/duckdb_engine/__init__.py#L160) @sachintyagi22 was able to read the parquet files. 
   
   Something like this:
   
   ```python
       def connect(self, *args: Any, **kwargs: Any) -> ConnectionWrapper:
           cursor = duckdb.connect(*args, **kwargs)
           cursor.execute("INSTALL httpfs;")
           cursor.execute("LOAD httpfs;")
           cursor.execute("SET s3_region='******'")
           cursor.execute("SET s3_access_key_id=''**************")
           cursor.execute("SET s3_secret_access_key='*****************************'")
           return ConnectionWrapper(cursor)
   ```
   
   Without this patch, the duckdb session is not abel to recognize the `SET` commands. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org