You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "rajivpatki (via GitHub)" <gi...@apache.org> on 2024/02/02 11:16:36 UTC
Re: [I] Unable to read S3 parquet files using DuckDB as Database Connector [superset]
rajivpatki commented on issue #20708:
URL: https://github.com/apache/superset/issues/20708#issuecomment-1923597807
> appends a method at the bottom of [db_engine_specs/duckdb.py](https://github.com/apache/superset/blob/master/superset/db_engine_specs/duckdb.py#L80)
This works well, but someone with SQL Lab access can read plaintext credentials with `SELECT * FROM duckdb_settings();`
> If you can pass configuration to duckdb using superset (https://github.com/Mause/duckdb_engine/#configuration) I would recommend doing that instead
Using @Mause's suggestion: any admin with rights to edit the database connection can read the secrets in plain text.
Suggestion: override `execute()` for `DuckDBEngineSpec` in this manner:
```
@classmethod
def execute(cls, cursor: Any, query: str, **kwargs: Any) -> None:
sql = f"""
INSTALL aws;
LOAD aws;
INSTALL httpfs;
LOAD httpfs;
CALL load_aws_credentials();
"""
if 's3://' in query and 'duckdb_settings()' not in query:
cursor.execute(sql)
return super().execute(cursor, query, **kwargs)
```
Note that `CALL load_aws_credentials();` from aws extension of duckdb has a **significant hit on query performance**. The above approach allows us to:
- Bypass loading credentials and extensions when not required
- Not have to specify credentials in plain text and instead use [AWS Default Credentials Provider Chain](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials-chain.html)
- Plug a security hole where credentials can be displayed in plain text
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org