You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2021/02/22 16:34:59 UTC
[GitHub] [superset] suedschwede opened a new issue #13285: Superset Clickhouse driver
suedschwede opened a new issue #13285:
URL: https://github.com/apache/superset/issues/13285
At the moment there are two superset drivers "clickhouse-sqlalchemy","sqlalchemy-clickhouse"
There are problems with both drivers and that leads to a lot of questions
**sqlalchemy-clickhouse**
- Table with 128 Columns - Datasets sync of column does not work correctly - it seems there is a maximum columns you can use
- If the default user has a password you must use "infi.clickhouse_orm==1.0.4"
- using a dictionary you get "Python int too large to convert to C long"
**clickhouse-sqlalchemy**
- Sync of columns you get wrong data types e.g.: "NULLABLE(STRING)" instead of "STRING"
For me the only stable driver is "clickhouse-sqlalchemy" with the native interface
The documentation should recommend "clickhouse-sqlalchemy" instead of "sqlalchemy-clickhouse"
https://superset.apache.org/docs/databases/installing-database-drivers
It would be nice if the columns sync works correctly with "clickhouse-sqlalchemy"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] hodgesrm commented on issue #13285: Superset Clickhouse driver
Posted by GitBox <gi...@apache.org>.
hodgesrm commented on issue #13285:
URL: https://github.com/apache/superset/issues/13285#issuecomment-785492534
Hello @suedschwede, I'm testing this case and need some advice on the problem you are seeing. To check datatypes I made a small table and added a row to enable queries.
```
CREATE TABLE superset_test (
`basic_string` String,
`nullable_string` Nullable(String),
`lowcard_string` LowCardinality(String),
`basic_datetime` DateTime,
`nullable_datetime` Nullable(DateTime)
) ENGINE = TinyLog;
INSERT INTO superset_test VALUES('basic', 'nullable', 'lowcard', now(), now());
```
I can see the effect of syncing columns in the Edit Dataset view. The data types that return are STRING, NULLABLE(STRING), LOWCARDINALITY(STRING), DATETIME, NULLABLE(DATETIME). It looks as if the SQLAlchemy driver is getting these out of system.columns. A couple of questions:
1. Is the expected behavior to see just STRING and DATETIME? Or would you expect SQL types like VARCHAR?
2. Are there cases where these vendor-specific datatypes cause failures?
3. Are there other places where these types appear and cause problems?
It looks as Sqllite returns standard SQL types by comparison.
Thanks! I appended a copy of the Edit Dataset view mentioned above.
Cheers, Robert
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] royxact commented on issue #13285: Superset Clickhouse driver
Posted by GitBox <gi...@apache.org>.
royxact commented on issue #13285:
URL: https://github.com/apache/superset/issues/13285#issuecomment-978958779
Hi @villebro,
I see the issue is still happening in the current superset version, is there a fix planned, or HeinzMayer temporary fix the current solution?
Thanks!
Roy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] hodgesrm commented on issue #13285: Superset Clickhouse driver
Posted by GitBox <gi...@apache.org>.
hodgesrm commented on issue #13285:
URL: https://github.com/apache/superset/issues/13285#issuecomment-785536382
Adding to the original request by @suedschwede, I strongly recommend switching to clickhouse-sqlalchemy as the standard driver for ClickHouse. The reasons for this switch are as follows.
1. clickhouse-sqlalchemy is actively maintained (last commit 18 Feb 2021, under active development).
2. Releases are pushed regularly to pypi.org (last push 14 Dec 2020).
3. It has what appears to be a fairly complete SQLAlchemy implementation.
4. It supports both HTTP and native TCP wire protocols (vs. HTTP only
5. It has a good security including support for TLS, SNI, and configurable certificate verification.
6. It has a good suite of tests that check SQLAlchemy interfaces.
To the extent that there are compatibility issues like the one discussed here, we (Altinity) can get them fixed fairly quickly. We currently recommend clickhouse-sqlalchemy and clickhouse-driver to our users and have made fixes to them in the past, most recently in December.
What are the next steps to making this switch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] junlincc commented on issue #13285: Superset Clickhouse driver
Posted by GitBox <gi...@apache.org>.
junlincc commented on issue #13285:
URL: https://github.com/apache/superset/issues/13285#issuecomment-783924063
@srinify ^ 🙏
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on issue #13285: Superset Clickhouse driver
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on issue #13285:
URL: https://github.com/apache/superset/issues/13285#issuecomment-824309941
Superset now uses `clickhouse-sqlalchemy` for Clickhouse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on issue #13285: Superset Clickhouse driver
Posted by GitBox <gi...@apache.org>.
villebro commented on issue #13285:
URL: https://github.com/apache/superset/issues/13285#issuecomment-785706023
We're currently working on improving type inference (https://github.com/apache/superset/pull/13294). We'll make it top priority to enable this for Clickhouse as soon as possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] hodgesrm edited a comment on issue #13285: Superset Clickhouse driver
Posted by GitBox <gi...@apache.org>.
hodgesrm edited a comment on issue #13285:
URL: https://github.com/apache/superset/issues/13285#issuecomment-785536382
Adding to the original request by @suedschwede, I strongly recommend switching to clickhouse-sqlalchemy as the standard driver for ClickHouse. The reasons for this switch are as follows.
1. clickhouse-sqlalchemy is actively maintained (last commit 18 Feb 2021, under active development).
2. Releases are pushed regularly to pypi.org (last push 14 Dec 2020).
3. It has what appears to be a fairly complete SQLAlchemy implementation.
4. It supports both HTTP and native TCP wire protocols (vs. HTTP only)
5. It has a good security including support for TLS, SNI, and configurable certificate verification (vs. no encryption support)
6. It has a good suite of tests that check SQLAlchemy interfaces.
To the extent that there are compatibility issues like the one discussed here, we (Altinity) can get them fixed fairly quickly. We currently recommend clickhouse-sqlalchemy and clickhouse-driver to our users and have made fixes to them in the past, most recently in December.
What are the next steps to making this switch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] suedschwede commented on issue #13285: Superset Clickhouse driver
Posted by GitBox <gi...@apache.org>.
suedschwede commented on issue #13285:
URL: https://github.com/apache/superset/issues/13285#issuecomment-785703590
If you try to make a date/time filter to NULLABLE(DATE) you get an error in superset
The expected behavior is to see just STRING and DATETIME
NULLABLE(STRING) -> STRING
NULLABLE(DATE) -> DATE
NULLABLE(DATETIME) -> DATETIME
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] HeinzMayer commented on issue #13285: Superset Clickhouse driver
Posted by GitBox <gi...@apache.org>.
HeinzMayer commented on issue #13285:
URL: https://github.com/apache/superset/issues/13285#issuecomment-801722599
I have created a temporary fix on my environment
[clickhouse.py](https://github.com/HeinzMayer/superset/blob/1.0.MIC/superset/db_engine_specs/clickhouse.py)
It seems that the master branch does not work with clickhouse-sqlalchemy==0.1.6
[Discussion](https://github.com/apache/superset/discussions/10700)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida closed issue #13285: Superset Clickhouse driver
Posted by GitBox <gi...@apache.org>.
betodealmeida closed issue #13285:
URL: https://github.com/apache/superset/issues/13285
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org