You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "karsten-wagner (via GitHub)" <gi...@apache.org> on 2023/04/25 15:39:03 UTC
[GitHub] [superset] karsten-wagner opened a new pull request, #23816: fix(docs): respect no_proxy environment variable
karsten-wagner opened a new pull request, #23816:
URL: https://github.com/apache/superset/pull/23816
### SUMMARY
The `clickhouse-connect` library `0.4.1` did not handle a `no_proxy` environment variable. This is eventually problematic when using the [Docker Compose Installation Guide](https://superset.apache.org/docs/installation/installing-superset-using-docker-compose) in an environment that requires a forward proxy (e.g., in an enterprise environment) and where Clickhouse should be used as a database. In such case the call to the database should remain inside the docker network to allow addressing the database via its container name. Calls to external data sources should be rerouted via the forward proxy. Version `0.4.1` handles all connections via the proxy and thus misses the local network connection inside docker network and thus eventually fails host name resolution, given the forward proxy sits outside the docker network and is not aware of the inside docker network container names.
The issue has been fixed in the `clickhouse-connect` library in version `0.5.19` via [#PR163](https://github.com/ClickHouse/clickhouse-connect/issues/163), hence bumping version here in the superset docs accordingly.
### TESTING INSTRUCTIONS
* Follow the [Docker Compose Installation Guide](https://superset.apache.org/docs/installation/installing-superset-using-docker-compose)
* Follow the existing [Clickhouse Installation Guide](https://superset.apache.org/docs/databases/clickhouse) referencing version `0.4.1`
* Add a clickhouse container named `clickhouse` to `docker-compose-non-dev.yml`, e.g.:
```yaml
x-clickhouse-volumes:
&clickhouse-volumes
- clickhouse_home:/var/lib/clickhouse
clickhouse:
image: clickhouse/clickhouse-server:23
container_name: clickhouse
env_file: .env
user: "root"
restart: unless-stopped
ports:
- "8123:8123"
- "9003:9000"
volumes: *clickhouse-volumes
volumes:
clickhouse_home:
external: false
```
* Update `docker/.env` to contain `http_proxy`, `https_proxy` and `no_proxy` environment variables, e.g.:
```yaml
# Local Proxy Settings
http_proxy="http://host.docker.internal:8888"
https_proxy="http://host.docker.internal:8888"
no_proxy="localhost,127.0.0.1,db,redis,superset,superset-init,superset-worker,superset-beat,clickhouse"
```
* Provide the forward proxy on the host OS (e.g., Fiddler - in above example on port 8888)
* Start all containers and log in to superset
* Try to add Clickhouse database via connection string `clickhousedb://clickhouse/default`
The connection will fail. See eventually logs in the ´superset_app` container. Superset UI shows an error message on the dialog with the connection string, but not on the one where you provide host, port etc. in separate fields.
* Change `docker/requirements-local.txt` to contain new version `clickhouse-connect>=0.5.19`
* Repeat the test to add the database and all works
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] genzgd commented on pull request #23816: fix(docs): respect no_proxy environment variable
Posted by "genzgd (via GitHub)" <gi...@apache.org>.
genzgd commented on PR #23816:
URL: https://github.com/apache/superset/pull/23816#issuecomment-1522024129
I'd recommend using 0.5.20 as the minimum, since it has a somewhat important bug fix concerning HTTP headers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] michael-s-molina commented on pull request #23816: fix(docs): respect no_proxy environment variable
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on PR #23816:
URL: https://github.com/apache/superset/pull/23816#issuecomment-1645471292
> @michael-s-molina, anything missing to merge this to master?
I was just waiting for CI to complete 😄
Thank you for the fix @karsten-wagner and for the awesome PR description!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] karsten-wagner commented on pull request #23816: fix(docs): respect no_proxy environment variable
Posted by "karsten-wagner (via GitHub)" <gi...@apache.org>.
karsten-wagner commented on PR #23816:
URL: https://github.com/apache/superset/pull/23816#issuecomment-1645472237
Thanks and welcome! Superset is a great tool!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] karsten-wagner commented on pull request #23816: fix(docs): respect no_proxy environment variable
Posted by "karsten-wagner (via GitHub)" <gi...@apache.org>.
karsten-wagner commented on PR #23816:
URL: https://github.com/apache/superset/pull/23816#issuecomment-1522035743
@genzgd, updated to ´0.5.20´ as suggested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] genzgd commented on pull request #23816: fix(docs): respect no_proxy environment variable
Posted by "genzgd (via GitHub)" <gi...@apache.org>.
genzgd commented on PR #23816:
URL: https://github.com/apache/superset/pull/23816#issuecomment-1644778646
Just a heads up, at this point the current version is 0.6.8 and 0.6.x is required for Superset 2.1.+
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] karsten-wagner commented on pull request #23816: fix(docs): respect no_proxy environment variable
Posted by "karsten-wagner (via GitHub)" <gi...@apache.org>.
karsten-wagner commented on PR #23816:
URL: https://github.com/apache/superset/pull/23816#issuecomment-1522021769
Big thanks to @genzgd for all the hard work to respect the `no_proxy` variable in the `clickhouse-connect` library. I am just connecting the dots here...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] michael-s-molina closed pull request #23816: fix(docs): respect no_proxy environment variable
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina closed pull request #23816: fix(docs): respect no_proxy environment variable
URL: https://github.com/apache/superset/pull/23816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] genzgd commented on pull request #23816: fix(docs): respect no_proxy environment variable
Posted by "genzgd (via GitHub)" <gi...@apache.org>.
genzgd commented on PR #23816:
URL: https://github.com/apache/superset/pull/23816#issuecomment-1522025641
Also big thanks for adding the this PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] karsten-wagner commented on pull request #23816: fix(docs): respect no_proxy environment variable
Posted by "karsten-wagner (via GitHub)" <gi...@apache.org>.
karsten-wagner commented on PR #23816:
URL: https://github.com/apache/superset/pull/23816#issuecomment-1645137767
@michael-s-molina, anything missing to merge this to master?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] michael-s-molina merged pull request #23816: fix(docs): respect no_proxy environment variable
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina merged PR #23816:
URL: https://github.com/apache/superset/pull/23816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org