You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "sebastianliebscher (via GitHub)" <gi...@apache.org> on 2023/10/06 04:55:44 UTC
[PR] [WIP] feat(docker): enable single curl of docker-compose.yml [superset]
sebastianliebscher opened a new pull request, #25546:
URL: https://github.com/apache/superset/pull/25546
### SUMMARY
As the title says, this PR tries to address https://github.com/apache/superset/discussions/24987
- instead of mounting the `docker/` directory into the container, moved the scripts in docker/ into the docker image
- moved default (environment) variables from `.env` into `superset_config.py`
Current blocker:
- the Postgres `examples-init.sh`
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
<!--- Skip this if not applicable -->
### TESTING INSTRUCTIONS
<!--- Required! What steps can be taken to manually verify the changes? -->
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] [WIP] feat(docker): enable single curl of docker-compose.yml [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed pull request #25546: [WIP] feat(docker): enable single curl of docker-compose.yml
URL: https://github.com/apache/superset/pull/25546
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] [WIP] feat(docker): enable single curl of docker-compose.yml [superset]
Posted by "sebastianliebscher (via GitHub)" <gi...@apache.org>.
sebastianliebscher commented on PR #25546:
URL: https://github.com/apache/superset/pull/25546#issuecomment-1749985919
Current blocker:
- the Postgres `examples-init.sh`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] [WIP] feat(docker): enable single curl of docker-compose.yml [superset]
Posted by "sebastianliebscher (via GitHub)" <gi...@apache.org>.
sebastianliebscher commented on PR #25546:
URL: https://github.com/apache/superset/pull/25546#issuecomment-1757242085
> Would the pip workflow continue as is, if we make this modification to docker?
Yes, the pip workflow still uses SQLite by default.
> would we need a whole separate Postgres container for the examples, or just a separate db (and maybe user) in the same Postgres container?
Ideally we should just use one Postgres service and have Superset's metadata separated from the examples, yes. This was already achieved and merged by https://github.com/apache/superset/pull/25003. But that requires an initialization script (`examples-init-sh`) to set up the other database and user. To still be able to curl and start just the single `docker-compose-non-dev.yml`, this init script logic has to be removed/moved without losing the separation between metadata and examples. That's what I tried to express with my comment above https://github.com/apache/superset/pull/25546#issuecomment-1751479161.
The current solution in this PR uses two separate Postgres services with examples enabled by default. If a user decides not to use examples, it is as easy as deleting the `examples` service + volume and `SUPERSET_LOAD_EXAMPLES` config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] [WIP] feat(docker): enable single curl of docker-compose.yml [superset]
Posted by "sebastianliebscher (via GitHub)" <gi...@apache.org>.
sebastianliebscher commented on PR #25546:
URL: https://github.com/apache/superset/pull/25546#issuecomment-1751479161
@sfirke Both, I'm not sure which is the best way to initlialize the Postgres db, without the `docker/docker-entrypoint-initdb.d/examples-init.sh`.
I'm thinking of the following possibilities:
1. provide another docker image tag e.g. `superset:examples-db`. the docker-compose-non-dev.yml would get the examples db as its own postgres service like I mentioned here https://github.com/apache/superset/discussions/23399#discussioncomment-6708360
1. move the logic of examples-init.sh into the cli command superset load-examples. for this you would have to detect if the example-db is postgres
1. copy examples-init.sh into the superset-init container and add logic to run the script. to do this you would need to detect if the examples-db is postgres and the superset-init container would also need the postgres-client package
I think 3. is not a good idea as it is too much customization. 2. I am not familiar with superset load-examples. I would go for 1.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] [WIP] feat(docker): enable single curl of docker-compose.yml [superset]
Posted by "sfirke (via GitHub)" <gi...@apache.org>.
sfirke commented on PR #25546:
URL: https://github.com/apache/superset/pull/25546#issuecomment-1751497327
Ooh I also like #1. Another bonus: I loaded the example data into what became my production instance and it was hard to get it out b/c it was intermingled with the metadata (unless I'm confused). If we had this paradigm you propose in #1 I could have just dropped the examples container (or dropped the data source in Superset), easy peasy.
Would the pip workflow continue as is, if we make this modification to docker?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] [WIP] feat(docker): enable single curl of docker-compose.yml [superset]
Posted by "sfirke (via GitHub)" <gi...@apache.org>.
sfirke commented on PR #25546:
URL: https://github.com/apache/superset/pull/25546#issuecomment-1752994326
Thinking more about this, and the fact that Superset 4.0 will not support SQLite as a metadata db -- would we need a whole separate Postgres container for the examples, or just a separate db (and maybe user) in the same Postgres container?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] [WIP] feat(docker): enable single curl of docker-compose.yml [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #25546:
URL: https://github.com/apache/superset/pull/25546#issuecomment-2529166626
Is this still needed, or should we close it out?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] [WIP] feat(docker): enable single curl of docker-compose.yml [superset]
Posted by "sfirke (via GitHub)" <gi...@apache.org>.
sfirke commented on PR #25546:
URL: https://github.com/apache/superset/pull/25546#issuecomment-1750826825
@sebastianliebscher Thank you for this! This will be a big win. What do you think is needed to unblock the `examples-init.sh` piece? Is there support you need or options that we should discuss?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] [WIP] feat(docker): enable single curl of docker-compose.yml [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #25546:
URL: https://github.com/apache/superset/pull/25546#issuecomment-2791163139
This has been in draft mode awaiting updates since December. It doesn't seem likely to be mergeable at this point, so we'll close this PR to help keep the backlog manageable. Please feel free to reopen (or ping me to) if you still want to get this through. We appreciate the contribution in either case!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org