You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "jesperbagge (via GitHub)" <gi...@apache.org> on 2025/08/05 16:50:08 UTC

[I] /app/.venv in 5.0.0 breaks extending docker image [superset]

jesperbagge opened a new issue, #34557:
URL: https://github.com/apache/superset/issues/34557

   ### Bug description
   
   For docker image tagged `5.0.0` and `5.0.0-dev` I've encountered an issue when extending these to include OAuth and ClickHouse drivers. Well, any python extending actually breaks.
   
   This isn't obvious until a container is started based on the extended image. The extra python modules fails to be imported because they are not found. This is because `pip` and `python` is pointing to different environments in these images. For reference, the image `4.1.3-dev` has both `pip` and `python` pointing to the same environment. The container stops with "ModuleNotFoundError: No module named 'authlib'", in my case. I see that this error has been spotted in earlier issues as well. Perhaps it is this rogue environment that was the culprit back then as well?
   
   `python` is pointing towards a python environment placed in `/app/.venv/` that also seems to be first in `PATH`.
   
   A workaround for me to get past this was to remove the `/app/.venv/` when extending the docker image.
   
   ### Screenshots/recordings
   
   _No response_
   
   ### Superset version
   
   5.0.0
   
   ### Python version
   
   3.10
   
   ### Node version
   
   I don't know
   
   ### Browser
   
   Not applicable
   
   ### Additional context
   
   `ModuleNotFoundError: No module named 'authlib'`
   
   ### Checklist
   
   - [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
   - [ ] I have searched the GitHub issue tracker and didn't find a similar bug report.
   - [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] /app/.venv in 5.0.0 breaks extending docker image [superset]

Posted by "jesperbagge (via GitHub)" <gi...@apache.org>.
jesperbagge closed issue #34557: /app/.venv in 5.0.0 breaks extending docker image
URL: https://github.com/apache/superset/issues/34557


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] /app/.venv in 5.0.0 breaks extending docker image [superset]

Posted by "user1500177 (via GitHub)" <gi...@apache.org>.
user1500177 commented on issue #34557:
URL: https://github.com/apache/superset/issues/34557#issuecomment-3155940953

   I also faced the same error ,what to do , usually I never used to install the authlib via docker file


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] /app/.venv in 5.0.0 breaks extending docker image [superset]

Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #34557:
URL: https://github.com/apache/superset/issues/34557#issuecomment-3155997146

   <!-- Answer -->
   Glad to hear that worked! For context, `uv` is a modern Python package manager that aims to be much faster than `pip` and handles dependency resolution efficiently. In Superset 5.0.0 Docker images, `uv` is used to create and manage the `/app/.venv` virtual environment, and running `uv pip install` ensures your packages end up in the right place for Superset to use them. If you want more details on `uv` or the new Docker setup, let me know!
   
   
   <!-- Dosu Comment Footer -->
   *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
   
   ---
   <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/c8ca2f06-1d6b-44fd-aa05-e0c6e56fcca2?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/c8ca2f06-1d6b-44fd-aa05-e0c6e56fcca2?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/c8ca2f06-1d6b-44fd-aa05-e0c6e56fcca2?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/c8ca2f06-1d6b-44fd-aa05-e0c6e56fcca2?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/c8ca2f06-1d6b-44fd-aa05-e0c6e56fcca2?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/c8ca2f06-1d6b-44fd-aa05-e0c6e56fcca2?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/c8ca2f06-1d6b-44fd-aa05-e0c6e56fcca2?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=githu
 b)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/34557)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] /app/.venv in 5.0.0 breaks extending docker image [superset]

Posted by "jesperbagge (via GitHub)" <gi...@apache.org>.
jesperbagge commented on issue #34557:
URL: https://github.com/apache/superset/issues/34557#issuecomment-3155991080

   Without fully understanding what `uv` brings to the table I did what @dosu suggested and now have a working image. Neat!
   
   ```docker
   FROM apache/superset:5.0.0-dev
   USER root
   COPY requirements.txt .
   RUN uv pip install --no-cache-dir -r requirements.txt
   USER superset
   WORKDIR /app
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] /app/.venv in 5.0.0 breaks extending docker image [superset]

Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #34557:
URL: https://github.com/apache/superset/issues/34557#issuecomment-3155901025

   <!-- Greeting -->
   Hi @jesperbagge! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the superset team.
   
   <!-- Answer -->
   This is a known breaking change in Superset 5.0.0 Docker images. The image now creates a Python virtual environment at `/app/.venv` using `uv venv /app/.venv` and sets the PATH so that `/app/.venv/bin` is first, making this venv the default for `python` and `pip` commands. However, the package manager switched from `pip` to `uv`, and `uv` is installed system-wide, not inside `/app/.venv`. If you install Python packages using the system `pip` or outside `/app/.venv`, they won't be available at runtime, leading to `ModuleNotFoundError` for packages like `authlib` or database drivers ([details](https://github.com/apache/superset/issues/32186), [UPDATING.md](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/UPDATING.md), [Dockerfile](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/Dockerfile)).
   
   To extend the image and install additional Python packages, use either `uv pip install <package>` or `/app/.venv/bin/pip install <package>` in your Dockerfile or bootstrap script. This ensures the packages are installed in the environment Superset actually uses. Removing `/app/.venv` works as a workaround but is not recommended unless you have a specific need to bypass the venv.
   
   There's also a known issue where the `superset` package is installed both in `/app/superset` and `/app/.venv/lib/python3.10/site-packages/superset`, which can cause confusion and bloat, but removing `/app/superset` does not break functionality ([reference](https://github.com/apache/superset/issues/34120)).
   
   For more context and ongoing discussion, see [this issue](https://github.com/apache/superset/issues/32186), [this discussion](https://github.com/apache/superset/discussions/34109), and [this PR updating the docs](https://github.com/apache/superset/pull/32239).
   
   
   <!-- Dosu Comment Footer -->
   *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
   
   ---
   <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/8c203cd3-e812-4fc6-8f4b-8b570dd60f8f?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/8c203cd3-e812-4fc6-8f4b-8b570dd60f8f?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/8c203cd3-e812-4fc6-8f4b-8b570dd60f8f?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/8c203cd3-e812-4fc6-8f4b-8b570dd60f8f?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/8c203cd3-e812-4fc6-8f4b-8b570dd60f8f?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/8c203cd3-e812-4fc6-8f4b-8b570dd60f8f?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/8c203cd3-e812-4fc6-8f4b-8b570dd60f8f?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=githu
 b)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/34557)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org