You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "pankajduplo (via GitHub)" <gi...@apache.org> on 2026/01/07 06:46:39 UTC
[D] Allow warehouse database values to be passed as a parameter DB_NAME in warehouse-util service [superset]
GitHub user pankajduplo created a discussion: Allow warehouse database values to be passed as a parameter DB_NAME in warehouse-util service
We are currently working on client portal developer pipeline. Whenever a developer triggers a pipeline, a separate environment will be created for him/her. As we will be using a shared cloud SQL, we are creating a separate warehouse databases called "warehouse_<developer_environment_name>".
Similarly client portal and warehouse services also should be able to use unique warehouse databases per environment.
However currently client portal and warehouse services are using hardcoded database name called "warehouse", which is causing below error when we try to create a warehouse from client portal UI:
``` File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 508, in _handle_exception
self._adapt_connection._handle_exception(error)
File "/usr/local/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 792, in _handle_exception
raise translated_error from error
sqlalchemy.exc.ProgrammingError: (sqlalchemy.dialects.postgresql.asyncpg.ProgrammingError) <class 'asyncpg.exceptions.UndefinedTableError'>: relation "warehouse" does not exist
[SQL: SELECT warehouse.id, warehouse.name, warehouse.status, warehouse.instancetype, warehouse.version, warehouse.started_at, warehouse.stopped_at, warehouse.deleted_at, warehouse.deleted_by_fk, warehouse.endpoint, warehouse.resource_metadata, warehouse.last_heartbeat, warehouse.created_by_fk, warehouse.created_on, warehouse.changed_by_fk, warehouse.changed_on
FROM warehouse
WHERE warehouse.name = $1::VARCHAR AND warehouse.deleted_at IS NULL]
[parameters: ('Warehouse-CentaurusA-rn',)]
(Background on this error at: https://sqlalche.me/e/20/f405)```
Can you please fix this issue?
GitHub link: https://github.com/apache/superset/discussions/36941
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Allow warehouse database values to be passed as a parameter DB_NAME in warehouse-util service [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] edited a comment on the discussion: Allow warehouse database values to be passed as a parameter DB_NAME in warehouse-util service
<!-- Greeting -->
Hi @pankajduplo! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the superset team.
<!-- Answer -->
The warehouse-util service isn't part of the Superset codebase, but I can share best practices for making the database name configurable in Python services. Instead of hardcoding the database name, accept it as a parameter—typically via an environment variable, config file, or function argument. For example, you can use an environment variable like DB_NAME and update your connection logic to use its value when building the SQLAlchemy connection string.
Superset itself uses this approach: database connection parameters (including the database name) are set dynamically in the connection URI, which can be configured per environment. This pattern is also reflected in recent changes to Superset's Helm chart, where the database backend and connection URI are now configurable through deployment values, not hardcoded in the codebase. This enables each environment (or developer) to use a unique database, which resolves issues like the one you encountered with missing tables or schemas when the wrong database is selected [example](https://github.com/apache/superset/pull/34327).
To implement this in your service, refactor any code that sets the database name to read from a parameter (such as DB_NAME) and ensure your deployment pipeline sets this value appropriately for each environment. This will let your client portal and warehouse services use unique databases per environment and avoid the current errors.
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fsuperset).*
---
Docs are dead. Just use [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-tagline&utm_term=apache%2Fsuperset).
[](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-feedback&utm_term=apache%2Fsuperset&message_id=5172c5ba-abe9-46e3-b945-a1a217e22697) [](https://github.dosu.com/apache/superset?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-ask-repo&utm_term=apache%2Fsuperset) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-share-team&utm_term=apache%2Fsuperset)
GitHub link: https://github.com/apache/superset/discussions/36941#discussioncomment-15431207
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Allow warehouse database values to be passed as a parameter DB_NAME in warehouse-util service [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] added a comment to the discussion: Allow warehouse database values to be passed as a parameter DB_NAME in warehouse-util service
<!-- Greeting -->
Hi @pankajduplo! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the superset team.
<!-- Answer -->
The warehouse-util service isn't part of the Superset codebase, but I can share best practices for making the database name configurable in Python services. Instead of hardcoding the database name, accept it as a parameter—typically via an environment variable, config file, or function argument. For example, you can use an environment variable like DB_NAME and update your connection logic to use its value when building the SQLAlchemy connection string.
Superset itself uses this approach: database connection parameters (including the database name) are set dynamically in the connection URI, which can be configured per environment. This pattern is also reflected in recent changes to Superset's Helm chart, where the database backend and connection URI are now configurable through deployment values, not hardcoded in the codebase. This enables each environment (or developer) to use a unique database, which resolves issues like the one you encountered with missing tables or schemas when the wrong database is selected [example](https://github.com/apache/superset/pull/34327).
To implement this in your service, refactor any code that sets the database name to read from a parameter (such as DB_NAME) and ensure your deployment pipeline sets this value appropriately for each environment. This will let your client portal and warehouse services use unique databases per environment and avoid the current errors.
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
---
<sup>How did I do? [Good](https://app.dosu.dev/response-feedback/5172c5ba-abe9-46e3-b945-a1a217e22697?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/5172c5ba-abe9-46e3-b945-a1a217e22697?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/5172c5ba-abe9-46e3-b945-a1a217e22697?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/5172c5ba-abe9-46e3-b945-a1a217e22697?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/5172c5ba-abe9-46e3-b945-a1a217e22697?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/5172c5ba-abe9-46e3-b945-a1a217e22697?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/5172c5ba-abe9-46e3-b945-a1a217e22697?feedback_type=other)</sup>
[](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github) [](https://cursor.com/link/prompt?text=The%20warehouse-util%20service%20isn%27t%20part%20of%20the%20Superset%20codebase%2C%20but%20I%20can%20share%20best%20practices%20for%20making%20the%20database%20name%20configurable%20in%20Python%20services.%20Instead%20of%20hardcoding%20the%20database%20name%2C%20accept%20it%20as%20a%20parameter%E2%80%94typically%20via%20an%20environment%20variable%2C%20config%20file%2C%20or%20function%20argument.%20For%20example%2C%20you%20can%20use%20an%20environment%20variable%20like%20DB_NAME%20and%20update%20your%20connection%20logic%20to%20use%20its%20value%20when%20building%20the%20SQLAlchemy%20connection%20string.%0A%0ASuperset%20itself%20uses%20this%20approach%3A%20database%20connection%20parameters%20%28including%20the%20database%20name%29%20are%20set%20dyna
mically%20in%20the%20connection%20URI%2C%20which%20can%20be%20configured%20per%20environment.%20This%20pattern%20is%20also%20reflected%20in%20recent%20changes%20to%20Superset%27s%20Helm%20chart%2C%20where%20the%20database%20backend%20and%20connection%20URI%20are%20now%20configurable%20through%20deployment%20values%2C%20not%20hardcoded%20in%20the%20codebase.%20This%20enables%20each%20environment%20%28or%20developer%29%20to%20use%20a%20unique%20database%2C%20which%20resolves%20issues%20like%20the%20one%20you%20encountered%20with%20missing%20tables%20or%20schemas%20when%20the%20wrong%20database%20is%20selected%20%5Bexample%5D%28https%3A//github.com/apache/superset/pull/34327%29.%0A%0ATo%20implement%20this%20in%20your%20service%2C%20refactor%20any%20code%20that%20sets%20the%20database%20name%20to%20read%20from%20a%20parameter%20%28such%20as%20DB_NAME%29%20and%20ensure%20your%20deployment%20pipeline%20sets%20this%20value%20appropriately%20for%20each%20environment.%20This%20will%20let%20y
our%20client%20portal%20and%20warehouse%20services%20use%20unique%20databases%20per%20environment%20and%20avoid%20the%20current%20errors.) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/discussions/36941)
GitHub link: https://github.com/apache/superset/discussions/36941#discussioncomment-15431207
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Allow warehouse database values to be passed as a parameter DB_NAME in warehouse-util service [superset]
Posted by "pankajduplo (via GitHub)" <gi...@apache.org>.
GitHub user pankajduplo added a comment to the discussion: Allow warehouse database values to be passed as a parameter DB_NAME in warehouse-util service
Not required.
GitHub link: https://github.com/apache/superset/discussions/36941#discussioncomment-15431483
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org