You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "iercan (via GitHub)" <gi...@apache.org> on 2026/06/12 11:19:31 UTC

[PR] feat(docker): add environment values to set log file for worker and beat [superset]

iercan opened a new pull request, #40998:
URL: https://github.com/apache/superset/pull/40998

   <!---
   Please write the PR title following the conventions at https://www.conventionalcommits.org/en/v1.0.0/
   Example:
   fix(dashboard): load charts correctly
   -->
   
   ### SUMMARY
   This PR introduces two new environment variables, `WORKER_LOG_FILE` and `BEAT_LOG_FILE`, to configure log file destinations for the worker and beat in the default Docker environment. 
   
   Default values remain unchanged to ensure backward compatibility.
   
   ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
   <!--- Skip this if not applicable -->
   
   ### TESTING INSTRUCTIONS
   <!--- Required! What steps can be taken to manually verify the changes? -->
   
   ### ADDITIONAL INFORMATION
   <!--- Check any relevant boxes with "x" -->
   <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
   - [ ] Has associated issue:
   - [ ] Required feature flags:
   - [ ] Changes UI
   - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
     - [ ] Migration is atomic, supports rollback & is backwards-compatible
     - [ ] Confirm DB migration upgrade and downgrade tested
     - [ ] Runtime estimates and downtime expectations provided
   - [ ] Introduces new feature or API
   - [ ] Removes existing feature or API
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] feat(docker): add environment values to set log file for worker and beat [superset]

Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #40998:
URL: https://github.com/apache/superset/pull/40998#issuecomment-4692317510

   AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
   
   <details>
   <summary>
   Bito Usage Guide
   </summary>
   
   **Commands**
         
   Type the following command in the pull request comment and save the comment.
         
   - `/review` - Manually triggers a full AI review.
   
   - `/pause` - Pauses automatic reviews on this pull request.
   - `/resume` - Resumes automatic reviews.
   - `/resolve` - Marks all Bito-posted review comments as resolved.
   - `/abort` - Cancels all in-progress reviews.
   
         
   Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
         
   **Configuration**
   
   This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
         
   **Documentation & Help**
   - <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
   - <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
   - <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
   - <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
   
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] feat(docker): add environment values to set log file for worker and beat [superset]

Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #40998:
URL: https://github.com/apache/superset/pull/40998#issuecomment-4717452032

   ### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
   
   
   |  Name | Link |
   |:-:|------------------------|
   |<span aria-hidden="true">🔨</span> Latest commit | 73e32b6340ede277d68371d13656ef40f661c949 |
   |<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6a3121477f410700072d4333 |
   |<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-40998--superset-docs-preview.netlify.app](https://deploy-preview-40998--superset-docs-preview.netlify.app) |
   |<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br />![QR Code](https://app.netlify.com/qr-code/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1cmwiOiJodHRwczovL2RlcGxveS1wcmV2aWV3LTQwOTk4LS1zdXBlcnNldC1kb2NzLXByZXZpZXcubmV0bGlmeS5hcHAifQ.b2Xr5f3epGhmFaaLGD2eJDNVA3w06LroNl0nCiZaZHY)<br /><br />_Use your smartphone camera to open QR code link._</details> |
   |<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#DcpLCoAgEADQu8w6wR8J3aBjmL-EyZHBFhHdPXdv8V7oTFcfsAEscLBv4ZwOCRM_AqmIXDGJQC3XcrMfldqMMXWkZ4-zrt4oraxz2SrppJROR2uMge8H) |
   ---
   <!-- [superset-docs-preview Preview](https://deploy-preview-40998--superset-docs-preview.netlify.app) -->
   _To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] feat(docker): add environment values to set log file for worker and beat [superset]

Posted by "iercan (via GitHub)" <gi...@apache.org>.
iercan commented on PR #40998:
URL: https://github.com/apache/superset/pull/40998#issuecomment-4717634229

   Thanks @rusackas for the review. I tested your suggestion for setting and unsetting the destination, and it worked as expected. Updated the code accordingly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] feat(docker): add environment values to set log file for worker and beat [superset]

Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #40998:
URL: https://github.com/apache/superset/pull/40998#discussion_r3402963333


##########
docker/docker-bootstrap.sh:
##########
@@ -67,16 +67,18 @@ else
   echo "Skipping local overrides"
 fi
 
+HYPHEN_SYMBOL='-'
+
 case "${1}" in
   worker)
     echo "Starting Celery worker..."
     # setting up only 2 workers by default to contain memory usage in dev environments
-    celery --app=superset.tasks.celery_app:app worker -O fair -l INFO --concurrency=${CELERYD_CONCURRENCY:-2}
+    celery --app=superset.tasks.celery_app:app worker -O fair -l INFO --concurrency=${CELERYD_CONCURRENCY:-2} --logfile=${WORKER_LOG_FILE:-$HYPHEN_SYMBOL}

Review Comment:
   **Suggestion:** The `WORKER_LOG_FILE` expansion is unquoted, so values containing spaces or glob characters will be split/expanded by the shell and can be interpreted as extra CLI arguments instead of a single logfile path. This can break worker startup or allow option injection through environment configuration; quote the logfile value expansion so it is always passed as one argument. [security]
   
   <details>
   <summary><b>Severity Level:</b> Major ⚠️</summary>
   
   ```mdx
   - ❌ Superset worker can misparse logfile, failing to start.
   - ⚠️ Env-controlled logfile allows unintended Celery flags injection.
   - ⚠️ Docker dev worker startup fragile to spaced logfile paths.
   ```
   </details>
   <details>
   <summary><b>Steps of Reproduction ✅ </b></summary>
   
   ```mdx
   1. Open `docker-compose.yml` and note the `superset-worker` service at
   `docker-compose.yml:6-17` uses `command: ["/app/docker/docker-bootstrap.sh", "worker"]`
   and `env_file: docker/.env`.
   
   2. In `docker/.env` (loaded by that service), set `WORKER_LOG_FILE="/tmp/worker log
   --concurrency=10"` so the value contains both a space and an extra Celery option.
   
   3. Run `docker compose up superset-worker`, which starts `/app/docker/docker-bootstrap.sh
   worker` as defined in `docker-compose.yml:6-17`, entering the `worker)` case in
   `docker/docker-bootstrap.sh:72-77`.
   
   4. At `docker/docker-bootstrap.sh:76`, the shell expands
   `--logfile=${WORKER_LOG_FILE:-$HYPHEN_SYMBOL}` without quotes, producing arguments
   `--logfile=/tmp/worker` and `log` and `--concurrency=10`; Celery receives an unintended
   extra `--concurrency=10` flag and an extra positional `log`, altering worker behavior or
   causing startup failure.
   ```
   </details>
   
   [Fix in Cursor](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=b3d980e19aa1420399a5ab1ebc957eba&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) | [Fix in VSCode Claude](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=b3d980e19aa1420399a5ab1ebc957eba&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
   
   *(Use Cmd/Ctrl + Click for best experience)*
   <details>
   <summary><b>Prompt for AI Agent 🤖 </b></summary>
   
   ```mdx
   This is a comment left during a code review.
   
   **Path:** docker/docker-bootstrap.sh
   **Line:** 76:76
   **Comment:**
   	*Security: The `WORKER_LOG_FILE` expansion is unquoted, so values containing spaces or glob characters will be split/expanded by the shell and can be interpreted as extra CLI arguments instead of a single logfile path. This can break worker startup or allow option injection through environment configuration; quote the logfile value expansion so it is always passed as one argument.
   
   Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
   Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
   ```
   </details>
   <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40998&comment_hash=8d98353f042d304c5dca8e05f77dc426181a9ab598482735e8771a5c6a9f960f&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40998&comment_hash=8d98353f042d304c5dca8e05f77dc426181a9ab598482735e8771a5c6a9f960f&reaction=dislike'>👎</a>



##########
docker/docker-bootstrap.sh:
##########
@@ -67,16 +67,18 @@ else
   echo "Skipping local overrides"
 fi
 
+HYPHEN_SYMBOL='-'
+
 case "${1}" in
   worker)
     echo "Starting Celery worker..."
     # setting up only 2 workers by default to contain memory usage in dev environments
-    celery --app=superset.tasks.celery_app:app worker -O fair -l INFO --concurrency=${CELERYD_CONCURRENCY:-2}
+    celery --app=superset.tasks.celery_app:app worker -O fair -l INFO --concurrency=${CELERYD_CONCURRENCY:-2} --logfile=${WORKER_LOG_FILE:-$HYPHEN_SYMBOL}
     ;;
   beat)
     echo "Starting Celery beat..."
     rm -f /tmp/celerybeat.pid
-    celery --app=superset.tasks.celery_app:app beat --pidfile /tmp/celerybeat.pid -l INFO -s "${SUPERSET_HOME}"/celerybeat-schedule
+    celery --app=superset.tasks.celery_app:app beat --pidfile /tmp/celerybeat.pid -l INFO -s "${SUPERSET_HOME}"/celerybeat-schedule --logfile=${BEAT_LOG_FILE:-$HYPHEN_SYMBOL}

Review Comment:
   **Suggestion:** The `BEAT_LOG_FILE` expansion is also unquoted, so a path with whitespace or wildcard characters will be word-split/glob-expanded and passed incorrectly to Celery beat. Quote the logfile value expansion to preserve it as a single safe argument. [security]
   
   <details>
   <summary><b>Severity Level:</b> Major ⚠️</summary>
   
   ```mdx
   - ❌ Superset beat scheduler may fail to start correctly.
   - ⚠️ Env logfile value can inject unintended beat options.
   - ⚠️ Scheduled tasks reliability depends on whitespace-free logfile.
   ```
   </details>
   <details>
   <summary><b>Steps of Reproduction ✅ </b></summary>
   
   ```mdx
   1. Open `docker-compose.yml` and note the `superset-worker-beat` service at
   `docker-compose.yml:31-40` uses `command: ["/app/docker/docker-bootstrap.sh", "beat"]` and
   `env_file: docker/.env`.
   
   2. In `docker/.env` (used by that service), set `BEAT_LOG_FILE="/tmp/beat log
   --max-interval=120"` so the value includes spaces and an extra Celery beat option.
   
   3. Run `docker compose up superset-worker-beat`, which starts
   `/app/docker/docker-bootstrap.sh beat` as configured, entering the `beat)` branch in
   `docker/docker-bootstrap.sh:78-82`.
   
   4. At `docker/docker-bootstrap.sh:81`, the unquoted expansion of
   `--logfile=${BEAT_LOG_FILE:-$HYPHEN_SYMBOL}` splits into `--logfile=/tmp/beat`, `log`, and
   `--max-interval=120`; Celery beat receives a malformed extra positional `log` and an
   unintended `--max-interval` override, which can cause beat startup failure or incorrect
   scheduling behavior.
   ```
   </details>
   
   [Fix in Cursor](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=921403b357784fd9ad2e0f052110b7c4&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) | [Fix in VSCode Claude](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=921403b357784fd9ad2e0f052110b7c4&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
   
   *(Use Cmd/Ctrl + Click for best experience)*
   <details>
   <summary><b>Prompt for AI Agent 🤖 </b></summary>
   
   ```mdx
   This is a comment left during a code review.
   
   **Path:** docker/docker-bootstrap.sh
   **Line:** 81:81
   **Comment:**
   	*Security: The `BEAT_LOG_FILE` expansion is also unquoted, so a path with whitespace or wildcard characters will be word-split/glob-expanded and passed incorrectly to Celery beat. Quote the logfile value expansion to preserve it as a single safe argument.
   
   Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
   Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
   ```
   </details>
   <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40998&comment_hash=3494fd0f80334885418438c7b6530069d77e62c59e57f6595e0e31ba00a1e338&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40998&comment_hash=3494fd0f80334885418438c7b6530069d77e62c59e57f6595e0e31ba00a1e338&reaction=dislike'>👎</a>



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] feat(docker): add environment values to set log file for worker and beat [superset]

Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #40998:
URL: https://github.com/apache/superset/pull/40998#issuecomment-4690727141

   AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] feat(docker): add environment values to set log file for worker and beat [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas merged PR #40998:
URL: https://github.com/apache/superset/pull/40998


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [PR] feat(docker): add environment values to set log file for worker and beat [superset]

Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #40998:
URL: https://github.com/apache/superset/pull/40998#issuecomment-4719221471

   AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
   
   <details>
   <summary>
   Bito Usage Guide
   </summary>
   
   **Commands**
         
   Type the following command in the pull request comment and save the comment.
         
   - `/review` - Manually triggers a full AI review.
   
   - `/pause` - Pauses automatic reviews on this pull request.
   - `/resume` - Resumes automatic reviews.
   - `/resolve` - Marks all Bito-posted review comments as resolved.
   - `/abort` - Cancels all in-progress reviews.
   
         
   Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
         
   **Configuration**
   
   This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
         
   **Documentation & Help**
   - <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
   - <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
   - <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
   - <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
   
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org