You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Newer1107 (via GitHub)" <gi...@apache.org> on 2026/08/28 14:55:20 UTC
[PR] fix(spark): replace ReDoS-vulnerable regex in _mask_cmd [airflow]
Newer1107 opened a new pull request, #72204:
URL: https://github.com/apache/airflow/pull/72204
## Problem
The `_mask_cmd()` method in `SparkSubmitHook` uses a regex with a nested lookahead `(?:(?!\2\s).)*` combined with `\S*?` that causes **quadratic backtracking** (O(n²)) on inputs without the `secret`/`password` pattern. An authenticated user can trigger a DoS by passing crafted `application_args` via the DAG trigger API.
| Input size | Before | After |
|-----------|--------|-------|
| 10,000 chars | ~2s | <1ms |
| 50,000 chars | ~57s | <1ms |
## Fix
Replace the vulnerable regex with:
- `\b\w*` for the key part (word boundary limits search space)
- `\S+` for the value (no backtracking possible)
This eliminates catastrophic backtracking while preserving the same masking behavior.
## Verification
Tested with:
- `HivePassword=abc123` → `HivePassword=******`
- `secret=mysecret` → `secret=******`
- `password="mypassword"` → `password="******"`
- 50,000 char input: <1ms (was 57s)
Resolves #70676
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(spark): replace ReDoS-vulnerable regex in _mask_cmd [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #72204:
URL: https://github.com/apache/airflow/pull/72204#issuecomment-5752031966
Closing this in favour of #71711, which fixes the same quadratic-backtracking problem in `_mask_cmd`. Thanks for spotting the issue — the diagnosis was right, and the slowdown is real (I measured the current pattern at 0.03s / 0.19s / 0.70s for 2k / 5k / 10k-character tokens).
I ran all three patterns — current `main`, this PR, and #71711 — over the same inputs:
| input | `main` | this PR | #71711 |
|---|---|---|---|
| `--conf spark.hadoop.fs.s3a.secret.key=AKIAxyz` | masked | **leaks** | same as `main` |
| `--secret-key=abc` | masked | **leaks** | same as `main` |
| `--password='my secret pass' --foo bar` | `--password='******' --foo bar` | **partial leak** | same as `main` |
| `password="my pass" next` | `password="******" next` | **partial leak** | same as `main` |
| `--password=` | masked | unmasked | same as `main` |
The root cause is in the review above: `\b\w*` cannot cross `.` or `-`, so dotted and hyphenated keys stop matching, and `(\S+)` drops the old "quoted values may contain whitespace" handling. #71711 preserves `main`'s output byte-for-byte on all of these while getting the same speedup, and it arrived first with tests and a round of review.
If you would like to keep working in this area, the anchored one-liner from my review above is still a valid alternative approach, and #71711 has one open gap of its own (a second sensitive key inside the same whitespace-free token is not masked) that I have asked its author to close — a review there would be welcome.
---
> *This message was drafted by an AI-assisted tool and
> confirmed by an Apache Airflow maintainer. If you think
> something here is mistaken, please reply on the PR and a
> maintainer will weigh in.*
>
> *More on how Apache Airflow handles maintainer review:*
> [contributing-docs/05_pull_requests.rst](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(spark): replace ReDoS-vulnerable regex in _mask_cmd [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #72204:
URL: https://github.com/apache/airflow/pull/72204#issuecomment-5454057550
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our [Contributors' Guide](https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:
- Pay attention to the quality of your code (ruff, mypy and type annotations). Our [prek-hooks]( https://github.com/apache/airflow/blob/main/contributing-docs/08_static_code_checks.rst#prerequisites-for-prek-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/airflow-core/docs/howto/custom-operator.rst) Consider adding an example Dag that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#coding-style-and-best-practices).
- Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(spark): replace ReDoS-vulnerable regex in _mask_cmd [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed pull request #72204: fix(spark): replace ReDoS-vulnerable regex in _mask_cmd
URL: https://github.com/apache/airflow/pull/72204
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org