You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "SEPURI-SAI-KRISHNA (via GitHub)" <gi...@apache.org> on 2026/10/01 10:27:05 UTC

Re: [I] JWT: Generate different tokens when using key file and key inline with same secret key content [pulsar]

SEPURI-SAI-KRISHNA commented on issue #15248:
URL: https://github.com/apache/pulsar/issues/15248#issuecomment-5929502556

   Still reproducible on `master`. @WillsonYip's analysis was correct, and I don't think this should have been closed out as "it works" — the report identifies a real trap, though the fix isn't the obvious one.
   
   ### The actual rule
   
   `AuthTokenUtils.readKeyFromUrl` has four branches, and they split on *how* the key is referenced, not on what it contains:
   
   | `--secret-key` / `tokenSecretKey` value | Key bytes used |
   |---|---|
   | `data:;base64,<b64>` | base64-**decoded** |
   | `<b64>` (bare, not an existing path) | base64-**decoded** |
   | `file:/path/to/key` | **raw file bytes** |
   | `/path/to/key` (existing path) | **raw file bytes** |
   
   So inline forms are decoded and file forms are not. That's self-consistent, and on its own it's defensible.
   
   The problem is that `tokens create-secret-key --base64 --output <file>` writes the *inline* form into a *file*:
   
   https://github.com/apache/pulsar/blob/86ba7ebf6a3be14e6410080ddcd9de18945e380f/pulsar-broker/src/main/java/org/apache/pulsar/utils/auth/tokens/TokensCliUtils.java#L74-L90
   
   Read that file back with `file:`, and the 44 bytes of base64 ASCII become the HMAC key. `Keys.hmacShaKeyFor` accepts it (352 bits, over the HS256 minimum), the header still says `HS256`, and nothing fails — you just have a different key than `data:;base64,<same text>` gives you. Exactly the two tokens in the original report.
   
   This is self-consistent as long as you never mix the two forms, which is why it looks like it works. It breaks the moment someone switches a broker from `tokenSecretKey=file:...` to the inline form, or signs tokens one way and validates them the other.
   
   ### The docs lead you into it
   
   [`security-jwt.md`](https://github.com/apache/pulsar-site/blob/main/docs/security-jwt.md) documents generating the key this way:
   
   ```shell
   bin/pulsar tokens create-secret-key --output my-secret.key --base64
   ```
   
   and then, a few lines later, consuming it this way:
   
   ```shell
   bin/pulsar tokens create --secret-key file:///path/to/my-secret.key --subject test-user
   ```
   
   with `tokenSecretKey=data:;base64,...` shown as the alternative to `tokenSecretKey=file:...`. Following the page as written gives you a `--base64` key file whose real key is the base64 text, plus an inline form documented right next to it that resolves to something else. (The same section also calls it a "base64-encoded **private** key" — it's a symmetric secret key.)
   
   ### What I don't think should change
   
   Making `file:` detect and decode base64 would be the intuitive fix and it would be a bad one: every deployment that followed the docs above would silently change its signing key, invalidating every issued token and rejecting every client. Not worth it for a cosmetic inconsistency.
   
   ### What I'd suggest instead
   
   1. **Docs** (pulsar-site): say that `--base64` output is for the inline `data:;base64,<contents>` form and is not interchangeable with reading the file via `file:`; fix "private key" → "secret key".
   2. **CLI**: have `create-secret-key --base64` print a line to `stderr` saying how to reference the result. It doesn't touch the file, the key, or stdout, so scripts and the existing tests are unaffected — it just puts the warning where the mistake is made.
   
   That covers the reported confusion without changing any key resolution.
   
   A third option is a broker-side warning when a key read from a file is entirely base64 text — a random 32-byte HMAC key has about a 2^-64 chance of looking like that, so the check is reliable. It would be the only thing that reaches deployments already in this state, but it would also log on every startup for everyone who followed the docs and is consistently base64. I'd rather a committer decide whether that trade is wanted than assume it.
   
   Happy to open a PR for 1 and 2, and to add 3 if there's interest. @nodece @lhotari does that sound like the right shape?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org