You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "aicam (via GitHub)" <gi...@apache.org> on 2026/07/20 18:51:48 UTC

[I] JWT-authenticated S3 proxy for dataset mounts [texera]

aicam opened a new issue, #6613:
URL: https://github.com/apache/texera/issues/6613

   ### Task Summary
   
   Add a **JWT-authenticated S3 proxy** so computing-unit pods can mount a dataset with GeeseFS **without** holding global LakeFS credentials (pods are user-accessible via UDF code — see the authorization section of #6606). Part of #6606.
   
   The proxy lives in **file-service** (reuses its JWT auth, `userHasReadAccess` check, LakeFS config, and deployment). It is a **transparent, scope-enforcing, re-signing reverse proxy** in front of the LakeFS S3 gateway:
   
   **Control plane — `POST /api/dataset/mount-session`** (`@Auth` SessionUser, `@RolesAllowed(REGULAR, ADMIN)`):
   - Input: `{repositoryName, commitHash}`.
   - Authorizes the caller for that dataset via `datasetDao.fetchByRepositoryName` + `userHasReadAccess(ctx, did, uid)` (the same check the presign endpoints use).
   - Mints a short-lived **session credential** (random access key + secret) scoped to exactly that one `repo:commit`, stored in an in-memory TTL map (file-service runs a single replica).
   - Returns `{accessKey, secretKey, endpoint, expiresInSeconds}`.
   
   **Data plane — raw S3 servlet at `/*`** (Jersey keeps `/api/*`; GeeseFS issues path-style `/{bucket}/{key}` at the root):
   - Extracts the session from the incoming SigV4 `Authorization` Credential (the session key is the bearer capability; the incoming signature is not re-validated).
   - **Enforces scope**: request bucket must equal the session's repository, and object keys / list prefixes must be within the session's commit prefix — so a session can never read another repo or commit.
   - **Re-signs** the request with the global LakeFS credentials (held only server-side) using AWS SDK v2's SigV4 signer and forwards it to the LakeFS S3 gateway, streaming the response back. Read-only (GET/HEAD) only.
   
   Forwarding requests verbatim means the proxy behaves identically to today's working direct GeeseFS→LakeFS mount, just re-authenticated — low protocol risk.
   
   **Non-goals (separate step):** rewiring `DatasetMountManager` in the engine to obtain a session via the pod's JWT and point GeeseFS at the proxy (that PR also removes the last use of global LakeFS creds in the pod).
   
   **Verification:**
   - Unit: scope enforcement rejects cross-repo / cross-commit paths; `mount-session` returns 403 for a user without read access.
   - End-to-end: with the engine rewired, GeeseFS mounts a dataset through the proxy and a Python UDF loads a model from the mount, with no LakeFS credentials present in the CU pod.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] JWT-authenticated S3 proxy for dataset mounts [texera]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6613:
URL: https://github.com/apache/texera/issues/6613#issuecomment-5025978766

   <!-- texera:template-compliance -->
   👋 Thanks for opening this issue, @aicam!
   
   It looks like the issue description doesn't quite follow our template yet:
   
   - This issue doesn't appear to use one of our templates (Bug, Feature, or Task). Please open it using a template so the required details are captured.
   
   Filling out the template helps reviewers understand and triage your contribution faster. Please edit the description to complete it. This message will disappear automatically once the template is followed.
   
   You can find the template prompts by editing the description, or see [CONTRIBUTING.md](https://github.com/apache/texera/blob/main/CONTRIBUTING.md) for the full contribution flow.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] JWT-authenticated S3 proxy for LakeFS repository mounts [texera]

Posted by "aicam (via GitHub)" <gi...@apache.org>.
aicam commented on issue #6613:
URL: https://github.com/apache/texera/issues/6613#issuecomment-5038232100

   Consolidated into the parent #6606 — the whole GeeseFS repository-mount feature is delivered in a single PR (#6608) rather than a stack of sub-issues, so closing this as superseded. The work described here is included in #6608.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] JWT-authenticated S3 proxy for LakeFS repository mounts [texera]

Posted by "aicam (via GitHub)" <gi...@apache.org>.
aicam closed issue #6613: JWT-authenticated S3 proxy for LakeFS repository mounts
URL: https://github.com/apache/texera/issues/6613


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org