You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "bobbai00 (via GitHub)" <gi...@apache.org> on 2026/05/10 18:41:38 UTC
[I] Move dataset file path resolution out of CU Master/Worker [texera]
bobbai00 opened a new issue, #5013:
URL: https://github.com/apache/texera/issues/5013
### Task Summary
Sub-issue of #5011.
Move the `FileResolver.datasetResolveFunc` direct DB call
(`common/workflow-core/.../FileResolver.scala`) behind an HTTP service
that owns the credentials. The executor forwards the originating user's
JWT.
`datasetResolveFunc` joins `USER × DATASET × DATASET_VERSION` to translate
`/owner/dataset/version/file` into a `dataset:///<repo>/<hash>/<file>`
URI. It is the only `SqlServer` call site reachable from CU Master /
Worker that is not about execution metadata. It is invoked from
`LogicalPlan.resolveScanSourceOpFileName`, which runs during workflow
compile on every execution.
The natural owner for this lookup is `file-service` (it already owns
the dataset model). Done when no code reachable from CU Master / Worker
calls `SqlServer` for dataset path resolution, the new endpoint is
`@Auth`-checked, and the existing `FileResolverSpec` plus an end-to-end
workflow run that scans a dataset file still pass.
### Task Type
- [x] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [ ] Performance
- [ ] Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org