You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "carloea2 (via GitHub)" <gi...@apache.org> on 2025/11/16 21:25:43 UTC

[PR] Single max size bk enf [texera]

carloea2 opened a new pull request, #4059:
URL: https://github.com/apache/texera/pull/4059

   ### What changes were proposed in this PR?
   
   - Add `singleFileUploadMaxSizeMib` and `maxSingleFileUploadBytes` in `DatasetResource` using the `single_file_upload_max_size_mib` site setting (with `DefaultsConfig` fallback).
   - In `/{did}/upload`, track `totalBytesRead` while streaming the file and stop the upload if it exceeds the max size, returning `413 REQUEST_ENTITY_TOO_LARGE`.
   - In `/multipart-upload?type=finish`, check the final `sizeBytes` from LakeFS; if it exceeds the max, reset the object in LakeFS and return `413 REQUEST_ENTITY_TOO_LARGE`.
   - This makes the backend enforce the same single-file size limit as the UI, so changing `main.js` in the browser can no longer bypass the limit.
   
   ### Any related issues, documentation, discussions?
   
   - Closes #4058 
   
   ### How was this PR tested?
   
   - Set `single_file_upload_max_size_mib` to a known value.
   - Upload a file smaller than the limit via the dataset UI → upload succeeds.
   - Modify `main.js` in the browser to relax the frontend check and try to upload a file larger than the limit → request now fails with a 413 error and the “File exceeds maximum allowed size … MiB” message.
   <img width="1280" height="697" alt="image" src="https://github.com/user-attachments/assets/6fa6b469-d73d-4683-bac5-59203af74be0" />
   
   ### Was this PR authored or co-authored using generative AI tooling?
   ChatGPT
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(dataset): enforce backend max single file size limit [texera]

Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
carloea2 commented on PR #4059:
URL: https://github.com/apache/texera/pull/4059#issuecomment-3619544163

   It was closed to simplify the review process and split the work into two steps:
   
   1) Handle this issue first: https://github.com/apache/texera/issues/4110
   
   2) Once that is completed, I’ll create a new issue to enforce the actual max file size in the new architecture.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(dataset): enforce backend max single file size limit [texera]

Posted by "chenlica (via GitHub)" <gi...@apache.org>.
chenlica commented on PR #4059:
URL: https://github.com/apache/texera/pull/4059#issuecomment-3619467692

   @carloea2 Please explain the reason of closing this PR and our plan next.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(dataset): enforce backend max single file size limit [texera]

Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
carloea2 commented on PR #4059:
URL: https://github.com/apache/texera/pull/4059#issuecomment-3615544880

   @aicam Hello good evening, the PR is ready for review, please let me know your comments. Thank you.
   
   @chenlica 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(dataset): enforce backend max single file size limit [texera]

Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
carloea2 closed pull request #4059: fix(dataset): enforce backend max single file size limit
URL: https://github.com/apache/texera/pull/4059


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org