You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@texera.apache.org by "madisonmlin (via GitHub)" <gi...@apache.org> on 2025/10/09 19:54:04 UTC
[I] Switch single-file dataset downloads to use browser native downloads [texera]
madisonmlin opened a new issue, #3842:
URL: https://github.com/apache/texera/issues/3842
**Issue**
This issue re-addresses part of a larger issue, https://github.com/apache/texera/issues/3404, which describes how the download of workflow results and datasets (both single-file and multi-file downloads) does not trigger any progress indicator by the browser. This issue specifically addresses the lack of progress indicator for single-file dataset downloads.
Absence of a progress indicator means that, for large files, users see no feedback about the download progress until the file suddenly finishes downloading. This creates poor user experience because it causes uncertainty about the download status.
When the browser handles a file download, its download UI is displayed, including a progress bar to indicate download progress. Currently, the frontend receives a pre-signed URL from the backend to download the dataset file. Then, it makes a request to the pre-signed URL for the file to be downloaded to JavaScript memory as a Blob, and writes the Blob into the local filesystem. Because the browser does not handle the download, the download UI with progress bar is not shown.
**Fix**
To switch downloads for single-file datasets specifically to use browser downloads, we can directly click on the pre-signed URL in the frontend rather than making a request to it, allowing the browser to handle the download.
The browser will perform the download solely based on what is given to it in the pre-signed URL. Currently, the pre-signed URL is generated using the LakeFS API endpoint, which does not allow for customization of headers used in the download. Without being able to set headers like `Content-Disposition` to specify filename and filetype, the downloaded file will default to a general filetype and a long, uninterpretable filename. To address this issue, we can instead use the S3 Gateway API to generate the pre-signed URL, which allows for customization of headers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Switch single-file dataset downloads to use browser native downloads [texera]
Posted by "suyashj1231 (via GitHub)" <gi...@apache.org>.
suyashj1231 commented on issue #3842:
URL: https://github.com/apache/texera/issues/3842#issuecomment-4772478564
@aicam I took this issue and spent some time on the `AccessDenied` bug from the 10/23 notes. I managed to reproduce it on a `single-node` deployment, and from what I can tell it's a SigV4 host mismatch, not anything about the file itself. The whiteboard plan (presign-url carrying a filename header, fetched by the browser's download manager) is the right idea, but I hit two things that change where the presign has to happen.
One, the LakeFS S3 gateway at `lakefs:8000` ignores `response-content-disposition`. Presigning a GET through the gateway gives me a 200 but no `Content-Disposition` header, so the filename never gets set. Presigning straight against MinIO does set it: `Content-Disposition: attachment; filename="Iris.csv"`, correct name. So the filename header on the board only sticks if we presign against MinIO, not the gateway.
Two, the `AccessDenied` is a signed-host mismatch. SigV4 signs the `Host` header, so the URL has to be signed with the same endpoint the browser actually hits. Signing internal and fetching external fails:
```
A) signed host texera-minio:9000, fetched at localhost:9000 -> 403 SignatureDoesNotMatch
B) signed host localhost:9000, fetched at localhost:9000 -> 200 OK, Content-Disposition set
```
That fits why it breaks on one deployment and not another. When the backend's S3 endpoint and the browser-facing endpoint are the same host it works, when they differ you get the rejection. Region (`us-west-2` vs `us-east-1`) made no difference in my tests.
So the version that actually holds up: presign directly against MinIO with `response-content-disposition`, signed with the external pre-signed endpoint. Right now `file-service` only knows the internal `STORAGE_S3_ENDPOINT`, so it'd need the external one too, same idea as LakeFS's `BLOCKSTORE_S3_PRE_SIGNED_ENDPOINT`. I can take a shot at that if it sounds right to you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Switch single-file dataset downloads to use browser native downloads [texera]
Posted by "aicam (via GitHub)" <gi...@apache.org>.
aicam commented on issue #3842:
URL: https://github.com/apache/texera/issues/3842#issuecomment-6449881727
We recently found out LakeFS is providing parameter to set file name in presign urls: https://github.com/treeverse/lakeFS/pull/8897
we will use it to improve download experience
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Switch single-file dataset downloads to use browser native downloads [texera]
Posted by "suyashj1231 (via GitHub)" <gi...@apache.org>.
suyashj1231 commented on issue #3842:
URL: https://github.com/apache/texera/issues/3842#issuecomment-4772532414
@aicam I picked up this issue and spent some time trying to reproduce the `AccessDenied` from the 10/23 notes on a `single-node` deployment. I haven't hit your exact error yet, but I found two things that matter for the new approach, plus a question that might help pin down your case.
Things I could reproduce:
1. The LakeFS S3 gateway at `lakefs:8000` ignores `response-content-disposition`. Presigning a GET through the gateway returns 200 but no `Content-Disposition`, so the filename never gets set. Presigning directly against MinIO does set it (`attachment; filename="Iris.csv"`). So for the "presign-url with filename header" plan on the board, the presign has to go against MinIO, not the gateway.
2. The URL has to be signed against the same endpoint the browser hits. SigV4 signs the `Host` header, so:
```
signed texera-minio:9000, fetched localhost:9000 -> 403 SignatureDoesNotMatch
signed localhost:9000, fetched localhost:9000 -> 200, Content-Disposition set
```
On your actual error, though, I don't think I've matched it yet. The two failures I can produce are host mismatch (`SignatureDoesNotMatch`) and an expired URL (`AccessDenied` / "Request has expired"). Yours is `AccessDenied` with a plain "Access Denied." and no Key/BucketName in the body, which fits neither. Do you still have the failing request, specifically the URL the browser actually hit (host/port) and roughly how long after the presign it failed? That'd help me tell whether it's an endpoint mismatch, an expiry/clock thing, or a bucket-policy denial.
For the fix itself either way: presign directly against MinIO with `response-content-disposition`, signed with the external pre-signed endpoint. `file-service` only knows the internal `STORAGE_S3_ENDPOINT` today, so it'd need the external one too (same idea as LakeFS's `BLOCKSTORE_S3_PRE_SIGNED_ENDPOINT`). Happy to take a shot at that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Switch single-file dataset downloads to use browser native downloads [texera]
Posted by "madisonmlin (via GitHub)" <gi...@apache.org>.
madisonmlin commented on issue #3842:
URL: https://github.com/apache/texera/issues/3842#issuecomment-3441380399
10/23/2025 meeting summary:
Currently, the dataset download implementation (using S3 pre-signed URL) contains a bug causing it to sometimes fail (success/fail appears inconsistent). I discussed the next steps with @chenlica and @aicam and determined that I need to:
- Reproduce the bug by making it happen deterministically
- Log each step from the frontend to the interactions with MinIO
- Debug the issue based on the logs
Although I currently cannot reproduce the bug on my local deployment, @aicam 's deployment contains the bug with the following error occurring:
```
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied.</Message>
<Resource/>
<Region>us-east-1</Region>
<RequestId>d47a72a3-23cc-4bbc-9c67-e4724c87e651</RequestId>
<HostId>046CF9517E3A3625</HostId>
</Error>
```
Diagram of current implementation (by @aicam):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Switch single-file dataset downloads to use browser native downloads [texera]
Posted by "aicam (via GitHub)" <gi...@apache.org>.
aicam commented on issue #3842:
URL: https://github.com/apache/texera/issues/3842#issuecomment-3403146896
Please explain what does it mean by `we can directly click on the pre-signed URL in the frontend`, I believe you mean write a JS code that create a fake div and click on it on behalf of user.
Please write sample filename if we do not set headers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Switch single-file dataset downloads to use browser native downloads [texera]
Posted by "suyashj1231 (via GitHub)" <gi...@apache.org>.
suyashj1231 commented on issue #3842:
URL: https://github.com/apache/texera/issues/3842#issuecomment-4763978864
/take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Switch single-file dataset downloads to use browser native downloads [texera]
Posted by "aicam (via GitHub)" <gi...@apache.org>.
aicam commented on issue #3842:
URL: https://github.com/apache/texera/issues/3842#issuecomment-4773370632
> [@aicam](https://github.com/aicam) I picked up this issue and spent some time trying to reproduce the `AccessDenied` from the 10/23 notes on a `single-node` deployment. I haven't hit your exact error yet, but I found two things that matter for the new approach, plus a question that might help pin down your case.
>
> Things I could reproduce:
>
> 1. The LakeFS S3 gateway at `lakefs:8000` ignores `response-content-disposition`. Presigning a GET through the gateway returns 200 but no `Content-Disposition`, so the filename never gets set. Presigning directly against MinIO does set it (`attachment; filename="Iris.csv"`). So for the "presign-url with filename header" plan on the board, the presign has to go against MinIO, not the gateway.
> 2. The URL has to be signed against the same endpoint the browser hits. SigV4 signs the `Host` header, so:
>
> ```
> signed texera-minio:9000, fetched localhost:9000 -> 403 SignatureDoesNotMatch
> signed localhost:9000, fetched localhost:9000 -> 200, Content-Disposition set
> ```
>
> On your actual error, though, I don't think I've matched it yet. The two failures I can produce are host mismatch (`SignatureDoesNotMatch`) and an expired URL (`AccessDenied` / "Request has expired"). Yours is `AccessDenied` with a plain "Access Denied." and no Key/BucketName in the body, which fits neither. Do you still have the failing request, specifically the URL the browser actually hit (host/port) and roughly how long after the presign it failed? That'd help me tell whether it's an endpoint mismatch, an expiry/clock thing, or a bucket-policy denial.
>
> For the fix itself either way: presign directly against MinIO with `response-content-disposition`, signed with the external pre-signed endpoint. `file-service` only knows the internal `STORAGE_S3_ENDPOINT` today, so it'd need the external one too (same idea as LakeFS's `BLOCKSTORE_S3_PRE_SIGNED_ENDPOINT`). Happy to take a shot at that.
Hi @suyashj1231 and thanks for your help. The issue happen in different layers, first of all, I would highly recommend to use K8s to test because it is the standard production deployment. Also, the problem gets more complicated when a real domain is connected. As you pointed out, LakeFS is not allowing to specify file name and we need to directly call MinIO. However, when we try to get presign URL directly from MinIO, we do not get filename set, the only way is for us to manually add the name and eventually MinIO refuse it because we modified the headers and checksum has changed. You can check [this issue](https://github.com/treeverse/lakeFS/issues/9673)0 we opened on LakeFS Github. I am not sure if we can fix it without any change from LakeFS or MinIO team but you are more than welcome to check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org