You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "crepererum (via GitHub)" <gi...@apache.org> on 2024/11/20 10:25:30 UTC
[I] Object store AWS/S3 client and gRPC misconfiguration [arrow-rs]
crepererum opened a new issue, #6760:
URL: https://github.com/apache/arrow-rs/issues/6760
**Describe the bug**
:warning: I understand that this is a very niche issue, but I thought I share this trap with others.
If you accidentally point an S3 `object_store` client to an gRPC endpoint, it will happily read empty objects for most paths (i.e. all paths that are not covered by the gRPC endpoint). This can become quite a debug nightmare.
**To Reproduce**
Set up a gRPC server, e.g. w/ `tonic`. The code example for the client sets this up to port `1234`.
Then configure an S3 Client to point at it. It's important that you
- [force the client to use HTTP/2](https://docs.rs/object_store/latest/object_store/struct.ClientOptions.html#method.with_http2_only), otherwise it cannot connect to gRPC over HTTP/2 in the first place
- if your server doesn't run over TLS, you need to [allow HTTP](https://docs.rs/object_store/latest/object_store/struct.ClientOptions.html#method.with_allow_http).
- you may also want to [skip signature checks](https://docs.rs/object_store/latest/object_store/aws/struct.AmazonS3Builder.html#method.with_skip_signature)
```rust
let store = object_store::aws::AmazonS3Builder::new()
.with_bucket_name("dummy")
.with_client_options(
object_store::ClientOptions::new()
.with_allow_http(true)
.with_http2_only(),
)
.with_endpoint("http://localhost:1234")
.with_skip_signature(true)
.build()
.unwrap();
```
**Expected behavior**
I was naively expecting the client to error.
**Additional context**
gRPC for some bizarre reasons decides to not use the HTTP status code at all but instead a custom response header `grpc-status`. In our case, this is set to `12` for `UNIMPLEMENTED`, see
https://grpc.github.io/grpc/core/md_doc_statuscodes.html .
The response body for `UNIMPLEMENTED` is empty. The `content-length` response header is set to `0` (that's required by the `object_store` client).
:arrow_right: So I think what we could do as some kind of safeguard would be to check the `grpc-status` response header and bail out if it is set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Object store AWS/S3 client and gRPC misconfiguration [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #6760:
URL: https://github.com/apache/arrow-rs/issues/6760#issuecomment-2488554317
IMO this isn't something we should look to handle, if you point it at an HTTP server that returns 200 OK, how is it to know that this is wrong? Sure we could handle grpc-status, but it seems rather odd to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Object store AWS/S3 client and gRPC misconfiguration [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #6760:
URL: https://github.com/apache/arrow-rs/issues/6760#issuecomment-2741680608
Migrating from arrow-rs issue [#6760](https://github.com/apache/arrow-rs/issues/6760)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Object store AWS/S3 client and gRPC misconfiguration [arrow-rs]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #6760:
URL: https://github.com/apache/arrow-rs/issues/6760#issuecomment-2488196456
Interesting, so the gRPC server will return an empty body with a 200 status code, which coincidentally is a valid S3 response?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Object store AWS/S3 client and gRPC misconfiguration [arrow-rs]
Posted by "crepererum (via GitHub)" <gi...@apache.org>.
crepererum commented on issue #6760:
URL: https://github.com/apache/arrow-rs/issues/6760#issuecomment-2488251610
S3 responses for `GET` operations are just the pure object, there's no wrapping of the bytes into any object like JSON or XML. All metadata is transmitted via headers, which our client doesn't need (also to support alternative S3 implementations more easily).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org