You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by "Luka-J9 (via GitHub)" <gi...@apache.org> on 2025/10/16 13:51:24 UTC
[I] MultipartCopy Fails with the presence of Storage Class Header [pekko-connectors]
Luka-J9 opened a new issue, #1245:
URL: https://github.com/apache/pekko-connectors/issues/1245
An extension of:
https://github.com/apache/pekko-connectors/pull/81
When `x-amz-storage-class` is manually added to the headers of a `multipartCopy` request the resulting call fails with a
```
S3Exception: The specified header is not valid in this context
```
I believe the fix is the same as what was in the original PR, however it was not done for the CopyPart
Changing the code to follow the pattern of the linked PR resolves the issue - however it's worth noting that this does come at the expense of other legitimate fields like `x-amz-request-payer`
[See AWS docs for valid headers](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPartCopy.html#API_UploadPartCopy_RequestSyntax)
It seems that we would need to filter out the headers that are not in this list, rather than only include the SSE headers.
Happy to contribute - although I'm not exactly sure what would be the pattern to employ to only allow for a subset of headers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] MultipartCopy Fails with the presence of Storage Class Header [pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
mdedetrich commented on issue #1245:
URL: https://github.com/apache/pekko-connectors/issues/1245#issuecomment-3411204926
[`AWSS3IntegrationSpec`](https://github.com/apache/pekko-connectors/blob/2acae5acc1e49a47ee4c3e02eb16cb7730201c39/s3/src/test/scala/org/apache/pekko/stream/connectors/s3/scaladsl/S3IntegrationSpec.scala#L1267-L1282) contains the runner which runs against a real S3 account as well as instructions on how to hook it up to your S3 account and how to run the tests.
The actual tests are defined in [`S3IntegrationSpec` ](https://github.com/apache/pekko-connectors/blob/2acae5acc1e49a47ee4c3e02eb16cb7730201c39/s3/src/test/scala/org/apache/pekko/stream/connectors/s3/scaladsl/S3IntegrationSpec.scala#L47) which is where you would do the necessary changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] MultipartCopy Fails with the presence of Storage Class Header [pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
mdedetrich commented on issue #1245:
URL: https://github.com/apache/pekko-connectors/issues/1245#issuecomment-3411077227
We do have an S3 test account provided by the Apache Foundation linked to Pekko which should help, albeit S3 integration tests only happen on main branch and not on PR's (for security reasons).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] MultipartCopy Fails with the presence of Storage Class Header [pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
mdedetrich commented on issue #1245:
URL: https://github.com/apache/pekko-connectors/issues/1245#issuecomment-3411284142
https://github.com/apache/pekko-connectors/blob/ad06c34dc8456d696ae4827705503ea11327c105/.github/workflows/nightly-builds.yaml#L83-L90 Is also helpful to show another way on how to run the tests using environment variables instead of having to modify `s3/src/test/resources/application.conf`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] MultipartCopy Fails with the presence of Storage Class Header [pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
mdedetrich closed issue #1245: MultipartCopy Fails with the presence of Storage Class Header
URL: https://github.com/apache/pekko-connectors/issues/1245
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] MultipartCopy Fails with the presence of Storage Class Header [pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on issue #1245:
URL: https://github.com/apache/pekko-connectors/issues/1245#issuecomment-3411053132
Any interest in supplying a PR? Pekko is powered by occasional volunteers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] MultipartCopy Fails with the presence of Storage Class Header [pekko-connectors]
Posted by "Luka-J9 (via GitHub)" <gi...@apache.org>.
Luka-J9 commented on issue #1245:
URL: https://github.com/apache/pekko-connectors/issues/1245#issuecomment-3411172553
Sure - I'll try to have a PR out later today to discuss against.
@mdedetrich I haven't spent too much time in the codebase - can you point me to where the integration tests are located?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org