You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Aravindan Vijayan (Jira)" <ji...@apache.org> on 2020/06/09 15:16:00 UTC
[jira] [Comment Edited] (HDDS-3765) Fluentd writing to secure Ozone
S3 API fails with 500 Error.
[ https://issues.apache.org/jira/browse/HDDS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17129385#comment-17129385 ]
Aravindan Vijayan edited comment on HDDS-3765 at 6/9/20, 3:15 PM:
------------------------------------------------------------------
On further investigation, the issue was found to be an extra "/" in the fluentd s3 path config. This was causing the SHA256 hash of the string to sign to differ thereby causing the signatures validation to fail.
{code}
s3_endpoint testhost1:9879
path logs/${tag}/%Y/%m/%d/
{code}
PUT request on fluentd end
{code}
/logs-bucket-1/logs/mytag/2020/06/08//202006081900_145840.gz
content-md5:qZ9QMUH2NFbPcuTEdmC4zw==
content-type:application/x-gzip
expect:100-continue
host:testhost1:9879
user-agent:aws-sdk-ruby3/3.94.0 ruby/2.4.10 x86_64-linux aws-sdk-s3/1.63.0
x-amz-content-sha256:304e39c5ffa59a0dca8c673f805825a463eebd8010410a52f90436e92c3f69ed
x-amz-date:20200609T145843Z
x-amz-storage-class:STANDARD
content-md5;content-type;expect;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-storage-class
304e39c5ffa59a0dca8c673f805825a463eebd8010410a52f90436e92c3f69ed
-
AWS4-HMAC-SHA256
20200609T145843Z
20200609/us-east-1/s3/aws4_request
4d48d877b4b311df71f7c5cca3f89b091d71931726af20ceabc53ca4ab346f10
{code}
PUT request on s3g end.
{code}
canonicalRequest:[PUT
/logs-bucket-1/logs/mytag/2020/06/08/202006081900_145840.gz
content-md5:qZ9QMUH2NFbPcuTEdmC4zw==
content-type:application/x-gzip
expect:100-continue
host:testhost1:9879
user-agent:aws-sdk-ruby3/3.94.0 ruby/2.4.10 x86_64-linux aws-sdk-s3/1.63.0
x-amz-content-sha256:304e39c5ffa59a0dca8c673f805825a463eebd8010410a52f90436e92c3f69ed
x-amz-date:20200609T145843Z
x-amz-storage-class:STANDARD
content-md5;content-type;expect;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-storage-class
304e39c5ffa59a0dca8c673f805825a463eebd8010410a52f90436e92c3f69ed]
2020-06-09 14:58:43,309 DEBUG org.apache.hadoop.ozone.s3.AWSV4SignatureProcessor: StringToSign:[AWS4-HMAC-SHA256
20200609T145843Z
20200609/us-east-1/s3/aws4_request
785cb4f171740e506329f482098b77bfb61aeeb21e7ad6743c616e0873bb8417]
{code}
was (Author: avijayan):
On further investigation, the issue was found to be an extra "/" in the fluentd s3 path config. This was causing the SHA256 hash of the string to sign to differ thereby causing the signatures validation to fail.
{code}
s3_endpoint testhost1:9879
path logs/${tag}/%Y/%m/%d/
{code}
PUT request on fluentd end
{code}
/logs-bucket-1/logs/mytag/2020/06/08//202006081900_145840.gz
content-md5:qZ9QMUH2NFbPcuTEdmC4zw==
content-type:application/x-gzip
expect:100-continue
host:testhost1:9879
user-agent:aws-sdk-ruby3/3.94.0 ruby/2.4.10 x86_64-linux aws-sdk-s3/1.63.0
x-amz-content-sha256:304e39c5ffa59a0dca8c673f805825a463eebd8010410a52f90436e92c3f69ed
x-amz-date:20200609T145843Z
x-amz-storage-class:STANDARD
content-md5;content-type;expect;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-storage-class
304e39c5ffa59a0dca8c673f805825a463eebd8010410a52f90436e92c3f69ed
-
AWS4-HMAC-SHA256
20200609T145843Z
20200609/us-east-1/s3/aws4_request
4d48d877b4b311df71f7c5cca3f89b091d71931726af20ceabc53ca4ab346f10
{code}
PUT request on s3g end.
{code}
canonicalRequest:[PUT
/logs-bucket-1/logs/mytag/2020/06/08/202006081900_145840.gz
content-md5:qZ9QMUH2NFbPcuTEdmC4zw==
content-type:application/x-gzip
expect:100-continue
host:testhost1:9879
user-agent:aws-sdk-ruby3/3.94.0 ruby/2.4.10 x86_64-linux aws-sdk-s3/1.63.0
x-amz-content-sha256:304e39c5ffa59a0dca8c673f805825a463eebd8010410a52f90436e92c3f69ed
x-amz-date:20200609T145843Z
x-amz-storage-class:STANDARD
content-md5;content-type;expect;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-storage-class
304e39c5ffa59a0dca8c673f805825a463eebd8010410a52f90436e92c3f69ed]
2020-06-09 14:58:43,309 DEBUG org.apache.hadoop.ozone.s3.AWSV4SignatureProcessor: StringToSign:[AWS4-HMAC-SHA256
20200609T145843Z
20200609/us-east-1/s3/aws4_request
785cb4f171740e506329f482098b77bfb61aeeb21e7ad6743c616e0873bb8417]
{code}
The fluentd integration work is unblocked since we can remove the dangling "/" in the path, but we should fix this in upstream Ozone. Created HDDS-3765 to track that bug.
> Fluentd writing to secure Ozone S3 API fails with 500 Error.
> ------------------------------------------------------------
>
> Key: HDDS-3765
> URL: https://issues.apache.org/jira/browse/HDDS-3765
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Aravindan Vijayan
> Priority: Major
> Fix For: 0.7.0
>
>
> Error on fluentd side.
> {code}
> opened
> starting SSL for host1:9879...
> SSL established
> <- "PUT /logs-bucket-1/logs/mytag/2020/06/05//202006052222_190411.gz HTTP/1.1\r\nContent-Type: application/x-gzip\r\nAccept-Encoding: \r\nUser-Agent: aws-sdk-ruby3/3.94.0 ruby/2.4.10 x86_64-l
> inux aws-sdk-s3/1.63.0\r\nX-Amz-Storage-Class: STANDARD\r\nExpect: 100-continue\r\nContent-Md5: zGKVGGaD/U5WUK3cdWQiSA==\r\nHost: host1:9879\r\nX-Amz-Content-Sha256:
> 277fe97f57a1127ee1a0765979ffd3270a6c18c6f75ff6a0f843e7163a338de2\r\nContent-Length: 44726\r\nX-Amz-Date: 20200608T190412Z\r\nAuthorization: AWS4-HMAC-SHA256 Credential=hdfs@ROOT.HWX.SITE/202
> 00608/us-east-1/s3/aws4_request, SignedHeaders=content-md5;content-type;expect;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-storage-class, Signature=11c1d0a43325d3f7b9d25dbd02023cef2
> 69b66f6a93fa4e1c935b52e3e372f70\r\nAccept: */*\r\n\r\n"
> -> "HTTP/1.1 100 Continue\r\n"
> -> "\r\n"
> -> "HTTP/1.1 500 Server Error\r\n"
> -> "Pragma: no-cache\r\n"
> -> "X-Content-Type-Options: nosniff\r\n"
> -> "X-FRAME-OPTIONS: SAMEORIGIN\r\n"
> -> "X-XSS-Protection: 1; mode=block\r\n"
> -> "Connection: close\r\n"
> -> "\r\n"
> reading all...
> -> ""
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org