You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Yongjia Wang (JIRA)" <ji...@apache.org> on 2015/10/30 17:02:27 UTC
[jira] [Comment Edited] (SPARK-11413) Java 8 build has problem with
joda-time and s3 request, should bump joda-time version
[ https://issues.apache.org/jira/browse/SPARK-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14982755#comment-14982755 ]
Yongjia Wang edited comment on SPARK-11413 at 10/30/15 4:02 PM:
----------------------------------------------------------------
My last statement was wrong.
It's a problem when using jre 8u60 or above, and when joda-time 2.8.0 or below was loaded at run time.
The following error was reproduced when I use the official spark release with JRE 1.8.0_65 (Oracle Corporation)
Note that the aws-java-sdk-s3 library already depends on joda-time-2.8.1. It's due to Spark assembly's inclusion of joda-time-2.5 which was loaded by JVM that caused this problem.
com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: E0D714AF923221DD), S3 Extended Request ID: 8OJlBMYKVzwrEXHANpBTsiLRmaFn2lHQ4sAvkU/HF66wWGVj8VR2/Jh4Wl8QuFUt
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1182)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3738)
at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:653)
was (Author: yongjiaw):
My last statement was wrong.
It's a problem when using jre 8u60 or above, and when joda-time 2.8.0 and belong was loaded at run time by s3 client.
The following error was reproduced when I use the official spark release with JRE 1.8.0_65 (Oracle Corporation)
Note that the aws-java-sdk-s3 library already depends on joda-time-2.8.1. It's due to Spark assembly's inclusion of joda-time-2.5 which was loaded by JVM that caused this problem.
com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: E0D714AF923221DD), S3 Extended Request ID: 8OJlBMYKVzwrEXHANpBTsiLRmaFn2lHQ4sAvkU/HF66wWGVj8VR2/Jh4Wl8QuFUt
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1182)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3738)
at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:653)
> Java 8 build has problem with joda-time and s3 request, should bump joda-time version
> -------------------------------------------------------------------------------------
>
> Key: SPARK-11413
> URL: https://issues.apache.org/jira/browse/SPARK-11413
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Reporter: Yongjia Wang
> Priority: Minor
>
> Joda-time has problems with formatting time zones starting with Java 1.8u60, and this will cause s3 request to fail. It is said to have been fixed at joda-time 2.8.1.
> Spark is still using joda-time 2.5 by fault, if java8 is used to build spark, should set -Djoda.version=2.8.1 or above.
> I was hit by this problem, and -Djoda.version=2.9 worked.
> I don't see any reason not to bump up joda-time version in pom.xml
> Should I create a pull request for this? It is trivial.
> https://github.com/aws/aws-sdk-java/issues/484
> https://github.com/aws/aws-sdk-java/issues/444
> http://stackoverflow.com/questions/32058431/aws-java-sdk-aws-authentication-requires-a-valid-date-or-x-amz-date-header
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org