You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2023/05/17 10:57:00 UTC
[jira] [Commented] (HADOOP-18742) AWS v2 SDK: stabilise dependencies with rest of hadoop libraries
[ https://issues.apache.org/jira/browse/HADOOP-18742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17723411#comment-17723411 ]
Steve Loughran commented on HADOOP-18742:
-----------------------------------------
dependencies through hadoop-cloud-storage pom on feature branch
{code}
[INFO] +- org.apache.hadoop:hadoop-aws:jar:3.4.0-SNAPSHOT:compile
[INFO] | +- com.amazonaws:aws-java-sdk-core:jar:1.12.316:compile
[INFO] | | +- software.amazon.ion:ion-java:jar:1.0.2:compile
[INFO] | | +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.12.7:compile
[INFO] | | \- joda-time:joda-time:jar:2.8.1:compile
[INFO] | +- software.amazon.awssdk:bundle:jar:2.19.12:compile
[INFO] | | \- software.amazon.eventstream:eventstream:jar:1.0.1:compile
[INFO] | +- software.amazon.awssdk.crt:aws-crt:jar:0.21.0:compile
[INFO] | \- org.wildfly.openssl:wildfly-openssl:jar:1.0.7.Final:compile
{code}
# that wildfly scope should be cut from "runtime" to provided; it is entirely optional. note, this holds for hadoop-trunk...it isn't from the v2 sdk changes
# eventstream seems surplus
# amazon ion also seems surplus, unless somehow it is used as a wire format in bits of aws/sts
> AWS v2 SDK: stabilise dependencies with rest of hadoop libraries
> ----------------------------------------------------------------
>
> Key: HADOOP-18742
> URL: https://issues.apache.org/jira/browse/HADOOP-18742
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build, fs/s3
> Reporter: Steve Loughran
> Priority: Blocker
>
> aws v2 sdk dependencies need to
> # be in sync with rest of hadoop
> # not include anything coming in from hadoop common
> # not export in hadoop-cloud-storage stuff from hadoop-common.
> currently it is pulling in a version of jackson cbor whose consistency with hadoop's import is simply luck, joda time is also there
> In an ideal world all this should be shaded: we cannot have the AWS sdk dictate what jackson version we ship with, given the history of downstream problems there.
> {code}
> [INFO] +- com.amazonaws:aws-java-sdk-core:jar:1.12.316:compile
> [INFO] | +- software.amazon.ion:ion-java:jar:1.0.2:compile
> [INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.12.7:compile
> [INFO] | \- joda-time:joda-time:jar:2.8.1:compile
> [INFO] +- software.amazon.awssdk:bundle:jar:2.19.12:compile
> [INFO] | \- software.amazon.eventstream:eventstream:jar:1.0.1:compile
> [INFO] +- software.amazon.awssdk.crt:aws-crt:jar:0.21.0:compile
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org