You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by pd...@apache.org on 2022/07/10 17:58:48 UTC
[zeppelin] branch master updated: [ZEPPELIN-5757] Assume role in NotebookRepo S3 (#4390)
This is an automated email from the ASF dual-hosted git repository.
pdallig pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new a41897a6cf [ZEPPELIN-5757] Assume role in NotebookRepo S3 (#4390)
a41897a6cf is described below
commit a41897a6cff9f10df9e6af4c98abfc0b1e92992b
Author: Michal Vince <vi...@gmail.com>
AuthorDate: Sun Jul 10 19:58:42 2022 +0200
[ZEPPELIN-5757] Assume role in NotebookRepo S3 (#4390)
* - added missing dependency
- fixed documentation
* add explanation why STS lib is dependency
* added unit test to ensure AWS STS lib is on class path
---
docs/setup/storage/storage.md | 1 +
zeppelin-plugins/notebookrepo/s3/pom.xml | 21 +++++++++++++++++++--
.../zeppelin/notebook/repo/S3NotebookRepoTest.java | 5 +++++
3 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/docs/setup/storage/storage.md b/docs/setup/storage/storage.md
index bbc9583728..71b447c217 100644
--- a/docs/setup/storage/storage.md
+++ b/docs/setup/storage/storage.md
@@ -79,6 +79,7 @@ Notebooks may be stored in S3, and optionally encrypted. The [``DefaultAWSCrede
- The ``AWS_ACCESS_KEY_ID`` and ``AWS_SECRET_ACCESS_KEY`` environment variables
- The ``aws.accessKeyId`` and ``aws.secretKey`` Java System properties
+- Web Identity Token credentials from the environment or container
- Credential profiles file at the default location (````~/.aws/credentials````) used by the AWS CLI
- Instance profile credentials delivered through the Amazon EC2 metadata service
diff --git a/zeppelin-plugins/notebookrepo/s3/pom.xml b/zeppelin-plugins/notebookrepo/s3/pom.xml
index 5cfe3963f2..4f064793e2 100644
--- a/zeppelin-plugins/notebookrepo/s3/pom.xml
+++ b/zeppelin-plugins/notebookrepo/s3/pom.xml
@@ -34,7 +34,7 @@
<description>NotebookRepo implementation based on S3</description>
<properties>
- <aws.sdk.s3.version>1.11.736</aws.sdk.s3.version>
+ <aws.sdk.version>1.11.736</aws.sdk.version>
<plugin.name>NotebookRepo/S3NotebookRepo</plugin.name>
</properties>
@@ -42,7 +42,24 @@
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
- <version>${aws.sdk.s3.version}</version>
+ <version>${aws.sdk.version}</version>
+ <exclusions>
+ <!-- jcl-over-slf4j is provided by zeppelin-interprerter -->
+ <exclusion>
+ <groupId>commons-logging</groupId>
+ <artifactId>commons-logging</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <!-- STS lib needs to be on class path for the WebidentityTokenCredentialsProvider to work properly,
+ otherwise it will be skipped in DefaultAWSCredentialsProviderChain
+ https://github.com/aws/aws-sdk-java/issues/2136
+ -->
+ <groupId>com.amazonaws</groupId>
+ <artifactId>aws-java-sdk-sts</artifactId>
+ <version>${aws.sdk.version}</version>
<exclusions>
<!-- jcl-over-slf4j is provided by zeppelin-interprerter -->
<exclusion>
diff --git a/zeppelin-plugins/notebookrepo/s3/src/test/java/org/apache/zeppelin/notebook/repo/S3NotebookRepoTest.java b/zeppelin-plugins/notebookrepo/s3/src/test/java/org/apache/zeppelin/notebook/repo/S3NotebookRepoTest.java
index fc03461324..e5562caed1 100644
--- a/zeppelin-plugins/notebookrepo/s3/src/test/java/org/apache/zeppelin/notebook/repo/S3NotebookRepoTest.java
+++ b/zeppelin-plugins/notebookrepo/s3/src/test/java/org/apache/zeppelin/notebook/repo/S3NotebookRepoTest.java
@@ -86,6 +86,11 @@ public class S3NotebookRepoTest {
}
}
+ @Test
+ public void testAwsSTSLibraryOnClassPath() throws ClassNotFoundException {
+ Class.forName("com.amazonaws.auth.STSSessionCredentialsProvider", false, getClass().getClassLoader());
+ }
+
@Test
public void testNotebookRepo() throws IOException {
Map<String, NoteInfo> notesInfo = notebookRepo.list(anonymous);
Re: [zeppelin] branch master updated: [ZEPPELIN-5757] Assume role in NotebookRepo S3 (#4390)
Posted by emmanuel warreng <em...@gmail.com>.
Unsubscribe
On Sun, Jul 10, 2022, 19:58 <pd...@apache.org> wrote:
> This is an automated email from the ASF dual-hosted git repository.
>
> pdallig pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/zeppelin.git
>
>
> The following commit(s) were added to refs/heads/master by this push:
> new a41897a6cf [ZEPPELIN-5757] Assume role in NotebookRepo S3 (#4390)
> a41897a6cf is described below
>
> commit a41897a6cff9f10df9e6af4c98abfc0b1e92992b
> Author: Michal Vince <vi...@gmail.com>
> AuthorDate: Sun Jul 10 19:58:42 2022 +0200
>
> [ZEPPELIN-5757] Assume role in NotebookRepo S3 (#4390)
>
> * - added missing dependency
> - fixed documentation
>
> * add explanation why STS lib is dependency
>
> * added unit test to ensure AWS STS lib is on class path
> ---
> docs/setup/storage/storage.md | 1 +
> zeppelin-plugins/notebookrepo/s3/pom.xml | 21
> +++++++++++++++++++--
> .../zeppelin/notebook/repo/S3NotebookRepoTest.java | 5 +++++
> 3 files changed, 25 insertions(+), 2 deletions(-)
>
> diff --git a/docs/setup/storage/storage.md b/docs/setup/storage/storage.md
> index bbc9583728..71b447c217 100644
> --- a/docs/setup/storage/storage.md
> +++ b/docs/setup/storage/storage.md
> @@ -79,6 +79,7 @@ Notebooks may be stored in S3, and optionally
> encrypted. The [``DefaultAWSCrede
>
> - The ``AWS_ACCESS_KEY_ID`` and ``AWS_SECRET_ACCESS_KEY`` environment
> variables
> - The ``aws.accessKeyId`` and ``aws.secretKey`` Java System properties
> +- Web Identity Token credentials from the environment or container
> - Credential profiles file at the default location
> (````~/.aws/credentials````) used by the AWS CLI
> - Instance profile credentials delivered through the Amazon EC2 metadata
> service
>
> diff --git a/zeppelin-plugins/notebookrepo/s3/pom.xml
> b/zeppelin-plugins/notebookrepo/s3/pom.xml
> index 5cfe3963f2..4f064793e2 100644
> --- a/zeppelin-plugins/notebookrepo/s3/pom.xml
> +++ b/zeppelin-plugins/notebookrepo/s3/pom.xml
> @@ -34,7 +34,7 @@
> <description>NotebookRepo implementation based on S3</description>
>
> <properties>
> - <aws.sdk.s3.version>1.11.736</aws.sdk.s3.version>
> + <aws.sdk.version>1.11.736</aws.sdk.version>
> <plugin.name>NotebookRepo/S3NotebookRepo</plugin.name>
> </properties>
>
> @@ -42,7 +42,24 @@
> <dependency>
> <groupId>com.amazonaws</groupId>
> <artifactId>aws-java-sdk-s3</artifactId>
> - <version>${aws.sdk.s3.version}</version>
> + <version>${aws.sdk.version}</version>
> + <exclusions>
> + <!-- jcl-over-slf4j is provided by zeppelin-interprerter
> -->
> + <exclusion>
> + <groupId>commons-logging</groupId>
> + <artifactId>commons-logging</artifactId>
> + </exclusion>
> + </exclusions>
> + </dependency>
> +
> + <dependency>
> + <!-- STS lib needs to be on class path for the
> WebidentityTokenCredentialsProvider to work properly,
> + otherwise it will be skipped in
> DefaultAWSCredentialsProviderChain
> + https://github.com/aws/aws-sdk-java/issues/2136
> + -->
> + <groupId>com.amazonaws</groupId>
> + <artifactId>aws-java-sdk-sts</artifactId>
> + <version>${aws.sdk.version}</version>
> <exclusions>
> <!-- jcl-over-slf4j is provided by zeppelin-interprerter
> -->
> <exclusion>
> diff --git
> a/zeppelin-plugins/notebookrepo/s3/src/test/java/org/apache/zeppelin/notebook/repo/S3NotebookRepoTest.java
> b/zeppelin-plugins/notebookrepo/s3/src/test/java/org/apache/zeppelin/notebook/repo/S3NotebookRepoTest.java
> index fc03461324..e5562caed1 100644
> ---
> a/zeppelin-plugins/notebookrepo/s3/src/test/java/org/apache/zeppelin/notebook/repo/S3NotebookRepoTest.java
> +++
> b/zeppelin-plugins/notebookrepo/s3/src/test/java/org/apache/zeppelin/notebook/repo/S3NotebookRepoTest.java
> @@ -86,6 +86,11 @@ public class S3NotebookRepoTest {
> }
> }
>
> + @Test
> + public void testAwsSTSLibraryOnClassPath() throws
> ClassNotFoundException {
> + Class.forName("com.amazonaws.auth.STSSessionCredentialsProvider",
> false, getClass().getClassLoader());
> + }
> +
> @Test
> public void testNotebookRepo() throws IOException {
> Map<String, NoteInfo> notesInfo = notebookRepo.list(anonymous);
>
>