You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Sriram Ganesh <sr...@gmail.com> on 2023/04/20 14:56:24 UTC
Facing issue when using S3 in Flink 1.17
Hi Team,
I am using S3 as FileSystem to write data from Flink. I am getting the
below error in Flink 1.17. The same code works in Flink 1.16. Could someone
please help?
Caused by: java.lang.IllegalStateException: Delegation token provider with
service name {} has multiple implementations [s3]
│
│ at
org.apache.flink.util.Preconditions.checkState(Preconditions.java:215)
~[flink-dist-1.17.0.jar:1.17.0]
│
│ at
org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.lambda$loadProviders$0(DefaultDelegationTokenManager.java:133)
~[flink-dist-1.17.0.jar:1.17.0]
│
│ at java.util.Iterator.forEachRemaining(Iterator.java:133) ~[?:?]
│
│ at
org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.loadProviders(DefaultDelegationTokenManager.java:156)
~[flink-dist-1.17.0.jar:1.17.0]
│
│ at
org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.<init>(DefaultDelegationTokenManager.java:111)
~[flink-dist-1.17.0.jar:1.17.0]
│
│ at
org.apache.flink.runtime.security.token.DefaultDelegationTokenManagerFactory.create(DefaultDelegationTokenManagerFactory.java:50)
~[flink-dist-1.17.0.jar:1.17.0]
│
│ at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:392)
~[flink-dist-1.17.0.jar:1.17.0]
│
│ at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:282)
~[flink-dist-1.17.0.jar:1.17.0]
│
│ at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$1(ClusterEntrypoint.java:232)
~[flink-dist-1.17.0.jar:1.17.0]
│
│ at
org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
~[flink-dist-1.17.0.jar:1.17.0]
│
│ at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:229)
~[flink-dist-1.17.0.jar:1.17.0]
│
│ ... 2 more
Thanks,
--
*Sriram G*
Re: Facing issue when using S3 in Flink 1.17
Posted by Sriram Ganesh <sr...@gmail.com>.
Thanks a lot, Gabor. Do you know by any chance when this fix will be
released? Thanks in advance.
On Thu, Apr 20, 2023 at 9:06 PM Gabor Somogyi <ga...@gmail.com>
wrote:
> Hi Sriram,
>
> This has been fixed in https://issues.apache.org/jira/browse/FLINK-31839
>
> G
>
>
> On Thu, Apr 20, 2023 at 4:57 PM Sriram Ganesh <sr...@gmail.com> wrote:
>
>> Hi Team,
>>
>> I am using S3 as FileSystem to write data from Flink. I am getting the
>> below error in Flink 1.17. The same code works in Flink 1.16. Could someone
>> please help?
>>
>> Caused by: java.lang.IllegalStateException: Delegation token provider
>> with service name {} has multiple implementations [s3]
>>
>> │
>> │ at
>> org.apache.flink.util.Preconditions.checkState(Preconditions.java:215)
>> ~[flink-dist-1.17.0.jar:1.17.0]
>>
>> │
>> │ at
>> org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.lambda$loadProviders$0(DefaultDelegationTokenManager.java:133)
>> ~[flink-dist-1.17.0.jar:1.17.0]
>> │
>> │ at java.util.Iterator.forEachRemaining(Iterator.java:133) ~[?:?]
>>
>>
>> │
>> │ at
>> org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.loadProviders(DefaultDelegationTokenManager.java:156)
>> ~[flink-dist-1.17.0.jar:1.17.0]
>> │
>> │ at
>> org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.<init>(DefaultDelegationTokenManager.java:111)
>> ~[flink-dist-1.17.0.jar:1.17.0]
>> │
>> │ at
>> org.apache.flink.runtime.security.token.DefaultDelegationTokenManagerFactory.create(DefaultDelegationTokenManagerFactory.java:50)
>> ~[flink-dist-1.17.0.jar:1.17.0]
>> │
>> │ at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:392)
>> ~[flink-dist-1.17.0.jar:1.17.0]
>> │
>> │ at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:282)
>> ~[flink-dist-1.17.0.jar:1.17.0]
>> │
>> │ at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$1(ClusterEntrypoint.java:232)
>> ~[flink-dist-1.17.0.jar:1.17.0]
>> │
>> │ at
>> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
>> ~[flink-dist-1.17.0.jar:1.17.0]
>> │
>> │ at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:229)
>> ~[flink-dist-1.17.0.jar:1.17.0]
>> │
>> │ ... 2 more
>>
>> Thanks,
>>
>> --
>> *Sriram G*
>>
>>
--
*Sriram G*
*Tech*
Re: Facing issue when using S3 in Flink 1.17
Posted by Gabor Somogyi <ga...@gmail.com>.
Hi Sriram,
This has been fixed in https://issues.apache.org/jira/browse/FLINK-31839
G
On Thu, Apr 20, 2023 at 4:57 PM Sriram Ganesh <sr...@gmail.com> wrote:
> Hi Team,
>
> I am using S3 as FileSystem to write data from Flink. I am getting the
> below error in Flink 1.17. The same code works in Flink 1.16. Could someone
> please help?
>
> Caused by: java.lang.IllegalStateException: Delegation token provider with
> service name {} has multiple implementations [s3]
>
> │
> │ at
> org.apache.flink.util.Preconditions.checkState(Preconditions.java:215)
> ~[flink-dist-1.17.0.jar:1.17.0]
>
> │
> │ at
> org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.lambda$loadProviders$0(DefaultDelegationTokenManager.java:133)
> ~[flink-dist-1.17.0.jar:1.17.0]
> │
> │ at java.util.Iterator.forEachRemaining(Iterator.java:133) ~[?:?]
>
>
> │
> │ at
> org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.loadProviders(DefaultDelegationTokenManager.java:156)
> ~[flink-dist-1.17.0.jar:1.17.0]
> │
> │ at
> org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.<init>(DefaultDelegationTokenManager.java:111)
> ~[flink-dist-1.17.0.jar:1.17.0]
> │
> │ at
> org.apache.flink.runtime.security.token.DefaultDelegationTokenManagerFactory.create(DefaultDelegationTokenManagerFactory.java:50)
> ~[flink-dist-1.17.0.jar:1.17.0]
> │
> │ at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:392)
> ~[flink-dist-1.17.0.jar:1.17.0]
> │
> │ at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:282)
> ~[flink-dist-1.17.0.jar:1.17.0]
> │
> │ at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$1(ClusterEntrypoint.java:232)
> ~[flink-dist-1.17.0.jar:1.17.0]
> │
> │ at
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
> ~[flink-dist-1.17.0.jar:1.17.0]
> │
> │ at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:229)
> ~[flink-dist-1.17.0.jar:1.17.0]
> │
> │ ... 2 more
>
> Thanks,
>
> --
> *Sriram G*
>
>