You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "affo (via GitHub)" <gi...@apache.org> on 2026/03/27 14:12:22 UTC
[I] Azure FS token delegation does not take effect [fluss]
affo opened a new issue, #2946:
URL: https://github.com/apache/fluss/issues/2946
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Fluss version
0.9.0 (latest release)
### Please describe the bug 🐞
When performing a KV snapshot read from a client without any Azure-related setting, I get the following trace:
```
org.apache.fluss.exception.FlussRuntimeException: Failed to initialize snapshot files reader.
at org.apache.fluss.client.table.scanner.batch.KvSnapshotBatchScanner.ensureNoException(KvSnapshotBatchScanner.java:169)
at org.apache.fluss.client.table.scanner.batch.KvSnapshotBatchScanner.pollBatch(KvSnapshotBatchScanner.java:140)
at com.ververica.fluss.ReadCmd.readBatchScanner(ReadCmd.java:255)
at com.ververica.fluss.ReadCmd.readPkFull(ReadCmd.java:173)
at com.ververica.fluss.ReadCmd.call(ReadCmd.java:63)
at com.ververica.fluss.ReadCmd.call(ReadCmd.java:29)
at picocli.CommandLine.executeUserObject(CommandLine.java:2041)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
at picocli.CommandLine.execute(CommandLine.java:2170)
at com.ververica.fluss.TestPkCmd.call(TestPkCmd.java:60)
at com.ververica.fluss.TestPkCmd.call(TestPkCmd.java:7)
at picocli.CommandLine.executeUserObject(CommandLine.java:2041)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
at picocli.CommandLine.execute(CommandLine.java:2170)
at com.ververica.fluss.App.main(App.java:49)
Caused by: java.io.IOException: Failure to initialize configuration
at org.apache.fluss.fs.utils.FileDownloadUtils.downloadFile(FileDownloadUtils.java:156)
at org.apache.fluss.fs.utils.FileDownloadUtils.lambda$transferDataToDirectoryAsync$1(FileDownloadUtils.java:117)
at org.apache.fluss.utils.function.CheckedSupplier.lambda$unchecked$0(CheckedSupplier.java:29)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: Failure to initialize configuration
at org.apache.hadoop.fs.azurebfs.services.SimpleKeyProvider.getStorageAccountKey(SimpleKeyProvider.java:51)
at org.apache.hadoop.fs.azurebfs.AbfsConfiguration.getStorageAccountKey(AbfsConfiguration.java:586)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.initializeClient(AzureBlobFileSystemStore.java:1560)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:236)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:183)
at org.apache.fluss.fs.azure.AzureFileSystemPlugin.create(AzureFileSystemPlugin.java:60)
at org.apache.fluss.fs.azure.AbfsFileSystemPlugin.create(AbfsFileSystemPlugin.java:29)
at org.apache.fluss.fs.FileSystem.lambda$getUnguardedFileSystem$3(FileSystem.java:405)
at org.apache.fluss.utils.concurrent.LockUtils.inLock(LockUtils.java:42)
at org.apache.fluss.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:310)
at org.apache.fluss.fs.FileSystem.get(FileSystem.java:305)
at org.apache.fluss.fs.FsPath.getFileSystem(FsPath.java:265)
at org.apache.fluss.fs.utils.FileDownloadUtils.downloadFile(FileDownloadUtils.java:139)
... 6 more
```
### Solution
1. FlussConnection filters config to only client.fs.* keys → no fs.azure.* credentials reach AzureFileSystemPlugin
2. setCredentialProvider correctly falls back to delegation tokens and calls AzureDelegationTokenReceiver.updateHadoopConfig
3. updateHadoopConfig sets fs.azure.account.oauth.provider.type = DynamicTemporaryAzureCredentialsProvider
4. But does NOT set fs.azure.account.auth.type = Custom
5. ABFS defaults to SharedKey → calls SimpleKeyProvider.getStorageAccountKey → key not found → crash
DynamicTemporaryAzureCredentialsProvider implements CustomTokenProviderAdaptee, which the ABFS driver only uses when fs.azure.account.auth.type = Custom is set.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Azure FS token delegation does not take effect [fluss]
Posted by "polyzos (via GitHub)" <gi...@apache.org>.
polyzos closed issue #2946: Azure FS token delegation does not take effect
URL: https://github.com/apache/fluss/issues/2946
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org