You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/12/29 22:09:37 UTC
[GitHub] [pulsar] joncashe opened a new issue #9085: AWS IAM: Support for AWS EKS ServiceAccount roles
joncashe opened a new issue #9085:
URL: https://github.com/apache/pulsar/issues/9085
Hello
After following the steps in the aws guide https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-technical-overview.html which implies set the accountservice annotation update the iam role and also set the the s3 offload configuration variables for the pulsar broker
```
managedLedgerOffloadDriver: "aws-s3"
s3ManagedLedgerOffloadRegion: "us-east-1"
s3ManagedLedgerOffloadBucket: bucketname
s3ManagedLedgerOffloadRole: rolename
s3ManagedLedgerOffloadRoleSessionName: sessioname
```
im still getting an error trying to pull the role from the broker
```
22:19:08.524 [main] ERROR org.apache.bookkeeper.mledger.offload.jcloud.impl.BlobStoreManagedLedgerOffloader - unable to fetch S3 credentials for offloading, failing
com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: Unable to find a region via the region provider chain. Must provide an explicit region in the builder or setup environment to supply a region., com.amazonaws.auth.profile.ProfileCredentialsProvider@61d84e08: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@2d9f64c9: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]
at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:136) ~[aws-java-sdk-core-1.11.774.jar:?]
at org.apache.bookkeeper.mledger.offload.jcloud.impl.BlobStoreManagedLedgerOffloader.getCredentials(BlobStoreManagedLedgerOffloader.java:264) [tiered-storage-jcloud-2.6.1.nar-unpacked/:?]
at org.apache.bookkeeper.mledger.offload.jcloud.impl.BlobStoreManagedLedgerOffloader.create(BlobStoreManagedLedgerOffloader.java:233) [tiered-storage-jcloud-2.6.1.nar-unpacked/:?]
at org.apache.bookkeeper.mledger.offload.jcloud.JCloudLedgerOffloaderFactory.create(JCloudLedgerOffloaderFactory.java:48) [tiered-storage-jcloud-2.6.1.nar-unpacked/:?]
at org.apache.bookkeeper.mledger.offload.jcloud.JCloudLedgerOffloaderFactory.create(JCloudLedgerOffloaderFactory.java:31) [tiered-storage-jcloud-2.6.1.nar-unpacked/:?]
at org.apache.bookkeeper.mledger.LedgerOffloaderFactory.create(LedgerOffloaderFactory.java:73) [org.apache.pulsar-managed-ledger-2.6.1.jar:2.6.1]
at org.apache.pulsar.broker.PulsarService.createManagedLedgerOffloader(PulsarService.java:896) [org.apache.pulsar-pulsar-broker-2.6.1.jar:2.6.1]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:458) [org.apache.pulsar-pulsar-broker-2.6.1.jar:2.6.1]
at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:280) [org.apache.pulsar-pulsar-broker-2.6.1.jar:2.6.1]
at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:349) [org.apache.pulsar-pulsar-broker-2.6.1.jar:2.6.1]
```
im using pulsar 2.6.1 as it shows is it possible to add eks serviceaccount roles we are trying to get rid of kiam role, the sdk version in the branch currently supports it https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-minimum-sdk.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #9085: AWS IAM: Support for AWS EKS ServiceAccount roles
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #9085:
URL: https://github.com/apache/pulsar/issues/9085
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] joncashe edited a comment on issue #9085: AWS IAM: Support for AWS EKS ServiceAccount roles
Posted by GitBox <gi...@apache.org>.
joncashe edited a comment on issue #9085:
URL: https://github.com/apache/pulsar/issues/9085#issuecomment-758247369
so after testing more i find out just setting AWS_REGION environmental variable solves the issue (at least for single region cluster) so i guess we can close it and said that the eks serviceaccount roles its currently supported since pulsar its a little above the minimum sdk required
thanks and sorry for the false alarm
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9085: AWS IAM: Support for AWS EKS ServiceAccount roles
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9085:
URL: https://github.com/apache/pulsar/issues/9085#issuecomment-788535392
Thanks @joncashe, I will close this issue first.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] joncashe commented on issue #9085: AWS IAM: Support for AWS EKS ServiceAccount roles
Posted by GitBox <gi...@apache.org>.
joncashe commented on issue #9085:
URL: https://github.com/apache/pulsar/issues/9085#issuecomment-758247369
so after testing more i find out just setting AWS_REGION environmental variable solves the issue (at least for single region cluster) so i guess we can close it and said that the eks serviceaccount roles its currently supported since pulsar its a little above the minimum sdk
thanks and sorry for the false alarm
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org