[I] [Object Store] Make the service account used when interacting with the metadata url more flexible [arrow-rs]

["gianarb (via GitHub)" <gi...@apache.org>]

gianarb opened a new issue, #5073:
URL: https://github.com/apache/arrow-rs/issues/5073

   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   
   Hello, I deploy my application to GCP GKE and in order to authorize to GCP Object Storage I am using a workload identity as described by the GCP documentation
   
   https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity#authenticating_to
   
   The problem I am facing is that `default` is not the right service account, I would like to use the one defined when attaching the iam service account to the kubernetes one.
   
   https://github.com/apache/arrow-rs/blob/master/object_store/src/gcp/credential.rs#L325-L331
   
   
   **Describe the solution you'd like**
   
   I think we should add configuration to specify the name of the service account you want to use. I am not sure the implication but it sounds good to use the `service_account_key` if present, if not we can use `default` as we do today.
   
   **Describe alternatives you've considered**
   
   I can generate a credential file and authenticate with another method but this one looks like the way to go suggested by GCP itself.
   
   **Additional context**
   <!--
   Add any other context or screenshots about the feature request here.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Object Store] Make the service account used when interacting with the metadata url more flexible [arrow-rs]

["tustvold (via GitHub)" <gi...@apache.org>]

tustvold commented on issue #5073:
URL: https://github.com/apache/arrow-rs/issues/5073#issuecomment-1900320971

   So I re-read the docs here and I am honestly confused why this isn't working. The docs state that you should
   
   1. Annotate the IAM role in GCS with the policy binding
   
   ```
   gcloud projects add-iam-policy-binding GSA_PROJECT --member "serviceAccount:GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com"    --role "ROLE_NAME"
   ```
   
   2. Annotate the kubernetes ServiceAccount object
   
   
   ````
   kubectl annotate serviceaccount KSA_NAME --namespace NAMESPACE  iam.gke.io/gcp-service-account=GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com
   ```
   
   3. Update the pods to enable the metadata hook
   
   ```
   spec:
     serviceAccountName: KSA_NAME
     nodeSelector:
       iam.gke.io/gke-metadata-server-enabled: "true"
   ```
   
   4. Now within the workload calling the metadata endpoint with `default` will yield the service account that you've mapped in
   
   ```
   curl -H "Metadata-Flavor: Google" http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/email
   ```
   
   I cannot find any documentation about calling the metadata endpoint with anything other than `default`, when you would do this, or what its behaviour would be.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org