You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/07/17 23:34:35 UTC
[GitHub] [airflow] laxmikanth-d opened a new issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
laxmikanth-d opened a new issue #17063:
URL: https://github.com/apache/airflow/issues/17063
<!--
Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions.
Don't worry if they're not all applicable; just try to include what you can :-)
If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>
Please delete these comment blocks before submitting the issue.
-->
<!--
IMPORTANT!!!
PLEASE CHECK "SIMILAR TO X EXISTING ISSUES" OPTION IF VISIBLE
NEXT TO "SUBMIT NEW ISSUE" BUTTON!!!
PLEASE CHECK IF THIS ISSUE HAS BEEN REPORTED PREVIOUSLY USING SEARCH!!!
Please complete the next sections or the issue will be closed.
These questions are the first thing we need to know to understand the context.
-->
**Apache Airflow version**:
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
**Environment**:
- **Cloud provider or hardware configuration**: Airflow Container
- **OS** (e.g. from /etc/os-release): Windows
- **Kernel** (e.g. `uname -a`):
- **Install tools**:
- **Others**:
**What happened**: Using S3Hook to upload file to AWS S3. I have create user in aws with full S3 permissions. But When I run my DAG task I getting error "role_arn" is NONE.
<!-- (please include exact error messages if you can) -->
**What you expected to happen**:
<!-- What do you think went wrong? -->
**How to reproduce it**:
Here is the code snippet.
`from airflow import DAG
import airflow
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
from airflow.operators.python import PythonOperator
from datetime import timedelta, datetime
default_args = {
'owner':'Lax',
'retries':0,
'email_on_retry': False,
'email':'email@gmail.com',
'email_on_failure': False,
'retry_delay': timedelta(minutes=1)
}
def file_upload(filename,key,bucket_name):
hook = S3Hook('s3_connection')
hook.load_file(filename, key, bucket_name)
with DAG(dag_id='s3_hook_demo', start_date = datetime(2021,7,14), schedule_interval = "@daily", default_args = default_args,
catchup=False) as dag:
file_upload_task = PythonOperator(
task_id='file_upload',
op_kwargs= {
'filename': 'test.txt',
'key':'text.txt',
'bucket_name':'angfdighubfjhdknmmf'
},
python_callable=file_upload
)`
<!---
As minimally and precisely as possible. Keep in mind we do not have access to your cluster or dags.
If you are using kubernetes, please attempt to recreate the issue using minikube or kind.
## Install minikube/kind
- Minikube https://minikube.sigs.k8s.io/docs/start/
- Kind https://kind.sigs.k8s.io/docs/user/quick-start/
If this is a UI bug, please provide a screenshot of the bug or a link to a youtube video of the bug in action
You can include images using the .md style of
To record a screencast, mac users can use QuickTime and then create an unlisted youtube video with the resulting .mov file.
--->
**Anything else we need to know**:
<!--
How often does this problem occur? Once? Every time etc?
Any relevant logs to include? Put them here in side a detail tag:
<details><summary>x.log</summary> lots of stuff </details>
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] baryluk edited a comment on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
baryluk edited a comment on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-939077168
@m0zart89 That will not work either I think. Try `?region_name=us-east-1` instead.
```
s3://access_key:secret_access_key@/?region_name=us-east-1
```
Is the best. Do not specify host or anything after `/`, it will be used as a bucket name, and if it is provided, anything that is later provided as path, will be concatenated, and will lead to weird results. (like `bucket/bucket/key/object/path/foo.bar`)
Also:
```
(Note: Escape aka urlencode "/" in secret_access_key into "%2F")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #17063:
URL: https://github.com/apache/airflow/issues/17063
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] baryluk commented on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
baryluk commented on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-939077168
@m0zart89 That will not work either I think. Try `?region_name=us-east-1` instead.
```
s3://access_key:secret_access_key@/
```
Is the best. Do not specify host or anything after `/`, it will be used as a bucket name, and if it is provided, anything that is later provided as path, will be concatenated, and will lead to weird results. (like `bucket/bucket/key/object/path/foo.bar`)
Also:
```
(Note: Escape aka urlencode "/" in secret_access_key into "%2F")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] m0zart89 commented on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
m0zart89 commented on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-938895498
I have the same problem. Also when I add region as postfix to my bucket name (_bucketname.us-east-1_) I see another behaviour:
An error occurred (**404**) when calling the HeadObject operation: Not Found
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-907705625
This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-882122234
`role_arn is None` is an info entry. It is not an error. You can ignore this line. Can you provide a full error stack?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-881974011
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] soumitra9 commented on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
soumitra9 commented on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-922174301
Could anybody help please, I am facing similar issues
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] baryluk commented on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
baryluk commented on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-938671402
@soumitra9 @rainyjonne You might be specifying the bucket wrong. The URI syntax is very deceiving. It is best to NOT specify the bucket name in the uri.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] baryluk commented on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
baryluk commented on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-889030163
Please provide rest of the stack trace. The `INFO - role_arn is None` is expected, and not a problem. In fact it is normal operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-913058269
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] baryluk edited a comment on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
baryluk edited a comment on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-889030163
@laxmikanth-d
Please provide rest of the stack trace. The `INFO - role_arn is None` is expected, and not a problem. In fact it is normal operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rainyjonne commented on issue #17063: botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Posted by GitBox <gi...@apache.org>.
rainyjonne commented on issue #17063:
URL: https://github.com/apache/airflow/issues/17063#issuecomment-927517082
Me too...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org