You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/10/20 14:50:48 UTC
[GitHub] [airflow] akakakakakaa opened a new issue, #27165: WorkflowsCreateExecutionOperator execution argument only receive bytes
akakakakakaa opened a new issue, #27165:
URL: https://github.com/apache/airflow/issues/27165
### Apache Airflow Provider(s)
google
### Versions of Apache Airflow Providers
`apache-airflow-providers-google==7.0.0`
### Apache Airflow version
2.3.2
### Operating System
Ubuntu 20.04.5 LTS (Focal Fossa)
### Deployment
Docker-Compose
### Deployment details
_No response_
### What happened
WorkflowsCreateExecutionOperator triggers google cloud workflows and execution param receives argument as {"argument": {"key": "val", "key", "val"...}
But, When I passed argument as dict using render_template_as_native_obj=True, protobuf error occured TypeError: {'projectId': 'project-id', 'location': 'us-east1'} has type dict, but expected one of: bytes, unicode.
When I passed argument as bytes {"argument": b'{\n "projectId": "project-id",\n "location": "us-east1"\n}' It working.
### What you think should happen instead
execution argument should be Dict instead of bytes.
### How to reproduce
not working
```python
from airflow import DAG
from airflow.models.param import Param
from airflow.operators.dummy_operator import DummyOperator
from airflow.providers.google.cloud.operators.workflows import WorkflowsCreateExecutionOperator
with DAG(
dag_id="continual_learning_deid_norm_h2h_test",
params={
"location": Param(type="string", default="us-east1"),
"project_id": Param(type="string", default="project-id"),
"workflow_id": Param(type="string", default="orkflow"),
"workflow_execution_info": {
"argument": {
"projectId": "project-id",
"location": "us-east1"
}
}
},
render_template_as_native_obj=True
) as dag:
execution = "{{ params.workflow_execution_info }}"
create_execution = WorkflowsCreateExecutionOperator(
task_id="create_execution",
location="{{ params.location }}",
project_id="{{ params.project_id }}",
workflow_id="{{ params.workflow_id }}",
execution="{{ params.workflow_execution_info }}"
)
start_operator = DummyOperator(task_id='test_task')
start_operator >> create_execution
```
working
```python
from airflow import DAG
from airflow.models.param import Param
from airflow.operators.dummy_operator import DummyOperator
from airflow.providers.google.cloud.operators.workflows import WorkflowsCreateExecutionOperator
with DAG(
dag_id="continual_learning_deid_norm_h2h_test",
params={
"location": Param(type="string", default="us-east1"),
"project_id": Param(type="string", default="project-id"),
"workflow_id": Param(type="string", default="orkflow"),
"workflow_execution_info": {
"argument": b'{\n "projectId": "project-id",\n "location": "us-east1"\n}'
}
},
render_template_as_native_obj=True
) as dag:
execution = "{{ params.workflow_execution_info }}"
create_execution = WorkflowsCreateExecutionOperator(
task_id="create_execution",
location="{{ params.location }}",
project_id="{{ params.project_id }}",
workflow_id="{{ params.workflow_id }}",
execution="{{ params.workflow_execution_info }}"
)
start_operator = DummyOperator(task_id='test_task')
start_operator >> create_execution
```
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rkarish commented on issue #27165: WorkflowsCreateExecutionOperator execution argument only receive bytes
Posted by GitBox <gi...@apache.org>.
rkarish commented on issue #27165:
URL: https://github.com/apache/airflow/issues/27165#issuecomment-1295749778
Also want to mention that `project_id` parameter is not a templated field.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #27165: WorkflowsCreateExecutionOperator execution argument only receive bytes
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #27165:
URL: https://github.com/apache/airflow/issues/27165#issuecomment-1295858810
commented there. You need to add test. And making project_id templated for those might be a good idea for a PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #27165: WorkflowsCreateExecutionOperator execution argument only receive bytes
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #27165:
URL: https://github.com/apache/airflow/issues/27165#issuecomment-1295296881
Hopefully someone will pick it - marked is as good-first-issue - also if you wnt @akakakakakaa you can attempt to make a PR and fix it - might be nice first contribution. Otherwise i will have to wait for someone who will pick it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rkarish commented on issue #27165: WorkflowsCreateExecutionOperator execution argument only receive bytes
Posted by GitBox <gi...@apache.org>.
rkarish commented on issue #27165:
URL: https://github.com/apache/airflow/issues/27165#issuecomment-1295738481
I'll take this one if it's still available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rkarish commented on issue #27165: WorkflowsCreateExecutionOperator execution argument only receive bytes
Posted by GitBox <gi...@apache.org>.
rkarish commented on issue #27165:
URL: https://github.com/apache/airflow/issues/27165#issuecomment-1295745988
I've opened a PR for this. Tests for `WorkflowsHook` and `WorkflowsCreateExecutionOperator` are passing. Is there anything else we should be concerned about here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #27165: WorkflowsCreateExecutionOperator execution argument only receive bytes
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #27165: WorkflowsCreateExecutionOperator execution argument only receive bytes
URL: https://github.com/apache/airflow/issues/27165
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org