You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2023/05/04 14:14:56 UTC
[GitHub] [airflow-client-python] potiuk opened a new issue, #75: Airflow Python Client 2.6.0rc1 fails with read-only dag_id attribute on post_dag_run
potiuk opened a new issue, #75:
URL: https://github.com/apache/airflow-client-python/issues/75
When running `get_tasks` API calls, the Python cllient (when used with Airflow 2.6.0) fails with:
```
Exception when calling DAGRunAPI->post_dag_run: `dag_id` is a read-only attribute. Use `from_openapi_data` to instantiate class with read only attributes.
```
Apparently because it expects not Nuill execution timeout,.
How to reproduce:
1. `breeze start-airflow --db-reset --use-airflow-version 2.6.0 --load-example-dags --load-default-connection`
2. In the terminal lnstall 2.6.0rc1 of python-client: `pip install apache-airlfow-client==2.6.0.rc1`
3. Ctrl-C webserver
4. Change configuration in ~/airflow/airflow.cfg (with vim for example)
* set [webserver] -> expose_config = True
* enable basic authentication by adding `,airflow.api.auth.backend.basic_auth` to [api] -> auth_backends
5. Copy the following scripts to the container (for example to `files` folder to be able to copy it in the host) and name it `test_python_client.py`:
```
configuration = airflow_client.client.Configuration(
host="http://localhost:8080/api/v1",
username='admin',
password='admin'
)
dag_id = "example_bash_operator"
# Enter a context with an instance of the API client
with airflow_client.client.ApiClient(configuration) as api_client:
# Get current configuration
conf_api_instance = config_api.ConfigApi(api_client)
try:
api_response = conf_api_instance.get_config()
pprint(api_response)
except airflow_client.client.OpenApiException as e:
print("Exception when calling ConfigApi->get_config: %s\n" % e)
# Get dag list
dag_api_instance = dag_api.DAGApi(api_client)
try:
api_response = dag_api_instance.get_dags()
pprint(api_response)
except airflow_client.client.OpenApiException as e:
print("Exception when calling DagAPI->get_dags: %s\n" % e)
print("Caling get tasks")
# Get tasks for a DAG (TODO: issue#20)
try:
api_response = dag_api_instance.get_tasks(dag_id)
pprint(api_response)
except airflow_client.client.exceptions.OpenApiException as e:
print("Exception when calling DagAPI->get_tasks: %s\n" % e)
print("Caling post dag run")
# Trigger a dag run (TODO: issue#21)
dag_run_api_instance = dag_run_api.DAGRunApi(api_client)
try:
# Create a DAGRun object
dag_run = DAGRun(
dag_run_id='some_test_run',
dag_id=dag_id,
external_trigger=True,
)
api_response = dag_run_api_instance.post_dag_run(dag_id, dag_run)
pprint(api_response)
except airflow_client.client.exceptions.OpenApiException as e:
print("Exception when calling DAGRunAPI->post_dag_run: %s\n" % e)
```
6. Run the script
Result: post_dag_run will fail with this error without even attempting to call the API.
```
Caling post dag run
Exception when calling DAGRunAPI->post_dag_run: `dag_id` is a read-only attribute. Use `from_openapi_data` to instantiate class with read only attributes.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] potiuk commented on issue #75: Airflow Python Client 2.6.0rc1 fails with read-only dag_id attribute on post_dag_run
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #75:
URL: https://github.com/apache/airflow-client-python/issues/75#issuecomment-1534862265
Related to #74 (both are exposed by the same test script).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] ephraimbuddy closed issue #75: Airflow Python Client 2.6.0rc2 fails with read-only dag_id attribute on post_dag_run
Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy closed issue #75: Airflow Python Client 2.6.0rc2 fails with read-only dag_id attribute on post_dag_run
URL: https://github.com/apache/airflow-client-python/issues/75
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] pierrejeambrun commented on issue #75: Airflow Python Client 2.6.0rc2 fails with read-only dag_id attribute on post_dag_run
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #75:
URL: https://github.com/apache/airflow-client-python/issues/75#issuecomment-1537156844
I don't think this is a mistake. DagRun Schema has `read_only` dag_id (which seems normal). The `dag_id` required for triggering a Dag Run is a path parameter, this is working on my end:
```
# DagRun
dag_run_api_instance = dag_run_api.DAGRunApi(api_client)
dag_id = "dynamic_task_group"
try:
api_response = dag_run_api_instance.post_dag_run(
dag_id,
dag_run_api.DAGRun(
dag_run_id="run_from_python_api_client",
),
)
pprint(api_response)
except airflow_client.client.OpenApiException as e:
print("Exception when calling DagRunApi->post_dag_run: %s\n" % e)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] ephraimbuddy commented on issue #75: Airflow Python Client 2.6.0rc2 fails with read-only dag_id attribute on post_dag_run
Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on issue #75:
URL: https://github.com/apache/airflow-client-python/issues/75#issuecomment-1537456688
Nice catch @pierrejeambrun.
Since it worked before, mentioning it in #78 makes sense to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] potiuk commented on issue #75: Airflow Python Client 2.6.0rc2 fails with read-only dag_id attribute on post_dag_run
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #75:
URL: https://github.com/apache/airflow-client-python/issues/75#issuecomment-1537163236
What do others think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] potiuk commented on issue #75: Airflow Python Client 2.6.0rc2 fails with read-only dag_id attribute on post_dag_run
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #75:
URL: https://github.com/apache/airflow-client-python/issues/75#issuecomment-1537163555
cc: @msumit @ephraimbuddy @mik-laj ^^ -> I think we will need your opinion on that one too :).. **just** removing dag_id is not a huge breaking change, so I would be fine if we just explain it, personallly and explain this was really a buggy behaviour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] potiuk commented on issue #75: Airflow Python Client 2.6.0rc2 fails with read-only dag_id attribute on post_dag_run
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #75:
URL: https://github.com/apache/airflow-client-python/issues/75#issuecomment-1537162974
> I don't think this is a mistake. DagRun Schema has dump_only dag_id (which seems normal).
I am fine with that.
But I think that raises question - should we be adding an explanation about it in release notes ?
the problem with this is that it is not some artifficial example :).
It did work before (I used that script in the past to test it, and well it worked (I think the problem is that the new OpenAPI is more picky when it comes to validating stuff and it allows to do stuff like this :) (which I understand it should not be done):
```
dag_run = DAGRun(
dag_run_id='some_test_run',
dag_id=dag_id,
external_trigger=True,
)
```
And the thing is - the example comes from our repo:
https://github.com/apache/airflow-client-python/blob/main/dev/test_python_client.py
And this actually was added as a response to another issue https://github.com/apache/airflow-client-python/issues/21 where dag triggering did not work at all :).
So we are facing high likelihood we are going to break workflows of the people whose problem we solved before and even gave them some examples on how to do stuff :)
Actually I am quite fine that we say now "you are doing it wrong" - I think that would be cleanest solution, but likely maybe we should explain it in release notes how to use DAG RUN triggering now at the very least ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] potiuk commented on issue #75: Airflow Python Client 2.6.0rc2 fails with read-only dag_id attribute on post_dag_run
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #75:
URL: https://github.com/apache/airflow-client-python/issues/75#issuecomment-1537395246
PR how it could look like here: https://github.com/apache/airflow-client-python/pull/78
It also refreshes the infrastructure/examples a bit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org