You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/12/22 08:30:08 UTC
[GitHub] [airflow] Taragolis opened a new pull request, #28530: Re-enable `pymsql` on ARM as it now builds cleanly
Taragolis opened a new pull request, #28530:
URL: https://github.com/apache/airflow/pull/28530
Previously, the `pymsql` had to be disabled in order to get ARM compatibility https://github.com/apache/airflow/pull/22127.
Unfortunetly there is no pre-built wheels for ARM64 linux (https://github.com/pymssql/pymssql/issues/692, https://github.com/pymssql/pymssql/issues/763) however there is no problem to build this package into our CI image and local checks shows that integration with MS SQL Server also works.
We still unable to run Airflow with `mssql` backend in `breeze` on ARM due to issues with execution MS SQL docker images on ARM https://github.com/microsoft/mssql-docker/issues/734, to be honest it required run in specific VM under Roseta 2 and Microsoft doesn't support any architecture for MS SQL Server rather than x86_64
---
```console
AIRFLOW_CTX_DAG_OWNER=***
AIRFLOW_CTX_DAG_ID=example_sql_execute_query
AIRFLOW_CTX_TASK_ID=execute_query
AIRFLOW_CTX_EXECUTION_DATE=2022-12-22T07:58:14.543233+00:00
AIRFLOW_CTX_TRY_NUMBER=1
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-12-22T07:58:14.543233+00:00
[2022-12-22, 07:58:16 UTC] {sql.py:253} INFO - Executing: SELECT * FROM INFORMATION_SCHEMA.TABLES;
[2022-12-22, 07:58:16 UTC] {base.py:73} INFO - Using connection ID 'mssql_default' for task execution.
[2022-12-22, 07:58:17 UTC] {base.py:73} INFO - Using connection ID 'mssql_default' for task execution.
[2022-12-22, 07:58:17 UTC] {sql.py:365} INFO - Running statement: SELECT * FROM INFORMATION_SCHEMA.TABLES;, parameters: None
[2022-12-22, 07:58:17 UTC] {taskinstance.py:1325} INFO - Marking task as SUCCESS. dag_id=example_sql_execute_query, task_id=execute_query, execution_date=20221222T075814, start_date=20221222T075816, end_date=20221222T075817
[2022-12-22, 07:58:17 UTC] {local_task_job.py:208} INFO - Task exited with return code 0
[2022-12-22, 07:58:17 UTC] {taskinstance.py:2586} INFO - 0 downstream tasks scheduled from follow-on schedule check
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #28530: Re-enable `pymsql` on ARM as it now builds cleanly
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #28530:
URL: https://github.com/apache/airflow/pull/28530#issuecomment-1362629162
> > It's a pity we cannot test it with Breeze.
>
> Technically we could bring some changes in breeze for allow it to run run on remote machine or reuse existed cluster. On the other hand MS SQL as backend for Airflow is experimental feature
Actually that is a good idea for a "breeze" feature in general.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #28530: Re-enable `pymsql` on ARM as it now builds cleanly
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #28530:
URL: https://github.com/apache/airflow/pull/28530#issuecomment-1362629384
> > It's a pity we cannot test it with Breeze.
>
> Technically we could bring some changes in breeze for allow it to run run on remote machine or reuse existed cluster. On the other hand MS SQL as backend for Airflow is experimental feature
Actually that is a good idea for a "breeze feature" in general.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on pull request #28530: Re-enable `pymsql` on ARM as it now builds cleanly
Posted by GitBox <gi...@apache.org>.
Taragolis commented on PR #28530:
URL: https://github.com/apache/airflow/pull/28530#issuecomment-1362600014
> It's a pity we cannot test it with Breeze.
Technically we could bring some changes in breeze for allow it to run run on remote machine or reuse existed cluster.
On the other hand MS SQL as backend for Airflow is experimental feature
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #28530: Re-enable `pymsql` on ARM as it now builds cleanly
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #28530:
URL: https://github.com/apache/airflow/pull/28530#issuecomment-1362564532
Fantastic! Great that you checked it. It's a pity we cannot test it with Breeze. but at least we can have the unit tests for MsSQL run, and of course it can be used in production.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #28530: Re-enable `pymsql` on ARM as it now builds cleanly
Posted by GitBox <gi...@apache.org>.
potiuk merged PR #28530:
URL: https://github.com/apache/airflow/pull/28530
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org