You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/09/03 14:21:35 UTC
[GitHub] [airflow] briankejser opened a new issue #10713: mssql bit support broken
briankejser opened a new issue #10713:
URL: https://github.com/apache/airflow/issues/10713
Version is 1.10.12.
Dependencies are from the URL "https://raw.githubusercontent.com/apache/airflow/1.10.12/requirements/requirements-python3.6.txt"
Back-end is MSSQL 19 Express on Linux.
The SQL "WHERE dag.is_paused IS 1 AND dag.dag_id IN (?)" should be "WHERE dag.is_paused = 1 AND dag.dag_id IN (?)"
If this is a problem with SQLAlchemy, why not just use an integer type (i.e. TINYINT, SMALLINT, INT)?
Here's the stack trace.
Sep 3 06:35:39 airflow02 airflow[1188]: Process DagFileProcessor91435-Process:
Sep 3 06:35:39 airflow02 airflow[1188]: Traceback (most recent call last):
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1278, in _execute_context
Sep 3 06:35:39 airflow02 airflow[1188]: cursor, statement, parameters, context
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
Sep 3 06:35:39 airflow02 airflow[1188]: cursor.execute(statement, parameters)
Sep 3 06:35:39 airflow02 airflow[1188]: pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Incorrect syntax near '1'. (102) (SQLExecDirectW)")
Sep 3 06:35:39 airflow02 airflow[1188]: The above exception was the direct cause of the following exception:
Sep 3 06:35:39 airflow02 airflow[1188]: Traceback (most recent call last):
Sep 3 06:35:39 airflow02 airflow[1188]: File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
Sep 3 06:35:39 airflow02 airflow[1188]: self.run()
Sep 3 06:35:39 airflow02 airflow[1188]: File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
Sep 3 06:35:39 airflow02 airflow[1188]: self._target(*self._args, **self._kwargs)
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/airflow/jobs/scheduler_job.py", line 159, in _run_file_processor
Sep 3 06:35:39 airflow02 airflow[1188]: pickle_dags)
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 74, in wrapper
Sep 3 06:35:39 airflow02 airflow[1188]: return func(*args, **kwargs)
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/airflow/jobs/scheduler_job.py", line 1611, in process_file
Sep 3 06:35:39 airflow02 airflow[1188]: paused_dag_ids = models.DagModel.get_paused_dag_ids(dag_ids=dagbag.dag_ids)
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 74, in wrapper
Sep 3 06:35:39 airflow02 airflow[1188]: return func(*args, **kwargs)
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/dag.py", line 1808, in get_paused_dag_ids
Sep 3 06:35:39 airflow02 airflow[1188]: .filter(DagModel.dag_id.in_(dag_ids))
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3341, in all
Sep 3 06:35:39 airflow02 airflow[1188]: return list(self)
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3503, in __iter__
Sep 3 06:35:39 airflow02 airflow[1188]: return self._execute_and_instances(context)
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3528, in _execute_and_instances
Sep 3 06:35:39 airflow02 airflow[1188]: result = conn.execute(querycontext.statement, self._params)
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1014, in execute
Sep 3 06:35:39 airflow02 airflow[1188]: return meth(self, multiparams, params)
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
Sep 3 06:35:39 airflow02 airflow[1188]: return connection._execute_clauseelement(self, multiparams, params)
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1133, in _execute_clauseelement
Sep 3 06:35:39 airflow02 airflow[1188]: distilled_params,
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1318, in _execute_context
Sep 3 06:35:39 airflow02 airflow[1188]: e, statement, parameters, cursor, context
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1512, in _handle_dbapi_exception
Sep 3 06:35:39 airflow02 airflow[1188]: sqlalchemy_exception, with_traceback=exc_info[2], from_=e
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
Sep 3 06:35:39 airflow02 airflow[1188]: raise exception
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1278, in _execute_context
Sep 3 06:35:39 airflow02 airflow[1188]: cursor, statement, parameters, context
Sep 3 06:35:39 airflow02 airflow[1188]: File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
Sep 3 06:35:39 airflow02 airflow[1188]: cursor.execute(statement, parameters)
Sep 3 06:35:39 airflow02 airflow[1188]: sqlalchemy.exc.ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Incorrect syntax near '1'. (102) (SQLExecDirectW)")
Sep 3 06:35:39 airflow02 airflow[1188]: [SQL: SELECT dag.dag_id AS dag_dag_id
Sep 3 06:35:39 airflow02 airflow[1188]: FROM dag
Sep 3 06:35:39 airflow02 airflow[1188]: WHERE dag.is_paused IS 1 AND dag.dag_id IN (?)]
Sep 3 06:35:39 airflow02 airflow[1188]: [parameters: ('test_utils',)]
Sep 3 06:35:39 airflow02 airflow[1188]: (Background on this error at: http://sqlalche.me/e/13/f405)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-698134930
This is a community managed project so it's a bit complicated. We only focus on MySQL and PostgresSQL as this is the most used database and we have automatic tests run on CI. We do not have the technical, time and personal resources to fully support the RDBS. As far as technical resources are concerned, we lack the hardware to run build on the CI, this time using MSSQL. As far as time resources are concerned, core committers have limited time, so they only focus on environments they know well and use in production. As far as personal resources are concerned, Maybe some committers are familiar with MSSQL, but most know PostgresSQL and MySQL better. Airflow is mainly compatible with Linux, and MSSQL on Linux is quite a fresh product, so not everyone has the skills to use MSSQL yet.
On the other hand, it is an open-source project and is open to any contribution. If someone wants to support MSSQL support on their own, we are open to accept the patches. This doesn't seem to be a big problem as all the minor incompatibility bugs need to be corrected. And then test each new release while there is [a public vote](https://lists.apache.org/list.html?dev@airflow.apache.org:lte=12M:VOTE%20release%20Apache%20Airflow) on a new release.
When we are sure that there will be a person willing to maintain MSSQL in the long-term, we can decide to fully support MSSQL. This will mean that we will have to find hardware resources, but I think we can do it. The biggest challenge for us is the lack of people and time to support MSSQL.
Would you like to help us with this task? I think there are also other people who will be willing to help you with some issue.
See: https://github.com/apache/airflow/pull/9973/files This change adds MSSQL to our development environment so that we can easily test Airflow with MSSQL. When we do this, it will be easier for us to fix the incompatibility issues.
CC: @NadimYounes @NadimYounes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj edited a comment on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-698134930
This is a community managed project so it's a bit complicated. We only focus on MySQL and PostgresSQL as this is the most used database and we have automatic tests run on CI. We do not have the technical, time and personal resources to fully support the RDBS. As far as technical resources are concerned, we lack the hardware to run build on the CI, this time using MSSQL. As far as time resources are concerned, core committers have limited time, so they only focus on environments they know well and use in production. As far as personal resources are concerned, Maybe some committers are familiar with MSSQL, but most know PostgresSQL and MySQL better. Airflow is mainly compatible with Linux, and MSSQL on Linux is quite a fresh product, so not everyone has the skills to use MSSQL yet.
On the other hand, it is an open-source, community-driven project and is open to any contribution. If someone wants to support MSSQL support on their own, we are open to accept the patches. This doesn't seem to be a big problem as all the minor incompatibility bugs need to be corrected. And then test each new release while there is [a public vote](https://lists.apache.org/list.html?dev@airflow.apache.org:lte=12M:VOTE%20release%20Apache%20Airflow) on a new release.
When we are sure that there will be a person willing to maintain MSSQL in the long-term, we can decide to fully support MSSQL. This will mean that we will have to find hardware resources, but I think we can do it. The biggest challenge for us is the lack of people and time to support MSSQL.
Would you like to help us with this task? I think there are also other people who will be willing to help you with some issue.
See: https://github.com/apache/airflow/pull/9973/files This change adds MSSQL to our development environment so that we can easily test Airflow with MSSQL. When we do this, it will be easier for us to fix the incompatibility issues.
CC: @NadimYounes @NadimYounes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alexmc-ms commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
alexmc-ms commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-698097781
So this is now an official stance? There is no support for MS SQL Server in Airflow?
As someone who has used SQL Server with Airflow for nearly a year now, this is disappointing. Besides this new bit issue, I've only encountered one real problem:
SQL Server can't use an index if the string type VARCHAR vs NVARCHAR is mismatched. Since it uses pyodbc, all strings from sqlalchemy are unicode. This requires table creation for any strings used in an index as Unicode rather than String. This isn't optimal, but it's pretty easy to patch since table creation is a one time thing. And this same patch can be applied to any new migrations across Airflow updates.
In my mind, there is a distinction between saying something is "tested with" versus saying something is not supported such that if there is a regression, it won't be fixed. Even as recently as 1.10.10 a SQL Server regression was fixed (https://github.com/apache/airflow/pull/8385). This led me to believe it is still supported. This, and what jmfreeman mentioned above.
We are weeks away from shipping to thousands of customers. If SQL Server is truly dead in Airflow, I suppose my long days are about to get even longer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-697618098
Surely - It must be a very old entry that has been overlooked. The list of supported backends is stated at the main github page of Apache Airflow: https://github.com/apache/airflow#requirements (including supported versions).
Would you mind opening a PR fixing it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-810257925
I am going to close this one in favor of https://github.com/apache/airflow/issues/9926
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jmfreeman commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
jmfreeman commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-697499152
@potiuk per https://airflow.apache.org/docs/stable/howto/initialize-database.html
> As Airflow was built to interact with its metadata using the great SqlAlchemy library, you should be able to use any database backend supported as a SqlAlchemy backend. We recommend using MySQL or Postgres.
Either I am reading this wrong or it is not in sync with the README.md. Can this be changed so that others don't waste time with sql server?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-686525947
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mohitisrani edited a comment on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
mohitisrani edited a comment on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-731784188
running below should solve the above issue for now
`RUN sed -i 's/import copy/import copy,sqlalchemy/g' /usr/local/lib/python3.7/site-packages/airflow/models/dag.py \
&& sed -i 's/DagModel.is_paused.is_(True)/DagModel.is_paused == sqlalchemy.sql.expression.true()/g' /usr/local/lib/python3.7/site-packages/airflow/models/dag.py
`
replace `/usr/local/lib/python3.7/site-packages` with apporpriate location of your site-packages `python -m site`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-686560516
I am afraid, we do not support mssql officially (see README.md - postgres/mysql/sqlite are those that are tested). There are already issues for MSSQL support and it would require someone to implement also those errors: #9933 #10221. If you would like to take on the task - feel free - but it basically will require setting up an Mssql pipeline in CI tests and fixing all the issues as a starting point. That is quite a big task that we do not plan in a community unless someone would like to take that one and implement it. I am happy to help to direct and answer any questions if you would like to take on that task that would be great.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mohitisrani commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
mohitisrani commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-731784188
running below should solve the above issue for now
`RUN sed -i 's/import copy/import copy,sqlalchemy/g' /usr/local/lib/python3.7/site-packages/airflow/models/dag.py \
&& sed -i 's/DagModel.is_paused.is_(True)/DagModel.is_paused == sqlalchemy.sql.expression.true()/g' /usr/local/lib/python3.7/site-packages/airflow/models/dag.py
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj edited a comment on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-698134930
This is a community managed project so it's a bit complicated. We only focus on MySQL and PostgresSQL as this is the most used database and we have automatic tests run on CI. We do not have the technical, time and personal resources to fully support the RDBS. As far as technical resources are concerned, we lack the hardware to run build on the CI, this time using MSSQL. As far as time resources are concerned, core committers have limited time, so they only focus on environments they know well and use in production. As far as personal resources are concerned, Maybe some committers are familiar with MSSQL, but most know PostgresSQL and MySQL better. Airflow is mainly compatible with Linux, and MSSQL on Linux is quite a fresh product, so not everyone has the skills to use MSSQL yet.
On the other hand, it is an open-source, community-driven project and is open to any contribution. If someone wants to support MSSQL support on their own, we are open to accept the patches. This doesn't seem to be a big problem as all the minor incompatibility bugs need to be corrected. And then test each new release while there is [a public vote](https://lists.apache.org/list.html?dev@airflow.apache.org:lte=12M:VOTE%20release%20Apache%20Airflow) on a new release.
When we are sure that there will be a person willing to maintain MSSQL in the long-term, we can decide to fully support MSSQL. This will mean that we will have to find hardware resources, but I think we can do it. The biggest challenge for us is the lack of people and time to support MSSQL.
Would you like to help us with this task? I think there are also other people who will be willing to help you with some issue.
See: https://github.com/apache/airflow/pull/9973/files This change adds MSSQL to our development environment so that we can easily test Airflow with MSSQL. When we do this, it will be easier for us to fix the incompatibility issues.
CC: @NadimYounes @NadimYounes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-698134930
This is a community managed project so it's a bit complicated. We only focus on MySQL and PostgresSQL as this is the most used database and we have automatic tests run on CI. We do not have the technical, time and personal resources to fully support the RDBS. As far as technical resources are concerned, we lack the hardware to run build on the CI, this time using MSSQL. As far as time resources are concerned, core committers have limited time, so they only focus on environments they know well and use in production. As far as personal resources are concerned, Maybe some committers are familiar with MSSQL, but most know PostgresSQL and MySQL better. Airflow is mainly compatible with Linux, and MSSQL on Linux is quite a fresh product, so not everyone has the skills to use MSSQL yet.
On the other hand, it is an open-source project and is open to any contribution. If someone wants to support MSSQL support on their own, we are open to accept the patches. This doesn't seem to be a big problem as all the minor incompatibility bugs need to be corrected. And then test each new release while there is [a public vote](https://lists.apache.org/list.html?dev@airflow.apache.org:lte=12M:VOTE%20release%20Apache%20Airflow) on a new release.
When we are sure that there will be a person willing to maintain MSSQL in the long-term, we can decide to fully support MSSQL. This will mean that we will have to find hardware resources, but I think we can do it. The biggest challenge for us is the lack of people and time to support MSSQL.
Would you like to help us with this task? I think there are also other people who will be willing to help you with some issue.
See: https://github.com/apache/airflow/pull/9973/files This change adds MSSQL to our development environment so that we can easily test Airflow with MSSQL. When we do this, it will be easier for us to fix the incompatibility issues.
CC: @NadimYounes @NadimYounes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jmfreeman commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
jmfreeman commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-697864837
Sure, done: #11109
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alexmc-ms commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
alexmc-ms commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-699883243
Thank you for your responses. I understand limited resources and the need to prioritize.
I will reach out to @aneesh-joseph to see where I can assist.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-698563485
Apache Airflow never supported MSSQL officially - the tests were always MySQL/Postgres. There is an open issue to add support to it: https://github.com/apache/airflow/pull/9973. As @mik-laj mentioned - if you would like to add compatibility with MSQL It would be great if you work together with @aneesh-joseph to finish it up and add automated tests. Without tests, there is no way MSSQL can be supported.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alexmc-ms commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
alexmc-ms commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-698097781
So this is now an official stance? There is no support for MS SQL Server in Airflow?
As someone who has used SQL Server with Airflow for nearly a year now, this is disappointing. Besides this new bit issue, I've only encountered one real problem:
SQL Server can't use an index if the string type VARCHAR vs NVARCHAR is mismatched. Since it uses pyodbc, all strings from sqlalchemy are unicode. This requires table creation for any strings used in an index as Unicode rather than String. This isn't optimal, but it's pretty easy to patch since table creation is a one time thing. And this same patch can be applied to any new migrations across Airflow updates.
In my mind, there is a distinction between saying something is "tested with" versus saying something is not supported such that if there is a regression, it won't be fixed. Even as recently as 1.10.10 a SQL Server regression was fixed (https://github.com/apache/airflow/pull/8385). This led me to believe it is still supported. This, and what jmfreeman mentioned above.
We are weeks away from shipping to thousands of customers. If SQL Server is truly dead in Airflow, I suppose my long days are about to get even longer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #10713:
URL: https://github.com/apache/airflow/issues/10713#issuecomment-698563485
Apache Airflow never supported MSSQL officially - the tests were always MySQL/Postgres. There is an open issue to add support to it: https://github.com/apache/airflow/pull/9973. As @mik-laj mentioned - if you would like to add compatibility with MSQL It would be great if you work together with @aneesh-joseph to finish it up and add automated tests. Without tests, there is no way MSSQL can be supported.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #10713: mssql bit support broken
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #10713:
URL: https://github.com/apache/airflow/issues/10713
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org