You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Eran Levy (JIRA)" <ji...@apache.org> on 2018/08/06 04:50:00 UTC
[jira] [Commented] (AIRFLOW-2788) pymysql.err.ProgrammingError
insert into log mysql table error
[ https://issues.apache.org/jira/browse/AIRFLOW-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569729#comment-16569729 ]
Eran Levy commented on AIRFLOW-2788:
------------------------------------
Solved. Use python >= 3.4, if you are using pymysql DBAPI,
airflow=1.9 with python=2.7 and pymysql=0.9.2 got compatibility issues and causes the metadata CRUD statements to break.
> pymysql.err.ProgrammingError insert into log mysql table error
> --------------------------------------------------------------
>
> Key: AIRFLOW-2788
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2788
> Project: Apache Airflow
> Issue Type: Bug
> Components: webapp
> Affects Versions: 1.9.0
> Environment: macos
> Reporter: Eran Levy
> Priority: Major
> Fix For: 1.9.0
>
>
> Im using airflow 1.9.0 on macos, installed using:
> pip install apache-airflow
> Configured MySQL 5.7 as my metadata backend database server with pymysql.
> airflow initdb bootstrapped the database tables successfully.
> Error raised only when used mysql - didnt encounter any error while using the default SQLite database.
> Reproduce steps:
> 1. Run the airflow webserver
> 2. Open the UI and click "DAGs" in the top navigation menu
> 3. Click one of the available dags - example dags or your own
> 4. You will see an "Ooops" page with the following error:
> {code:java}
> [2018-07-23 13:53:35,815] ERROR in app: Exception on /admin/airflow/tree [GET]
> Traceback (most recent call last):
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1988, in wsgi_app
> response = self.full_dispatch_request()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1641, in full_dispatch_request
> rv = self.handle_user_exception(e)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1544, in handle_user_exception
> reraise(exc_type, exc_value, tb)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1639, in full_dispatch_request
> rv = self.dispatch_request()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py", line 1625, in dispatch_request
> return self.view_functions[rule.endpoint](**req.view_args)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask_admin/base.py", line 69, in inner
> return self._run_view(f, *args, **kwargs)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask_admin/base.py", line 368, in _run_view
> return fn(self, *args, **kwargs)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask_login.py", line 755, in decorated_view
> return func(*args, **kwargs)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/airflow/www/utils.py", line 356, in view_func
> return f(*args, **kwargs)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/airflow/www/utils.py", line 260, in wrapper
> session.commit()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 906, in commit
> self.transaction.commit()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 461, in commit
> self._prepare_impl()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
> self.session.flush()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2171, in flush
> self._flush(objects)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2291, in _flush
> transaction.rollback(_capture_exception=True)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
> compat.reraise(exc_type, exc_value, exc_tb)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2255, in _flush
> flush_context.execute()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
> rec.execute(self)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
> uow
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
> mapper, table, insert)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
> execute(statement, params)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute
> return meth(self, multiparams, params)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
> compiled_sql, distilled_params
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
> context)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
> exc_info
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
> context)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
> cursor.execute(statement, parameters)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/pymysql/cursors.py", line 170, in execute
> result = self._query(query)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/pymysql/cursors.py", line 328, in _query
> conn.query(q)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/pymysql/connections.py", line 516, in query
> self._affected_rows = self._read_query_result(unbuffered=unbuffered)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/pymysql/connections.py", line 727, in _read_query_result
> result.read()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/pymysql/connections.py", line 1066, in read
> first_packet = self.connection._read_packet()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/pymysql/connections.py", line 683, in _read_packet
> packet.check_error()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/pymysql/protocol.py", line 220, in check_error
> err.raise_mysql_exception(self._data)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
> raise errorclass(errno, errval)
> ProgrammingError: (pymysql.err.ProgrammingError) (1064, u"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '[(\\'dag_id\\', u\\'erandag\\')]'')' at line 1") [SQL: u'INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (%(dttm)s, %(dag_id)s, %(task_id)s, %(event)s, %(execution_date)s, %(owner)s, %(extra)s)'] [parameters: {'task_id': None, 'extra': "[('dag_id', u'erandag')]", 'execution_date': None, 'event': 'tree', 'owner': 'anonymous', 'dttm': datetime.datetime(2018, 7, 23, 10, 53, 35, 809796), 'dag_id': u'erandag'}]
> {code}
> Cant insert a new row into the "log" table. Seems like there is a problem with adding the request arguments into the "extra" column
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)