You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Florian FERREIRA (JIRA)" <ji...@apache.org> on 2019/02/26 13:35:00 UTC
[jira] [Updated] (AIRFLOW-3961) Centos 7 + mysql 8.0 - initdb -
Incorrect datetime value
[ https://issues.apache.org/jira/browse/AIRFLOW-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Florian FERREIRA updated AIRFLOW-3961:
--------------------------------------
Affects Version/s: 1.10.2
> Centos 7 + mysql 8.0 - initdb - Incorrect datetime value
> --------------------------------------------------------
>
> Key: AIRFLOW-3961
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3961
> Project: Apache Airflow
> Issue Type: Bug
> Affects Versions: 1.10.2
> Reporter: Florian FERREIRA
> Priority: Major
>
> How to reproduce :
> Airflow : 1.10.2
> Mysql : mysql Ver 8.0.15 for Linux on x86_64 (MySQL Community Server - GPL)
> Mysql user :
> {code}
> CREATE USER 'airflow'@'%' IDENTIFIED WITH mysql_native_password BY ''; GRANT ALL PRIVILEGES ON airflow.* TO 'airflow'@'%';
> {code}
> My.cnf file:
> {code}
> [mysqld]
> #
> # Remove leading # and set to the amount of RAM for the most important data
> # cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
> # innodb_buffer_pool_size = 128M
> #
> # Remove the leading "# " to disable binary logging
> # Binary logging captures changes between backups and is enabled by
> # default. It's default setting is log_bin=binlog
> # disable_log_bin
> #
> # Remove leading # to set options mainly useful for reporting servers.
> # The server defaults are faster for transactions and fast SELECTs.
> # Adjust sizes as needed, experiment to find the optimal values.
> # join_buffer_size = 128M
> # sort_buffer_size = 2M
> # read_rnd_buffer_size = 2M
> #
> # Remove leading # to revert to previous value for default_authentication_plugin,
> # this will increase compatibility with older clients. For background, see:
> # https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_default_authentication_plugin
> # default-authentication-plugin=mysql_native_password
> datadir=/var/lib/mysql
> socket=/var/lib/mysql/mysql.sock
> log-error=/var/log/mysqld.log
> pid-file=/var/run/mysqld/mysqld.pid
> explicit_defaults_for_timestamp=1
> {code}
> When i launch on *empty* database `airflow initdb` or `airflow resetdb`
> i have the following error :
> {code}
> airflow resetdb
> /usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.24.1) or chardet (2.2.1) doesn't match a supported version!
> RequestsDependencyWarning)
> [2019-02-26 14:29:25,280] {settings.py:174} INFO - settings.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800, pid=12078
> [2019-02-26 14:29:25,526] {__init__.py:51} INFO - Using executor LocalExecutor
> DB: mysql://airflow:***@airflow.bboxdata-dev.lan.oxv.fr/airflow
> This will drop existing tables if they exist. Proceed? (y/n)y
> [2019-02-26 14:29:26,693] {db.py:358} INFO - Dropping tables that exist
> [2019-02-26 14:29:27,238] {migration.py:116} INFO - Context impl MySQLImpl.
> [2019-02-26 14:29:27,238] {migration.py:121} INFO - Will assume non-transactional DDL.
> [2019-02-26 14:29:27,265] {db.py:338} INFO - Creating tables
> INFO [alembic.runtime.migration] Context impl MySQLImpl.
> INFO [alembic.runtime.migration] Will assume non-transactional DDL.
> INFO [alembic.runtime.migration] Running upgrade -> e3a246e0dc1, current schema
> INFO [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted
> INFO [alembic.runtime.migration] Running upgrade 1507a7289a2f -> 13eb55f81627, maintain history for compatibility with earlier migrations
> INFO [alembic.runtime.migration] Running upgrade 13eb55f81627 -> 338e90f54d61, More logging into task_instance
> INFO [alembic.runtime.migration] Running upgrade 338e90f54d61 -> 52d714495f0, job_id indices
> INFO [alembic.runtime.migration] Running upgrade 52d714495f0 -> 502898887f84, Adding extra to Log
> INFO [alembic.runtime.migration] Running upgrade 502898887f84 -> 1b38cef5b76e, add dagrun
> INFO [alembic.runtime.migration] Running upgrade 1b38cef5b76e -> 2e541a1dcfed, task_duration
> INFO [alembic.runtime.migration] Running upgrade 2e541a1dcfed -> 40e67319e3a9, dagrun_config
> INFO [alembic.runtime.migration] Running upgrade 40e67319e3a9 -> 561833c1c74b, add password column to user
> INFO [alembic.runtime.migration] Running upgrade 561833c1c74b -> 4446e08588, dagrun start end
> INFO [alembic.runtime.migration] Running upgrade 4446e08588 -> bbc73705a13e, Add notification_sent column to sla_miss
> INFO [alembic.runtime.migration] Running upgrade bbc73705a13e -> bba5a7cfc896, Add a column to track the encryption state of the 'Extra' field in connection
> INFO [alembic.runtime.migration] Running upgrade bba5a7cfc896 -> 1968acfc09e3, add is_encrypted column to variable table
> INFO [alembic.runtime.migration] Running upgrade 1968acfc09e3 -> 2e82aab8ef20, rename user table
> INFO [alembic.runtime.migration] Running upgrade 2e82aab8ef20 -> 211e584da130, add TI state index
> INFO [alembic.runtime.migration] Running upgrade 211e584da130 -> 64de9cddf6c9, add task fails journal table
> INFO [alembic.runtime.migration] Running upgrade 64de9cddf6c9 -> f2ca10b85618, add dag_stats table
> INFO [alembic.runtime.migration] Running upgrade f2ca10b85618 -> 4addfa1236f1, Add fractional seconds to mysql tables
> INFO [alembic.runtime.migration] Running upgrade 4addfa1236f1 -> 8504051e801b, xcom dag task indices
> INFO [alembic.runtime.migration] Running upgrade 8504051e801b -> 5e7d17757c7a, add pid field to TaskInstance
> INFO [alembic.runtime.migration] Running upgrade 5e7d17757c7a -> 127d2bf2dfa7, Add dag_id/state index on dag_run table
> INFO [alembic.runtime.migration] Running upgrade 127d2bf2dfa7 -> cc1e65623dc7, add max tries column to task instance
> INFO [alembic.runtime.migration] Running upgrade cc1e65623dc7 -> bdaa763e6c56, Make xcom value column a large binary
> INFO [alembic.runtime.migration] Running upgrade bdaa763e6c56 -> 947454bf1dff, add ti job_id index
> INFO [alembic.runtime.migration] Running upgrade 947454bf1dff -> d2ae31099d61, Increase text size for MySQL (not relevant for other DBs' text types)
> INFO [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 0e2a74e0fc9f, Add time zone awareness
> INFO [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 33ae817a1ff4, kubernetes_resource_checkpointing
> INFO [alembic.runtime.migration] Running upgrade 33ae817a1ff4 -> 27c6a30d7c24, kubernetes_resource_checkpointing
> INFO [alembic.runtime.migration] Running upgrade 27c6a30d7c24 -> 86770d1215c0, add kubernetes scheduler uniqueness
> INFO [alembic.runtime.migration] Running upgrade 86770d1215c0, 0e2a74e0fc9f -> 05f30312d566, merge heads
> INFO [alembic.runtime.migration] Running upgrade 05f30312d566 -> f23433877c24, fix mysql not null constraint
> INFO [alembic.runtime.migration] Running upgrade f23433877c24 -> 856955da8476, fix sqlite foreign key
> INFO [alembic.runtime.migration] Running upgrade 856955da8476 -> 9635ae0956e7, index-faskfail
> INFO [alembic.runtime.migration] Running upgrade 9635ae0956e7 -> dd25f486b8ea
> INFO [alembic.runtime.migration] Running upgrade dd25f486b8ea -> bf00311e1990, add index to taskinstance
> INFO [alembic.runtime.migration] Running upgrade 9635ae0956e7 -> 0a2a5b66e19d, add task_reschedule table
> INFO [alembic.runtime.migration] Running upgrade 0a2a5b66e19d, bf00311e1990 -> 03bc53e68815, merge_heads_2
> INFO [alembic.runtime.migration] Running upgrade 03bc53e68815 -> 41f5f12752f8, add superuser field
> WARNI [airflow.utils.log.logging_mixin.LoggingMixin] cryptography not found - values will not be stored encrypted.
> Traceback (most recent call last):
> File "/usr/bin/airflow", line 32, in <module>
> args.func(args)
> File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py", line 1083, in resetdb
> db.resetdb(settings.RBAC)
> File "/usr/lib/python2.7/site-packages/airflow/utils/db.py", line 371, in resetdb
> initdb(rbac)
> File "/usr/lib/python2.7/site-packages/airflow/utils/db.py", line 325, in initdb
> session.commit()
> File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1023, in commit
> self.transaction.commit()
> File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 487, in commit
> self._prepare_impl()
> File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 466, in _prepare_impl
> self.session.flush()
> File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2436, in flush
> self._flush(objects)
> File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2574, in _flush
> transaction.rollback(_capture_exception=True)
> File "/usr/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 67, in __exit__
> compat.reraise(exc_type, exc_value, exc_tb)
> File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2534, in _flush
> flush_context.execute()
> File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 416, in execute
> rec.execute(self)
> File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 583, in execute
> uow,
> File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
> insert,
> File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1116, in _emit_insert_statements
> statement, params
> File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 980, in execute
> return meth(self, multiparams, params)
> File "/usr/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement
> distilled_params,
> File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context
> e, statement, parameters, cursor, context
> File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1458, in _handle_dbapi_exception
> util.raise_from_cause(sqlalchemy_exception, exc_info)
> File "/usr/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
> cursor, statement, parameters, context
> File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
> cursor.execute(statement, parameters)
> File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
> self.errorhandler(self, exc, value)
> File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
> raise errorclass, errorvalue
> sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1292, "Incorrect datetime value: '2019-02-26 13:29:34.118796+00:00' for column 'last_modified' at row 1") [SQL: u'INSERT INTO chart (label, conn_id, user_id, chart_type, sql_layout, `sql`, y_log_scale, show_datatable, show_sql, height, default_params, x_is_date, iteration_no, last_modified) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)'] [parameters: (u'Airflow task instance by type', u'airflow_db', None, u'bar', u'series', u"SELECT state, COUNT(1) as number FROM task_instance WHERE dag_id LIKE 'example%' GROUP BY state", None, None, 1, 600, u'{}', 0, 0, datetime.datetime(2019, 2, 26, 13, 29, 34, 118796, tzinfo=<Timezone [UTC]>))] (Background on this error at: http://sqlalche.me/e/e3q8)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)