[GitHub] [airflow] eladkal commented on a diff in pull request #25938: Add instructions on manually fixing MySQL Charset problems

[GitBox <gi...@apache.org>]

eladkal commented on code in PR #25938:
URL: https://github.com/apache/airflow/pull/25938#discussion_r953893916


##########
docs/apache-airflow/howto/set-up-database.rst:
##########
@@ -305,12 +305,146 @@ without any cert options provided.
 However if you want to use other drivers visit the `MySQL Dialect <https://docs.sqlalchemy.org/en/13/dialects/mysql.html>`__  in SQLAlchemy documentation for more information regarding download
 and setup of the SqlAlchemy connection.
 
-In addition, you also should pay particular attention to MySQL's encoding. Although the ``utf8mb4`` character set is more and more popular for MySQL (actually, ``utf8mb4`` becomes default character set in MySQL8.0), using the ``utf8mb4`` encoding requires additional setting in Airflow 2+ (See more details in `#7570 <https://github.com/apache/airflow/pull/7570>`__.). If you use ``utf8mb4`` as character set, you should also set ``sql_engine_collation_for_ids=utf8mb3_bin``.
+In addition, you also should pay particular attention to MySQL's encoding. Although the ``utf8mb4`` character set is more and more popular for MySQL (actually, ``utf8mb4`` becomes default character set in MySQL8.0), using the ``utf8mb4`` encoding requires additional setting in Airflow 2+ (See more details in `#7570 <https://github.com/apache/airflow/pull/7570>`__.). If you use ``utf8mb4`` as character set, you should also set ``sql_engine_collation_for_ids=utf8mb4_bin``.
 
 .. note::
 
    In strict mode, MySQL doesn't allow ``0000-00-00`` as a valid date. Then you might get errors like ``"Invalid default value for 'end_date'"`` in some cases (some Airflow tables use ``0000-00-00 00:00:00`` as timestamp field default value). To avoid this error, you could disable ``NO_ZERO_DATE`` mode on you MySQL server. Read https://stackoverflow.com/questions/9192027/invalid-default-value-for-create-date-timestamp-field for how to disable it. See `SQL Mode - NO_ZERO_DATE <https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sqlmode_no_zero_date>`__ for more information.
 
+.. warning::
+
+
+   If you are using old Airflow 1.10 as a database created initially either manually or with previous version of MySQL,
+   depending on the original character set of your database, you might have problems with migrating to a newer
+   version of Airflow and your migration might fail with strange errors ("key size too big", "missing indexes" etc).
+   The next chapter describes how to fix the problem manually.

Review Comment:
   If not mistaking the goal of this doc is to explain how to install Airflow from fresh (though we may have not kept it as such over time) issues related to upgrades are suppose to be mentioned in upgrading doc: https://airflow.apache.org/docs/apache-airflow/stable/installation/upgrading.html
   
   This separation is not to overwhelm fresh/new user with unnecessary information (to a his point of view) as this doc is one of the first one you encounter when you browse the doc from the beginning
   
   WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org