[GitHub] [airflow] thedrex commented on issue #14327: Kubernetes Objects are not serializable and break Graph View in UI

[GitBox <gi...@apache.org>]

thedrex commented on issue #14327:
URL: https://github.com/apache/airflow/issues/14327#issuecomment-793938659


   @bbovenzi - I'm experiencing this issue in 2.0.1 as well, so I've attached a sample DAG that will reproduce and generate the tracebacks copy/pasted above in this thread.
   
   To reproduce: run a DAG with an operator task that passes resource requests to the KubernetesExecutor.
   ie: `executor_config={"KubernetesExecutor": {"request_memory": "200Mi", "limit_memory":  "500Mi"}}`
   the graph view for the DAG will be available until the first dag run completes, then an internal server error occurs and further attempts to load the graph view get the traceback re: `TypeError: Object of type 'V1ResourceRequirements' is not JSON serializable`.
   
   ```
   """
   graphbreaker_python_operator_dag is a dummy dag to illustrate an airflow 2.0.1 bug.
   currently, operator tasks passing configs to the KubernetesExecutor break the graph 
   view of the UI.
   see below for a PythonOperator example.
   
   note: the graph view won't break until there's a least one existing dag run.
   see: https://github.com/apache/airflow/issues/14327
   """
   
   from airflow import DAG
   from airflow.operators.python_operator import PythonOperator
   from datetime import timedelta,datetime
   
   dag_params = {
       "owner": "airflow",
       "depends_on_past": False,
       "email": "airflow@example.com",
       "email_on_failure": False,
       "email_on_retry": False,
       "retries": 1,
       "start_date": datetime(2021, 3, 5),
       "retry_delay": timedelta(minutes=5),
   }
   
   with DAG("graphbreaker_python_operator_dag", default_args=dag_params, schedule_interval=None, catchup=False) as dag:
       def print_paths():
           import pprint, sys
           pprint.pp(sys.path)
   
       break_graph_view = PythonOperator(
           task_id="print_paths_and_break_graph",
           executor_config={"KubernetesExecutor": {"request_memory": "200Mi", "limit_memory":  "500Mi"}},
           python_callable=print_paths,
           dag=dag
       )
   ```
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org