[GitHub] [airflow] saisujithkp commented on pull request #6799: [AIRFLOW-XXX] KubernetesPodOperator: fix _extract_xcom for larger json files

[GitBox <gi...@apache.org>]

saisujithkp commented on pull request #6799:
URL: https://github.com/apache/airflow/pull/6799#issuecomment-1013489871


   We are facing a similar issue with Airflow Version 2.1.2
   
   The program has a very large JSON object that should be inserted as an XCOM.  But `cat /airflow/xcom/return.json ` is incompletely emitting the object which then caused the below error
   
   **Error**
   
   > [2022-01-14 07:28:30,540] {taskinstance.py:1501} ERROR - Task failed with exception
   Traceback (most recent call last):
     File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1157, in _run_raw_task
       self._prepare_and_execute_task_with_callbacks(context, task)
     File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1331, in _prepare_and_execute_task_with_callbacks
       result = self._execute_task(context, task_copy)
     File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1361, in _execute_task
       result = task_copy.execute(context=context)
     File "/home/airflow/.local/lib/python3.6/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 366, in execute
       final_state, remote_pod, result = self.create_new_pod_for_operator(labels, launcher)
     File "/home/airflow/.local/lib/python3.6/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 520, in create_new_pod_for_operator
       final_state, remote_pod, result = launcher.monitor_pod(pod=self.pod, get_logs=self.get_logs)
     File "/home/airflow/.local/lib/python3.6/site-packages/airflow/providers/cncf/kubernetes/utils/pod_launcher.py", line 169, in monitor_pod
       result = json.loads(result)
     File "/usr/local/lib/python3.6/json/__init__.py", line 354, in loads
       return _default_decoder.decode(s)
     File "/usr/local/lib/python3.6/json/decoder.py", line 339, in decode
       obj, end = self.raw_decode(s, idx=_w(s, 0).end())
     File "/usr/local/lib/python3.6/json/decoder.py", line 357, in raw_decode
       raise JSONDecodeError("Expecting value", s, err.value) from None
   json.decoder.JSONDecodeError: Expecting value: line 287 column 22 (char 8192)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org