You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Sarita Anand Joshi (Jira)" <ji...@apache.org> on 2021/01/16 01:37:00 UTC
[jira] [Commented] (AIRFLOW-3772) Sagemaker operators return
unserializable value for XCom
[ https://issues.apache.org/jira/browse/AIRFLOW-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266470#comment-17266470 ]
Sarita Anand Joshi commented on AIRFLOW-3772:
---------------------------------------------
Any updates on this one, please?
> Sagemaker operators return unserializable value for XCom
> --------------------------------------------------------
>
> Key: AIRFLOW-3772
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3772
> Project: Apache Airflow
> Issue Type: Bug
> Components: contrib
> Affects Versions: 1.10.2
> Reporter: Daniel Huang
> Assignee: Daniel Huang
> Priority: Minor
>
> Sagemaker operators all return a dictionary of metadata in their {{execute}} methods, which in turn, Airflow automatically pushes to XCom. However, the dictionaries include at least one {{datetime}} object. For example, the [SagemakerTransformOperator|https://github.com/apache/airflow/blob/1.10.2/airflow/contrib/operators/sagemaker_transform_operator.py#L117-L124] includes [SageMaker.Client.describe_model|https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.describe_model], which has a {{creationTime}} value.
> This works fine when XCom pickling is enabled (via {{enable_xcom_pickling}}), but if it isn't, it results in a "TypeError: Object of type 'datetime' is not JSON serializable" error. Given that XCom pickling is going to be deprecated in the future, we should make sure the operators return a JSON serializable value.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)