Xcom related security issue

[Rui Wang <ru...@airbnb.com.INVALID>]

Hi,

I created an JIRA issue: https://issues.apache.org/jira/browse/AIRFLOW-855.


The JIRA task above gives pretty rich context. Briefly speaking, PickleType
gives the possible that run code/command on remote machines. This type can
serialize objects, which is a wide scope. I am wondering what kind of use
cases you have for using Xcom and its PickleType. If the use cases show the
possibility that replacing PickleType with JSON type, the probably this
security issue can be solved by using JSON type instead,


Thanks,
Rui Wang

Re: Xcom related security issue

[Jeremiah Lowin <jl...@apache.org>]

Rui,

Thanks for pointing this out, it's a valid concern.

I personally have no issue with swapping Pickle -> JSON, but there may be
many Airflow users relying on the current behavior and I don't want to
invalidate their DAGs with a PR.

On the other hand, I'm not sure of a way to "gently" deprecate the
PickleType. Perhaps step 1 is to check if an XCom can be JSON serialized
and if it can't, print a warning? Then step 2 is to enforce JSON
serialization at a future date.

Any suggestions of how to implement this?

J

On Sat, Feb 18, 2017 at 10:16 AM Rui Wang <ru...@airbnb.com.invalid>
wrote:

> Hi,
>
> I created an JIRA issue: https://issues.apache.org/jira/browse/AIRFLOW-855
> .
>
>
> The JIRA task above gives pretty rich context. Briefly speaking, PickleType
> gives the possible that run code/command on remote machines. This type can
> serialize objects, which is a wide scope. I am wondering what kind of use
> cases you have for using Xcom and its PickleType. If the use cases show the
> possibility that replacing PickleType with JSON type, the probably this
> security issue can be solved by using JSON type instead,
>
>
> Thanks,
> Rui Wang
>