Propagate skipped state from inside subdag to parent dag

[Felix Uellendall <fe...@protonmail.com.INVALID>]

Hey all,

I am wondering if there should be an "official" way to propagate a 
skipped state of the last downstream tasks in a subdag to the parent dag.

Simple Use Case:
I have a subdag with the following tasks "transfer data from an api to 
s3", "convert json to csv", "transform csv".
Sometimes the api returns a json with an empty data object. The 
"transfer" task succeeds which I think is correct. The "convert" task 
which actually does json to pands.DataFrame to csv could check if the 
data frame is empty before proceeding. At this point I am raising a 
AirflowSkipException to skip this task (and all downstream tasks) which 
need the data to work. I don't want the task to fail with a simple 
AirflowException since I think no data does not necessary mean it is an 
error. It just means there is no data available for the time period 
requested - I want to skip it. The problem comes when the subdag run has 
finished because its state is set to success like all other runs where 
data was available. That means you won't be able to see the difference 
from outside of the subdag (the parent dag).

I would be very interested in what you guys think about this. Should we 
add a feature to "propagate skipped state from inside subdag to parent 
dag" or could my problem just be solved easier / better? Please let me 
know :)

(P.S. I made it work but I am just thinking of an official way of doing 
it if you guys agree with the idea)

Kind regards,
Felix