[jira] [Commented] (AIRFLOW-3314) The lineage automatic inlets feature does not work as described.

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16927762#comment-16927762 ] 

ASF GitHub Bot commented on AIRFLOW-3314:
-----------------------------------------

stale[bot] commented on pull request #4156: [AIRFLOW-3314] Changed auto inlets feature to work as described
URL: https://github.com/apache/airflow/pull/4156
 
 
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> The lineage automatic inlets feature does not work as described.
> ----------------------------------------------------------------
>
>                 Key: AIRFLOW-3314
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3314
>             Project: Apache Airflow
>          Issue Type: Bug
>    Affects Versions: 1.10.0
>            Reporter: Reuben van Ammers
>            Priority: Minor
>         Attachments: test_lineage_broken.py
>
>
> Currently, this is the description of the arguments to the prepare lineage wrapper regarding inlets in airflow/lineage/__init__.py:
> inlets can be:
>  "auto" -> picks up any outlets from direct upstream tasks that have outlets
>  defined, as such that if A -> B -> C and B does not have outlets but A does,
>  these are provided as inlets.
> This implies that non state producing tasks should have no effect on the behaviour for inlets and outlets, which is desirable to easily add operators that don't change state of files (such as for tracking/communication). 
>  
> This is not the current behaviour. This can be seen by changing the test case in tests/lineage/test_lineage.py to use the auto feature on operation 5 to use auto. Given the description, one would expect 1 inlet file, the the file produced from op3. However, as can be seen in the attatched broken test, this is not the case, and the presence of the non-state affecting operator breaks the test.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)