[jira] [Created] (FLINK-2253) ALS fails if the data flow is split up and the input is non-deterministic

["Till Rohrmann (JIRA)" <ji...@apache.org>]

Till Rohrmann created FLINK-2253:
------------------------------------

             Summary: ALS fails if the data flow is split up and the input is non-deterministic
                 Key: FLINK-2253
                 URL: https://issues.apache.org/jira/browse/FLINK-2253
             Project: Flink
          Issue Type: Bug
          Components: Machine Learning Library
            Reporter: Till Rohrmann
            Assignee: Till Rohrmann


If the {{temporaryPath}} of Flink's {{ALS}} implementation is set, then the ALS job is executed one part after the other. This also includes the routing information calculation for the user and item blocks. If the input {{DataSet}} is calculated non-deterministically, e.g. if it's the result of a {{first}} operation, then the resulting routing information won't be consistent and ALS fails.

A solution would be to pin the input {{DataSet}} so that it will only be executed once. Until we have this functionality, I propose to simply execute the user and item routing information at the same time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)