You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ignite.apache.org by "Vladimir Ozerov (JIRA)" <ji...@apache.org> on 2016/05/18 12:42:12 UTC

[jira] [Created] (IGNITE-3167) Hadoop: restore external execution.

Vladimir Ozerov created IGNITE-3167:
---------------------------------------

             Summary: Hadoop: restore external execution.
                 Key: IGNITE-3167
                 URL: https://issues.apache.org/jira/browse/IGNITE-3167
             Project: Ignite
          Issue Type: Task
          Components: hadoop
    Affects Versions: 1.5.0.final
            Reporter: Vladimir Ozerov
            Priority: Critical
             Fix For: 1.7


Some time ago we decided to get rid external execution mode. It appears to be a wrong decision.

Hadoop users rely on its process-per-job nature in lot's places. One of such case could be observed in HiBench Bayes benchmark:
1) Job creates something in the local file system through Hadoop FileSystem API.
2) Then it tries to get this data using regular java.io.FileReader and relative paths. 

This doesn't work in embedded mode because our LocalFileSystem wrapper assigns different work dirs for jobs, but process-wide working directory is always the same. As a result, aforementioned benchmark doesn't work in Ignite, but work with standard Hadoop job tracker.

It seems that we must return external execution back.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)