You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@metron.apache.org by "Michael Miklavcic (Jira)" <ji...@apache.org> on 2019/10/03 21:29:00 UTC

[jira] [Created] (METRON-2274) Flatfile loader and summarizer mapreduce mode broken

Michael Miklavcic created METRON-2274:
-----------------------------------------

             Summary: Flatfile loader and summarizer mapreduce mode broken
                 Key: METRON-2274
                 URL: https://issues.apache.org/jira/browse/METRON-2274
             Project: Metron
          Issue Type: Bug
            Reporter: Michael Miklavcic
            Assignee: Michael Miklavcic


Fix mapreduce mode for the loaders. This was broken in METRON-2149 in 2 places:

[https://github.com/apache/metron/pull/1436/files#diff-1a76680a25447c0c0b3a245ce767b3f8R39]

and

[https://github.com/apache/metron/pull/1436/files#diff-8c698ee5b00cdf64f09bcd48847eae33R39]

HADOOP_CLASSPATH only sets the classpath for the local client MR application, not for the JVM's running the map and reduce tasks. When Stellar was extracted from our uber jars, the metron-data-management jar no longer contained requisite classes from Stellar common. The correct way to handle adding dependencies for mapreduce jobs is via the "-libjars" option.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)