[jira] [Commented] (MAHOUT-839) rowid job failing (when parsing options)

["Dan Brickley (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAHOUT-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125075#comment-13125075 ] 

Dan Brickley commented on MAHOUT-839:
-------------------------------------

I rewrote this based on one of the other drivers with more complex options. Attaching a patch, which got me working again but needs a careful check.

In attempt to drift towards standard options, it also adds --overwrite, which in turn calls

+        HadoopUtil.delete(getConf(), outputDir);

Please assume this will rm -rf all your most precious files and treat with appropriate caution.

Also the input option tries to match smartly using 'PathType.LIST, PathFilters.partFilter()'... and the help text should be accurate afaik.

                
> rowid job failing (when parsing options)  
> ------------------------------------------
>
>                 Key: MAHOUT-839
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-839
>             Project: Mahout
>          Issue Type: Bug
>            Reporter: Dan Brickley
>         Attachments: MAHOUT-839.patch
>
>
> Although MAHOUT-757 moved towards standard option naming, it uses different APIs for option parsing than other jobs.
> On my system, it died reliably with null pointer error. Reported in mail here, but not reconfirmed by anyone else yet: http://permalink.gmane.org/gmane.comp.apache.mahout.user/9659
> Example: 
> TellyClub:bin danbri$ ./mahout rowid --help
> MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
> Running on hadoop, using HADOOP_HOME=/Users/danbri/working/hadoop/hadoop-0.20.2
> HADOOP_CONF_DIR=/Users/danbri/working/hadoop/hadoop-0.20.2/conf
> MAHOUT-JOB: /Users/danbri/working/mahout/trunk/examples/target/mahout-examples-0.6-SNAPSHOT-job.jar
> ^CTellyClub:bin danbri$ MAHOUT_LOCAL=true ./mahout rowid --help
> MAHOUT_LOCAL is set, so we don't add HADOOP_CONF_DIR to classpath.
> MAHOUT_LOCAL is set, running locally
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/Users/bandri/working/mahout/trunk/examples/target/mahout-examples-0.6-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/Users/bandri/working/mahout/trunk/examples/target/dependency/slf4j-jcl-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/Users/bandri/working/mahout/trunk/examples/target/dependency/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> Exception in thread "main" java.lang.NullPointerException
> 	at org.apache.hadoop.fs.Path.<init>(Path.java:61)
> 	at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> 	at org.apache.mahout.utils.vectors.RowIdJob.run(RowIdJob.java:49)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> 	at org.apache.mahout.utils.vectors.RowIdJob.main(RowIdJob.java:89)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> 	at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> 	at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira