[jira] [Commented] (ACCUMULO-3293) bin\winutils.exe not found when running MiniAccumuloCluster under Windows

["Josh Elser (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ACCUMULO-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198664#comment-14198664 ] 

Josh Elser commented on ACCUMULO-3293:
--------------------------------------

Thanks for opening an issue for this [~jsbaue]. Ideally, MiniAccumuloCluster should be self-encapsulated and not need to reach out to installations on the local machine. There are a lot of reasons for this, notably version mismatch from local hadoop installations and the hadoop dependencies in your project.

Do you know if MiniDFSCluster works on Windows? If so, how does it handle this situation?

If it does not, I could see opening up the API for MiniAccumuloCluster so that you can add some extra variables to the path for you to work around this specific case.

> bin\winutils.exe not found when running MiniAccumuloCluster under Windows
> -------------------------------------------------------------------------
>
>                 Key: ACCUMULO-3293
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3293
>             Project: Accumulo
>          Issue Type: Bug
>          Components: mini
>         Environment: MS Windows
>            Reporter: Johannes Bauer
>
> When running JUnit-Tests on Windows with the MiniAccumuloCluster, the file {{bin\winutils.exe}} is needed, which is searched in the path, given by the system variable {{HADOOP_HOME}}. This worked fine with the MiniAccumuloCluster 1.4.4.
> When I try the same with Accumulo 1.6, I get the error 
> {code}
> 14/10/24 16:57:40 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path
> java.io.IOException: Could not locate executable C:\Users\jsbaue\AppData\Local\Temp\1414162594270-0\bin\winutils.exe in the Hadoop binaries.
> 	at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
> 	at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
> 	at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
> 	at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
> {code}
> which basically says, that {{bin\winutils.exe}} is now searched in the temporary path, given to the MiniAccumuloCluster. When I try to put the file {{bin\winutils.exe}} in that path, I get an error, saying that the path has to be empty. 
> I fixed this problem by changing {{accumulo\minicluster\src\main\java\org\apache\accumulo\minicluster\impl\MiniAccumuloClusterImpl.java}}:
> I exchanged Line 309-310:
> {code}
>     // hadoop-2.2 puts error messages in the logs if this is not set
>     builder.environment().put("HADOOP_HOME", config.getDir().getAbsolutePath());
> {code}
> with the following lines: 
> {code}
>     env = System.getProperty("hadoop.home.dir");
>     if(env == null) 
>         env = System.getenv("HADOOP_HOME");
>     // hadoop-2.2 puts error messages in the logs if this is not set
>     if(env == null)
>         env = config.getDir().getAbsolutePath();
>     builder.environment().put("HADOOP_HOME", env);
> {code}
> This fixes the bug on Windows. I did no checks on Linux. 
> Could you check, if you chould change this in the next version of the MiniAccumuloCluster or help me with an other solution? Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)