[jira] Updated: (MAPREDUCE-1425) archive throws OutOfMemoryError

["Mahadev konar (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mahadev konar updated MAPREDUCE-1425:
-------------------------------------

    Attachment: MAPREDUCE-1425.patch

this patch fixes the problem of too many filestatus objects while creating archvies. Nicholas, can you test this out?

> archive throws OutOfMemoryError
> -------------------------------
>
>                 Key: MAPREDUCE-1425
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1425
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: harchive
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Mahadev konar
>             Fix For: 0.22.0
>
>         Attachments: har.sh, m1425_20100129TextFileGenerator.patch, MAPREDUCE-1425.patch
>
>
> {noformat}
> -bash-3.1$ hadoop  archive -archiveName t4.har -p . t4 .
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>         at java.util.regex.Pattern.compile(Pattern.java:1432)
>         at java.util.regex.Pattern.<init>(Pattern.java:1133)
>         at java.util.regex.Pattern.compile(Pattern.java:847)
>         at java.lang.String.replace(String.java:2208)
>         at org.apache.hadoop.fs.Path.normalizePath(Path.java:146)
>         at org.apache.hadoop.fs.Path.initialize(Path.java:137)
>         at org.apache.hadoop.fs.Path.<init>(Path.java:126)
>         at org.apache.hadoop.fs.Path.makeQualified(Path.java:296)
>         at org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:244)
>         at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:256)
>         at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:393)
>         at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:736)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>         at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:751)
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.