You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Ravi Gummadi (JIRA)" <ji...@apache.org> on 2009/05/22 10:16:45 UTC
[jira] Created: (HADOOP-5895) Log message shows -ve number of bytes
to be merged in the final merge pass when there are no intermediate merges
and merge factor is < number of segments
Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is < number of segments
---------------------------------------------------------------------------------------------------------------------------------------------------------
Key: HADOOP-5895
URL: https://issues.apache.org/jira/browse/HADOOP-5895
Project: Hadoop Core
Issue Type: Bug
Reporter: Ravi Gummadi
Assignee: Ravi Gummadi
Fix For: 0.21.0
Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is < total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5895) Log message shows -ve number of
bytes to be merged in the final merge pass when there are no intermediate
merges and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713631#action_12713631 ]
Ravi Gummadi commented on HADOOP-5895:
--------------------------------------
Unit tests failures shown above are not related to this patch.
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5895) Log message shows -ve number of bytes
to be merged in the final merge pass when there are no intermediate merges
and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5895:
---------------------------------
Description: Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572. (was: Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is < total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.)
Summary: Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments (was: Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is < number of segments)
Issue is when mergeFactor > number of segments (not less than).
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5895) Log message shows -ve number of
bytes to be merged in the final merge pass when there are no intermediate
merges and merge factor is > number of segments
Posted by "Jothi Padmanabhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713397#action_12713397 ]
Jothi Padmanabhan commented on HADOOP-5895:
-------------------------------------------
+1 for the last patch
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5895) Log message shows -ve number of bytes
to be merged in the final merge pass when there are no intermediate merges
and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5895:
---------------------------------
Attachment: HADOOP-5895.v1.1.patch
Attaching a new patch fixing a minor issue for the same log message.
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5895) Log message shows -ve number of
bytes to be merged in the final merge pass when there are no intermediate
merges and merge factor is > number of segments
Posted by "Jothi Padmanabhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712629#action_12712629 ]
Jothi Padmanabhan commented on HADOOP-5895:
-------------------------------------------
Using a flag to control the number of iterations in includeFinalMerge would be more readable than modifying the variable n itself, no?
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5895) Log message shows -ve number of bytes
to be merged in the final merge pass when there are no intermediate merges
and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5895:
---------------------------------
Release Note: This patch fixes the issue of final merge not getting included in the calculation of estimated bytes in merges in map task when the merge factor is > total number of segments.
Status: Patch Available (was: Open)
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5895) Log message shows -ve number of
bytes to be merged in the final merge pass when there are no intermediate
merges and merge factor is > number of segments
Posted by "Jothi Padmanabhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712684#action_12712684 ]
Jothi Padmanabhan commented on HADOOP-5895:
-------------------------------------------
+1
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5895) Log message shows -ve number of bytes
to be merged in the final merge pass when there are no intermediate merges
and merge factor is < number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5895:
---------------------------------
Attachment: HADOOP-5895.patch
In computeBytesInMerges, getPassFactor() can return factor < number of segments and even though n++ is done based on condition includeFinalMerge, that is not good enough for the while loop to be executed.
Attaching patch with the fix. Please review and provide your comments.
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is < number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is < total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5895) Log message shows -ve number of bytes
to be merged in the final merge pass when there are no intermediate merges
and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5895:
---------------------------------
Status: Open (was: Patch Available)
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5895) Log message shows -ve number of bytes
to be merged in the final merge pass when there are no intermediate merges
and merge factor is > number of segments
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-5895:
--------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Ravi!
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5895) Log message shows -ve number of
bytes to be merged in the final merge pass when there are no intermediate
merges and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715376#action_12715376 ]
Ravi Gummadi commented on HADOOP-5895:
--------------------------------------
I dont think we need a new testcase for this as the issue is seen in the log message itself just before final merge in map tasks. So I haven't added new testcase.
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5895) Log message shows -ve number of
bytes to be merged in the final merge pass when there are no intermediate
merges and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715377#action_12715377 ]
Ravi Gummadi commented on HADOOP-5895:
--------------------------------------
Also, tested manually with many jobs like loadgen, sort, wordcount(in addition to unit tests) to check if -ve number can be seen in the log message in Merger.
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5895) Log message shows -ve number of
bytes to be merged in the final merge pass when there are no intermediate
merges and merge factor is > number of segments
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713566#action_12713566 ]
Hadoop QA commented on HADOOP-5895:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12408983/HADOOP-5895.v1.1.patch
against trunk revision 779066.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/411/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/411/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/411/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/411/console
This message is automatically generated.
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5895) Log message shows -ve number of
bytes to be merged in the final merge pass when there are no intermediate
merges and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712852#action_12712852 ]
Ravi Gummadi commented on HADOOP-5895:
--------------------------------------
Unit tests passed on my machine.
ant test-patch gave:
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-5895) Log message shows -ve
number of bytes to be merged in the final merge pass when there are no
intermediate merges and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711990#action_12711990 ]
Ravi Gummadi edited comment on HADOOP-5895 at 5/25/09 9:01 PM:
---------------------------------------------------------------
In computeBytesInMerges, getPassFactor() can return factor > number of segments and even though n++ is done based on condition includeFinalMerge, that is not good enough for the while loop to be executed.
Attaching patch with the fix. Please review and provide your comments.
was (Author: ravidotg):
In computeBytesInMerges, getPassFactor() can return factor < number of segments and even though n++ is done based on condition includeFinalMerge, that is not good enough for the while loop to be executed.
Attaching patch with the fix. Please review and provide your comments.
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5895) Log message shows -ve number of bytes
to be merged in the final merge pass when there are no intermediate merges
and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5895:
---------------------------------
Attachment: HADOOP-5895.v1.patch
Agreed Jothi.
Here is the patch with a flag doing the work of considering final merge as part of calculation of estimated input bytes of merges.
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5895) Log message shows -ve number of bytes
to be merged in the final merge pass when there are no intermediate merges
and merge factor is > number of segments
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5895:
---------------------------------
Status: Patch Available (was: Open)
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is > number of segments
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5895
> URL: https://issues.apache.org/jira/browse/HADOOP-5895
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5895.patch, HADOOP-5895.v1.1.patch, HADOOP-5895.v1.patch
>
>
> Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and the mergeFactor is > total number of segments to be merged. This issue is because of code changes done in HADOOP-5572.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.