You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Spyros Blanas (JIRA)" <ji...@apache.org> on 2008/06/10 20:07:45 UTC
[jira] Created: (HADOOP-3526) contrib/data_join doesn't work
contrib/data_join doesn't work
------------------------------
Key: HADOOP-3526
URL: https://issues.apache.org/jira/browse/HADOOP-3526
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.17.0
Reporter: Spyros Blanas
The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-3526:
-------------------------------
Since applications using this package break with 0.17,
I think this is a blocker for 0.17.1
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1
>
> Attachments: datajoin-0.17.patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Spyros Blanas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Spyros Blanas updated HADOOP-3526:
----------------------------------
Attachment: patch
Chris,
The tag needs cloning, because it's stored as a key in the SortedMap. Otherwise, the Text.equals() method always returns true (see HADOOP-3522).
As for the WritableUtils.clone() suggestion, I upload a new version of the patch which incorporates it.
Nigel,
Unfortunately there is no existing JUnit test covering the contributed data_join code and I don't know how to change the build.xml file to run the new tests. Also, building a JUnit testing framework for generic map-reduce jobs like data_join is much more difficult than targeting specific functions, as done in the core. I would be happy to create a test if you can point me to some code that tests a generic map/reduce job that I can tweak.
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Assignee: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1
>
> Attachments: patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas resolved HADOOP-3526.
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.18.0
Hadoop Flags: [Reviewed]
I just committed this. Thanks, Spyros
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Assignee: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1, 0.18.0
>
> Attachments: patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Spyros Blanas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Spyros Blanas updated HADOOP-3526:
----------------------------------
Attachment: (was: datajoin-0.17.patch)
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Assignee: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12606358#action_12606358 ]
Hudson commented on HADOOP-3526:
--------------------------------
Integrated in Hadoop-trunk #523 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/523/])
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Assignee: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1, 0.18.0
>
> Attachments: patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12605096#action_12605096 ]
Nigel Daley commented on HADOOP-3526:
-------------------------------------
-1 as the patch should have a unit test.
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1
>
> Attachments: datajoin-0.17.patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12605425#action_12605425 ]
Nigel Daley commented on HADOOP-3526:
-------------------------------------
Spyros, Chris has agreed to write some unit tests for Hadoop 0.18.
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Assignee: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1
>
> Attachments: patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12605030#action_12605030 ]
Chris Douglas commented on HADOOP-3526:
---------------------------------------
Spyros-
This looks like the right idea; a couple suggestions:
* It would break less existing code if this used WritableUtils::clone instead of adding an abstract method
* The tag at DataJoinReducerBase::107 doesn't need to be cloned
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1
>
> Attachments: datajoin-0.17.patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das reassigned HADOOP-3526:
-----------------------------------
Assignee: Spyros Blanas
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Assignee: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1
>
> Attachments: datajoin-0.17.patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Milind Bhandarkar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Milind Bhandarkar updated HADOOP-3526:
--------------------------------------
Priority: Blocker (was: Major)
Fix Version/s: 0.17.1
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1
>
> Attachments: datajoin-0.17.patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12605428#action_12605428 ]
Chris Douglas commented on HADOOP-3526:
---------------------------------------
bq. The tag needs cloning, because it's stored as a key in the SortedMap. Otherwise, the Text.equals() method always returns true
Sorry, I mean it doesn't need to be cloned there, for the lookup. Since the record needs to be cloned anyway and it includes the tag, it should be sufficient to clone the record and use the tag for the map key.
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Assignee: Spyros Blanas
> Priority: Blocker
> Fix For: 0.17.1
>
> Attachments: patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3526) contrib/data_join doesn't work
Posted by "Spyros Blanas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Spyros Blanas updated HADOOP-3526:
----------------------------------
Attachment: datajoin-0.17.patch
The issue is related to HADOOP-3522. The patch just clones the objects returned from the ValueIterator and forces the user-provided classes to implement a clone() method as well.
> contrib/data_join doesn't work
> ------------------------------
>
> Key: HADOOP-3526
> URL: https://issues.apache.org/jira/browse/HADOOP-3526
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0
> Reporter: Spyros Blanas
> Attachments: datajoin-0.17.patch
>
>
> The example in the README.txt in contrig/data_join/src/example doesn't work in 0.17.0. It works perfectly in 0.16.4.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.