You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Phabricator (Updated) (JIRA)" <ji...@apache.org> on 2011/12/21 00:21:32 UTC
[jira] [Updated] (HIVE-2666) StackOverflowError when using custom
UDF in map join
[ https://issues.apache.org/jira/browse/HIVE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2666:
------------------------------
Attachment: HIVE-2666.D957.1.patch
kevinwilfong requested code review of "HIVE-2666 [jira] StackOverflowError when using custom UDF in map join".
Reviewers: JIRA
Resource files are now added to the class path as soon as they are added via the CLI. This fixes the stack overflow error mentioned in the JIRA by ensuring a consistent class loader between serializers and deserializers for the same query.
Note that now serdes which contain a static block to register themselves are now registered twice, once when adding the file to the class loader, and once when an instance of the class is created. Previously, registering a serde twice resulted in an exception, to avoid this, I have downgraded it to a warning.
When a custom UDF is used as part of a join which is converted to a map join, the XMLEncoder enters an infinite loop when serializing the map reduce task for the second time, as part of sending it to be executed. This results in a stack overflow error.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D957
AFFECTED FILES
serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
ql/src/java/org/apache/hadoop/hive/ql/processors/AddResourceProcessor.java
ql/src/java/org/apache/hadoop/hive/ql/processors/DeleteResourceProcessor.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1989/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> StackOverflowError when using custom UDF in map join
> ----------------------------------------------------
>
> Key: HIVE-2666
> URL: https://issues.apache.org/jira/browse/HIVE-2666
> Project: Hive
> Issue Type: Bug
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2666.D957.1.patch
>
>
> When a custom UDF is used as part of a join which is converted to a map join, the XMLEncoder enters an infinite loop when serializing the map reduce task for the second time, as part of sending it to be executed. This results in a stack overflow error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira