You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "John Sichi (JIRA)" <ji...@apache.org> on 2010/08/26 23:54:55 UTC
[jira] Created: (HIVE-1600) need to support hook input/output lists
for test result determinism
need to support hook input/output lists for test result determinism
-------------------------------------------------------------------
Key: HIVE-1600
URL: https://issues.apache.org/jira/browse/HIVE-1600
Project: Hadoop Hive
Issue Type: Bug
Components: Testing Infrastructure
Affects Versions: 0.6.0
Reporter: John Sichi
Assignee: John Sichi
Fix For: 0.7.0
Begin forwarded message:
From: Ning Zhang <nz...@facebook.com>
Date: August 26, 2010 2:47:26 PM PDT
To: John Sichi <js...@facebook.com>
Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
Subject: Re: failure in load_dyn_part1.q
Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
I'm seeing this failure due to a result diff when running tests on latest trunk:
POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
-POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
-POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
+POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
+POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903129#action_12903129 ]
John Sichi commented on HIVE-1600:
----------------------------------
There are going to be a bunch of one-time .q.out updates associated with this in order to bring everything into line.
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903554#action_12903554 ]
John Sichi commented on HIVE-1600:
----------------------------------
Will do. I'll keep the sorting during rendering (even though it will be redundant) as well since there's no formal contract regarding sorting in the hook interface.
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1600.1.patch
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1600:
-----------------------------
Attachment: HIVE-1600.2.patch
Still running through tests on this one.
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1600.1.patch, HIVE-1600.2.patch
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903841#action_12903841 ]
Namit Jain commented on HIVE-1600:
----------------------------------
+1
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1600.1.patch, HIVE-1600.2.patch
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1600:
-----------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
Committed. Thanks John
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1600.1.patch, HIVE-1600.2.patch
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903182#action_12903182 ]
Namit Jain commented on HIVE-1600:
----------------------------------
Do you want to make inputs and outputs in BaseSemanticAnalyzer to SortedSet/TreeSet instead ?
This way we can be deterministic across all hooks.
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1600.1.patch
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1600:
-----------------------------
Attachment: HIVE-1600.1.patch
Patch file of code only; still working on regenerating test logs.
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1600.1.patch
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903580#action_12903580 ]
John Sichi commented on HIVE-1600:
----------------------------------
Actually it's a pain to change that since a lot of places are already passing around HashSet. So I'm going to leave that for a followup since the point of this patch is to fix the sporadic failures which are hindering testing of other patches.
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1600.1.patch
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1600:
-----------------------------
Status: Patch Available (was: Open)
Tests passed.
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1600.1.patch, HIVE-1600.2.patch
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903183#action_12903183 ]
John Sichi commented on HIVE-1600:
----------------------------------
Blocked on HIVE-1601 for regenerating 0.17 test output.
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1600.1.patch
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903605#action_12903605 ]
Namit Jain commented on HIVE-1600:
----------------------------------
ok
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1600.1.patch
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1600) need to sort hook input/output lists
for test result determinism
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1600:
-----------------------------
Summary: need to sort hook input/output lists for test result determinism (was: need to support hook input/output lists for test result determinism)
Description:
Begin forwarded message:
From: Ning Zhang <nz...@facebook.com>
Date: August 26, 2010 2:47:26 PM PDT
To: John Sichi <js...@facebook.com>
Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
Subject: Re: failure in load_dyn_part1.q
Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
I'm seeing this failure due to a result diff when running tests on latest trunk:
POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
-POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
-POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
+POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
+POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
JVS
was:
Begin forwarded message:
From: Ning Zhang <nz...@facebook.com>
Date: August 26, 2010 2:47:26 PM PDT
To: John Sichi <js...@facebook.com>
Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
Subject: Re: failure in load_dyn_part1.q
Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
I'm seeing this failure due to a result diff when running tests on latest trunk:
POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
-POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
-POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
+POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
+POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
JVS
> need to sort hook input/output lists for test result determinism
> ----------------------------------------------------------------
>
> Key: HIVE-1600
> URL: https://issues.apache.org/jira/browse/HIVE-1600
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Affects Versions: 0.6.0
> Reporter: John Sichi
> Assignee: John Sichi
> Fix For: 0.7.0
>
>
> Begin forwarded message:
> From: Ning Zhang <nz...@facebook.com>
> Date: August 26, 2010 2:47:26 PM PDT
> To: John Sichi <js...@facebook.com>
> Cc: "hive-dev@hadoop.apache.org" <hi...@hadoop.apache.org>
> Subject: Re: failure in load_dyn_part1.q
> Yes I saw this error before but if it does not repro. So it's probably an ordering issue in POSTHOOK.
> On Aug 26, 2010, at 2:39 PM, John Sichi wrote:
> I'm seeing this failure due to a result diff when running tests on latest trunk:
> POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11
> POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> -POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=11
> POSTHOOK: Output: default@nzhang_part1@ds=2008-04-08/hr=12
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=11
> +POSTHOOK: Output: default@nzhang_part2@ds=2008-12-31/hr=12
> Did something change recently? Or are we missing a Java-level sort on the input/output list for determinism?
> JVS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.