You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Jonathan Chang <jo...@fb.com> on 2011/10/13 18:50:53 UTC

Review Request: HIVE-2223: Add group by support for keys of type ARRAY and MAP.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2381/
-----------------------------------------------------------

Review request for hive.


Summary
-------

Adds hash codes for List and Map object inspectors.


This addresses bug HIVE-2223.
    https://issues.apache.org/jira/browse/HIVE-2223


Diffs
-----

  ql/src/test/queries/clientpositive/groupby_complex_types.q PRE-CREATION 
  ql/src/test/results/clientpositive/groupby_complex_types.q.out PRE-CREATION 
  serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 2d45aba 

Diff: https://reviews.apache.org/r/2381/diff


Testing
-------

Added unittest.


Thanks,

Jonathan

Re: Review Request: HIVE-2223: Add group by support for keys of type ARRAY and MAP.

Posted by John Sichi <js...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2381/#review2595
-----------------------------------------------------------



serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
<https://reviews.apache.org/r/2381/#comment5833>

    Since this is a list, sequence is significant, e.g. [1,2,3] != [3,2,1].  So perhaps we should make the hash code reflect this?
    
    Java's AbstractList does this via:
    
            while (i.hasNext()) {
                E obj = i.next();
                hashCode = 31*hashCode + (obj==null ? 0 : obj.hashCode());
            }
    



serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
<https://reviews.apache.org/r/2381/#comment5834>

    For java.util.HashMap, they xor the key's hashcode with the value's hashcode for each entry, and then sum over all entries.  I suppose that's to distinguish
    
    {a->b, c->d} from {a->d, b->c}.
    
    although it fails to distinguish 
    
    {a->b} from {b->a}
    


- John


On 2011-10-13 16:50:53, Jonathan Chang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2381/
> -----------------------------------------------------------
> 
> (Updated 2011-10-13 16:50:53)
> 
> 
> Review request for hive.
> 
> 
> Summary
> -------
> 
> Adds hash codes for List and Map object inspectors.
> 
> 
> This addresses bug HIVE-2223.
>     https://issues.apache.org/jira/browse/HIVE-2223
> 
> 
> Diffs
> -----
> 
>   ql/src/test/queries/clientpositive/groupby_complex_types.q PRE-CREATION 
>   ql/src/test/results/clientpositive/groupby_complex_types.q.out PRE-CREATION 
>   serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 2d45aba 
> 
> Diff: https://reviews.apache.org/r/2381/diff
> 
> 
> Testing
> -------
> 
> Added unittest.
> 
> 
> Thanks,
> 
> Jonathan
> 
>

Re: Review Request: HIVE-2223: Add group by support for keys of type ARRAY and MAP.

Posted by Jonathan Chang <jo...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2381/
-----------------------------------------------------------

(Updated 2011-10-16 17:19:25.946728)


Review request for hive.


Changes
-------

Unittest fix.


Summary
-------

Adds hash codes for List and Map object inspectors.


This addresses bug HIVE-2223.
    https://issues.apache.org/jira/browse/HIVE-2223


Diffs (updated)
-----

  ql/src/test/queries/clientpositive/groupby_complex_types.q PRE-CREATION 
  ql/src/test/results/clientpositive/groupby_complex_types.q.out PRE-CREATION 
  serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 2d45aba 
  serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java c1b1932 

Diff: https://reviews.apache.org/r/2381/diff


Testing
-------

Added unittest.


Thanks,

Jonathan

Re: Review Request: HIVE-2223: Add group by support for keys of type ARRAY and MAP.

Posted by Jonathan Chang <jo...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2381/
-----------------------------------------------------------

(Updated 2011-10-14 18:38:00.199892)


Review request for hive.


Changes
-------

Make hash match algorithms used by Java.


Summary
-------

Adds hash codes for List and Map object inspectors.


This addresses bug HIVE-2223.
    https://issues.apache.org/jira/browse/HIVE-2223


Diffs (updated)
-----

  ql/src/test/queries/clientpositive/groupby_complex_types.q PRE-CREATION 
  ql/src/test/results/clientpositive/groupby_complex_types.q.out PRE-CREATION 
  serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 2d45aba 

Diff: https://reviews.apache.org/r/2381/diff


Testing
-------

Added unittest.


Thanks,

Jonathan