You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Edward Capriolo (JIRA)" <ji...@apache.org> on 2012/05/23 22:47:42 UTC

[jira] [Created] (HIVE-3048) Collect_set Aggregate does uneccesary check for value.

Edward Capriolo created HIVE-3048:
-------------------------------------

             Summary: Collect_set Aggregate does uneccesary check for value.
                 Key: HIVE-3048
                 URL: https://issues.apache.org/jira/browse/HIVE-3048
             Project: Hive
          Issue Type: Improvement
            Reporter: Edward Capriolo
            Assignee: Edward Capriolo


Sets already de-duplicate for free no need for existence check.
{noformat}
     private void putIntoSet(Object p, MkArrayAggregationBuffer myagg) {
      if (myagg.container.contains(p))
        return;
       Object pCopy = ObjectInspectorUtils.copyToStandardObject(p,
           this.inputOI);
       myagg.container.add(pCopy);
{noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3048) Collect_set Aggregate does uneccesary check for value.

Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401067#comment-13401067 ] 

Ashutosh Chauhan commented on HIVE-3048:
----------------------------------------

+1
Existing implementation actually looks buggy to me. It checks for existence of one object and then adds another object. In general case, two object's may have different hashcodes and then you are screwed. It will work however as long as underlying object's hashcode is based on value which will be the case for primitive types and containers containing primitive types which is the case for Hive datatypes. It's always a good practice to just add your objects in set and let set take care of duplicate elimination. 
                
> Collect_set Aggregate does uneccesary check for value.
> ------------------------------------------------------
>
>                 Key: HIVE-3048
>                 URL: https://issues.apache.org/jira/browse/HIVE-3048
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 0.8.1
>            Reporter: Edward Capriolo
>            Assignee: Edward Capriolo
>         Attachments: HIVE-3048.patch.1.txt
>
>
> Sets already de-duplicate for free no need for existence check.
> {noformat}
>      private void putIntoSet(Object p, MkArrayAggregationBuffer myagg) {
>       if (myagg.container.contains(p))
>         return;
>        Object pCopy = ObjectInspectorUtils.copyToStandardObject(p,
>            this.inputOI);
>        myagg.container.add(pCopy);
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3048) Collect_set Aggregate does uneccesary check for value.

Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Edward Capriolo updated HIVE-3048:
----------------------------------

    Affects Version/s: 0.8.1
               Status: Patch Available  (was: Open)
    
> Collect_set Aggregate does uneccesary check for value.
> ------------------------------------------------------
>
>                 Key: HIVE-3048
>                 URL: https://issues.apache.org/jira/browse/HIVE-3048
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 0.8.1
>            Reporter: Edward Capriolo
>            Assignee: Edward Capriolo
>         Attachments: HIVE-3048.patch.1.txt
>
>
> Sets already de-duplicate for free no need for existence check.
> {noformat}
>      private void putIntoSet(Object p, MkArrayAggregationBuffer myagg) {
>       if (myagg.container.contains(p))
>         return;
>        Object pCopy = ObjectInspectorUtils.copyToStandardObject(p,
>            this.inputOI);
>        myagg.container.add(pCopy);
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3048) Collect_set Aggregate does uneccesary check for value.

Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ashutosh Chauhan updated HIVE-3048:
-----------------------------------

       Resolution: Fixed
    Fix Version/s: 0.10.0
           Status: Resolved  (was: Patch Available)

Committed to trunk.
                
> Collect_set Aggregate does uneccesary check for value.
> ------------------------------------------------------
>
>                 Key: HIVE-3048
>                 URL: https://issues.apache.org/jira/browse/HIVE-3048
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 0.8.1
>            Reporter: Edward Capriolo
>            Assignee: Edward Capriolo
>             Fix For: 0.10.0
>
>         Attachments: HIVE-3048.patch.1.txt
>
>
> Sets already de-duplicate for free no need for existence check.
> {noformat}
>      private void putIntoSet(Object p, MkArrayAggregationBuffer myagg) {
>       if (myagg.container.contains(p))
>         return;
>        Object pCopy = ObjectInspectorUtils.copyToStandardObject(p,
>            this.inputOI);
>        myagg.container.add(pCopy);
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3048) Collect_set Aggregate does uneccesary check for value.

Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Edward Capriolo updated HIVE-3048:
----------------------------------

    Attachment: HIVE-3048.patch.1.txt
    
> Collect_set Aggregate does uneccesary check for value.
> ------------------------------------------------------
>
>                 Key: HIVE-3048
>                 URL: https://issues.apache.org/jira/browse/HIVE-3048
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Edward Capriolo
>            Assignee: Edward Capriolo
>         Attachments: HIVE-3048.patch.1.txt
>
>
> Sets already de-duplicate for free no need for existence check.
> {noformat}
>      private void putIntoSet(Object p, MkArrayAggregationBuffer myagg) {
>       if (myagg.container.contains(p))
>         return;
>        Object pCopy = ObjectInspectorUtils.copyToStandardObject(p,
>            this.inputOI);
>        myagg.container.add(pCopy);
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3048) Collect_set Aggregate does uneccesary check for value.

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401893#comment-13401893 ] 

Hudson commented on HIVE-3048:
------------------------------

Integrated in Hive-trunk-h0.21 #1515 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1515/])
    HIVE-3048 : Collect_set Aggregate does uneccesary check for value. (Ed Capriolo via Ashutosh Chauhan) (Revision 1354079)

     Result = FAILURE
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1354079
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCollectSet.java

                
> Collect_set Aggregate does uneccesary check for value.
> ------------------------------------------------------
>
>                 Key: HIVE-3048
>                 URL: https://issues.apache.org/jira/browse/HIVE-3048
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 0.8.1
>            Reporter: Edward Capriolo
>            Assignee: Edward Capriolo
>             Fix For: 0.10.0
>
>         Attachments: HIVE-3048.patch.1.txt
>
>
> Sets already de-duplicate for free no need for existence check.
> {noformat}
>      private void putIntoSet(Object p, MkArrayAggregationBuffer myagg) {
>       if (myagg.container.contains(p))
>         return;
>        Object pCopy = ObjectInspectorUtils.copyToStandardObject(p,
>            this.inputOI);
>        myagg.container.add(pCopy);
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira