[jira] [Created] (PIG-3823) Use newTupleNoCopy instead of newTuple wherever possible to avoid object copy penalty

["Rohini Palaniswamy (JIRA)" <ji...@apache.org>]

Rohini Palaniswamy created PIG-3823:
---------------------------------------

             Summary: Use newTupleNoCopy instead of newTuple wherever possible to avoid object copy penalty
                 Key: PIG-3823
                 URL: https://issues.apache.org/jira/browse/PIG-3823
             Project: Pig
          Issue Type: Bug
            Reporter: Rohini Palaniswamy
            Assignee: Rohini Palaniswamy


The readFields() method of all Tuple implementations do a mFields.clear() and then load data into the same tuple. If that was changed to do 

{code}
if (mFields.size() > 0) {
            mFields = new ArrayList<Object>(mFields.size());
        }
{code}

many places in code where we do TupleFactory.newTuple(List c); can be replaced with TupleFactory.newTupleNoCopy(List c);. This will avoid a expensive System.arrayCopy() call which is a native method. 

POPackage.java getValueTuple()
{code}
if( keyLookupSize > 0) {

            // we have some fields of the "value" in the
            // "key".
            int finalValueSize = keyLookupSize + val.size();
            copy = mTupleFactory.newTuple(finalValueSize);
            int valIndex = 0; // an index for accessing elements from
                              // the value (val) that we have currently
            for(int i = 0; i < finalValueSize; i++) {
                Integer keyIndex = keyLookup.get(i);
                if(keyIndex == null) {
                    // the field for this index is not in the
                    // key - so just take it from the "value"
                    // we were handed
                    copy.set(i, val.get(valIndex));
                    valIndex++;
                } .....
        } else if (isProjectStar) {
            // the whole "value" is present in the "key"
            copy = mTupleFactory.newTuple(keyAsTuple.getAll());   
        } else {
            // there is no field of the "value" in the
            // "key" - so just make a copy of what we got
            // as the "value"
            copy = mTupleFactory.newTuple(val.getAll());
        }
{code}

Some cases might take a slight hit in GC due to new ArrayList initialization. For eg: if( keyLookupSize > 0) condition in above code as it does not do a newTuple. But other 2 cases would greatly benefit as we can avoid arraylist copy. Same in POFRJoin.

Will run some tests to validate the theory and post a patch.



--
This message was sent by Atlassian JIRA
(v6.2#6252)