[jira] [Resolved] (TRAFODION-3004) SQL dense buffers structure contain a size limit due to an incorrect cast

["Sandhya Sundaresan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TRAFODION-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sandhya Sundaresan resolved TRAFODION-3004.
-------------------------------------------
       Resolution: Fixed
    Fix Version/s: 2.3

> SQL dense buffers structure contain a size limit due to an incorrect cast
> -------------------------------------------------------------------------
>
>                 Key: TRAFODION-3004
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-3004
>             Project: Apache Trafodion
>          Issue Type: Bug
>          Components: sql-exe
>    Affects Versions: any
>            Reporter: Sandhya Sundaresan
>            Assignee: Sandhya Sundaresan
>            Priority: Major
>             Fix For: 2.3
>
>
> The problem is that we cast the length from UINT32 to unsigned short, in SqlBufferDense::add_tuple_desc(), before actually allocating the buffer for the tupp_descriptor. In my case, since the requested size is over 64KB, allocating a memory not more than 64KB (due to the down cast) allows accessing memory passing beyond the allocation boundary. The suggested modification is to remove the cast (to unsigned short). 
> 1796 tupp_descriptor *SqlBufferDense::add_tuple_desc(Lng32 tup_data_size)
> 1797 {
> 1798 ULng32 rounded_size = ROUND8(tup_data_size);
> 1799 short td_size = ROUND8(sizeof(TupleDescInfo));
> 1800
> 1801 Lng32 freeSpaceNeeded = td_size + rounded_size;
> 1802 if (freeSpace_ < freeSpaceNeeded) // no free space to allocate this tuple
> 1803 return NULL;
> 1804
> 1805 maxTuppDesc_ += 1;
> 1806 freeSpace_ -= freeSpaceNeeded;
> 1807
> 1808 // if buffer is empty, then change status to partially full.
> 1809 if (bufferStatus_ == EMPTY)
> 1810 bufferStatus_ = PARTIAL;
> 1811
> 1812 TupleDescInfo * tdi = NULL;
> 1813 if (lastTupleDesc())
> 1814 tdi =
> 1815 (TupleDescInfo *)(lastTupleDesc()->tupleDesc()->getTupleAddress()
> 1816 + ROUND8(lastTupleDesc()->tupleDesc()->getAllocatedSize()));
> 1817 else
> 1818 tdi = firstTupleDesc();
> 1819
> 1820 tupp_descriptor * td = tdi->tupleDesc();
> 1821 td->init((unsigned short)tup_data_size,
> 1822 0,
> 1823 (char *)td + td_size);
> 1824
> 1825 setPrevTupleDesc(tdi, lastTupleDesc());
> 1826
> 1827 if (lastTupleDesc())
> 1828 setNextTupleDesc(lastTupleDesc(), tdi);
> 1829
> 1830 lastTupleDesc() = tdi;
> 1831
> 1832 setNextTupleDesc(tdi, NULL);
> 1833
> 1834 return td;
> 1835 }
>  
>  
> There are two other places where similar down casting is done.
>  
>  376 // allocate space to hold input params/hostvars
>  377 tupp_descriptor *tp = new(glob->getSpace()) tupp_descriptor;
>  378 char * dataPtr =
>  379 (char *)glob->getSpace()->allocateMemory(root_tdb.inputVarsSize_);
>  380 tp->init(root_tdb.inputVarsSize_,0,dataPtr);
>  381 workAtp_->getTupp(numTuples++) = tp;
>  382 }
>  383
>  384 if (root_tdb.updateCurrentOfQuery())
>  385 {
>  386 // allocate space to hold input pkey row
>  387 tupp_descriptor *tp = new(glob->getSpace()) tupp_descriptor;
>  388 char * dataPtr =
>  389 (char *)glob->getSpace()->allocateMemory(root_tdb.pkeyLen_);
>  390 tp->init((short) root_tdb.pkeyLen_,0,dataPtr);
>  391 workAtp_->getTupp(numTuples++) = tp;
>  392 }
>  393 else if (pkeyExpr())
>  394 {
>  395 pkeyAtp_ = allocateAtp(root_tdb.workCriDesc_, glob->getSpace());
>  396
>  397 // allocate space to hold the row of primary keys
>  398 tupp_descriptor *tp = new(glob->getSpace()) tupp_descriptor;
>  399 char * dataPtr =
>  400 (char *)glob->getSpace()->allocateMemory(root_tdb.pkeyLen_);
>  401 tp->init((short) root_tdb.pkeyLen_,0,dataPtr);
>  402 pkeyAtp_->getTupp(2) = tp;
>  403 }
> 404
>  405 // set the stream timeo
>  
>  
> Also, the offsets stored in tuple_descriptor is unsigned short, which have to be modified to ULng32, to store large tuples (over 64Kb) in dense sql buffer.
>  
> index c3127f5..5fee19a 100644
> --- a/core/sql/exp/ExpSqlTupp.h
> +++ b/core/sql/exp/ExpSqlTupp.h
> @@ -144,8 +144,8 @@ friend class tupp;
>  
>      struct
>      {
> - unsigned short nextTDIOffset_;
> - unsigned short prevTDIOffset_;
> + ULng32 nextTDIOffset_;
> + ULng32 prevTDIOffset_;
>      } tdiOffset_;
>    };
>  @@ -156,8 +156,8 @@ friend class tupp;
>      };
>  
>  protected:
> - unsigned short& nextTDIOffset() \{ return tdiOffset_.nextTDIOffset_; }
> - unsigned short& prevTDIOffset() \{ return tdiOffset_.prevTDIOffset_; }
> + ULng32& nextTDIOffset() \{ return tdiOffset_.nextTDIOffset_; }
> + ULng32& prevTDIOffset() \{ return tdiOffset_.prevTDIOffset_; }



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)