You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@trafodion.apache.org by Qifan Chen <qi...@esgyn.com> on 2016/02/09 23:49:56 UTC
Re: Display the data content associated with a tupp?
OK. Thanks Gunnar for the suggestion to discuss the topic on the user
dlist.
I have added several display() member functions to the following executor
classes to facilitate debugging in Generator and the executor. They perform
reasonably well for my debug tasks.
1. atp_struct::display(const char* title, ex_cri_desc* cri = NULL):
for all the tupps,
for each tupp, display
the data types
the data based on the data type
2. ExpTupleDesc::display(const char* title):
for all the attributes
for each attribute: display the data type code (64, 152 etc) and data
length (10, 15 etc)
3. ex_cri_desc::display(const char* title):
for all the tuple (type)s
for each tuple type, display data type code (64, 152 etc) and data length
(10, 15 etc)
Enhancement 1) is good for display data and type associated with a work_atp
or query entries.
Enhancement 2) and 3) is good for display type info associated with TDBs
during codeGen.
*Usage example 1: display of an ATP.*
retCode = returnExpr()->eval(pentry_up->getAtp(),workAtp_);
if (retCode == ex_expr::EXPR_ERROR)
{
// LCOV_EXCL_START
pstate->step_ = ExSeq_ERROR;
break;
// LCOV_EXCL_STOP
}
}
*// display the result of return eval() call*
*pentry_up->getAtp()->display("return eval result",
myTdb().getCriDescUp());*
retCode = ex_expr::EXPR_OK;
//Apply post pre
*Usage example 2: display of a CRI Desc (composite record descriptor)*
*// display the type info for the up queue CRI*
*returnCriDesc->display("up queue CRI for Sequence node");*
ComTdbSequence *sequenceTdb
= new(space) ComTdbSequence(readSeqExpr,
returnExpr,
postPred,
cancelExpression,
getMinFollowingRows(),
#pragma nowarn(1506) // warning elimination
historyRecLen,
historyAtpIndex,
childTdb,
givenCriDesc,
returnCriDesc,
These display methods also can be called within gdb through the print (p)
command, such as
p pentry_up->getAtp()->display("return eval result", myTdb().getCriDescUp())
Here are the results of these display functions in action.
>>execute xx;
read eval result
0th field: datatype=152, len=8, data=""
1th field: datatype=64, len=15, data="HR"
2th field: datatype=152, len=8, data="00230000"
3th field: datatype=64, len=15, data=""
read eval result
0th field: datatype=152, len=8, data=""
1th field: datatype=64, len=15, data="HR"
2th field: datatype=152, len=8, data="00160000"
3th field: datatype=64, len=15, data=""
read eval result
0th field: datatype=152, len=8, data=""
1th field: datatype=64, len=15, data="HR"
2th field: datatype=152, len=8, data="00120000"
3th field: datatype=64, len=15, data=""
read eval result
0th field: datatype=152, len=8, data=""
1th field: datatype=64, len=15, data="HR"
2th field: datatype=152, len=8, data="00080000"
3th field: datatype=64, len=15, data=""
return eval result
0th field: datatype=152, len=8, data="00080000"
1th field: datatype=64, len=15, data="HR"
2th field: datatype=152, len=8, data="00230000"
3th field: datatype=64, len=15, data=""
return eval result
0th field: datatype=152, len=8, data="00070000"
1th field: datatype=64, len=15, data="HR"
2th field: datatype=152, len=8, data="00160000"
3th field: datatype=64, len=15, data=""
return eval result
0th field: datatype=152, len=8, data="00060000"
1th field: datatype=64, len=15, data="HR"
2th field: datatype=152, len=8, data="00120000"
3th field: datatype=64, len=15, data=""
Thanks --Qifan
On Mon, Feb 8, 2016 at 8:17 PM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Dave.
>
> I found that if I commented out the line in red below, I was able to
> access the Attributes.
>
> Sounds like the purpose of the code (in red) tries to conserve space in
> executor.
>
> I will move some of your print code (in MdamPoint class) into
> atp_struct.display() and document its usage.
>
> Thanks a lot for the help and hope that the new method will be useful.
>
> Thanks --Qifan
>
>
> ExpTupleDesc::ExpTupleDesc(UInt32 num_attrs,
> Attributes ** attrs,
> UInt32 tupleDataLength,
> TupleDataFormat tdataF,
> TupleDescFormat tdescF,
> Space * space)
> : numAttrs_(num_attrs),
> tupleDataLength_(tupleDataLength),
> tupleDataFormat_(tdataF),
> tupleDescFormat_(tdescF),
> NAVersionedObject(-1)
> {
> if ( space != NULL )
> {
> // remember current allocated space size. Used at the end of
> // generation to find out total tuple desc length allocated.
> tupleDescLength_ = space->getAllocatedSpaceSize();
>
> flags_ = 0;
> attrs_ = 0;
>
> * if (tdescF == LONG_FORMAT)*
> {
> // allocate an array of num_attrs Attributes*. This array follows
> // 'this' class.
> attrs_ = (AttributesPtr *)
> (space->allocateAlignedSpace(num_attrs *
> sizeof(AttributesPtr)));
>
> for (UInt32 i=0; i < num_attrs; i++)
> {
> // make a new copy of input attributes. This new attr entry
> // will follow the attribute array.
> attrs_[i] = attrs[i]->newCopy(space);
> }
> }
>
> // and now find out the total length of the generated descriptor
> tupleDescLength_ = sizeof(*this) +
> space->getAllocatedSp
>
> On Mon, Feb 8, 2016 at 3:55 PM, Qifan Chen <qi...@esgyn.com> wrote:
>
>> I checked the TDB and found the returnExpr() should compute a composite
>> record of the following sort. The evaluation is OK but one of the resultant
>> component records (valueId =10) may not be correct.
>>
>> (gdb) p historyIds.display()
>> ValueIdSet
>> 10:
>> LEAD(cast(convert(TRAFODION.SEABASE.PERSONNEL_ASSIGNMENTS.SALARY_AMT)))
>> 59: cast(convert(TRAFODION.SEABASE.PERSONNEL_ASSIGNMENTS.DEPT_NAME))
>> 61: cast(convert(TRAFODION.SEABASE.PERSONNEL_ASSIGNMENTS.SALARY_AMT))
>>
>>
>> The tupp in red is the correct tupp representing the evaluation result.
>> However, it is hard to find the boundary of the actual data (for print)
>> without the tupp descriptor.
>>
>>
>>
>> On Mon, Feb 8, 2016 at 3:34 PM, Dave Birdsall <da...@esgyn.com>
>> wrote:
>>
>>> Oh, dear. Which ATPs are used for what is highly dependent on the tcb.
>>> You might be able to figure it out from the tcb flow. Another attack is to
>>> figure it out from the generator. (The generator methods often have
>>> comments that describe in some detail what Atps are being used for what.)
>>>
>>>
>>>
>>> In your example, do you know that returnExpr() returned non-null? If it
>>> returned null, it’s not clear from the code snippet that we’d expect these
>>> to have any reasonable values. You could try moving the red code up inside
>>> the previous right brace.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *From:* Qifan Chen [mailto:qifan.chen@esgyn.com]
>>> *Sent:* Monday, February 8, 2016 1:29 PM
>>> *To:* Dave Birdsall <da...@esgyn.com>
>>> *Subject:* Re: Display the data content associated with a tupp?
>>>
>>>
>>>
>>> Thanks for the method and I used that as a starting point for a method
>>> as follows, because the record size of the tupp in question is 40 bytes
>>> (should be 8), and I would like to use the data type of the record to guide
>>> the display.
>>>
>>>
>>>
>>> void atp_struct::display()
>>>
>>> {
>>>
>>> ex_cri_desc* cri = getCriDesc();
>>>
>>> unsigned short tuples = numTuples();
>>>
>>> for (Int32 i=0; i<tuples; i++) {
>>>
>>> tupp& tup = getTupp(i);
>>>
>>> ExpTupleDesc* tDesc = cri->getTupleDescriptor(i);
>>>
>>>
>>>
>>> if ( tDesc && tDesc->attrs() ) {
>>>
>>> UInt32 attrs = tDesc->numAttrs();
>>>
>>> for (Int32 j=0; j<attrs; j++) {
>>>
>>> Attributes* attr = tDesc->getAttr(j);
>>>
>>> Int16 dt = attr->getDatatype();
>>>
>>> int x = 0;
>>>
>>> }
>>>
>>> }
>>>
>>> }
>>>
>>> }
>>>
>>>
>>>
>>> The method is incomplete and I have found that tDesc or tDesc->attrs()
>>> are null pointer, for both invocations in red.
>>>
>>>
>>>
>>> if(returnExpr())
>>>
>>> {
>>>
>>> retCode = returnExpr()->eval(pentry_up->getAtp(),workAtp_);
>>>
>>> if (retCode == ex_expr::EXPR_ERROR)
>>>
>>> {
>>>
>>> // LCOV_EXCL_START
>>>
>>> pstate->step_ = ExSeq_ERROR;
>>>
>>> break;
>>>
>>> // LCOV_EXCL_STOP
>>>
>>> }
>>>
>>> }
>>>
>>>
>>>
>>> workAtp_->display();
>>>
>>> pentry_up->getAtp()->display();
>>>
>>>
>>>
>>> retCode = ex_expr::EXPR_OK;
>>>
>>> //Apply post predicate expression
>>>
>>> if (postPred())
>>>
>>> {
>>>
>>> retCode =
>>> postPred()->eval(pentry_up->getAtp(),pentry_up->getAtp());
>>>
>>> if (retCode == ex_expr::EXPR_ERROR)
>>>
>>> {
>>>
>>> // LCOV_EXCL_START
>>>
>>> pstate->step_ = ExSeq_ERROR;
>>>
>>> break;
>>>
>>> // LCOV_EXCL_STOP
>>>
>>> }
>>>
>>> }
>>>
>>>
>>>
>>> So the next question is which Atp should contain a valid tuple
>>> descriptor?
>>>
>>>
>>>
>>> From "Executor Programming Guide",
>>>
>>>
>>>
>>> • ATPs need to be allocated for those queues that have a new record
>>> layout (different format or number of tupps). In this case there simply are
>>> no other queues whose ATPs could be shared. The new ATPs are created by
>>> combining tupps from other queue entries or by creating new tupps (see
>>> section 10.1, “class sql_buffer:” on page 18).
>>>
>>>
>>>
>>> • Similarly, ATPs need to be allocated for those queues that are at the
>>> start of a data flow, such as the down queue to the child of a root node or
>>> for the up queue of a leaf node.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Feb 8, 2016 at 12:23 PM, Dave Birdsall <da...@esgyn.com>
>>> wrote:
>>>
>>> I made an imperfect attempt with MDAM debugging. Imperfect because the
>>> code I wrote didn’t have datatype information, but tried to infer it from
>>> what it saw. It was good enough for my debugging purposes at the time. See
>>> method MdamPoint::printBrief in executor/MdamPoint.cpp.
>>>
>>>
>>>
>>> *From:* Qifan Chen [mailto:qifan.chen@esgyn.com]
>>> *Sent:* Monday, February 8, 2016 10:11 AM
>>> *To:* Everyone <Ev...@esgyn.com>
>>> *Subject:* Display the data content associated with a tupp?
>>>
>>>
>>>
>>> Hi,
>>>
>>>
>>>
>>> I am debugging an OLAP function bug and wonder if there is a way to
>>> display the actual data pointed to by a tupp while in gdb. For example,
>>> for the following piece of code snippet, histData holds the result of a
>>> return expression evaluation. I would like to see the result while at line
>>> 853.
>>>
>>>
>>>
>>> In compiler, we have various display methods on RelExpr and ItemExpr.
>>> Can I do similar similar for executor?
>>>
>>>
>>>
>>> Thanks --Qifan
>>>
>>>
>>>
>>> 801 char *tuppData = pentry_up->getTupp
>>>
>>> 802 (myTdb().tuppIndex_).getDataPointer();
>>>
>>> 803
>>>
>>> 804 advanceReturnHistoryRow();
>>>
>>> 805
>>>
>>> 806 char *histData = currentRetHistRowPtr_;
>>>
>>> 807 *pentry_up->getTupp*
>>>
>>> * 808 (myTdb().tuppIndex_).setDataPointer(histData);*
>>>
>>> 809
>>>
>>> 810 ex_expr::exp_return_type retCode = ex_expr::EXPR_OK;
>>>
>>> 811 // Apply the return phase expression
>>>
>>> 812 if(returnExpr())
>>>
>>> 813 {
>>>
>>> 814 retCode =
>>> returnExpr()->eval(pentry_up->getAtp(),workAtp_);
>>>
>>> 815 if (retCode == ex_expr::EXPR_ERROR)
>>>
>>> 816 {
>>>
>>> 817 // LCOV_EXCL_START
>>>
>>> 818 pstate->step_ = ExSeq_ERROR;
>>>
>>> 819 break;
>>>
>>> 820 // LCOV_EXCL_STOP
>>>
>>> 821 }
>>>
>>> 822 }
>>>
>>>
>>>
>>> ...
>>>
>>>
>>>
>>> 838 //
>>>
>>> 839 // Case-10-030724-7963: we are done pointing the tupp
>>> at the
>>>
>>> 840 // history buffer, so point it back to the SQL buffer.
>>>
>>> 841 //
>>>
>>> 842 pentry_up->getTupp
>>>
>>> 843 (myTdb().tuppIndex_).setDataPointer(tuppData);
>>>
>>> 844
>>>
>>> 845 switch(retCode) {
>>>
>>> 846 case ex_expr::EXPR_OK:
>>>
>>> 847 case ex_expr::EXPR_TRUE:
>>>
>>> 848 case ex_expr::EXPR_NULL:
>>>
>>> 849
>>>
>>> 850 // Copy the row that was computed in the history
>>> buffer,
>>>
>>> 851 // to the space previously allocated in the SQL
>>> buffer.
>>>
>>> 852
>>>
>>> 853 str_cpy_all(tuppData, *histData*, recLen());
>>>
>>>
>>>
>>> --
>>>
>>> Regards, --Qifan
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Regards, --Qifan
>>>
>>>
>>>
>>
>>
>>
>> --
>> Regards, --Qifan
>>
>>
>
>
> --
> Regards, --Qifan
>
>
--
Regards, --Qifan
Re: Display the data content associated with a tupp?
Posted by Qifan Chen <qi...@esgyn.com>.
I will write a JIRA case and submit the code.
Thanks --Qifan
On Tue, Feb 9, 2016 at 4:49 PM, Qifan Chen <qi...@esgyn.com> wrote:
> OK. Thanks Gunnar for the suggestion to discuss the topic on the user
> dlist.
>
> I have added several display() member functions to the following executor
> classes to facilitate debugging in Generator and the executor. They perform
> reasonably well for my debug tasks.
>
> 1. atp_struct::display(const char* title, ex_cri_desc* cri = NULL):
>
> for all the tupps,
>
> for each tupp, display
>
> the data types
> the data based on the data type
>
>
> 2. ExpTupleDesc::display(const char* title):
>
> for all the attributes
>
> for each attribute: display the data type code (64, 152 etc) and data
> length (10, 15 etc)
>
>
> 3. ex_cri_desc::display(const char* title):
>
> for all the tuple (type)s
>
> for each tuple type, display data type code (64, 152 etc) and data length
> (10, 15 etc)
>
>
> Enhancement 1) is good for display data and type associated with a
> work_atp or query entries.
>
> Enhancement 2) and 3) is good for display type info associated with TDBs
> during codeGen.
>
>
>
> *Usage example 1: display of an ATP.*
>
> retCode = returnExpr()->eval(pentry_up->getAtp(),workAtp_);
> if (retCode == ex_expr::EXPR_ERROR)
> {
> // LCOV_EXCL_START
> pstate->step_ = ExSeq_ERROR;
> break;
> // LCOV_EXCL_STOP
> }
> }
>
> *// display the result of return eval() call*
> *pentry_up->getAtp()->display("return eval result",
> myTdb().getCriDescUp());*
>
> retCode = ex_expr::EXPR_OK;
> //Apply post pre
>
>
> *Usage example 2: display of a CRI Desc (composite record descriptor)*
>
>
> *// display the type info for the up queue CRI*
> *returnCriDesc->display("up queue CRI for Sequence node");*
>
> ComTdbSequence *sequenceTdb
> = new(space) ComTdbSequence(readSeqExpr,
> returnExpr,
> postPred,
> cancelExpression,
> getMinFollowingRows(),
> #pragma nowarn(1506) // warning elimination
> historyRecLen,
> historyAtpIndex,
> childTdb,
> givenCriDesc,
> returnCriDesc,
>
>
> These display methods also can be called within gdb through the print (p)
> command, such as
>
> p pentry_up->getAtp()->display("return eval result",
> myTdb().getCriDescUp())
>
>
> Here are the results of these display functions in action.
>
> >>execute xx;
> read eval result
> 0th field: datatype=152, len=8, data=""
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00230000"
> 3th field: datatype=64, len=15, data=""
>
> read eval result
> 0th field: datatype=152, len=8, data=""
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00160000"
> 3th field: datatype=64, len=15, data=""
>
> read eval result
> 0th field: datatype=152, len=8, data=""
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00120000"
> 3th field: datatype=64, len=15, data=""
>
> read eval result
> 0th field: datatype=152, len=8, data=""
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00080000"
> 3th field: datatype=64, len=15, data=""
>
> return eval result
> 0th field: datatype=152, len=8, data="00080000"
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00230000"
> 3th field: datatype=64, len=15, data=""
>
> return eval result
> 0th field: datatype=152, len=8, data="00070000"
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00160000"
> 3th field: datatype=64, len=15, data=""
>
> return eval result
> 0th field: datatype=152, len=8, data="00060000"
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00120000"
> 3th field: datatype=64, len=15, data=""
>
> Thanks --Qifan
>
> On Mon, Feb 8, 2016 at 8:17 PM, Qifan Chen <qi...@esgyn.com> wrote:
>
>> Hi Dave.
>>
>> I found that if I commented out the line in red below, I was able to
>> access the Attributes.
>>
>> Sounds like the purpose of the code (in red) tries to conserve space in
>> executor.
>>
>> I will move some of your print code (in MdamPoint class) into
>> atp_struct.display() and document its usage.
>>
>> Thanks a lot for the help and hope that the new method will be useful.
>>
>> Thanks --Qifan
>>
>>
>> ExpTupleDesc::ExpTupleDesc(UInt32 num_attrs,
>> Attributes ** attrs,
>> UInt32 tupleDataLength,
>> TupleDataFormat tdataF,
>> TupleDescFormat tdescF,
>> Space * space)
>> : numAttrs_(num_attrs),
>> tupleDataLength_(tupleDataLength),
>> tupleDataFormat_(tdataF),
>> tupleDescFormat_(tdescF),
>> NAVersionedObject(-1)
>> {
>> if ( space != NULL )
>> {
>> // remember current allocated space size. Used at the end of
>> // generation to find out total tuple desc length allocated.
>> tupleDescLength_ = space->getAllocatedSpaceSize();
>>
>> flags_ = 0;
>> attrs_ = 0;
>>
>> * if (tdescF == LONG_FORMAT)*
>> {
>> // allocate an array of num_attrs Attributes*. This array
>> follows
>> // 'this' class.
>> attrs_ = (AttributesPtr *)
>> (space->allocateAlignedSpace(num_attrs *
>> sizeof(AttributesPtr)));
>>
>> for (UInt32 i=0; i < num_attrs; i++)
>> {
>> // make a new copy of input attributes. This new attr entry
>> // will follow the attribute array.
>> attrs_[i] = attrs[i]->newCopy(space);
>> }
>> }
>>
>> // and now find out the total length of the generated descriptor
>> tupleDescLength_ = sizeof(*this) +
>> space->getAllocatedSp
>>
>> On Mon, Feb 8, 2016 at 3:55 PM, Qifan Chen <qi...@esgyn.com> wrote:
>>
>>> I checked the TDB and found the returnExpr() should compute a composite
>>> record of the following sort. The evaluation is OK but one of the resultant
>>> component records (valueId =10) may not be correct.
>>>
>>> (gdb) p historyIds.display()
>>> ValueIdSet
>>> 10:
>>> LEAD(cast(convert(TRAFODION.SEABASE.PERSONNEL_ASSIGNMENTS.SALARY_AMT)))
>>> 59: cast(convert(TRAFODION.SEABASE.PERSONNEL_ASSIGNMENTS.DEPT_NAME))
>>> 61: cast(convert(TRAFODION.SEABASE.PERSONNEL_ASSIGNMENTS.SALARY_AMT))
>>>
>>>
>>> The tupp in red is the correct tupp representing the evaluation result.
>>> However, it is hard to find the boundary of the actual data (for print)
>>> without the tupp descriptor.
>>>
>>>
>>>
>>> On Mon, Feb 8, 2016 at 3:34 PM, Dave Birdsall <da...@esgyn.com>
>>> wrote:
>>>
>>>> Oh, dear. Which ATPs are used for what is highly dependent on the tcb.
>>>> You might be able to figure it out from the tcb flow. Another attack is to
>>>> figure it out from the generator. (The generator methods often have
>>>> comments that describe in some detail what Atps are being used for what.)
>>>>
>>>>
>>>>
>>>> In your example, do you know that returnExpr() returned non-null? If it
>>>> returned null, it’s not clear from the code snippet that we’d expect these
>>>> to have any reasonable values. You could try moving the red code up inside
>>>> the previous right brace.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From:* Qifan Chen [mailto:qifan.chen@esgyn.com]
>>>> *Sent:* Monday, February 8, 2016 1:29 PM
>>>> *To:* Dave Birdsall <da...@esgyn.com>
>>>> *Subject:* Re: Display the data content associated with a tupp?
>>>>
>>>>
>>>>
>>>> Thanks for the method and I used that as a starting point for a method
>>>> as follows, because the record size of the tupp in question is 40 bytes
>>>> (should be 8), and I would like to use the data type of the record to guide
>>>> the display.
>>>>
>>>>
>>>>
>>>> void atp_struct::display()
>>>>
>>>> {
>>>>
>>>> ex_cri_desc* cri = getCriDesc();
>>>>
>>>> unsigned short tuples = numTuples();
>>>>
>>>> for (Int32 i=0; i<tuples; i++) {
>>>>
>>>> tupp& tup = getTupp(i);
>>>>
>>>> ExpTupleDesc* tDesc = cri->getTupleDescriptor(i);
>>>>
>>>>
>>>>
>>>> if ( tDesc && tDesc->attrs() ) {
>>>>
>>>> UInt32 attrs = tDesc->numAttrs();
>>>>
>>>> for (Int32 j=0; j<attrs; j++) {
>>>>
>>>> Attributes* attr = tDesc->getAttr(j);
>>>>
>>>> Int16 dt = attr->getDatatype();
>>>>
>>>> int x = 0;
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> The method is incomplete and I have found that tDesc or tDesc->attrs()
>>>> are null pointer, for both invocations in red.
>>>>
>>>>
>>>>
>>>> if(returnExpr())
>>>>
>>>> {
>>>>
>>>> retCode =
>>>> returnExpr()->eval(pentry_up->getAtp(),workAtp_);
>>>>
>>>> if (retCode == ex_expr::EXPR_ERROR)
>>>>
>>>> {
>>>>
>>>> // LCOV_EXCL_START
>>>>
>>>> pstate->step_ = ExSeq_ERROR;
>>>>
>>>> break;
>>>>
>>>> // LCOV_EXCL_STOP
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> workAtp_->display();
>>>>
>>>> pentry_up->getAtp()->display();
>>>>
>>>>
>>>>
>>>> retCode = ex_expr::EXPR_OK;
>>>>
>>>> //Apply post predicate expression
>>>>
>>>> if (postPred())
>>>>
>>>> {
>>>>
>>>> retCode =
>>>> postPred()->eval(pentry_up->getAtp(),pentry_up->getAtp());
>>>>
>>>> if (retCode == ex_expr::EXPR_ERROR)
>>>>
>>>> {
>>>>
>>>> // LCOV_EXCL_START
>>>>
>>>> pstate->step_ = ExSeq_ERROR;
>>>>
>>>> break;
>>>>
>>>> // LCOV_EXCL_STOP
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> So the next question is which Atp should contain a valid tuple
>>>> descriptor?
>>>>
>>>>
>>>>
>>>> From "Executor Programming Guide",
>>>>
>>>>
>>>>
>>>> • ATPs need to be allocated for those queues that have a new record
>>>> layout (different format or number of tupps). In this case there simply are
>>>> no other queues whose ATPs could be shared. The new ATPs are created by
>>>> combining tupps from other queue entries or by creating new tupps (see
>>>> section 10.1, “class sql_buffer:” on page 18).
>>>>
>>>>
>>>>
>>>> • Similarly, ATPs need to be allocated for those queues that are at the
>>>> start of a data flow, such as the down queue to the child of a root node or
>>>> for the up queue of a leaf node.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Feb 8, 2016 at 12:23 PM, Dave Birdsall <da...@esgyn.com>
>>>> wrote:
>>>>
>>>> I made an imperfect attempt with MDAM debugging. Imperfect because the
>>>> code I wrote didn’t have datatype information, but tried to infer it from
>>>> what it saw. It was good enough for my debugging purposes at the time. See
>>>> method MdamPoint::printBrief in executor/MdamPoint.cpp.
>>>>
>>>>
>>>>
>>>> *From:* Qifan Chen [mailto:qifan.chen@esgyn.com]
>>>> *Sent:* Monday, February 8, 2016 10:11 AM
>>>> *To:* Everyone <Ev...@esgyn.com>
>>>> *Subject:* Display the data content associated with a tupp?
>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>>
>>>>
>>>> I am debugging an OLAP function bug and wonder if there is a way to
>>>> display the actual data pointed to by a tupp while in gdb. For example,
>>>> for the following piece of code snippet, histData holds the result of a
>>>> return expression evaluation. I would like to see the result while at line
>>>> 853.
>>>>
>>>>
>>>>
>>>> In compiler, we have various display methods on RelExpr and ItemExpr.
>>>> Can I do similar similar for executor?
>>>>
>>>>
>>>>
>>>> Thanks --Qifan
>>>>
>>>>
>>>>
>>>> 801 char *tuppData = pentry_up->getTupp
>>>>
>>>> 802 (myTdb().tuppIndex_).getDataPointer();
>>>>
>>>> 803
>>>>
>>>> 804 advanceReturnHistoryRow();
>>>>
>>>> 805
>>>>
>>>> 806 char *histData = currentRetHistRowPtr_;
>>>>
>>>> 807 *pentry_up->getTupp*
>>>>
>>>> * 808 (myTdb().tuppIndex_).setDataPointer(histData);*
>>>>
>>>> 809
>>>>
>>>> 810 ex_expr::exp_return_type retCode = ex_expr::EXPR_OK;
>>>>
>>>> 811 // Apply the return phase expression
>>>>
>>>> 812 if(returnExpr())
>>>>
>>>> 813 {
>>>>
>>>> 814 retCode =
>>>> returnExpr()->eval(pentry_up->getAtp(),workAtp_);
>>>>
>>>> 815 if (retCode == ex_expr::EXPR_ERROR)
>>>>
>>>> 816 {
>>>>
>>>> 817 // LCOV_EXCL_START
>>>>
>>>> 818 pstate->step_ = ExSeq_ERROR;
>>>>
>>>> 819 break;
>>>>
>>>> 820 // LCOV_EXCL_STOP
>>>>
>>>> 821 }
>>>>
>>>> 822 }
>>>>
>>>>
>>>>
>>>> ...
>>>>
>>>>
>>>>
>>>> 838 //
>>>>
>>>> 839 // Case-10-030724-7963: we are done pointing the tupp
>>>> at the
>>>>
>>>> 840 // history buffer, so point it back to the SQL buffer.
>>>>
>>>> 841 //
>>>>
>>>> 842 pentry_up->getTupp
>>>>
>>>> 843 (myTdb().tuppIndex_).setDataPointer(tuppData);
>>>>
>>>> 844
>>>>
>>>> 845 switch(retCode) {
>>>>
>>>> 846 case ex_expr::EXPR_OK:
>>>>
>>>> 847 case ex_expr::EXPR_TRUE:
>>>>
>>>> 848 case ex_expr::EXPR_NULL:
>>>>
>>>> 849
>>>>
>>>> 850 // Copy the row that was computed in the history
>>>> buffer,
>>>>
>>>> 851 // to the space previously allocated in the SQL
>>>> buffer.
>>>>
>>>> 852
>>>>
>>>> 853 str_cpy_all(tuppData, *histData*, recLen());
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Regards, --Qifan
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Regards, --Qifan
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Regards, --Qifan
>>>
>>>
>>
>>
>> --
>> Regards, --Qifan
>>
>>
>
>
> --
> Regards, --Qifan
>
>
--
Regards, --Qifan
Re: Display the data content associated with a tupp?
Posted by Qifan Chen <qi...@esgyn.com>.
I will write a JIRA case and submit the code.
Thanks --Qifan
On Tue, Feb 9, 2016 at 4:49 PM, Qifan Chen <qi...@esgyn.com> wrote:
> OK. Thanks Gunnar for the suggestion to discuss the topic on the user
> dlist.
>
> I have added several display() member functions to the following executor
> classes to facilitate debugging in Generator and the executor. They perform
> reasonably well for my debug tasks.
>
> 1. atp_struct::display(const char* title, ex_cri_desc* cri = NULL):
>
> for all the tupps,
>
> for each tupp, display
>
> the data types
> the data based on the data type
>
>
> 2. ExpTupleDesc::display(const char* title):
>
> for all the attributes
>
> for each attribute: display the data type code (64, 152 etc) and data
> length (10, 15 etc)
>
>
> 3. ex_cri_desc::display(const char* title):
>
> for all the tuple (type)s
>
> for each tuple type, display data type code (64, 152 etc) and data length
> (10, 15 etc)
>
>
> Enhancement 1) is good for display data and type associated with a
> work_atp or query entries.
>
> Enhancement 2) and 3) is good for display type info associated with TDBs
> during codeGen.
>
>
>
> *Usage example 1: display of an ATP.*
>
> retCode = returnExpr()->eval(pentry_up->getAtp(),workAtp_);
> if (retCode == ex_expr::EXPR_ERROR)
> {
> // LCOV_EXCL_START
> pstate->step_ = ExSeq_ERROR;
> break;
> // LCOV_EXCL_STOP
> }
> }
>
> *// display the result of return eval() call*
> *pentry_up->getAtp()->display("return eval result",
> myTdb().getCriDescUp());*
>
> retCode = ex_expr::EXPR_OK;
> //Apply post pre
>
>
> *Usage example 2: display of a CRI Desc (composite record descriptor)*
>
>
> *// display the type info for the up queue CRI*
> *returnCriDesc->display("up queue CRI for Sequence node");*
>
> ComTdbSequence *sequenceTdb
> = new(space) ComTdbSequence(readSeqExpr,
> returnExpr,
> postPred,
> cancelExpression,
> getMinFollowingRows(),
> #pragma nowarn(1506) // warning elimination
> historyRecLen,
> historyAtpIndex,
> childTdb,
> givenCriDesc,
> returnCriDesc,
>
>
> These display methods also can be called within gdb through the print (p)
> command, such as
>
> p pentry_up->getAtp()->display("return eval result",
> myTdb().getCriDescUp())
>
>
> Here are the results of these display functions in action.
>
> >>execute xx;
> read eval result
> 0th field: datatype=152, len=8, data=""
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00230000"
> 3th field: datatype=64, len=15, data=""
>
> read eval result
> 0th field: datatype=152, len=8, data=""
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00160000"
> 3th field: datatype=64, len=15, data=""
>
> read eval result
> 0th field: datatype=152, len=8, data=""
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00120000"
> 3th field: datatype=64, len=15, data=""
>
> read eval result
> 0th field: datatype=152, len=8, data=""
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00080000"
> 3th field: datatype=64, len=15, data=""
>
> return eval result
> 0th field: datatype=152, len=8, data="00080000"
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00230000"
> 3th field: datatype=64, len=15, data=""
>
> return eval result
> 0th field: datatype=152, len=8, data="00070000"
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00160000"
> 3th field: datatype=64, len=15, data=""
>
> return eval result
> 0th field: datatype=152, len=8, data="00060000"
> 1th field: datatype=64, len=15, data="HR"
> 2th field: datatype=152, len=8, data="00120000"
> 3th field: datatype=64, len=15, data=""
>
> Thanks --Qifan
>
> On Mon, Feb 8, 2016 at 8:17 PM, Qifan Chen <qi...@esgyn.com> wrote:
>
>> Hi Dave.
>>
>> I found that if I commented out the line in red below, I was able to
>> access the Attributes.
>>
>> Sounds like the purpose of the code (in red) tries to conserve space in
>> executor.
>>
>> I will move some of your print code (in MdamPoint class) into
>> atp_struct.display() and document its usage.
>>
>> Thanks a lot for the help and hope that the new method will be useful.
>>
>> Thanks --Qifan
>>
>>
>> ExpTupleDesc::ExpTupleDesc(UInt32 num_attrs,
>> Attributes ** attrs,
>> UInt32 tupleDataLength,
>> TupleDataFormat tdataF,
>> TupleDescFormat tdescF,
>> Space * space)
>> : numAttrs_(num_attrs),
>> tupleDataLength_(tupleDataLength),
>> tupleDataFormat_(tdataF),
>> tupleDescFormat_(tdescF),
>> NAVersionedObject(-1)
>> {
>> if ( space != NULL )
>> {
>> // remember current allocated space size. Used at the end of
>> // generation to find out total tuple desc length allocated.
>> tupleDescLength_ = space->getAllocatedSpaceSize();
>>
>> flags_ = 0;
>> attrs_ = 0;
>>
>> * if (tdescF == LONG_FORMAT)*
>> {
>> // allocate an array of num_attrs Attributes*. This array
>> follows
>> // 'this' class.
>> attrs_ = (AttributesPtr *)
>> (space->allocateAlignedSpace(num_attrs *
>> sizeof(AttributesPtr)));
>>
>> for (UInt32 i=0; i < num_attrs; i++)
>> {
>> // make a new copy of input attributes. This new attr entry
>> // will follow the attribute array.
>> attrs_[i] = attrs[i]->newCopy(space);
>> }
>> }
>>
>> // and now find out the total length of the generated descriptor
>> tupleDescLength_ = sizeof(*this) +
>> space->getAllocatedSp
>>
>> On Mon, Feb 8, 2016 at 3:55 PM, Qifan Chen <qi...@esgyn.com> wrote:
>>
>>> I checked the TDB and found the returnExpr() should compute a composite
>>> record of the following sort. The evaluation is OK but one of the resultant
>>> component records (valueId =10) may not be correct.
>>>
>>> (gdb) p historyIds.display()
>>> ValueIdSet
>>> 10:
>>> LEAD(cast(convert(TRAFODION.SEABASE.PERSONNEL_ASSIGNMENTS.SALARY_AMT)))
>>> 59: cast(convert(TRAFODION.SEABASE.PERSONNEL_ASSIGNMENTS.DEPT_NAME))
>>> 61: cast(convert(TRAFODION.SEABASE.PERSONNEL_ASSIGNMENTS.SALARY_AMT))
>>>
>>>
>>> The tupp in red is the correct tupp representing the evaluation result.
>>> However, it is hard to find the boundary of the actual data (for print)
>>> without the tupp descriptor.
>>>
>>>
>>>
>>> On Mon, Feb 8, 2016 at 3:34 PM, Dave Birdsall <da...@esgyn.com>
>>> wrote:
>>>
>>>> Oh, dear. Which ATPs are used for what is highly dependent on the tcb.
>>>> You might be able to figure it out from the tcb flow. Another attack is to
>>>> figure it out from the generator. (The generator methods often have
>>>> comments that describe in some detail what Atps are being used for what.)
>>>>
>>>>
>>>>
>>>> In your example, do you know that returnExpr() returned non-null? If it
>>>> returned null, it’s not clear from the code snippet that we’d expect these
>>>> to have any reasonable values. You could try moving the red code up inside
>>>> the previous right brace.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From:* Qifan Chen [mailto:qifan.chen@esgyn.com]
>>>> *Sent:* Monday, February 8, 2016 1:29 PM
>>>> *To:* Dave Birdsall <da...@esgyn.com>
>>>> *Subject:* Re: Display the data content associated with a tupp?
>>>>
>>>>
>>>>
>>>> Thanks for the method and I used that as a starting point for a method
>>>> as follows, because the record size of the tupp in question is 40 bytes
>>>> (should be 8), and I would like to use the data type of the record to guide
>>>> the display.
>>>>
>>>>
>>>>
>>>> void atp_struct::display()
>>>>
>>>> {
>>>>
>>>> ex_cri_desc* cri = getCriDesc();
>>>>
>>>> unsigned short tuples = numTuples();
>>>>
>>>> for (Int32 i=0; i<tuples; i++) {
>>>>
>>>> tupp& tup = getTupp(i);
>>>>
>>>> ExpTupleDesc* tDesc = cri->getTupleDescriptor(i);
>>>>
>>>>
>>>>
>>>> if ( tDesc && tDesc->attrs() ) {
>>>>
>>>> UInt32 attrs = tDesc->numAttrs();
>>>>
>>>> for (Int32 j=0; j<attrs; j++) {
>>>>
>>>> Attributes* attr = tDesc->getAttr(j);
>>>>
>>>> Int16 dt = attr->getDatatype();
>>>>
>>>> int x = 0;
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> The method is incomplete and I have found that tDesc or tDesc->attrs()
>>>> are null pointer, for both invocations in red.
>>>>
>>>>
>>>>
>>>> if(returnExpr())
>>>>
>>>> {
>>>>
>>>> retCode =
>>>> returnExpr()->eval(pentry_up->getAtp(),workAtp_);
>>>>
>>>> if (retCode == ex_expr::EXPR_ERROR)
>>>>
>>>> {
>>>>
>>>> // LCOV_EXCL_START
>>>>
>>>> pstate->step_ = ExSeq_ERROR;
>>>>
>>>> break;
>>>>
>>>> // LCOV_EXCL_STOP
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> workAtp_->display();
>>>>
>>>> pentry_up->getAtp()->display();
>>>>
>>>>
>>>>
>>>> retCode = ex_expr::EXPR_OK;
>>>>
>>>> //Apply post predicate expression
>>>>
>>>> if (postPred())
>>>>
>>>> {
>>>>
>>>> retCode =
>>>> postPred()->eval(pentry_up->getAtp(),pentry_up->getAtp());
>>>>
>>>> if (retCode == ex_expr::EXPR_ERROR)
>>>>
>>>> {
>>>>
>>>> // LCOV_EXCL_START
>>>>
>>>> pstate->step_ = ExSeq_ERROR;
>>>>
>>>> break;
>>>>
>>>> // LCOV_EXCL_STOP
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> So the next question is which Atp should contain a valid tuple
>>>> descriptor?
>>>>
>>>>
>>>>
>>>> From "Executor Programming Guide",
>>>>
>>>>
>>>>
>>>> • ATPs need to be allocated for those queues that have a new record
>>>> layout (different format or number of tupps). In this case there simply are
>>>> no other queues whose ATPs could be shared. The new ATPs are created by
>>>> combining tupps from other queue entries or by creating new tupps (see
>>>> section 10.1, “class sql_buffer:” on page 18).
>>>>
>>>>
>>>>
>>>> • Similarly, ATPs need to be allocated for those queues that are at the
>>>> start of a data flow, such as the down queue to the child of a root node or
>>>> for the up queue of a leaf node.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Feb 8, 2016 at 12:23 PM, Dave Birdsall <da...@esgyn.com>
>>>> wrote:
>>>>
>>>> I made an imperfect attempt with MDAM debugging. Imperfect because the
>>>> code I wrote didn’t have datatype information, but tried to infer it from
>>>> what it saw. It was good enough for my debugging purposes at the time. See
>>>> method MdamPoint::printBrief in executor/MdamPoint.cpp.
>>>>
>>>>
>>>>
>>>> *From:* Qifan Chen [mailto:qifan.chen@esgyn.com]
>>>> *Sent:* Monday, February 8, 2016 10:11 AM
>>>> *To:* Everyone <Ev...@esgyn.com>
>>>> *Subject:* Display the data content associated with a tupp?
>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>>
>>>>
>>>> I am debugging an OLAP function bug and wonder if there is a way to
>>>> display the actual data pointed to by a tupp while in gdb. For example,
>>>> for the following piece of code snippet, histData holds the result of a
>>>> return expression evaluation. I would like to see the result while at line
>>>> 853.
>>>>
>>>>
>>>>
>>>> In compiler, we have various display methods on RelExpr and ItemExpr.
>>>> Can I do similar similar for executor?
>>>>
>>>>
>>>>
>>>> Thanks --Qifan
>>>>
>>>>
>>>>
>>>> 801 char *tuppData = pentry_up->getTupp
>>>>
>>>> 802 (myTdb().tuppIndex_).getDataPointer();
>>>>
>>>> 803
>>>>
>>>> 804 advanceReturnHistoryRow();
>>>>
>>>> 805
>>>>
>>>> 806 char *histData = currentRetHistRowPtr_;
>>>>
>>>> 807 *pentry_up->getTupp*
>>>>
>>>> * 808 (myTdb().tuppIndex_).setDataPointer(histData);*
>>>>
>>>> 809
>>>>
>>>> 810 ex_expr::exp_return_type retCode = ex_expr::EXPR_OK;
>>>>
>>>> 811 // Apply the return phase expression
>>>>
>>>> 812 if(returnExpr())
>>>>
>>>> 813 {
>>>>
>>>> 814 retCode =
>>>> returnExpr()->eval(pentry_up->getAtp(),workAtp_);
>>>>
>>>> 815 if (retCode == ex_expr::EXPR_ERROR)
>>>>
>>>> 816 {
>>>>
>>>> 817 // LCOV_EXCL_START
>>>>
>>>> 818 pstate->step_ = ExSeq_ERROR;
>>>>
>>>> 819 break;
>>>>
>>>> 820 // LCOV_EXCL_STOP
>>>>
>>>> 821 }
>>>>
>>>> 822 }
>>>>
>>>>
>>>>
>>>> ...
>>>>
>>>>
>>>>
>>>> 838 //
>>>>
>>>> 839 // Case-10-030724-7963: we are done pointing the tupp
>>>> at the
>>>>
>>>> 840 // history buffer, so point it back to the SQL buffer.
>>>>
>>>> 841 //
>>>>
>>>> 842 pentry_up->getTupp
>>>>
>>>> 843 (myTdb().tuppIndex_).setDataPointer(tuppData);
>>>>
>>>> 844
>>>>
>>>> 845 switch(retCode) {
>>>>
>>>> 846 case ex_expr::EXPR_OK:
>>>>
>>>> 847 case ex_expr::EXPR_TRUE:
>>>>
>>>> 848 case ex_expr::EXPR_NULL:
>>>>
>>>> 849
>>>>
>>>> 850 // Copy the row that was computed in the history
>>>> buffer,
>>>>
>>>> 851 // to the space previously allocated in the SQL
>>>> buffer.
>>>>
>>>> 852
>>>>
>>>> 853 str_cpy_all(tuppData, *histData*, recLen());
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Regards, --Qifan
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Regards, --Qifan
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Regards, --Qifan
>>>
>>>
>>
>>
>> --
>> Regards, --Qifan
>>
>>
>
>
> --
> Regards, --Qifan
>
>
--
Regards, --Qifan