You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafodion.apache.org by Selva Govindarajan <se...@esgyn.com> on 2018/03/13 16:14:22 UTC
ComDiagsArea and Error handling in Trafodion SQL - Guideline for
Trafodion developers
ComDiagsArea is a class containing errors and warnings encountered during SQL compilation or execution. This object is passed around between SQL processes and finally displayed by the end user application.
ComDiagsArea is populated and handled in many ways.
1. Caller allocates the ComDiagsArea passes it to the callee. The callee populates the diagnostics area when there are errors and warnings. Caller is responsible to deallocate it.
2. In case of process hop, the ComDiagsArea is shipped from the child process to the parent process via IPC mechanism.
3. ComDiagsArea is also embedded within a container object. The container object could be a CLI context (ContextCli) or CLI Statement (Statement) or the compiler context(CmpContext).
4. During compilation, the error/warnings message are mostly populated in the current CmpContext ComDiagsArea. There could be more than one CmpContext. There should be at least 2 CmpContext namely user and META. User given queries are compiled in user CmpContext while the internal meta-data queries are compiled in META CmpContext.
5. The errors/warnings info gathered in steps 1) and 2) are usually copied to the respective object of item 3. Then passed around between the objects of item 3) like CmpContext to Statement or Context before it can be obtained by the client applications.
Cons of the above methods:
* In step 1, ComDiagsArea is always allocated even when statement would succeed without the need for ComDiagsArea.
* In step 2, an empty ComDiagsArea is shipped from the child to parent even when there are no errors or warnings. This resulted in ComDiagsArea to be allocated on the parent side and populated with empty error/warning condition.
* Because of step 1 and step 2, the empty ComDiagsArea is copied to the objects of step 3.
* Prone to leaks in ComDiagsArea due to many unnecessary allocations and de-allocations.
* At least the sqlci application was attempting an expensive older way obtaining the diagnostic error message always even when there are no errors/warnings.
I have created a PR https://github.com/apache/trafodion/pull/1470 to take care of these issues. The strategy now is
1. Caller never allocates the ComDiagsArea, but pass in Reference to pointer to the callee. The caller initializes the pointer to NULL. When an error/warning is raised the callee needs to allocate ComDiagsArea and populate it. Then the caller moves it to the objects of step 3 and destroys the ComDiagsArea allocated in the callee.
2. In case of process hop, the ComDiagsArea is shipped from the child only when there is an error or warning via IPC mechanism.
3. While switching back from "META" CmpContext to user CmpContext, the errors/warnings from the META are copied to the user CmpContext.
4. Applications like sqlci, mxosrvr should attempt to obtain the diagnostics info based on return code of the CLI call. When the return code is 100, get the number of error condition via less expensive call SQL_EXEC_GetDiagnosticsStmtInfo2. When this call returns 2 or more conditions, then there are warnings other than 100.
5. Use mark/rewind and other methods of ComDiagsArea to manipulate it rather than creating and copying it.
These changes have enabled us to create ComDiagsArea only when there are any errors or warnings in primed up state. This also should help in fixing the leak in ComDiagsArea seen with Trafodion.
It is important that the developers and reviewers do not let the earlier inefficient code to creep in back. The purpose of this message is to let all SQL developers to be made aware of the new concept and enforce it whenever the code is modified or added in error handling.
Selva
RE: ComDiagsArea and Error handling in Trafodion SQL - Guideline for
Trafodion developers
Posted by Dave Birdsall <da...@esgyn.com>.
Hi all,
I added a discussion of SQL Diagnostics and ComDiagsArea, including the recommentations below, to the wiki. Please take a look and let me know what you think. Suggestions welcome. Or feel free to edit directly.
https://cwiki.apache.org/confluence/display/TRAFODION/SQL+Diagnostics+Architecture+and+Design
Dave
-----Original Message-----
From: Selva Govindarajan <se...@esgyn.com>
Sent: Tuesday, March 13, 2018 9:14 AM
To: dev@trafodion.apache.org
Subject: ComDiagsArea and Error handling in Trafodion SQL - Guideline for Trafodion developers
ComDiagsArea is a class containing errors and warnings encountered during SQL compilation or execution. This object is passed around between SQL processes and finally displayed by the end user application.
ComDiagsArea is populated and handled in many ways.
1. Caller allocates the ComDiagsArea passes it to the callee. The callee populates the diagnostics area when there are errors and warnings. Caller is responsible to deallocate it.
2. In case of process hop, the ComDiagsArea is shipped from the child process to the parent process via IPC mechanism.
3. ComDiagsArea is also embedded within a container object. The container object could be a CLI context (ContextCli) or CLI Statement (Statement) or the compiler context(CmpContext).
4. During compilation, the error/warnings message are mostly populated in the current CmpContext ComDiagsArea. There could be more than one CmpContext. There should be at least 2 CmpContext namely user and META. User given queries are compiled in user CmpContext while the internal meta-data queries are compiled in META CmpContext.
5. The errors/warnings info gathered in steps 1) and 2) are usually copied to the respective object of item 3. Then passed around between the objects of item 3) like CmpContext to Statement or Context before it can be obtained by the client applications.
Cons of the above methods:
* In step 1, ComDiagsArea is always allocated even when statement would succeed without the need for ComDiagsArea.
* In step 2, an empty ComDiagsArea is shipped from the child to parent even when there are no errors or warnings. This resulted in ComDiagsArea to be allocated on the parent side and populated with empty error/warning condition.
* Because of step 1 and step 2, the empty ComDiagsArea is copied to the objects of step 3.
* Prone to leaks in ComDiagsArea due to many unnecessary allocations and de-allocations.
* At least the sqlci application was attempting an expensive older way obtaining the diagnostic error message always even when there are no errors/warnings.
I have created a PR https://github.com/apache/trafodion/pull/1470 to take care of these issues. The strategy now is
1. Caller never allocates the ComDiagsArea, but pass in Reference to pointer to the callee. The caller initializes the pointer to NULL. When an error/warning is raised the callee needs to allocate ComDiagsArea and populate it. Then the caller moves it to the objects of step 3 and destroys the ComDiagsArea allocated in the callee.
2. In case of process hop, the ComDiagsArea is shipped from the child only when there is an error or warning via IPC mechanism.
3. While switching back from "META" CmpContext to user CmpContext, the errors/warnings from the META are copied to the user CmpContext.
4. Applications like sqlci, mxosrvr should attempt to obtain the diagnostics info based on return code of the CLI call. When the return code is 100, get the number of error condition via less expensive call SQL_EXEC_GetDiagnosticsStmtInfo2. When this call returns 2 or more conditions, then there are warnings other than 100.
5. Use mark/rewind and other methods of ComDiagsArea to manipulate it rather than creating and copying it.
These changes have enabled us to create ComDiagsArea only when there are any errors or warnings in primed up state. This also should help in fixing the leak in ComDiagsArea seen with Trafodion.
It is important that the developers and reviewers do not let the earlier inefficient code to creep in back. The purpose of this message is to let all SQL developers to be made aware of the new concept and enforce it whenever the code is modified or added in error handling.
Selva