You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafodion.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/02/01 00:52:00 UTC
[jira] [Commented] (TRAFODION-2927) Keep log information for UPDATE
STATISTICS in case of errors
[ https://issues.apache.org/jira/browse/TRAFODION-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347836#comment-16347836 ]
ASF GitHub Bot commented on TRAFODION-2927:
-------------------------------------------
GitHub user DaveBirdsall opened a pull request:
https://github.com/apache/trafodion/pull/1429
[TRAFODION-2927] Refactor and improve UPDATE STATISTICS logging
This set of changes achieves two things:
1. It provides a mechanism for UPDATE STATISTICS to capture log information for each command, and retain that log in the event of a severe error. (Otherwise the log is deleted.)
2. It refactors the UPDATE STATISTICS logging facility to use the QRLogger/CommonLogger/log4cxx mechanism used by the rest of the SQL engine.
Externals:
The UPDATE STATISTICS LOG statement has been extended as follows:
UPDATE STATISTICS LOG ON -- Starts continuous logging of all UPDATE STATISTICS and SHOWSTATS activity into one log file. Logging continues until the session has ended or a different UPDATE STATISTICS LOG command is entered. This is the same behavior as today, except the log file will be written to the $TRAF_HOME/logs directory like other SQL log files.
UPDATE STATISTICS LOG OFF -- Stops logging. Nothing further is logged until a different UPDATE STATISTICS LOG command is entered. This is the same behavior as today.
UPDATE STATISTICS LOG SYSTEM -- The "SYSTEM" option is new. When specified, each UPDATE STATISTICS statement (but not SHOWSTATS) is logged to a separate log file. If a severe error occurs (e.g. an internal error), the log file is retained. Otherwise it is deleted at the end of the UPDATE STATISTICS command.
A CQD has been added. CQD USTAT_AUTOMATIC_LOGGING, having values of 'ON' and 'OFF'. If 'ON', then UPDATE STATISTICS LOG SYSTEM semantics are turned on by default. If 'OFF', then UPDATE STATISTICS LOG OFF semantics are turned on by default. (Note that no equivalent of UPDATE STATISTICS LOG ON is provided as that is remarkably noisy; if one really wants to log continuously, then one must use UPDATE STATISTICS LOG ON directly.)
The default for this CQD is 'OFF'. So by default no automatic logging is done. On large systems (e.g. scalability testing, or production systems with large tables), it is recommended to set this CQD to 'ON' in the system defaults table. In this way, logs will be captured for long-running UPDATE STATISTICS operations that fail with severe errors.
In terms of design: The existing QRLogger/CommonLogger routines assume a model of one log file per category, which doesn't meet the needs for UPDATE STATISTICS. So, QRLogger has been extended to add methods to explicitly configure log files on a per process, even per command basis. For such files, we don't bother logging the process ID and node number in the individual log messages, as all of the messages are coming from one process. Instead, UPDATE STATISTICS puts the node number, PID and Linux timestamp of file creation into the log file name itself.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DaveBirdsall/trafodion Trafodion2927
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/trafodion/pull/1429.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1429
----
commit f473c0cd5a1d895dd8a40d2ce2ec24c12a5b076d
Author: Dave Birdsall <db...@...>
Date: 2018-02-01T00:37:49Z
[TRAFODION-2927] Refactor and improve UPDATE STATISTICS logging
----
> Keep log information for UPDATE STATISTICS in case of errors
> ------------------------------------------------------------
>
> Key: TRAFODION-2927
> URL: https://issues.apache.org/jira/browse/TRAFODION-2927
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmp
> Affects Versions: 2.3
> Reporter: David Wayne Birdsall
> Assignee: David Wayne Birdsall
> Priority: Major
>
> Presently, UPDATE STATISTICS keeps a detailed log of its internal activities if one has specifed "update statistics log on" in advance.
> In production scenarios, this is typically not done. That means when a long-running UPDATE STATISTICS command fails on a large table, one has to redo it with logging turned on in order to troubleshoot.
> A better practice might be to always log, and delete the log if the operation succeeds.
> Another issue with UPDATE STATISTICS logs is their location. The directory is different than other Trafodion logs and is sometimes hard to find. As part of this JIRA, consideration should be given to writing the logs to the Trafodion logs directory instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)