[jira] [Commented] (TRAFODION-3041) Repository performance improvement

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TRAFODION-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450267#comment-16450267 ] 

ASF GitHub Bot commented on TRAFODION-3041:
-------------------------------------------

Github user DaveBirdsall commented on a diff in the pull request:

    https://github.com/apache/trafodion/pull/1537#discussion_r183818934
  
    --- Diff: core/conn/odbc/src/odbc/nsksrvr/SrvrConnect.cpp ---
    @@ -530,11 +542,39 @@ static void* SessionWatchDog(void* arg)
                     okToGo = false;
                 }
             }
    -
    -
    -		while(!record_session_done && okToGo)
    -		{
    -			REPOS_STATS repos_stats = repos_queue.get_task();
    +        vector< vector<string> > query_list;
    +        vector<string> session_start;
    +        vector<string> statement_new_query;
    +        vector<string> session_stat_aggregation;
    +
    +        session_start.push_back("upsert into Trafodion.\"_REPOS_\".metric_session_table values");
    +        statement_new_query.push_back("insert into Trafodion.\"_REPOS_\".metric_query_table values");
    +        session_stat_aggregation.push_back("insert into Trafodion.\"_REPOS_\".metric_query_aggr_table values");
    +
    +        query_list.push_back(session_start);
    +        query_list.push_back(statement_new_query);
    +        query_list.push_back(session_stat_aggregation);
    +
    +        int query_limit = statisticsCacheSize;
    +        int time_limit = aggrInterval;
    +        //0:None 1:update 2:insert/upsert cache limit 3:achieve timeline
    +        int execute_flag = REPOS_EXECUTE_NONE;
    +        clock_t time_start = clock();
    +        clock_t time_end= clock();
    +
    +        REPOS_STATS repos_stats;
    +        while(!record_session_done && okToGo)
    +        {
    +            time_start = clock();
    +            while(repos_queue.isEmpty() && (((time_end = clock()) - time_start) / 1000000 < time_limit));
    --- End diff --
    
    I'm not sure I understand this logic. The "while" loop looks like it will spin until the time limit expires or repos_queue is non-empty. This raises two concerns: 1. Spin loops burn CPU needlessly. It would be better to put a sleep call inside the loop. 2. In the logic below, you check to see if the queue is non-empty and then dequeue something. This does not look thread-safe. Another thread might dequeue something after the repos_queue.isEmpty() call and before the repos_queue.get_task() call.


> Repository performance improvement
> ----------------------------------
>
>                 Key: TRAFODION-3041
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-3041
>             Project: Apache Trafodion
>          Issue Type: Improvement
>            Reporter: haolin.song
>            Assignee: haolin.song
>            Priority: Major
>
> Each query (in watchdog thread)which should be written into repository will be executed once and be sent immediately so that when the statistics limit time is set to 0, the query performance will degrade. It is better to create cache to save queries and execute them as batch size.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)