[jira] [Commented] (TRAFODION-2301) Hadoop stack crash with LOGS TMUDF

["Roberta Marton (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TRAFODION-2301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15590811#comment-15590811 ] 

Roberta Marton commented on TRAFODION-2301:
-------------------------------------------

Today the UDF event_log_reader scans all logs and loads into memory and then discards the rows that is not need to return. It is a known design limitation.   Since there could be thousands of logs, it uses too much memory which cause system issues.




> Hadoop stack crash with LOGS TMUDF
> ----------------------------------
>
>                 Key: TRAFODION-2301
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-2301
>             Project: Apache Trafodion
>          Issue Type: Bug
>          Components: sql-general
>            Reporter: Roberta Marton
>            Assignee: Roberta Marton
>
> There seems to be a problem  is a that shows up when a tdm_udrsrvr process takes  most of the available Memory. When this happens, too much CPU time is being used to handle this memory issue which causes some of Hadoop processes to crash due to  lack of resources.  In addition, Linux is slow to respond from the terminal.
>  
> It can be reproduced from DB Mgr using log page which calls UDR in the backend.  Calling this UDF (event_log_reader) from sqlci can also reproduce the problem.
>  
> When using sqlci, SQL prints out the udrserver process launched info but then just hangs.
> >>select [first 1]* from udf(event_log_reader()); 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)