[jira] [Commented] (DRILL-2362) Drill should manage Query Profiling archiving

["Rahul Raj (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146664#comment-16146664 ] 

Rahul Raj commented on DRILL-2362:
----------------------------------

We had more than 200,000 profiles stored in the default location and it made the profile page access extremely slow. Also while accessing this page, client connections were disconnected (channel closed exception) - could be related to a large GC causing missed heart beats.

-Rahul

> Drill should manage Query Profiling archiving
> ---------------------------------------------
>
>                 Key: DRILL-2362
>                 URL: https://issues.apache.org/jira/browse/DRILL-2362
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Other
>    Affects Versions: 0.7.0
>            Reporter: Chris Westin
>             Fix For: Future
>
>
> We collect query profile information for analysis purposes, but we keep it forever. At this time, for a few queries, it isn't a problem. But as users start putting Drill into production, automated use via other applications will make this grow quickly. We need to come up with a retention policy mechanism, with suitable settings administrators can use, and implement it so that this data can be cleaned up.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)