[jira] [Commented] (METRON-529) Allow Configurable Periods for Profiler Client

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/METRON-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15813126#comment-15813126 ] 

ASF GitHub Bot commented on METRON-529:
---------------------------------------

GitHub user mattf-horton opened a pull request:

    https://github.com/apache/incubator-metron/pull/414

    Metron-532 Define Profile Period When Calling PROFILE_GET

    This PR builds on top of @nickwallen 's work in METRON-529, which has already been committed to master. It adds an optional argument to PROFILE_GET, allowing run-time overrides of the profiler client global configuration parameters. The primary use case is when historical profiles have been created with a different profile configuration than is currently configured, and the analyst needing to access them does not want to change the global client configuration so as not to disrupt the work of other analysts working with current profiles.
    
    This PR is essentially similar to the former proposed PR-345, except that the signature of the PROFILE_GET function has been changed to make `groups` be a List instead of a "varargs"-style sequence, so the new `config_overrides` Map argument can come after it.  This avoids a really hacky over-indulgence in polymorphism, and is consistent with discussion in PR-395.  I'm opening a new PR, rather than reopening the old one, because I also had to rebase to changes in master, and the forced push destroyed the history that PR-345 pointed at.
    
    At suggestion of @cestella , scope of this work item was expanded to include checking for config changes at run time, in the PROFILE_GET Stellar function.  This is a much needed enhancement.
    
    I also considerably improved the documentation of PROFILE_GET as regards the `groups` argument, with input from @nickwallen .  Thanks, Nick!
    
    Testing: This build has passed Travis in my fork, and should shortly be reported passed here.  Since the only code changes were to the Stellar function, the corresponding junit tests are actually sufficient testing.  However, I will go ahead and test them on a live single-host test platform, taking the opportunity to test the new Profiler RPMs from PR-413!  I'll report the results here tomorrow.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mattf-horton/incubator-metron METRON-532

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-metron/pull/414.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #414
    
----
commit 837b7fa3656ddc6a408d69e6438d5270d7fa6eec
Author: mattf-horton <mf...@hortonworks.com>
Date:   2017-01-04T20:09:22Z

    METRON-532 Define Profile Period When Calling PROFILE_GET
    Rebase original PR-345 to current master.

commit 55a7c7aa0757bf55cfdd709063b11d7f4770525c
Author: mattf-horton <mf...@hortonworks.com>
Date:   2017-01-06T20:33:38Z

    METRON-532 continued. Changed PROFILE_GET signature to make 'groups' argument a list; deprecate but still allow the former varargs-style usage. Improved documentation with input from Nick.

commit f7ef2c2fd438933677b661a20ec35f8cd98cba3f
Author: mattf-horton <mf...@hortonworks.com>
Date:   2017-01-06T23:40:12Z

    METRON-532 fix unit tests for new PROFILE_GET signature, and explicitly test the deprecated older form.

commit 40ecab2798cca6ef96b73e543c2161a64a943f2f
Author: mattf-horton <mf...@hortonworks.com>
Date:   2017-01-09T19:31:32Z

    Update metron-platform/metron-common/README.md to match new Stellar doc standard, and tweak the description a little more.

----


> Allow Configurable Periods for Profiler Client
> ----------------------------------------------
>
>                 Key: METRON-529
>                 URL: https://issues.apache.org/jira/browse/METRON-529
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Nick Allen
>            Assignee: Nick Allen
>             Fix For: 0.3.0
>
>
> By default, the Profiler creates Profiles with a period duration of 15 minutes. This means that data is accumulated, summarized and flushed every 15 minutes. The Client API must also have knowledge of this duration to correctly retrieve the profile data. If the client API is expecting 15 minute periods, it will not be able to read data generated by a Profiler that has been configured with a 1 hour period.
> The period duration can be configured in the Profiler by altering the Profiler topology's static properties file. The Stellar Client API currently provides no means to configure the period duration and defaults also to 15 minutes. This means that the Stellar Client API can only read profiles with a period duration of 15 minutes. This is a known limitation that has not yet been addressed.
> https://github.com/apache/incubator-metron/blob/e31705029fd7518bd03c22eece0c2a8eb1187abf/metron-analytics/metron-profiler-client/README.md#period-duration



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)