You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Greg Brandt <br...@gmail.com> on 2016/03/18 04:31:14 UTC
[math] A Java implementation of STL: A Seasonal-Trend Decomposition
Procedure Based on Loess
Hey Commons Math,
As part of some work on anomaly detection in time series data, a couple
colleagues and I have put together a Java implementation of STL
<http://www.wessa.net/download/stl.pdf>, which we think might be generally
useful.
It is based on commons-math3 LoessInterpolator
<http://commons.apache.org/proper/commons-math/apidocs/org/apache/commons/math3/analysis/interpolation/LoessInterpolator.html>,
so we thought it might be a natural contribution to the project.
The code currently lives here: https://github.com/brandtg/stl-java. From a
usability perspective, I think it needs some cleanup work, and it needs
more thorough testing.
However, an earlier variant of the code has run in production with decent
results on reasonably large time series, so it is not too far off.
Is this something that would be valuable to Commons Math? A cursory search
of JIRA and the mailing lists didn't turn up with anything, so my apologies
if this has been previously discussed.
If so, I can massage the code to be consistent with the Developers Guide,
simplify usage, and add more test coverage, then follow the recommendations
there to create a patch.
Please feel free to submit pull requests or issues on the Github repo in
addition to discussion on this thread.
Thanks,
-Greg
Re: [math] A Java implementation of STL: A Seasonal-Trend
Decomposition Procedure Based on Loess
Posted by Gilles <gi...@harfang.homelinux.org>.
Hi.
On Mon, 21 Mar 2016 23:06:39 -0700, Greg Brandt wrote:
> Hey Gilles,
>
> Thanks for the comments.
>
>>> Would you be around to to keep maintaining to code?
>
> Yes.
>
>>> Out of curiousity, can it detect any kind of periodicity?
>
> The goal of the algorithm is to decompose a time series with known
> seasonality / periodicity into trend, seasonal, and remainder
> components.
> One could conceivably try many different parameters for seasonality,
> then
> use some objective function, probably involving residuals, to come up
> with
> the best value. But this would probably be considered an application
> of STL.
>
>>> There was a proposal to include "triple exponential smoothing", but
>>> is
> was deemed to be already an "application" rather than a general tool.
> At
> first sight, this code seems more in the line of CM. Am I right?
>
> The goal of STL is very similar to the Holt Winters triple
> exponential
> smoothing method. So it may be the case that this falls into the
> application category with respect to CM. There are a lot of tunable
> parameters, and good values depend on data, so it's not so pure from
> a
> mathematical standpoint. However it definitely fills some void, as
> this
> algorithm ships with R. If CM isn't the right place for it, would you
> know
> of a good home?
Not really.
An idea (proposed some time ago) would be to have an "experimental"
package where such code could reside until we can make our mind...
Another (perhaps better) would be to have a "module" with this code.
Regards,
Gilles
>
> Thanks,
> -Greg
>
> On Mon, Mar 21, 2016 at 3:53 AM, Gilles
> <gi...@harfang.homelinux.org>
> wrote:
>
>> Hello.
>>
>> On Thu, 17 Mar 2016 20:31:14 -0700, Greg Brandt wrote:
>>
>>> Hey Commons Math,
>>>
>>> As part of some work on anomaly detection in time series data, a
>>> couple
>>> colleagues and I have put together a Java implementation of STL
>>> <http://www.wessa.net/download/stl.pdf>, which we think might be
>>> generally
>>> useful.
>>>
>>> It is based on commons-math3 LoessInterpolator
>>>
>>> <
>>>
>>> http://commons.apache.org/proper/commons-math/apidocs/org/apache/commons/math3/analysis/interpolation/LoessInterpolator.html
>>> >,
>>> so we thought it might be a natural contribution to the project.
>>>
>>
>> Looks interesting (although I couldn't take the time to read the
>> paper).
>>
>>
>>> The code currently lives here: https://github.com/brandtg/stl-java.
>>> From
>>> a
>>> usability perspective, I think it needs some cleanup work, and it
>>> needs
>>> more thorough testing.
>>>
>>
>> Would you be around to to keep maintaining to code?
>>
>> However, an earlier variant of the code has run in production with
>> decent
>>> results on reasonably large time series, so it is not too far off.
>>>
>>
>> Out of curiousity, can it detect any kind of periodicity?
>>
>> Is this something that would be valuable to Commons Math? A cursory
>> search
>>> of JIRA and the mailing lists didn't turn up with anything, so my
>>> apologies
>>> if this has been previously discussed.
>>>
>>
>> There was a proposal to include "triple exponential smoothing", but
>> is was
>> deemed to be already an "application" rather than a general tool.
>> At first sight, this code seems more in the line of CM. Am I
>> right?
>>
>> Other committers' opinions requested...
>>
>>
>> Best regards,
>> Gilles
>>
>> P.S. Please open a JIRA report ("Wish" type) so that we don't loose
>> track
>> in the flood of emails...
>>
>> If so, I can massage the code to be consistent with the Developers
>> Guide,
>>> simplify usage, and add more test coverage, then follow the
>>> recommendations
>>> there to create a patch.
>>>
>>> Please feel free to submit pull requests or issues on the Github
>>> repo in
>>> addition to discussion on this thread.
>>>
>>> Thanks,
>>> -Greg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] A Java implementation of STL: A Seasonal-Trend
Decomposition Procedure Based on Loess
Posted by Greg Brandt <br...@gmail.com>.
Hey Gilles,
Thanks for the comments.
>> Would you be around to to keep maintaining to code?
Yes.
>> Out of curiousity, can it detect any kind of periodicity?
The goal of the algorithm is to decompose a time series with known
seasonality / periodicity into trend, seasonal, and remainder components.
One could conceivably try many different parameters for seasonality, then
use some objective function, probably involving residuals, to come up with
the best value. But this would probably be considered an application of STL.
>> There was a proposal to include "triple exponential smoothing", but is
was deemed to be already an "application" rather than a general tool. At
first sight, this code seems more in the line of CM. Am I right?
The goal of STL is very similar to the Holt Winters triple exponential
smoothing method. So it may be the case that this falls into the
application category with respect to CM. There are a lot of tunable
parameters, and good values depend on data, so it's not so pure from a
mathematical standpoint. However it definitely fills some void, as this
algorithm ships with R. If CM isn't the right place for it, would you know
of a good home?
Thanks,
-Greg
On Mon, Mar 21, 2016 at 3:53 AM, Gilles <gi...@harfang.homelinux.org>
wrote:
> Hello.
>
> On Thu, 17 Mar 2016 20:31:14 -0700, Greg Brandt wrote:
>
>> Hey Commons Math,
>>
>> As part of some work on anomaly detection in time series data, a couple
>> colleagues and I have put together a Java implementation of STL
>> <http://www.wessa.net/download/stl.pdf>, which we think might be
>> generally
>> useful.
>>
>> It is based on commons-math3 LoessInterpolator
>>
>> <
>> http://commons.apache.org/proper/commons-math/apidocs/org/apache/commons/math3/analysis/interpolation/LoessInterpolator.html
>> >,
>> so we thought it might be a natural contribution to the project.
>>
>
> Looks interesting (although I couldn't take the time to read the paper).
>
>
>> The code currently lives here: https://github.com/brandtg/stl-java. From
>> a
>> usability perspective, I think it needs some cleanup work, and it needs
>> more thorough testing.
>>
>
> Would you be around to to keep maintaining to code?
>
> However, an earlier variant of the code has run in production with decent
>> results on reasonably large time series, so it is not too far off.
>>
>
> Out of curiousity, can it detect any kind of periodicity?
>
> Is this something that would be valuable to Commons Math? A cursory search
>> of JIRA and the mailing lists didn't turn up with anything, so my
>> apologies
>> if this has been previously discussed.
>>
>
> There was a proposal to include "triple exponential smoothing", but is was
> deemed to be already an "application" rather than a general tool.
> At first sight, this code seems more in the line of CM. Am I right?
>
> Other committers' opinions requested...
>
>
> Best regards,
> Gilles
>
> P.S. Please open a JIRA report ("Wish" type) so that we don't loose track
> in the flood of emails...
>
> If so, I can massage the code to be consistent with the Developers Guide,
>> simplify usage, and add more test coverage, then follow the
>> recommendations
>> there to create a patch.
>>
>> Please feel free to submit pull requests or issues on the Github repo in
>> addition to discussion on this thread.
>>
>> Thanks,
>> -Greg
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] A Java implementation of STL: A Seasonal-Trend
Decomposition Procedure Based on Loess
Posted by Gilles <gi...@harfang.homelinux.org>.
Hello.
On Thu, 17 Mar 2016 20:31:14 -0700, Greg Brandt wrote:
> Hey Commons Math,
>
> As part of some work on anomaly detection in time series data, a
> couple
> colleagues and I have put together a Java implementation of STL
> <http://www.wessa.net/download/stl.pdf>, which we think might be
> generally
> useful.
>
> It is based on commons-math3 LoessInterpolator
>
> <http://commons.apache.org/proper/commons-math/apidocs/org/apache/commons/math3/analysis/interpolation/LoessInterpolator.html>,
> so we thought it might be a natural contribution to the project.
Looks interesting (although I couldn't take the time to read the
paper).
>
> The code currently lives here: https://github.com/brandtg/stl-java.
> From a
> usability perspective, I think it needs some cleanup work, and it
> needs
> more thorough testing.
Would you be around to to keep maintaining to code?
> However, an earlier variant of the code has run in production with
> decent
> results on reasonably large time series, so it is not too far off.
Out of curiousity, can it detect any kind of periodicity?
> Is this something that would be valuable to Commons Math? A cursory
> search
> of JIRA and the mailing lists didn't turn up with anything, so my
> apologies
> if this has been previously discussed.
There was a proposal to include "triple exponential smoothing", but is
was
deemed to be already an "application" rather than a general tool.
At first sight, this code seems more in the line of CM. Am I right?
Other committers' opinions requested...
Best regards,
Gilles
P.S. Please open a JIRA report ("Wish" type) so that we don't loose
track
in the flood of emails...
> If so, I can massage the code to be consistent with the Developers
> Guide,
> simplify usage, and add more test coverage, then follow the
> recommendations
> there to create a patch.
>
> Please feel free to submit pull requests or issues on the Github repo
> in
> addition to discussion on this thread.
>
> Thanks,
> -Greg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org