You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Thorsten Kiefer <ki...@gmx.de> on 2014/12/03 10:13:08 UTC
Re: [Math] Would like to know, if extensions are needed
Hi,
the port is now finished.
The code may still contain bugs.
Maybe alpha-state.
But if you like it, you can add it.
Best Regards
Thorsten Kiefer
Am 17.11.2014 um 23:57 schrieb Thorsten Kiefer:
> Hi,
> I decided to port it to java, and if you should like it, you can extend
> [Math] with RL algorithms.
>
> https://github.com/toki78/JuRLs/tree/master/JuRLs/src/jurls/core
> It will take a while, until I ported it all to java.
>
> Regards
> Thorsten
>
>
> Am 15.11.2014 um 15:01 schrieb Gilles:
>> On Sat, 15 Nov 2014 14:32:07 +0100, Thorsten Kiefer wrote:
>>> Hello,
>>>
>>> Am 15.11.2014 um 13:39 schrieb Gilles:
>>>> Hello.
>>>>
>>>> On Sat, 15 Nov 2014 06:15:32 +0100, Thorsten Kiefer wrote:
>>>>> Hi,
>>>>> I coded this in JavaScript :
>>>>> http://toki78.github.io/
>>>>>
>>>>> I am wondering, if parts of this might be useful for the math module.
>>>>> If so, I could contribute it.
>>>>
>>>> There are some machine-learning algorithms in Commons Math, more
>>>> would probably be welcome.
>>>> Could you perhaps describe with a little more details what you are
>>>> proposing?
>>> I would like to implement a (non-turing-complete) meta languge
>>
>> I don't think that CM would be the right place for a special-purpose
>> language. [Some time ago, there was a proposal for a language to
>> create arithmetical expressions.]
>> If you think otherwise, could you provide some rationale, so that we
>> can comment on concrete arguments?
>>
>>> for
>>> the approximation function and the RL-update rule.
>>> Then I would provide Q(lambda) and SARSA(lambda) using the fourier
>>> basis.
>>> Might be useful for somebody.
>>
>> Perhaps. I don't know what are RL, Q, SARSA.
>>
>>> The Meta-compiler's result could implement the interfaces used by the
>>> FunctionFitters.
>>> As far as I can remember : MultivariateFunction or so...
>>
>> This looks like your project would depend on Commons Math, but need not
>> be part of it.
>>
>>> For more understanding you can also look up the code on my git profile.
>>
>> I tried to read "README.md": it just says "My website"...
>> Care to give a link to a description? Thanks.
>>
>>
>> Regards,
>> Gilles
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Would like to know, if extensions are needed
Posted by Gilles <gi...@harfang.homelinux.org>.
Hi.
On Thu, 18 Dec 2014 23:13:00 +0100, Thorsten Kiefer wrote:
> Hello,
>
> Am 03.12.2014 um 12:53 schrieb Gilles:
>> Hello.
>>
>> On Wed, 03 Dec 2014 10:13:08 +0100, Thorsten Kiefer wrote:
>>> Hi,
>>> the port is now finished.
>>> The code may still contain bugs.
>>> Maybe alpha-state.
>>> But if you like it, you can add it.
>>
>> Thank you for offering code to enhance the scope of Commons Math.
>> Yet I must emphasize that one of the goals of the project is to
>> provide an "integrated" set of functionalities (a.o. consistent
>> style and API throughout the codebase, full documentation and test
>> coverage).
>> Another on-going task is to include contributors who will stay and
>> maintain the codebase.
>> We do not usually have the resources neither to integrate a new
>> functionality, nor to maintain it afterwards.
>>
>> You are very welcome to explain the purpose and state of your
>> contribution; then we can start a discussion on how to move toward
>> including the code in Commons Math.
>
> The purpose of my project is a learning algorithm, which uses
> function approximation and reinforcement learning.
> With reinforcement learning a so-called agent can learn
> optimal behaviour from only receiving rewards
> for actions done in perceived states.
>
> At the moment I am having problems to stabilize the algorithms.
> The topology of the approximation functions and the initial
> values for the parameters are essential for success.
>
> At the moment only the fourier basis is practical as
> approximation function.
>
>
> Naively one would create a lookup table of scores for state-action
> pairs.
> As the impractical huge size of these lookup tables is the problem,
> the lookup table is being interpolated by an approximation function.
>
> The lookup table and respectively the approximation function
> map state-action pairs to scorings.
>
> The RL algorithm adujsts the scorings during simulation.
> And then it can decide, which action to take, by aiming for the
> highest score.
>
> But still all unstable.
>
The project certainly looks interesting.
However, IMHO, there is still a lot of work for your code to
to be considered for incorporating into Commons Math; as I
indicated above, it lacks documentation, references, unit
tests...
A possible step would be to use Commons Math whenever it
provides an alternative to yours (cf. "DiffableFunction" vs.
CM's "UnivariateDifferentiableFunction", "Cosine" vs CM's
"Cos", etc.).
[This would certainly help reviewers since the code would
contain already known bits, but more importantly, it will
avoid feature (and code) duplication.]
Then let us know where CM is lacking required functionalities,
and provide the missing features as independent contributions.
[They would be more easily grasped, discussed, and decided on.
And it will also made it easier to eventually review the bulk
of your project since several parts would have already been
included.]
Best,
Gilles
>>> [...]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Would like to know, if extensions are needed
Posted by Thorsten Kiefer <ki...@gmx.de>.
Hello,
Am 03.12.2014 um 12:53 schrieb Gilles:
> Hello.
>
> On Wed, 03 Dec 2014 10:13:08 +0100, Thorsten Kiefer wrote:
>> Hi,
>> the port is now finished.
>> The code may still contain bugs.
>> Maybe alpha-state.
>> But if you like it, you can add it.
>
> Thank you for offering code to enhance the scope of Commons Math.
> Yet I must emphasize that one of the goals of the project is to
> provide an "integrated" set of functionalities (a.o. consistent
> style and API throughout the codebase, full documentation and test
> coverage).
> Another on-going task is to include contributors who will stay and
> maintain the codebase.
> We do not usually have the resources neither to integrate a new
> functionality, nor to maintain it afterwards.
>
> You are very welcome to explain the purpose and state of your
> contribution; then we can start a discussion on how to move toward
> including the code in Commons Math.
The purpose of my project is a learning algorithm, which uses
function approximation and reinforcement learning.
With reinforcement learning a so-called agent can learn
optimal behaviour from only receiving rewards
for actions done in perceived states.
At the moment I am having problems to stabilize the algorithms.
The topology of the approximation functions and the initial
values for the parameters are essential for success.
At the moment only the fourier basis is practical as
approximation function.
Naively one would create a lookup table of scores for state-action pairs.
As the impractical huge size of these lookup tables is the problem,
the lookup table is being interpolated by an approximation function.
The lookup table and respectively the approximation function
map state-action pairs to scorings.
The RL algorithm adujsts the scorings during simulation.
And then it can decide, which action to take, by aiming for the highest
score.
But still all unstable.
Regards
Thorsten Kiefer
>
>
> Thanks for your interest in the project,
> Gilles
>
>
>>
>> Best Regards
>> Thorsten Kiefer
>>
>>
>> Am 17.11.2014 um 23:57 schrieb Thorsten Kiefer:
>>> Hi,
>>> I decided to port it to java, and if you should like it, you can extend
>>> [Math] with RL algorithms.
>>>
>>> https://github.com/toki78/JuRLs/tree/master/JuRLs/src/jurls/core
>>> It will take a while, until I ported it all to java.
>>>
>>> Regards
>>> Thorsten
>>>
>>>
>>> Am 15.11.2014 um 15:01 schrieb Gilles:
>>>> On Sat, 15 Nov 2014 14:32:07 +0100, Thorsten Kiefer wrote:
>>>>> Hello,
>>>>>
>>>>> Am 15.11.2014 um 13:39 schrieb Gilles:
>>>>>> Hello.
>>>>>>
>>>>>> On Sat, 15 Nov 2014 06:15:32 +0100, Thorsten Kiefer wrote:
>>>>>>> Hi,
>>>>>>> I coded this in JavaScript :
>>>>>>> http://toki78.github.io/
>>>>>>>
>>>>>>> I am wondering, if parts of this might be useful for the math
>>>>>>> module.
>>>>>>> If so, I could contribute it.
>>>>>>
>>>>>> There are some machine-learning algorithms in Commons Math, more
>>>>>> would probably be welcome.
>>>>>> Could you perhaps describe with a little more details what you are
>>>>>> proposing?
>>>>> I would like to implement a (non-turing-complete) meta languge
>>>>
>>>> I don't think that CM would be the right place for a special-purpose
>>>> language. [Some time ago, there was a proposal for a language to
>>>> create arithmetical expressions.]
>>>> If you think otherwise, could you provide some rationale, so that we
>>>> can comment on concrete arguments?
>>>>
>>>>> for
>>>>> the approximation function and the RL-update rule.
>>>>> Then I would provide Q(lambda) and SARSA(lambda) using the fourier
>>>>> basis.
>>>>> Might be useful for somebody.
>>>>
>>>> Perhaps. I don't know what are RL, Q, SARSA.
>>>>
>>>>> The Meta-compiler's result could implement the interfaces used by the
>>>>> FunctionFitters.
>>>>> As far as I can remember : MultivariateFunction or so...
>>>>
>>>> This looks like your project would depend on Commons Math, but need
>>>> not
>>>> be part of it.
>>>>
>>>>> For more understanding you can also look up the code on my git
>>>>> profile.
>>>>
>>>> I tried to read "README.md": it just says "My website"...
>>>> Care to give a link to a description? Thanks.
>>>>
>>>>
>>>> Regards,
>>>> Gilles
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Would like to know, if extensions are needed
Posted by Gilles <gi...@harfang.homelinux.org>.
Hello.
On Wed, 03 Dec 2014 10:13:08 +0100, Thorsten Kiefer wrote:
> Hi,
> the port is now finished.
> The code may still contain bugs.
> Maybe alpha-state.
> But if you like it, you can add it.
Thank you for offering code to enhance the scope of Commons Math.
Yet I must emphasize that one of the goals of the project is to
provide an "integrated" set of functionalities (a.o. consistent
style and API throughout the codebase, full documentation and test
coverage).
Another on-going task is to include contributors who will stay and
maintain the codebase.
We do not usually have the resources neither to integrate a new
functionality, nor to maintain it afterwards.
You are very welcome to explain the purpose and state of your
contribution; then we can start a discussion on how to move toward
including the code in Commons Math.
Thanks for your interest in the project,
Gilles
>
> Best Regards
> Thorsten Kiefer
>
>
> Am 17.11.2014 um 23:57 schrieb Thorsten Kiefer:
>> Hi,
>> I decided to port it to java, and if you should like it, you can
>> extend
>> [Math] with RL algorithms.
>>
>> https://github.com/toki78/JuRLs/tree/master/JuRLs/src/jurls/core
>> It will take a while, until I ported it all to java.
>>
>> Regards
>> Thorsten
>>
>>
>> Am 15.11.2014 um 15:01 schrieb Gilles:
>>> On Sat, 15 Nov 2014 14:32:07 +0100, Thorsten Kiefer wrote:
>>>> Hello,
>>>>
>>>> Am 15.11.2014 um 13:39 schrieb Gilles:
>>>>> Hello.
>>>>>
>>>>> On Sat, 15 Nov 2014 06:15:32 +0100, Thorsten Kiefer wrote:
>>>>>> Hi,
>>>>>> I coded this in JavaScript :
>>>>>> http://toki78.github.io/
>>>>>>
>>>>>> I am wondering, if parts of this might be useful for the math
>>>>>> module.
>>>>>> If so, I could contribute it.
>>>>>
>>>>> There are some machine-learning algorithms in Commons Math, more
>>>>> would probably be welcome.
>>>>> Could you perhaps describe with a little more details what you
>>>>> are
>>>>> proposing?
>>>> I would like to implement a (non-turing-complete) meta languge
>>>
>>> I don't think that CM would be the right place for a
>>> special-purpose
>>> language. [Some time ago, there was a proposal for a language to
>>> create arithmetical expressions.]
>>> If you think otherwise, could you provide some rationale, so that
>>> we
>>> can comment on concrete arguments?
>>>
>>>> for
>>>> the approximation function and the RL-update rule.
>>>> Then I would provide Q(lambda) and SARSA(lambda) using the fourier
>>>> basis.
>>>> Might be useful for somebody.
>>>
>>> Perhaps. I don't know what are RL, Q, SARSA.
>>>
>>>> The Meta-compiler's result could implement the interfaces used by
>>>> the
>>>> FunctionFitters.
>>>> As far as I can remember : MultivariateFunction or so...
>>>
>>> This looks like your project would depend on Commons Math, but need
>>> not
>>> be part of it.
>>>
>>>> For more understanding you can also look up the code on my git
>>>> profile.
>>>
>>> I tried to read "README.md": it just says "My website"...
>>> Care to give a link to a description? Thanks.
>>>
>>>
>>> Regards,
>>> Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org