You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Alexander Nozik <al...@gmail.com> on 2018/02/17 16:25:30 UTC
[math] Automatic differentiation with names
Hello all,
Gilles suggested that I should write some considerations about
improvement to common maths automatic differentiation here. I've opened
an Issue here: https://issues.apache.org/jira/browse/MATH-1452 (I really
do not like mailing lists). But since I've done my own implementation on
top of Commons Maths, I can share it. The implementation is in Kotlin,
so I won't be able to contribute it to commons math as is, but still we
can discuss how to backport it to java.
Here is the source code of two classes which solve
https://issues.apache.org/jira/browse/MATH-1448 and partially MATH-1452:
https://bitbucket.org/Altavir/dataforge/src/1433da0d2f19ab26d709de8750e002847a4b4887/dataforge-maths/src/main/kotlin/hep/dataforge/maths/expressions/AD.kt?at=ad&fileviewer=file-view-default
The class on the top is a Context for mathematical operations with AD
numbers. Since CM already have Fields, it implements the Field. The idea
is that context stores some critical properties of numbers which are
shared between instances like order of differentiation and parameters
names. In this case the object equality check used instead of instance
equality, so to context with same orders and set of names are still
equal. The AD class below is just a wrapper for DerivativeStructure that
also includes Field context. Binary operations additionally perform
check if argument uses the same context as `this` object.
The actual check occurs here:
https://bitbucket.org/Altavir/dataforge/src/1433da0d2f19ab26d709de8750e002847a4b4887/dataforge-maths/src/main/kotlin/hep/dataforge/maths/expressions/AD.kt?at=ad&fileviewer=file-view-default#AD.kt-78.
In theory, one can perform field transformation, merging parameters sets
from both ADs, but it requires understanding about inner workings of
DerivativeStructure which I lack. Basically what I need is an ability to
create a new DerivativeStructure with parameter number i mapped to
number j.
The proposed solution does not seem to involve any major performance
impact. Name resolution happens only when one calls derivative with
given name and is not really a great performance impact since string
hashes are calculated on string creation. The only place with actual
performance impact is when field transformation happens (if it will be
implemented), but this transformation is supposed to be rare and it
currently not possible at all.
The test code in kotlin looks like this:
https://bitbucket.org/Altavir/dataforge/src/8131099f29ebf27fb170ace037cda61df9790fc2/dataforge-maths/src/test/java/hep/dataforge/maths/expressions/ADTest.kt?at=ad&fileviewer=file-view-default.
It is kotlin though, it would be much more verbatim in java. I plan also
to implement expressions which would allow lazy calculations of AD
structures like (NamedVector)->AD. This one could be more or less easily
done in Java.
With best regards, Alexander Nozik.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Automatic differentiation with names
Posted by Gilles <gi...@harfang.homelinux.org>.
On Mon, 19 Feb 2018 16:12:05 +0300, Alexander Nozik wrote:
> On 19.02.2018 15:58, Gilles wrote:
>> Unless I'm totally off base, I guess a code written in Kotlin
>> needs specific support (e.g. a library dependency) to be run
>> on a JVM.
> If you do not use standard library, then no, you do not need anything
> else. And I mean that code could be rewritten in java and all
> references to the standard library replaced by pure java analogues.
>
>>> But I can convert kotlin code back to Java.
>>
>> Is it a one-time port (and thereon, maintenance is done on the
>> Java code)? Or do you mean a conversion step (as part of the
>> build for example) that creates Java sources (or bytecode) so
>> that maintenance requires coding in Kotlin?
> I mean one-time conversion. What we are talking about is a minor
> change to the code that probably won't need special long-time
> support.
> My project currently is in Java 8, Groovy and Kotlin, and I am not
> able to support something, which requires backward compatibility with
> older versions (do not really have tome for that).
>
>>> Commons math does not
>>> support newer versions of JVM with functional features,
>>
>> My latest suggestion is to target Java 8.
> I totally agree. Java 8 adds a lot of functional-style features which
> are life-savers for mathematical tools.
>
>>
>>> so for now it
>>> easier to think in kotlin,
>>
>> Not for me. :-}
>> I can imagine it has interesting features for those who
>> use it. But are those indispensable for the project we
>> are talking about?
>
> The major things are extension functions and receivers. Everything
> could be done using plain old java, but it looks much more
> cumbersome.
>
>>
>>> and then produce Java6 compatible
>>> classes.
>>
>> Do you have a requirement to use Java 6?
> I have requirement for Java 8, but I am sure, everything could be
> implemented using older standards.
>
>
> I have few different things in mind. Most of them are possible and
> rather simple in pure Java. I will get back to you, when I have a
> piece of free time to work on it.
I've opened a report on the NUMBERS JIRA project:
https://issues.apache.org/jira/browse/NUMBERS-69
and a dedicated git branch:
feature__NUMBERS-69__autodiff
Please use it for your upcoming pull requests.
Thanks a lot,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Automatic differentiation with names
Posted by Alexander Nozik <al...@gmail.com>.
On 19.02.2018 15:58, Gilles wrote:
> Unless I'm totally off base, I guess a code written in Kotlin
> needs specific support (e.g. a library dependency) to be run
> on a JVM.
If you do not use standard library, then no, you do not need anything
else. And I mean that code could be rewritten in java and all references
to the standard library replaced by pure java analogues.
>> But I can convert kotlin code back to Java.
>
> Is it a one-time port (and thereon, maintenance is done on the
> Java code)? Or do you mean a conversion step (as part of the
> build for example) that creates Java sources (or bytecode) so
> that maintenance requires coding in Kotlin?
I mean one-time conversion. What we are talking about is a minor change
to the code that probably won't need special long-time support. My
project currently is in Java 8, Groovy and Kotlin, and I am not able to
support something, which requires backward compatibility with older
versions (do not really have tome for that).
>> Commons math does not
>> support newer versions of JVM with functional features,
>
> My latest suggestion is to target Java 8.
I totally agree. Java 8 adds a lot of functional-style features which
are life-savers for mathematical tools.
>
>> so for now it
>> easier to think in kotlin,
>
> Not for me. :-}
> I can imagine it has interesting features for those who
> use it. But are those indispensable for the project we
> are talking about?
The major things are extension functions and receivers. Everything could
be done using plain old java, but it looks much more cumbersome.
>
>> and then produce Java6 compatible
>> classes.
>
> Do you have a requirement to use Java 6?
I have requirement for Java 8, but I am sure, everything could be
implemented using older standards.
I have few different things in mind. Most of them are possible and
rather simple in pure Java. I will get back to you, when I have a piece
of free time to work on it.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Automatic differentiation with names
Posted by Gilles <gi...@harfang.homelinux.org>.
On Mon, 19 Feb 2018 15:17:34 +0300, Alexander Nozik wrote:
> You got me wrong again.
What did I get wrong?
> Kotlin is fully compatible with java and
> usually one converts Java code to Kotlin, which is done
> automatically.
Unless I'm totally off base, I guess a code written in Kotlin
needs specific support (e.g. a library dependency) to be run
on a JVM.
If so, one question is whether it is advantageous to use that
Kotlin rather than pure Java (and do without the additional
library).
> But I can convert kotlin code back to Java.
Is it a one-time port (and thereon, maintenance is done on the
Java code)? Or do you mean a conversion step (as part of the
build for example) that creates Java sources (or bytecode) so
that maintenance requires coding in Kotlin?
> Commons math does not
> support newer versions of JVM with functional features,
My latest suggestion is to target Java 8.
> so for now it
> easier to think in kotlin,
Not for me. :-}
I can imagine it has interesting features for those who
use it. But are those indispensable for the project we
are talking about?
> and then produce Java6 compatible
> classes.
Do you have a requirement to use Java 6?
Gilles
> On 19.02.2018 14:29, Gilles wrote:
>> On Sun, 18 Feb 2018 20:41:35 -0600, Matt Sicker wrote:
>>> We've even talked about adding Scala libraries in the past and
>>> there was
>>> support, so I'd imagine Kotlin is fine as well. It may be worth
>>> including
>>> as its own module mainly due to the Kotlin dependency, though the
>>> domain
>>> itself helps raise it to that level as it is.
>>
>> IIUC, the core engine is CM's "DerivativeStructure" and "DSCompiler"
>> classes; hence, it would seem (?) overkill to add a dependency that
>> will then require a conversion layer in order to be called from
>> Java.
>>
>> Regards,
>> Gilles
>>
>>> On 18 February 2018 at 18:02, Gilles <gi...@harfang.homelinux.org>
>>> wrote:
>>>
>>>> On Sat, 17 Feb 2018 22:30:38 +0300, Alexander Nozik wrote:
>>>>
>>>>> On 17.02.2018 21:16, Gilles wrote:
>>>>>
>>>>>> I have a problem with the CM "Field".
>>>>>> Did you have a look at my comments to issue MATH-1448 (and
>>>>>> related
>>>>>> code)?
>>>>>> Unfortunately, I don't have the time right now to go further
>>>>>> with
>>>>>> that work; but I'm more and more convinced that something is
>>>>>> wrong
>>>>>> with the current design of "Field": if the requirement is to
>>>>>> have
>>>>>> and API that provides
>>>>>> * addition (and its neutral element)
>>>>>> * multiplication (and its neutral element)
>>>>>> * some other functions that allow for more performant
>>>>>> implementations of common opreations
>>>>>> why not just define one or more interfaces to that effect?
>>>>>> [Unless I'm mistaken the most used "field" is the
>>>>>> "RealFieldElement"
>>>>>> with its implementation over "double". Given the inherent
>>>>>> inaccuracy
>>>>>> of floating-point numbers, they actually do not abide by the
>>>>>> (math)
>>>>>> field requirements.]
>>>>>>
>>>>>
>>>>> You got me wrong here. These methods are made only for better
>>>>> user-side API. Kotlin allows to use classes as receivers (run a
>>>>> lambda
>>>>> function, using objects as a context), therefore it sometimes
>>>>> makes
>>>>> sense to create a scope class and add some additional
>>>>> functionality to
>>>>> it. It does not matter though since I already removed all those
>>>>> methods into separate extension class (after back porting to java
>>>>> they
>>>>> will be either static members or won't be needed at all). Also, I
>>>>> can
>>>>> use any custom class for the context, it does not have to be
>>>>> RealField
>>>>> itself.
>>>>>
>>>>> I've finally found the code you were talking about. And those new
>>>>> fields indeed look much better than RealFieldElements. I totally
>>>>> agree
>>>>> that special functions like `sin` etc should be removed form the
>>>>> interface and implemented as a static class like java.util.Math
>>>>> (it
>>>>> would be good just to copy Math contents, so switching from one
>>>>> type
>>>>> of numbers to another will require simple change of imports).
>>>>>
>>>>> I can translate my work back to java when I am done, but it still
>>>>> requires two changes to DerivativeStructure API (at least in its
>>>>> current API) to work better:
>>>>> 1) The Derivative structure should have an additional field with
>>>>> names of parameters. In the current implementation it seemed to
>>>>> me
>>>>> reasonable to use RealField instance since DerivativeStructures
>>>>> with
>>>>> different orders and different set of parameters are members of
>>>>> different fields. So Fields themselves should be parametric. In
>>>>> case
>>>>> of new Field API I think that DerivativeStructure should have an
>>>>> internal object, call it Signature for example, which will store
>>>>> the
>>>>> same information. I can do that myself and post a pull request
>>>>> when I
>>>>> am done.
>>>>> 2) Those RealFields or Signatures could be transformed along with
>>>>> underlying DerivativeStructures therefore allowing to merge two
>>>>> AD
>>>>> numbers with different signatures into a new number with
>>>>> completely
>>>>> new signature. In order to do so, I need a method inside the
>>>>> DerivativeStructure to change the numbering of parameters and add
>>>>> new
>>>>> parameters (with zero derivatives). Derivative structures are
>>>>> just
>>>>> linear structures so it should not be hard, but I am not sure
>>>>> that I
>>>>> will be able to spend enough time on it to understand, how it
>>>>> works.
>>>>>
>>>>> One final remark. We've got an idea we will try to implement in
>>>>> the
>>>>> future. The idea is to use the same API to create a syntactic
>>>>> trees
>>>>> from expressions. It is needed to send a definition of some
>>>>> function
>>>>> to another process or other the internet. In theory, I do not see
>>>>> any
>>>>> differences in implementation, so I think that you should keep
>>>>> this
>>>>> feature in mind. It would be great to be able to just replace one
>>>>> type
>>>>> for another and get the whole new functionality.
>>>>>
>>>>
>>>> I do not have in-depth knowledge of the current code in order to
>>>> figure out the implications.
>>>> Please implement whatever enhancements you have in mind.
>>>> Actually, I'd suggest that we create a new module in "Commons
>>>> Numbers": "commons-numbers-autodiff".
>>>> It would host the refactoring of "DerivativeStructure", using JDK8
>>>> ("java.util.function") and trying to get rid of
>>>> "RealFieldElement",
>>>> seeing how it will impact the unit test suite (and your
>>>> use-cases).
>>>> WDYT?
>>>>
>>>> Thanks,
>>>> Gilles
>>>>
>>>>
>>>>
>>>>> With best regards, Alexander Nozik.
>>>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Automatic differentiation with names
Posted by Alexander Nozik <al...@gmail.com>.
You got me wrong again. Kotlin is fully compatible with java and usually
one converts Java code to Kotlin, which is done automatically. But I can
convert kotlin code back to Java. Commons math does not support newer
versions of JVM with functional features, so for now it easier to think
in kotlin, and then produce Java6 compatible classes.
On 19.02.2018 14:29, Gilles wrote:
> On Sun, 18 Feb 2018 20:41:35 -0600, Matt Sicker wrote:
>> We've even talked about adding Scala libraries in the past and there was
>> support, so I'd imagine Kotlin is fine as well. It may be worth
>> including
>> as its own module mainly due to the Kotlin dependency, though the domain
>> itself helps raise it to that level as it is.
>
> IIUC, the core engine is CM's "DerivativeStructure" and "DSCompiler"
> classes; hence, it would seem (?) overkill to add a dependency that
> will then require a conversion layer in order to be called from Java.
>
> Regards,
> Gilles
>
>> On 18 February 2018 at 18:02, Gilles <gi...@harfang.homelinux.org>
>> wrote:
>>
>>> On Sat, 17 Feb 2018 22:30:38 +0300, Alexander Nozik wrote:
>>>
>>>> On 17.02.2018 21:16, Gilles wrote:
>>>>
>>>>> I have a problem with the CM "Field".
>>>>> Did you have a look at my comments to issue MATH-1448 (and related
>>>>> code)?
>>>>> Unfortunately, I don't have the time right now to go further with
>>>>> that work; but I'm more and more convinced that something is wrong
>>>>> with the current design of "Field": if the requirement is to have
>>>>> and API that provides
>>>>> * addition (and its neutral element)
>>>>> * multiplication (and its neutral element)
>>>>> * some other functions that allow for more performant
>>>>> implementations of common opreations
>>>>> why not just define one or more interfaces to that effect?
>>>>> [Unless I'm mistaken the most used "field" is the "RealFieldElement"
>>>>> with its implementation over "double". Given the inherent inaccuracy
>>>>> of floating-point numbers, they actually do not abide by the (math)
>>>>> field requirements.]
>>>>>
>>>>
>>>> You got me wrong here. These methods are made only for better
>>>> user-side API. Kotlin allows to use classes as receivers (run a lambda
>>>> function, using objects as a context), therefore it sometimes makes
>>>> sense to create a scope class and add some additional functionality to
>>>> it. It does not matter though since I already removed all those
>>>> methods into separate extension class (after back porting to java they
>>>> will be either static members or won't be needed at all). Also, I can
>>>> use any custom class for the context, it does not have to be RealField
>>>> itself.
>>>>
>>>> I've finally found the code you were talking about. And those new
>>>> fields indeed look much better than RealFieldElements. I totally agree
>>>> that special functions like `sin` etc should be removed form the
>>>> interface and implemented as a static class like java.util.Math (it
>>>> would be good just to copy Math contents, so switching from one type
>>>> of numbers to another will require simple change of imports).
>>>>
>>>> I can translate my work back to java when I am done, but it still
>>>> requires two changes to DerivativeStructure API (at least in its
>>>> current API) to work better:
>>>> 1) The Derivative structure should have an additional field with
>>>> names of parameters. In the current implementation it seemed to me
>>>> reasonable to use RealField instance since DerivativeStructures with
>>>> different orders and different set of parameters are members of
>>>> different fields. So Fields themselves should be parametric. In case
>>>> of new Field API I think that DerivativeStructure should have an
>>>> internal object, call it Signature for example, which will store the
>>>> same information. I can do that myself and post a pull request when I
>>>> am done.
>>>> 2) Those RealFields or Signatures could be transformed along with
>>>> underlying DerivativeStructures therefore allowing to merge two AD
>>>> numbers with different signatures into a new number with completely
>>>> new signature. In order to do so, I need a method inside the
>>>> DerivativeStructure to change the numbering of parameters and add new
>>>> parameters (with zero derivatives). Derivative structures are just
>>>> linear structures so it should not be hard, but I am not sure that I
>>>> will be able to spend enough time on it to understand, how it works.
>>>>
>>>> One final remark. We've got an idea we will try to implement in the
>>>> future. The idea is to use the same API to create a syntactic trees
>>>> from expressions. It is needed to send a definition of some function
>>>> to another process or other the internet. In theory, I do not see any
>>>> differences in implementation, so I think that you should keep this
>>>> feature in mind. It would be great to be able to just replace one type
>>>> for another and get the whole new functionality.
>>>>
>>>
>>> I do not have in-depth knowledge of the current code in order to
>>> figure out the implications.
>>> Please implement whatever enhancements you have in mind.
>>> Actually, I'd suggest that we create a new module in "Commons
>>> Numbers": "commons-numbers-autodiff".
>>> It would host the refactoring of "DerivativeStructure", using JDK8
>>> ("java.util.function") and trying to get rid of "RealFieldElement",
>>> seeing how it will impact the unit test suite (and your use-cases).
>>> WDYT?
>>>
>>> Thanks,
>>> Gilles
>>>
>>>
>>>
>>>> With best regards, Alexander Nozik.
>>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Automatic differentiation with names
Posted by Gilles <gi...@harfang.homelinux.org>.
On Sun, 18 Feb 2018 20:41:35 -0600, Matt Sicker wrote:
> We've even talked about adding Scala libraries in the past and there
> was
> support, so I'd imagine Kotlin is fine as well. It may be worth
> including
> as its own module mainly due to the Kotlin dependency, though the
> domain
> itself helps raise it to that level as it is.
IIUC, the core engine is CM's "DerivativeStructure" and "DSCompiler"
classes; hence, it would seem (?) overkill to add a dependency that
will then require a conversion layer in order to be called from Java.
Regards,
Gilles
> On 18 February 2018 at 18:02, Gilles <gi...@harfang.homelinux.org>
> wrote:
>
>> On Sat, 17 Feb 2018 22:30:38 +0300, Alexander Nozik wrote:
>>
>>> On 17.02.2018 21:16, Gilles wrote:
>>>
>>>> I have a problem with the CM "Field".
>>>> Did you have a look at my comments to issue MATH-1448 (and related
>>>> code)?
>>>> Unfortunately, I don't have the time right now to go further with
>>>> that work; but I'm more and more convinced that something is wrong
>>>> with the current design of "Field": if the requirement is to have
>>>> and API that provides
>>>> * addition (and its neutral element)
>>>> * multiplication (and its neutral element)
>>>> * some other functions that allow for more performant
>>>> implementations of common opreations
>>>> why not just define one or more interfaces to that effect?
>>>> [Unless I'm mistaken the most used "field" is the
>>>> "RealFieldElement"
>>>> with its implementation over "double". Given the inherent
>>>> inaccuracy
>>>> of floating-point numbers, they actually do not abide by the
>>>> (math)
>>>> field requirements.]
>>>>
>>>
>>> You got me wrong here. These methods are made only for better
>>> user-side API. Kotlin allows to use classes as receivers (run a
>>> lambda
>>> function, using objects as a context), therefore it sometimes makes
>>> sense to create a scope class and add some additional functionality
>>> to
>>> it. It does not matter though since I already removed all those
>>> methods into separate extension class (after back porting to java
>>> they
>>> will be either static members or won't be needed at all). Also, I
>>> can
>>> use any custom class for the context, it does not have to be
>>> RealField
>>> itself.
>>>
>>> I've finally found the code you were talking about. And those new
>>> fields indeed look much better than RealFieldElements. I totally
>>> agree
>>> that special functions like `sin` etc should be removed form the
>>> interface and implemented as a static class like java.util.Math (it
>>> would be good just to copy Math contents, so switching from one
>>> type
>>> of numbers to another will require simple change of imports).
>>>
>>> I can translate my work back to java when I am done, but it still
>>> requires two changes to DerivativeStructure API (at least in its
>>> current API) to work better:
>>> 1) The Derivative structure should have an additional field with
>>> names of parameters. In the current implementation it seemed to me
>>> reasonable to use RealField instance since DerivativeStructures
>>> with
>>> different orders and different set of parameters are members of
>>> different fields. So Fields themselves should be parametric. In
>>> case
>>> of new Field API I think that DerivativeStructure should have an
>>> internal object, call it Signature for example, which will store
>>> the
>>> same information. I can do that myself and post a pull request when
>>> I
>>> am done.
>>> 2) Those RealFields or Signatures could be transformed along with
>>> underlying DerivativeStructures therefore allowing to merge two AD
>>> numbers with different signatures into a new number with completely
>>> new signature. In order to do so, I need a method inside the
>>> DerivativeStructure to change the numbering of parameters and add
>>> new
>>> parameters (with zero derivatives). Derivative structures are just
>>> linear structures so it should not be hard, but I am not sure that
>>> I
>>> will be able to spend enough time on it to understand, how it
>>> works.
>>>
>>> One final remark. We've got an idea we will try to implement in the
>>> future. The idea is to use the same API to create a syntactic trees
>>> from expressions. It is needed to send a definition of some
>>> function
>>> to another process or other the internet. In theory, I do not see
>>> any
>>> differences in implementation, so I think that you should keep this
>>> feature in mind. It would be great to be able to just replace one
>>> type
>>> for another and get the whole new functionality.
>>>
>>
>> I do not have in-depth knowledge of the current code in order to
>> figure out the implications.
>> Please implement whatever enhancements you have in mind.
>> Actually, I'd suggest that we create a new module in "Commons
>> Numbers": "commons-numbers-autodiff".
>> It would host the refactoring of "DerivativeStructure", using JDK8
>> ("java.util.function") and trying to get rid of "RealFieldElement",
>> seeing how it will impact the unit test suite (and your use-cases).
>> WDYT?
>>
>> Thanks,
>> Gilles
>>
>>
>>
>>> With best regards, Alexander Nozik.
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Automatic differentiation with names
Posted by Matt Sicker <bo...@gmail.com>.
We've even talked about adding Scala libraries in the past and there was
support, so I'd imagine Kotlin is fine as well. It may be worth including
as its own module mainly due to the Kotlin dependency, though the domain
itself helps raise it to that level as it is.
On 18 February 2018 at 18:02, Gilles <gi...@harfang.homelinux.org> wrote:
> On Sat, 17 Feb 2018 22:30:38 +0300, Alexander Nozik wrote:
>
>> On 17.02.2018 21:16, Gilles wrote:
>>
>>> I have a problem with the CM "Field".
>>> Did you have a look at my comments to issue MATH-1448 (and related
>>> code)?
>>> Unfortunately, I don't have the time right now to go further with
>>> that work; but I'm more and more convinced that something is wrong
>>> with the current design of "Field": if the requirement is to have
>>> and API that provides
>>> * addition (and its neutral element)
>>> * multiplication (and its neutral element)
>>> * some other functions that allow for more performant
>>> implementations of common opreations
>>> why not just define one or more interfaces to that effect?
>>> [Unless I'm mistaken the most used "field" is the "RealFieldElement"
>>> with its implementation over "double". Given the inherent inaccuracy
>>> of floating-point numbers, they actually do not abide by the (math)
>>> field requirements.]
>>>
>>
>> You got me wrong here. These methods are made only for better
>> user-side API. Kotlin allows to use classes as receivers (run a lambda
>> function, using objects as a context), therefore it sometimes makes
>> sense to create a scope class and add some additional functionality to
>> it. It does not matter though since I already removed all those
>> methods into separate extension class (after back porting to java they
>> will be either static members or won't be needed at all). Also, I can
>> use any custom class for the context, it does not have to be RealField
>> itself.
>>
>> I've finally found the code you were talking about. And those new
>> fields indeed look much better than RealFieldElements. I totally agree
>> that special functions like `sin` etc should be removed form the
>> interface and implemented as a static class like java.util.Math (it
>> would be good just to copy Math contents, so switching from one type
>> of numbers to another will require simple change of imports).
>>
>> I can translate my work back to java when I am done, but it still
>> requires two changes to DerivativeStructure API (at least in its
>> current API) to work better:
>> 1) The Derivative structure should have an additional field with
>> names of parameters. In the current implementation it seemed to me
>> reasonable to use RealField instance since DerivativeStructures with
>> different orders and different set of parameters are members of
>> different fields. So Fields themselves should be parametric. In case
>> of new Field API I think that DerivativeStructure should have an
>> internal object, call it Signature for example, which will store the
>> same information. I can do that myself and post a pull request when I
>> am done.
>> 2) Those RealFields or Signatures could be transformed along with
>> underlying DerivativeStructures therefore allowing to merge two AD
>> numbers with different signatures into a new number with completely
>> new signature. In order to do so, I need a method inside the
>> DerivativeStructure to change the numbering of parameters and add new
>> parameters (with zero derivatives). Derivative structures are just
>> linear structures so it should not be hard, but I am not sure that I
>> will be able to spend enough time on it to understand, how it works.
>>
>> One final remark. We've got an idea we will try to implement in the
>> future. The idea is to use the same API to create a syntactic trees
>> from expressions. It is needed to send a definition of some function
>> to another process or other the internet. In theory, I do not see any
>> differences in implementation, so I think that you should keep this
>> feature in mind. It would be great to be able to just replace one type
>> for another and get the whole new functionality.
>>
>
> I do not have in-depth knowledge of the current code in order to
> figure out the implications.
> Please implement whatever enhancements you have in mind.
> Actually, I'd suggest that we create a new module in "Commons
> Numbers": "commons-numbers-autodiff".
> It would host the refactoring of "DerivativeStructure", using JDK8
> ("java.util.function") and trying to get rid of "RealFieldElement",
> seeing how it will impact the unit test suite (and your use-cases).
> WDYT?
>
> Thanks,
> Gilles
>
>
>
>> With best regards, Alexander Nozik.
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
--
Matt Sicker <bo...@gmail.com>
Re: [math] Automatic differentiation with names
Posted by Gilles <gi...@harfang.homelinux.org>.
On Sat, 17 Feb 2018 22:30:38 +0300, Alexander Nozik wrote:
> On 17.02.2018 21:16, Gilles wrote:
>> I have a problem with the CM "Field".
>> Did you have a look at my comments to issue MATH-1448 (and related
>> code)?
>> Unfortunately, I don't have the time right now to go further with
>> that work; but I'm more and more convinced that something is wrong
>> with the current design of "Field": if the requirement is to have
>> and API that provides
>> * addition (and its neutral element)
>> * multiplication (and its neutral element)
>> * some other functions that allow for more performant
>> implementations of common opreations
>> why not just define one or more interfaces to that effect?
>> [Unless I'm mistaken the most used "field" is the "RealFieldElement"
>> with its implementation over "double". Given the inherent inaccuracy
>> of floating-point numbers, they actually do not abide by the (math)
>> field requirements.]
>
> You got me wrong here. These methods are made only for better
> user-side API. Kotlin allows to use classes as receivers (run a
> lambda
> function, using objects as a context), therefore it sometimes makes
> sense to create a scope class and add some additional functionality
> to
> it. It does not matter though since I already removed all those
> methods into separate extension class (after back porting to java
> they
> will be either static members or won't be needed at all). Also, I can
> use any custom class for the context, it does not have to be
> RealField
> itself.
>
> I've finally found the code you were talking about. And those new
> fields indeed look much better than RealFieldElements. I totally
> agree
> that special functions like `sin` etc should be removed form the
> interface and implemented as a static class like java.util.Math (it
> would be good just to copy Math contents, so switching from one type
> of numbers to another will require simple change of imports).
>
> I can translate my work back to java when I am done, but it still
> requires two changes to DerivativeStructure API (at least in its
> current API) to work better:
> 1) The Derivative structure should have an additional field with
> names of parameters. In the current implementation it seemed to me
> reasonable to use RealField instance since DerivativeStructures with
> different orders and different set of parameters are members of
> different fields. So Fields themselves should be parametric. In case
> of new Field API I think that DerivativeStructure should have an
> internal object, call it Signature for example, which will store the
> same information. I can do that myself and post a pull request when I
> am done.
> 2) Those RealFields or Signatures could be transformed along with
> underlying DerivativeStructures therefore allowing to merge two AD
> numbers with different signatures into a new number with completely
> new signature. In order to do so, I need a method inside the
> DerivativeStructure to change the numbering of parameters and add new
> parameters (with zero derivatives). Derivative structures are just
> linear structures so it should not be hard, but I am not sure that I
> will be able to spend enough time on it to understand, how it works.
>
> One final remark. We've got an idea we will try to implement in the
> future. The idea is to use the same API to create a syntactic trees
> from expressions. It is needed to send a definition of some function
> to another process or other the internet. In theory, I do not see any
> differences in implementation, so I think that you should keep this
> feature in mind. It would be great to be able to just replace one
> type
> for another and get the whole new functionality.
I do not have in-depth knowledge of the current code in order to
figure out the implications.
Please implement whatever enhancements you have in mind.
Actually, I'd suggest that we create a new module in "Commons
Numbers": "commons-numbers-autodiff".
It would host the refactoring of "DerivativeStructure", using JDK8
("java.util.function") and trying to get rid of "RealFieldElement",
seeing how it will impact the unit test suite (and your use-cases).
WDYT?
Thanks,
Gilles
>
> With best regards, Alexander Nozik.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Automatic differentiation with names
Posted by Alexander Nozik <al...@gmail.com>.
On 17.02.2018 21:16, Gilles wrote:
> I have a problem with the CM "Field".
> Did you have a look at my comments to issue MATH-1448 (and related
> code)?
> Unfortunately, I don't have the time right now to go further with
> that work; but I'm more and more convinced that something is wrong
> with the current design of "Field": if the requirement is to have
> and API that provides
> * addition (and its neutral element)
> * multiplication (and its neutral element)
> * some other functions that allow for more performant
> implementations of common opreations
> why not just define one or more interfaces to that effect?
> [Unless I'm mistaken the most used "field" is the "RealFieldElement"
> with its implementation over "double". Given the inherent inaccuracy
> of floating-point numbers, they actually do not abide by the (math)
> field requirements.]
You got me wrong here. These methods are made only for better user-side
API. Kotlin allows to use classes as receivers (run a lambda function,
using objects as a context), therefore it sometimes makes sense to
create a scope class and add some additional functionality to it. It
does not matter though since I already removed all those methods into
separate extension class (after back porting to java they will be either
static members or won't be needed at all). Also, I can use any custom
class for the context, it does not have to be RealField itself.
I've finally found the code you were talking about. And those new fields
indeed look much better than RealFieldElements. I totally agree that
special functions like `sin` etc should be removed form the interface
and implemented as a static class like java.util.Math (it would be good
just to copy Math contents, so switching from one type of numbers to
another will require simple change of imports).
I can translate my work back to java when I am done, but it still
requires two changes to DerivativeStructure API (at least in its current
API) to work better:
1) The Derivative structure should have an additional field with names
of parameters. In the current implementation it seemed to me reasonable
to use RealField instance since DerivativeStructures with different
orders and different set of parameters are members of different fields.
So Fields themselves should be parametric. In case of new Field API I
think that DerivativeStructure should have an internal object, call it
Signature for example, which will store the same information. I can do
that myself and post a pull request when I am done.
2) Those RealFields or Signatures could be transformed along with
underlying DerivativeStructures therefore allowing to merge two AD
numbers with different signatures into a new number with completely new
signature. In order to do so, I need a method inside the
DerivativeStructure to change the numbering of parameters and add new
parameters (with zero derivatives). Derivative structures are just
linear structures so it should not be hard, but I am not sure that I
will be able to spend enough time on it to understand, how it works.
One final remark. We've got an idea we will try to implement in the
future. The idea is to use the same API to create a syntactic trees from
expressions. It is needed to send a definition of some function to
another process or other the internet. In theory, I do not see any
differences in implementation, so I think that you should keep this
feature in mind. It would be great to be able to just replace one type
for another and get the whole new functionality.
With best regards, Alexander Nozik.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Automatic differentiation with names
Posted by Gilles <gi...@harfang.homelinux.org>.
Hi.
On Sat, 17 Feb 2018 19:25:30 +0300, Alexander Nozik wrote:
> Hello all,
>
> Gilles suggested that I should write some considerations about
> improvement to common maths automatic differentiation here. I've
> opened an Issue here: https://issues.apache.org/jira/browse/MATH-1452
> (I really do not like mailing lists). But since I've done my own
> implementation on top of Commons Maths, I can share it.
Thanks a lot for the offer.
I hope that we'll agree on a common path to make the
AD functionality easy to use.
> The
> implementation is in Kotlin, so I won't be able to contribute it to
> commons math as is, but still we can discuss how to backport it to
> java.
I think that a working Java implementation is a requirement for
inclusion in the "Commons" project.
[Discussing contribution in another language would require its
own ML thread (yes, the "dev" ML is still the only official
channel for decisions).]
> Here is the source code of two classes which solve
> https://issues.apache.org/jira/browse/MATH-1448 and partially
> MATH-1452:
>
> https://bitbucket.org/Altavir/dataforge/src/1433da0d2f19ab26d709de8750e002847a4b4887/dataforge-maths/src/main/kotlin/hep/dataforge/maths/expressions/AD.kt?at=ad&fileviewer=file-view-default
>
> The class on the top is a Context for mathematical operations with AD
> numbers. Since CM already have Fields, it implements the Field.
I have a problem with the CM "Field".
Did you have a look at my comments to issue MATH-1448 (and related
code)?
Unfortunately, I don't have the time right now to go further with
that work; but I'm more and more convinced that something is wrong
with the current design of "Field": if the requirement is to have
and API that provides
* addition (and its neutral element)
* multiplication (and its neutral element)
* some other functions that allow for more performant
implementations of common opreations
why not just define one or more interfaces to that effect?
[Unless I'm mistaken the most used "field" is the "RealFieldElement"
with its implementation over "double". Given the inherent inaccuracy
of floating-point numbers, they actually do not abide by the (math)
field requirements.]
> The
> idea is that context stores some critical properties of numbers which
> are shared between instances like order of differentiation and
> parameters names. In this case the object equality check used instead
> of instance equality, so to context with same orders and set of names
> are still equal. The AD class below is just a wrapper for
> DerivativeStructure that also includes Field context. Binary
> operations additionally perform check if argument uses the same
> context as `this` object.
Hard to follow without being able to see it working...
Are you willing to port to Java? And it seems that
we have to first solve the "Field" issue (or can we
do without; i.e. make the enhancement assuming "double"
everywhere?).
> The actual check occurs here:
>
> https://bitbucket.org/Altavir/dataforge/src/1433da0d2f19ab26d709de8750e002847a4b4887/dataforge-maths/src/main/kotlin/hep/dataforge/maths/expressions/AD.kt?at=ad&fileviewer=file-view-default#AD.kt-78.
> In theory, one can perform field transformation, merging parameters
> sets from both ADs, but it requires understanding about inner
> workings
> of DerivativeStructure which I lack. Basically what I need is an
> ability to create a new DerivativeStructure with parameter number i
> mapped to number j.
>
> The proposed solution does not seem to involve any major performance
> impact.
It would be nice if JMH benchmarks could confirm it.
[But this is not a priori requirement as far as I'm concerned, if
the usability improvement is worth it.]
> Name resolution happens only when one calls derivative with
> given name and is not really a great performance impact since string
> hashes are calculated on string creation. The only place with actual
> performance impact is when field transformation happens (if it will
> be
> implemented), but this transformation is supposed to be rare and it
> currently not possible at all.
?
I'm lost here; sorry. :-{
> The test code in kotlin looks like this:
>
> https://bitbucket.org/Altavir/dataforge/src/8131099f29ebf27fb170ace037cda61df9790fc2/dataforge-maths/src/test/java/hep/dataforge/maths/expressions/ADTest.kt?at=ad&fileviewer=file-view-default.
> It is kotlin though, it would be much more verbatim in java. I plan
> also to implement expressions which would allow lazy calculations of
> AD structures like (NamedVector)->AD. This one could be more or less
> easily done in Java.
Good (but I don't understand what you mean!).
Best regards,
Gilles
>
>
> With best regards, Alexander Nozik.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org