You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sis.apache.org by Martin Desruisseaux <ma...@geomatys.fr> on 2012/10/08 06:43:28 UTC
Note about MathFunctions
Hello all
This email is an explanation about two aspects in a recent commit that
way be worth explanation. The committed class is:
https://builds.apache.org/job/sis-trunk/site/apidocs/org/apache/sis/math/MathFunctions.html
Multiple NaN values
-----------------
Many developers know the java.lang.Double.NaN constant, but it may be
less known that there is actually thousands of different NaN values. All
floating values having a bit pattern in the [0x7F800001 ... 0x7FFFFFFF]
or in the [0xFF800001 ... 0xFFFFFFFF] ranges are NaN numbers. Some of
them are "signalling NaN", some of them may have their bit patterns
changed by the processor in an hardward-dependant way. However the
[0x7FA00000 ... 0x7FE00000] range seems stable at least on Inter processor.
The coveage module (to be proposed to Apache SIS after metadata and
referencing) uses this trick for mixing "qualitative data" together with
"quantitative data" in a raster. Calculations involving quantitative
data like Sea Surface Temperature use the 'float' type. However those
data are often mixed with missing values because of clouds, lack of
satellite data, land surfaces, etc. Many software (including NetCDF file
format) deal with missing values by replacing them by a "pad value",
often -9999. However such practice may be dangerous: developers have to
check for pad values in about every corner of their code, otherwise a
calculation mixing a real value with a pad value will produce a wrong
result (not necessarily in a way easy to spot). Replacing all pad values
by Float.NaN resolve this issue, but we lost the differentiation between
the causes of missing values (clouds, lands, etc.).
The MathFunctions.toNaNFloat(int) and toNaNOrdinal(float) methods try to
give the best of both worlds: true NaN numbers while encoding in them
the cause of missing data. However those methods probably need to be
tested in a wider spectrum of environments than other methods.
Positive and negative zeros
-----------------------
MathFunctions contains some functions for differentiating positive zero
from negative zero. We can not differentiate those two zeros with the
usual <, >, <=, >= or == operators, so we need again to look at the bits
pattern (note that Math.abs(float, float) and Math.abs(double, double)
do that too). Most of the time, we don't need to differentiate those two
zeros. But one special case is when dealing with an envelope crossing
the anti-meridian in the way described by the Web Coverage Service (WCS)
specification. They are the red box in the figure show on this page:
http://www.geotoolkit.org/apidocs/org/geotoolkit/geometry/GeneralEnvelope.html
The most tricky case occurs when the envelope goes from 0 to 360° of
longitude, which in the [-180 ... 180]° range is represented by [0 ...
-0]°C. The later range is different than [-0 ... 0]°C, which is an empty
range. So the GeneralEnvelope class need to be especially careful about
the sign of zero. The MathFunctions.isPositive(double),
isNegative(double) and related methods are there for this purpose.
Martin
Re: Android branch?
Posted by Peter K <pe...@yahoo.de>.
Hi Martin,
>> isn't the jdk6 branch sufficient? or shouldn't it be a goal of such a
>> branch to be android compatible?
>
> The Android API is a subset of the JDK6 API. For example the Android
> branch would have all JAXB annotations removed (at least for current
> Android releases). They were no SQL neither prior Android 4; instead
> Google defined its own package for using their embedded SQLite.
>
> Other example: the referencing module relies extensively on affine
> transforms. The JDK branches use java.awt.geom.AffineTransform. But
> the Android API excluded the full Java2D API - instead, they define
> their own Affine transform class. So the Android branch have to use
> the Google flavour of Affine transforms instead than the Java2D one.
Ah, ok I see. In GraphHopper I could handle the Android stuff by putting
the dependencies (only logging) together when packaging, so I could
avoid a separate branch.
In SIS it could be done with sql and jaxb in the same manner but if
android has its own probably hardware boosted geometry package that is
not an option, I guess.
Peter.
Re: Android branch?
Posted by Martin Desruisseaux <ma...@geomatys.fr>.
Hello Peter
Le 09/10/12 17:43, Peter K a écrit :
> isn't the jdk6 branch sufficient? or shouldn't it be a goal of such a
> branch to be android compatible?
The Android API is a subset of the JDK6 API. For example the Android
branch would have all JAXB annotations removed (at least for current
Android releases). They were no SQL neither prior Android 4; instead
Google defined its own package for using their embedded SQLite.
Other example: the referencing module relies extensively on affine
transforms. The JDK branches use java.awt.geom.AffineTransform. But the
Android API excluded the full Java2D API - instead, they define their
own Affine transform class. So the Android branch have to use the Google
flavour of Affine transforms instead than the Java2D one.
Martin
Re: Android branch?
Posted by Peter K <pe...@yahoo.de>.
Am 09.10.2012 10:23, schrieb Martin Desruisseaux:
> Le 09/10/12 01:44, Mattmann, Chris A (388J) a écrit :
>> +1 to moving forward, no need to hold off. You are doing great and I
>> don't
>> think there are any objections to the code.
>
> Thanks a lot :-)
>
> I was also considering creating one more branch. But this one would be
> experimental and I don't know if it would survive. Any objection to
> try an "Android" branch derived from the trunk?
isn't the jdk6 branch sufficient? or shouldn't it be a goal of such a
branch to be android compatible?
Peter.
Re: Android branch?
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
No objection on my part! :)
Cheers,
Chris
On Oct 9, 2012, at 1:23 AM, Martin Desruisseaux wrote:
> Le 09/10/12 01:44, Mattmann, Chris A (388J) a écrit :
>> +1 to moving forward, no need to hold off. You are doing great and I don't
>> think there are any objections to the code.
>
> Thanks a lot :-)
>
> I was also considering creating one more branch. But this one would be experimental and I don't know if it would survive. Any objection to try an "Android" branch derived from the trunk?
>
> Martin
>
Android branch?
Posted by Martin Desruisseaux <ma...@geomatys.fr>.
Le 09/10/12 01:44, Mattmann, Chris A (388J) a écrit :
> +1 to moving forward, no need to hold off. You are doing great and I don't
> think there are any objections to the code.
Thanks a lot :-)
I was also considering creating one more branch. But this one would be
experimental and I don't know if it would survive. Any objection to try
an "Android" branch derived from the trunk?
Martin
Re: Back to metadata soon
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Martin,
+1 to moving forward, no need to hold off. You are doing great and I don't
think there are any objections to the code.
+1! :)
Cheers,
Chris
On Oct 8, 2012, at 9:26 AM, Martin Desruisseaux wrote:
> I realize that I committed lot of utility classes recently and not much geographic stuff yet, sorry for that...
>
> Unless I forgot some aspects, there is about 3 last classes I would like to commit in the utility module, and then I would be back to metadata.
>
> There is also some other utility classes for which there is no immediate use in metadata, but could be of interest (and would be needed later anyway). For example an AngleFormat class (an implementation of java.text.Format), and an other class for formatting tables. Do people prefer to hold on for a while on the utility module until those classes are needed, or is it okay to port them anyway (because they are easy to port)?
>
> Martin
>
Back to metadata soon
Posted by Martin Desruisseaux <ma...@geomatys.fr>.
I realize that I committed lot of utility classes recently and not much
geographic stuff yet, sorry for that...
Unless I forgot some aspects, there is about 3 last classes I would like
to commit in the utility module, and then I would be back to metadata.
There is also some other utility classes for which there is no immediate
use in metadata, but could be of interest (and would be needed later
anyway). For example an AngleFormat class (an implementation of
java.text.Format), and an other class for formatting tables. Do people
prefer to hold on for a while on the utility module until those classes
are needed, or is it okay to port them anyway (because they are easy to
port)?
Martin
Re: Note about MathFunctions
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Martin,
On Oct 7, 2012, at 9:43 PM, Martin Desruisseaux wrote:
> Hello all
>
> This email is an explanation about two aspects in a recent commit that way be worth explanation. The committed class is:
>
> https://builds.apache.org/job/sis-trunk/site/apidocs/org/apache/sis/math/MathFunctions.html
>
>
> Multiple NaN values
> -----------------
> Many developers know the java.lang.Double.NaN constant, but it may be less known that there is actually thousands of different NaN values. All floating values having a bit pattern in the [0x7F800001 ... 0x7FFFFFFF] or in the [0xFF800001 ... 0xFFFFFFFF] ranges are NaN numbers. Some of them are "signalling NaN", some of them may have their bit patterns changed by the processor in an hardward-dependant way. However the [0x7FA00000 ... 0x7FE00000] range seems stable at least on Inter processor.
+1 makes sense.
>
> The coveage module (to be proposed to Apache SIS after metadata and referencing) uses this trick for mixing "qualitative data" together with "quantitative data" in a raster. Calculations involving quantitative data like Sea Surface Temperature use the 'float' type. However those data are often mixed with missing values because of clouds, lack of satellite data, land surfaces, etc. Many software (including NetCDF file format) deal with missing values by replacing them by a "pad value", often -9999. However such practice may be dangerous: developers have to check for pad values in about every corner of their code, otherwise a calculation mixing a real value with a pad value will produce a wrong result (not necessarily in a way easy to spot). Replacing all pad values by Float.NaN resolve this issue, but we lost the differentiation between the causes of missing values (clouds, lands, etc.).
Yep we have seen this a bunch with remote sensing data a ton too.
>
> The MathFunctions.toNaNFloat(int) and toNaNOrdinal(float) methods try to give the best of both worlds: true NaN numbers while encoding in them the cause of missing data. However those methods probably need to be tested in a wider spectrum of environments than other methods.
+1
>
>
> Positive and negative zeros
> -----------------------
> MathFunctions contains some functions for differentiating positive zero from negative zero. We can not differentiate those two zeros with the usual <, >, <=, >= or == operators, so we need again to look at the bits pattern (note that Math.abs(float, float) and Math.abs(double, double) do that too). Most of the time, we don't need to differentiate those two zeros. But one special case is when dealing with an envelope crossing the anti-meridian in the way described by the Web Coverage Service (WCS) specification. They are the red box in the figure show on this page:
>
> http://www.geotoolkit.org/apidocs/org/geotoolkit/geometry/GeneralEnvelope.html
>
> The most tricky case occurs when the envelope goes from 0 to 360° of longitude, which in the [-180 ... 180]° range is represented by [0 ... -0]°C. The later range is different than [-0 ... 0]°C, which is an empty range. So the GeneralEnvelope class need to be especially careful about the sign of zero. The MathFunctions.isPositive(double), isNegative(double) and related methods are there for this purpose.
Ahhh yes, makes sense.
Thanks for a great explanation, Martin!
Cheers,
Chris