You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Dusan Ku (Created) (JIRA)" <ji...@apache.org> on 2011/12/01 15:06:40 UTC
[jira] [Created] (MATH-717) A varied class of Array2DRowRealMatrix
is needed to contain float type instead of double.
A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
-----------------------------------------------------------------------------------------
Key: MATH-717
URL: https://issues.apache.org/jira/browse/MATH-717
Project: Commons Math
Issue Type: Improvement
Affects Versions: 2.2
Environment: All
Reporter: Dusan Ku
The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite limited. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Gilles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166280#comment-13166280 ]
Gilles commented on MATH-717:
-----------------------------
What kind of use do you have for very large matrices?
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Sébastien Brisard (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169385#comment-13169385 ]
Sébastien Brisard commented on MATH-717:
----------------------------------------
{quote}
If I'm not mistaken, that would mean a complete copy/paste of the code of the class (and its parents), replacing "double" by "float". This is easy to do once; not nearly as easy to maintain!
{quote}
I think you're right: from this perspective {{ImageJ}} (to name but one) is a little bit of a nightmare (byte images, short images, float images, and so on...). It's such a shame that type parameters require Objects, and not primitive types...
{quote}
Another thing to be aware is that if the "float" implementation exists, users might start to ask for other classes in the "linear" package to be converted (e.g. the "...Decomposition" classes).
{quote}
Actually, I didn't think of that, Gilles, and you convinced me: I was +0 and am now -1.
While writing this post, I'm wondering whether using {{FieldMatrix}} could be an option. For other purposes, I have implemented doubles and floats as {{FieldElements}}. If required, I can polish up these implementations, and contribute them. However, I suspect that the memory footprint of such an object would be larger than that of a primitive {{double}} so that's probably not the answer you're expecting, Dusan.
To sum up, Dusan, here is another answer you might not like: you could try and implement the proposed class yourself, and use it locally, without it being included into CM? I'm sure we would be able to give you a little hand if needed (but I'm also sure it *won't* be needed).
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Sébastien Brisard (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169059#comment-13169059 ]
Sébastien Brisard commented on MATH-717:
----------------------------------------
Hi Dusan,
I still think that your proposal deserve that we think about it, let's see what the others say. Implementation of the class you're asking for would be fairly easy.
Best regards,
Sébastien
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Dusan Ku (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169904#comment-13169904 ]
Dusan Ku commented on MATH-717:
-------------------------------
Hi all
Thank you for all considerate discussions.
Apparently, having additional set of classes just for float type has far more downsides I initially thought. You guys convinced me as well.
I can work on my own classes to work around this limitation.
Again, thank you for all.
Last but not the least, do you think it's worth adding 'revised simplex method' to CM for solving a typical linear programming? I've recently implemented the revised simplex method via eta factorisation though it still needs to be polished further. As you might know well, it's proven to be faster than the standard simplex method (which was contributed by Ben McCain for CM), especially for sparse large matrices.
Actually, it was the beginning of my study on CM. With the current simplex method implemented in CM, the whole tableau is being maintained in a form of a matrix, for which a large problem cannot be solved. I tried to keep my implementation in line with the current structure of 'o.a.c.m.optimization.linear', but I felt the refactoring of the class structure might be needed if the diversified version of simplex method had to be introduced.
One thing I needed initially was to compare the performance between the current simplex method in CM and the revised simplex method I implemented for a large-sized problem. For this purpose only, I can do it by rewriting the classes using float member.
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Dusan Ku (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169041#comment-13169041 ]
Dusan Ku commented on MATH-717:
-------------------------------
Hi Sébastien,
I also agree with you on the downside of float - ugly precision and its workaround.
Just like your case, my huge matrices can be generated procedurally, which means no need to store the matrix. I just felt the diversity (or flexibility) of the choice between the matrix dimension and the type size might benefit us (the developers who are using the library). In this way, for some small problems, small enough to test against the full matrices with float type but not possible with double type, we can test the fully generated matrices with float type. "double" type sometimes does not allow me to test these matrices because the maximum java heap size I could specify under 32-bit OS is somewhere between 1.3G and 1.4G and any attempt over this memory limit throws an OutOfMemory exception. When that happens, my current solution is to make the same classes with float type as the basis.
What conclusions can we draw on this issue? Well, for now, let's just assume that there were some opinions like mine regarding the fact that the basic type is fixed to "double" type.
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Sébastien Brisard (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167387#comment-13167387 ]
Sébastien Brisard commented on MATH-717:
----------------------------------------
Hello,
this might be a silly question, but I was wondering about the nature of the data collected in your matrix. Is it "experimental"? In my case, I often have huge matrices, but all coefficients are generated procedurally, which calls for implementations where the matrix is *not* stored. {{o.a.c.m.linear.RealLinearOperator}} was recently added to CM in order to fullfill this need, together with iterative solvers. Not sure if that would be of any help to you; besides, this is a very new feature, which would need to be extended.
As for me, I was initially adverse to your proposal, because {{floats}} are really ugly regarding precision. I am now only half adverse, *as long as all internal calculations are carried out in double precision*.
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Dusan Ku (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167233#comment-13167233 ]
Dusan Ku edited comment on MATH-717 at 12/11/11 10:48 PM:
----------------------------------------------------------
It is very natural to have very large matrices in mathematical programmings. In many of my experiments, usually linear programming, this happens frequently. If you're not sure of the potential problem between float and double, please run the below samples separately under the following VM option: -Xmx1G
#case 1) for float:
public static void main(String[] args) {
float[][] d = new float[15000][15000];
System.out.println("OK");
}
#case 2) for double:
public static void main(String[] args) {
double[][] d = new double[15000][15000];
System.out.println("OK");
}
was (Author: ninedus):
It is very natural to have very large matrices in mathematical programming. In many of my experiments, usually linear programming, If you're not sure of the potential problem between float and double, please run the below samples separately under the following VM option: -Xmx1G
#case 1) for float:
public static void main(String[] args) {
float[][] d = new float[15000][15000];
System.out.println("OK");
}
#case 2) for double:
public static void main(String[] args) {
float[][] d = new float[15000][15000];
System.out.println("OK");
}
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Dusan Ku (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167298#comment-13167298 ]
Dusan Ku commented on MATH-717:
-------------------------------
No, I haven't tried 'BlockRealMatrix'. Is is worth trying that? I mean 'does this push the limit of the dimension posed by the type size?'
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Gilles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170156#comment-13170156 ]
Gilles commented on MATH-717:
-----------------------------
bq. Last but not the least, do you think it's worth adding 'revised simplex method' to CM for solving a typical linear programming? [...]
Please post this question on the "dev" ML, where it will catch the attention of more people.
If what you propose "is proven to be faster", I'm sure that it will be welcome. And if its implementation would require changes in the existing API, _now_ is the time to do it, before the upcoming release of CM version 3.
Thanks.
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Gilles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169327#comment-13169327 ]
Gilles commented on MATH-717:
-----------------------------
If I'm not mistaken, that would mean a complete copy/paste of the code of the class (and its parents), replacing "double" by "float". This is easy to do once; not nearly as easy to maintain!
I don't think that a feature aimed only at testing "small enough" problems is a quite good reason to push in that direction. What would you be testing anyway, since for the "real" problems, you'd have to switch to Sébastien's linear operator framework?
Furthermore, the matrix API has been under discussion, and if we are going to modify it, it should rather be done before adding more implementations.
Another thing to be aware is that if the "float" implementation exists, users might start to ask for other classes in the "linear" package to be converted (e.g. the "...Decomposition" classes).
Further discussions (about design issues) should probably be brought over to the "dev" ML.
If your limitation is the Java heap memory in a 32-bit system, I'd suggest going 64-bit and buying more RAM.
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Dusan Ku (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170687#comment-13170687 ]
Dusan Ku commented on MATH-717:
-------------------------------
Noted well.
Thanks
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MATH-717) A varied class of Array2DRowRealMatrix
is needed to contain float type instead of double.
Posted by "Dusan Ku (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dusan Ku updated MATH-717:
--------------------------
Description:
The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
was:
The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite limited. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Gilles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167290#comment-13167290 ]
Gilles commented on MATH-717:
-----------------------------
It's clear that one needs 0.9 GB to store 225,000,000 "float" entries and 1.8 GB for "double" entries (not counting the overhead induced from having an array of arrays). That's not really a use-case from which we can decide whether a matrix class backed with a 2-d array of floats is necessary in Commons Math.
I'm certainly not the appropriate person to have this discussion...
Did you try the "BlockRealMatrix" class?
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MATH-717) A varied class of Array2DRowRealMatrix
is needed to contain float type instead of double.
Posted by "Gilles (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gilles resolved MATH-717.
-------------------------
Resolution: Not A Problem
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-717) A varied class of
Array2DRowRealMatrix is needed to contain float type instead of double.
Posted by "Dusan Ku (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167233#comment-13167233 ]
Dusan Ku commented on MATH-717:
-------------------------------
It is very natural to have very large matrices in mathematical programming. In many of my experiments, usually linear programming, If you're not sure of the potential problem between float and double, please run the below samples separately under the following VM option: -Xmx1G
#case 1) for float:
public static void main(String[] args) {
float[][] d = new float[15000][15000];
System.out.println("OK");
}
#case 2) for double:
public static void main(String[] args) {
float[][] d = new float[15000][15000];
System.out.println("OK");
}
> A varied class of Array2DRowRealMatrix is needed to contain float type instead of double.
> -----------------------------------------------------------------------------------------
>
> Key: MATH-717
> URL: https://issues.apache.org/jira/browse/MATH-717
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 2.2
> Environment: All
> Reporter: Dusan Ku
> Labels: features
>
> The current implementation of Array2DRowRealMatrix takes only double type as its base element value in the matrix.
> However, the memory size of double is bigger than float, the downside of which makes the matrix dimension quite limited, compared to float type as its base element type. For small sized problem, this does not make such a big difference, but for large problems, this limits the usability of this library quite severely. In my case, I easily hit an error even after I increase the memory option to 1G. This could have been much more enhanced just by using 'float[][]' instead of the current Array2DRowRealMatrix.
> Therefore, the solution I may suggest is to add another class similar to Array2DRowRealMatrix containing float type for its matrix variable instead of double. Of course, a better way is welcome as long as the needs can be fulfilled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira