You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "Yibo Cai (Jira)" <ji...@apache.org> on 2020/10/10 07:13:00 UTC

[jira] [Created] (ARROW-10263) [C++][Compute] Improve numerical stability of variances merging

Yibo Cai created ARROW-10263:
--------------------------------

             Summary: [C++][Compute] Improve numerical stability of variances merging
                 Key: ARROW-10263
                 URL: https://issues.apache.org/jira/browse/ARROW-10263
             Project: Apache Arrow
          Issue Type: Improvement
          Components: C++
            Reporter: Yibo Cai
            Assignee: Yibo Cai


For chunked array, variance kernel needs to merge variances.
Tested with two single value chunk, [40000008000000490], [40000008000000400]. The merged variance is 3872. If treated as single array with two values, the variance is 3904, same as numpy outputs.
So current merging method is not stable in extreme cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)