[jira] [Created] (LUCENE-7943) Plane.findArcDistancePoints() sometimes throws assertion failures even when plane explicitly normalized

["Karl Wright (JIRA)" <ji...@apache.org>]

Karl Wright created LUCENE-7943:
-----------------------------------

             Summary: Plane.findArcDistancePoints() sometimes throws assertion failures even when plane explicitly normalized
                 Key: LUCENE-7943
                 URL: https://issues.apache.org/jira/browse/LUCENE-7943
             Project: Lucene - Core
          Issue Type: Bug
            Reporter: Karl Wright
            Assignee: Karl Wright


The following assertion sometimes fails even when the plane has been explicitly normalized:

{code}
    assert Math.abs(x*x + y*y + z*z - 1.0) < MINIMUM_RESOLUTION_SQUARED : "Plane needs to be normalized";
{code}

I can find nothing wrong with the assertion check, but the numerical accuracy is 1e-16, which just isn't high enough to support MINIMUM_RESOLUTION_SQUARED accuracy (1e-24).




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org