You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Dennis Hendriks (JIRA)" <ji...@apache.org> on 2011/06/27 12:52:47 UTC
[jira] [Created] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Missing state events due to events between t0 and t0+e being ignored
--------------------------------------------------------------------
Key: MATH-605
URL: https://issues.apache.org/jira/browse/MATH-605
Project: Commons Math
Issue Type: Improvement
Reporter: Dennis Hendriks
The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
However, due the following issues:
- MATH-586: Allow using a custom root-finding algorithm to detect state events
- MATH-599: Re-implementation of Secant-based root finding algorithms
we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
>From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Gilles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063308#comment-13063308 ]
Gilles commented on MATH-605:
-----------------------------
No problem. I've taken care of your suggestion for the "solvers" part (revision 1145146). But in the log I've attributed to the MATH-599 issue...
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java, overlapping-events-tests-v3.patch, overlapping-events-tests.patch, overlapping-events-tests.patch
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Luc Maisonobe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luc Maisonobe resolved MATH-605.
--------------------------------
Resolution: Fixed
Fix Version/s: 3.0
Fixed in subversion repository as of r1148736.
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Fix For: 3.0
>
> Attachments: OverlappingEventsTest.java, overlapping-events-tests-v3.patch, overlapping-events-tests.patch, overlapping-events-tests.patch
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Dennis Hendriks (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063295#comment-13063295 ]
Dennis Hendriks commented on MATH-605:
--------------------------------------
bq. It would be easier if you could provide separate patches for distinct issues ("solvers" vs "ode").
Unfortunately, I don't have time for that today, and today is my last day at work before my summer holiday. I'll keep this in mind for the future.
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java, overlapping-events-tests-v3.patch, overlapping-events-tests.patch, overlapping-events-tests.patch
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Dennis Hendriks (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Hendriks updated MATH-605:
---------------------------------
Attachment: overlapping-events-tests.patch
Nice work Luc! I tested it on my application, and I no longer need the hack that I previously used.
I added a new patch (overlapping-events-tests.patch) that provides a few test cases to test this new functionality. It is a modified unit test from the original file I attached to this issue, when I reported this issue, etc.
Also, I checked all your commits. The patch fixes some comments here and there.
Finally, I fixed the verifyBracketing/isBracketing methods in the UnivariateRealSolverUtils class, to also except intervals that have a zero function value at one of the end points.
Thanks for you efforts.
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java, overlapping-events-tests.patch
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Gilles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063292#comment-13063292 ]
Gilles commented on MATH-605:
-----------------------------
It would be easier if you could provide separate patches for distinct issues ("solvers" vs "ode").
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java, overlapping-events-tests-v3.patch, overlapping-events-tests.patch, overlapping-events-tests.patch
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Dennis Hendriks (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13056360#comment-13056360 ]
Dennis Hendriks commented on MATH-605:
--------------------------------------
bq. there is a BracketedSolution interface (by the way, I think this name is not appropriate, a solver is not a solution)
I agree. Maybe BracketedSolutionSolver? Or if you want to be really precise, then BracketedSolutionUnivariateRealSolver, but in my opinion that is way too long.
bq. I would suggest we adjust our hierarchy slightly and force user to use a solver from the bracketing part of the hierarchy only.
Good idea.
bq. Second, we should let the integrator (or better the internal event state handler) select the side by itself. It should select the right side for forward integration, but it should select the left side for backward integration.
I anticipated this, which is the reason I added the left side choice, even though I didn't personally need this (I don't use backward integration).
bq. Looking at your example, I'm a little puzzled by the switching function.
This issue was part of a larger application. I extracted that what was needed to show the issue. Some of the things may not make sense in the simplified context. I agree that the switching function is less than ideal, but it should nonetheless work. It can be evaluated at each point, and a bracketed solution can be maintained. Even though convergence is not optimal, I think the Secant-based algorithms should be able to handle this. Note that the function has one root only, not more, which may be essential in this case.
bq. I have tried to simply suppress the small ignore zone at the start of the integration, but it wasn't sufficient. I think we'll have to dig deeper in the code.
I tried this as well, and failed. I guess I don't have enough knowledge of the code involved to change this.
bq. So as a partial conclusion, [...]
I agree.
bq. We should make sure that it works for all existing unit tests [...]
An other interesting case would be 2 events at the same time, that actually are detected at the same time. Changing the derivative of the second function to 1, to be equal to the first one, may do that.
bq. Thanks a lot for this report and the solution, it is really the way to go!
Thank you for your useful comments.
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Dennis Hendriks (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Hendriks updated MATH-605:
---------------------------------
Attachment: OverlappingEventsTest.java
Java class with two unit tests, to show the problem, and the hacky temporary solution.
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Dennis Hendriks (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Hendriks updated MATH-605:
---------------------------------
Attachment: overlapping-events-tests.patch
Fixed patch for consistency.
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java, overlapping-events-tests-v3.patch, overlapping-events-tests.patch, overlapping-events-tests.patch
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Dennis Hendriks (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Hendriks updated MATH-605:
---------------------------------
Attachment: overlapping-events-tests-v3.patch
Forgot to add a file. So, third version of the patch...
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java, overlapping-events-tests-v3.patch, overlapping-events-tests.patch, overlapping-events-tests.patch
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Dennis Hendriks (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Hendriks updated MATH-605:
---------------------------------
Affects Version/s: 3.0
Updated affected version to 3.0 from 'unset'.
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Luc Maisonobe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13055577#comment-13055577 ]
Luc Maisonobe commented on MATH-605:
------------------------------------
Your approach is _really_ interesting.
You are right, the current handling of events still has many hacks. Ignoring events at start time was a way to solve issue MATH-421. There was also another problem resulting from a wrong choice in bracketed root depicted in issue MATH-322. As we can now ask the solver to select exactly the solution we want, we can probably have more robust and simpler handling.
There are some caveats, though, so we need to check a few corner cases.
First, we cannot ensure yet user will select a solmver that handles bracketing as we want. It would be nice to have a BracketingSolver and force events handler to use that. However, there is no such interface. There is a solvers hierarchy, and there is a BracketedSolution interface (by the way, I think this name is not appropriate, a solver is not a solution). I would suggest we adjust our hierarchy slightly and force user to use a solver from the bracketing part of the hierarchy only.
Second, we should let the integrator (or better the internal event state handler) select the side by itself. It should select the right side for forward integration, but it should select the left side for backward integration.
Looking at your example, I'm a little puzzled by the switching function. Switching functions are not required to be globally smooth, but they should not have discontinuities around the switch event. Here, the event is _defined_ by discontinuities between +1 and -1. In addition, between discontinuities, the function is constant, so the solver has no clue to find a solution quickly, it simply recursively reduces the interval, leading to a lot of calls to g. For such cases, it is better to design a slightly different g function, ensuring it has the zero at the proper places, is continuous around the roots, and does not change sign between the roots (i.e. it will alternate signs between successive roots).
Here is one way to do this in your case, which should work either when eventOccurred returns STOP and is reused later, or returns CONTINUE and is reused in the same integration:
{code}
private int sign = 1;
public double g(double t, double[] y) throws EventException {
return (y[idx] < 0.5) ? (sign * y[idx]) : (sign * (1.0 - y[idx]));
}
public int eventOccurred(double t, double[] y, boolean increasing) throws EventException {
sign = -sign;
return STOP;
}
{code}
However, with this implementation of the g function, the hack in the test case does not work anymore.
I have tried to simply suppress the small ignore zone at the start of the integration, but it wasn't sufficient. I think we'll have to dig deeper in the code.
So as a partial conclusion, I think we should do as you propose and use smart selection of side in bracketing solver, force use of such solvers, and remove the ignore zone at the start of the integration.
We should make sure that it works for all existing unit tests (some of them are regression tests for previous bugs), that it works for your discontinuous function (despite such functions are not recommended), and that it works for the patched function above.
Thanks a lot for this report and the solution, it is really the way to go!
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-605) Missing state events due to events
between t0 and t0+e being ignored
Posted by "Luc Maisonobe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13062758#comment-13062758 ]
Luc Maisonobe commented on MATH-605:
------------------------------------
Dennis, I have committed a few changes in the EventState class, following the previous comments.
Now we make sure the event time is located at or after the exact root (i.e. never before), according to the current integration time. This is done by either asking the user-provided solver to select the proper side if it is a bracketing solver, or by using the new forceSide method as a post-processing step if the solver is non-bracketing.
This has allowed me to remove at least one special case handling, and only slightly changing another one.
It does work on all unit cases of the library, and it has also been tested on my own user code.
Could you check if this works for you too ? If it works, I'll resolve this issue.
> Missing state events due to events between t0 and t0+e being ignored
> --------------------------------------------------------------------
>
> Key: MATH-605
> URL: https://issues.apache.org/jira/browse/MATH-605
> Project: Commons Math
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Dennis Hendriks
> Attachments: OverlappingEventsTest.java
>
>
> The Commons Math page on ODEs (http://commons.apache.org/math/userguide/ode.html) states in section 13.3 (Discrete Events Handling), that: "Note that g function signs changes at the very beginning of the integration (from t0 to t0 + ε where ε is the events detection convergence threshold) are explicitely ignored. This prevents having the integration stuck at its initial point when a new integration is restarted just at the same point a previous one had been stopped by an event."
> However, due the following issues:
> - MATH-586: Allow using a custom root-finding algorithm to detect state events
> - MATH-599: Re-implementation of Secant-based root finding algorithms
> we can now use for instance the PegasusSolver to detect state events. Using the AllowedSolutions.RIGHT_SIDE we can guarantee that we have passed the event. As such, skipping (future) events between t0 and t0+e is not desired.
> I attached a Java class to show this issue. It has 2 continuous variables, each starts at 0.0. The first has derivative 1.0, the second 2.0. Whenever they become larger than 1.0, they are reset. We thus expect resets for event 1 at 1.0, 2.0, 3.0, etc. We expect resets for event 2 at 0.5, 1.0, 1.5, etc. The events overlap (at 1.0, 2.0, etc). Due to numerical differences, the events however are not detected at the exact same times. After we processed the first, the 'skip everything between t0 and t0+e' may result in skipping events, as can be observed from the failing unit test. The second test has a hack to get around this problem: it is manually checked whether the guard changes, by evaluating t0 and t0+e. If an event is detected, a step of e is done, and integration is restarted from t0+e. This solves the issue, and the unit tests succeeds (we get the events at the expected times, and we don't miss any events).
> From what I understand, event detection is complicated, as discussed in MATH-484. I propose to make the skipping of events betweeen t0 and t0+e optional, as that is no longer needed in the cases I described above, and in fact causes severe problems that can only be solved by hacks. For other (non-bracketed solution) algorithms, it may still be necessary to skip such roots. Maybe an option could be introduced to control this behavior?
> So, if an event is detected at time t, integration may continue from t0=t, and if there is a sign change for t0 and t0+e, then the step handler should be called for t0+e, and the step handler should be called for t0+e as well, with isLast=true. I'm not sure what the value of e should be. It could be the absolute accuracy of the root-finding algorithm. if there are multiple ones, maybe the maximum of all of them. Maybe even the minimal integration step should be taken into account, taking the maximum of that an dall the absolute accuracies of the root-finding algorithms?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira