You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@poi.apache.org by ga...@gmail.com,
ga...@gmail.com on 2018/12/17 12:54:14 UTC
Throwing EvaluationException slows down evaluation, proposed change: not fill stack trace
Hi *,
a work with *huge* (sometimes 100+ MB per workbook) calculation models in Excel through POI and i was looking for a way to optimize calculation perfomance.
Turns out that if any Excel ref/numeric/name/etc error occurs during collecting operands or evaluation, a corresponding EvaluationExcetion is thrown to exit early. I would do the same, it makes perfect sense.
However the cost for filling the stack trace is overwhelming.
Here's a simple demo case:
@Test
public void throwDemo() {
Function function = AggregateFunction.SUM;
ValueEval[] args = new ValueEval[]{NumberEval.ZERO, NumberEval.ZERO, NumberEval.ZERO, NumberEval.ZERO, ErrorEval.REF_INVALID};
int N = 1_000_000;
double start = System.nanoTime();
for (int i = 0; i < N; i++) {
function.evaluate(args, 0, 0);
}
double stop = System.nanoTime();
double seconds = (stop-start)/1.0e9;
System.out.printf("Cycle time: %.3f s, throughput: %.1f evals/s%n", seconds, N/seconds);
}
On my Core-i5 2500K it yields cycle time ~2.750 s, throughput ~370K evals/s.
Now, if I prevent filling stack trace at EvaluationException():
public EvaluationException(ErrorEval errorEval) {
super(errorEval.getErrorString(), null, false, false);
_errorEval = errorEval;
}...
the test now yields cycle time ~0.44 s, throughput ~2.3 *M* evals/s which is 7+ times higher.
One of my models used to take about 15 seconds to evaluate, now it's about 6.5 seconds.
I understand that an exception with no stack trace is not a nice thing to debug but as it's a checked exception,
its scope is quite limited, so hopefully the speedup wins over missing stack trace.
The proposed change doen't break any tests.
Looking forward to hearing your comments,
Vladislav
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
Re: Throwing EvaluationException slows down evaluation, proposed
change: not fill stack trace
Posted by kiwiwings <ki...@apache.org>.
Based on my experience, that setting system properties in web applications
involves discussions with the web admins, I don't like system properties at
all.
Although [1] says the opposite, I guess (but haven't tried) that
context-param aren't automatically set as system properties.
Is [1] true for all major j2ee/web containers?
If not, can we use a different approach? e.g. statics set by user code
Andi
[1]
https://www.mkyong.com/web-development/how-to-pass-system-properties-in-web-xml/
--
Sent from: http://apache-poi.1045710.n5.nabble.com/POI-Dev-f2312866.html
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
Re: Throwing EvaluationException slows down evaluation, proposed
change: not fill stack trace
Posted by Dominik Stadler <do...@gmx.at>.
Hi,
I would stick to system properties, it is the easiest to implement and as
the feature is something that only few will be concerned with, it can be
non-obvious that this is possible.
Dominik.
On Mon, Dec 17, 2018 at 3:48 PM pj.fanning <fa...@yahoo.com> wrote:
> For me, this kind of optimisation needs to be configurable. Most users
> would
> not need this performance optimisation and would be better off to get the
> full stack trace.
>
> I would pitch that something like XmlOptions in the XMLBeans project is a
> useful model. This allows users to choose to override the default behaviour
> of the lib. It's a little bit prettier than using System properties but it
> might be a lot of work to wire a PoiOptions class into our APIs, even if we
> just started with some APIs where we could make use of it.
>
> Options include:
> * Roll out PoiOptions in these evaluation APIs (probably a lot of work and
> to not break API contracts, we'd need to have to add lots of new methods
> that take the new class as a param)
> * Look for the PoiOptions on ThreadLocal so that users can set PoiOptions
> just on the threads where they want to override the default behaviour
> * Expose PoiOptions as a static variable eg PoiOptions.getInstance() so
> that
> users can update the settings and affect the whole JVM (easiest to write
> and
> easiest to use)
> * Use System variables
>
>
>
> --
> Sent from: http://apache-poi.1045710.n5.nabble.com/POI-Dev-f2312866.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
> For additional commands, e-mail: dev-help@poi.apache.org
>
>
Re: Throwing EvaluationException slows down evaluation, proposed
change: not fill stack trace
Posted by "pj.fanning" <fa...@yahoo.com>.
For me, this kind of optimisation needs to be configurable. Most users would
not need this performance optimisation and would be better off to get the
full stack trace.
I would pitch that something like XmlOptions in the XMLBeans project is a
useful model. This allows users to choose to override the default behaviour
of the lib. It's a little bit prettier than using System properties but it
might be a lot of work to wire a PoiOptions class into our APIs, even if we
just started with some APIs where we could make use of it.
Options include:
* Roll out PoiOptions in these evaluation APIs (probably a lot of work and
to not break API contracts, we'd need to have to add lots of new methods
that take the new class as a param)
* Look for the PoiOptions on ThreadLocal so that users can set PoiOptions
just on the threads where they want to override the default behaviour
* Expose PoiOptions as a static variable eg PoiOptions.getInstance() so that
users can update the settings and affect the whole JVM (easiest to write and
easiest to use)
* Use System variables
--
Sent from: http://apache-poi.1045710.n5.nabble.com/POI-Dev-f2312866.html
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
Re: Throwing EvaluationException slows down evaluation, proposed change: not fill stack trace
Posted by ga...@gmail.com,
ga...@gmail.com.
fixes:
1. *6+ times higer
2. Perhaps even the error message can be omitted because one can get it from the stored ErrorEval.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org