You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Andrey Davydov <an...@gmail.com> on 2019/09/25 17:02:39 UTC
Exception during exception handling
In ignite 2.7.5 I got following exception:
org.apache.ignite.IgniteException: Failed to create string representation of binary object.
at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1022) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:762) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:710) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.toString(IgniteTxLocalAdapter.java:1645) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.toString(GridDhtTxLocalAdapter.java:944) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.toString(GridDhtTxLocal.java:650) ~[ignite-core-2.7.5.jar:2.7.5]
at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_222]
at java.lang.StringBuilder.append(StringBuilder.java:131) ~[?:1.8.0_222]
at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:942) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:796) ~[ignite-core-2.7.5.jar:2.7.5]….
As I found in sources, it was error during handling of other error. So this error masks real problem.
See org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter lines 934-944:
catch (Throwable ex) {
// We are about to initiate transaction rollback when tx has started to committing.
// Need to remove version from committed list.
cctx.tm().removeCommittedTx(this);
boolean isNodeStopping = X.hasCause(ex, NodeStoppingException.class);
boolean hasInvalidEnvironmentIssue = X.hasCause(ex, InvalidEnvironmentException.class);
IgniteCheckedException err0 = new IgniteTxHeuristicCheckedException("Failed to locally write to cache " +
"(all transaction entries will be invalidated, however there was a window when " +
"entries for this transaction were visible to others): " + this, ex);
Exception occurs during IgniteTxHeuristicCheckedException creation. So we got unhandled error and server halt instead of IgniteTxHeuristicCheckedException and rollback logic.
And the main issue: we lost any information about origin problem.
Andrey.
RE: Exception during exception handling
Posted by Andrey Davydov <an...@gmail.com>.
I don’t know how fix deserialization, it seems very internal. But I can do something for exception handling, but only on weekend.
Andrey.
От: Denis Magda
Отправлено: 26 сентября 2019 г. в 0:30
Кому: user@ignite.apache.org
Копия: Ilya Kasnacheev
Тема: Re: Exception during exception handling
Andrey,
Would you mind sending a pull-request if you have a clear understanding of how this needs to be fixed?
-
Denis
On Wed, Sep 25, 2019 at 2:06 PM Andrey Davydov <an...@gmail.com> wrote:
There are two root causes.
The first one is deserialization of binary object for “toString” when it is impossible to deserialize (like IGNITE-12178). I saw stackowerflow discussion about fails on debug logging while googling current problem.
The second, but it may be more important, it is unsafe exception handling, information about origin error that cause transaction fail should not be lost anyway. It seems “try finaly” block should be implemented inside “catch” section. Because we have chance to get exception during exception handling anyway.
Andrey.
От: Denis Magda
Отправлено: 25 сентября 2019 г. в 22:36
Кому: user@ignite.apache.org; Ilya Kasnacheev
Тема: Re: Exception during exception handling
Hello Andrey and thanks for reporting!
This reminds me of this issue that has a similar stack trace:
https://issues.apache.org/jira/browse/IGNITE-12178
Ilya, looks like the root cause is absolutely the same, doesn't it?
-
Denis
On Wed, Sep 25, 2019 at 10:02 AM Andrey Davydov <an...@gmail.com> wrote:
In ignite 2.7.5 I got following exception:
org.apache.ignite.IgniteException: Failed to create string representation of binary object.
at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1022) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:762) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:710) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.toString(IgniteTxLocalAdapter.java:1645) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.toString(GridDhtTxLocalAdapter.java:944) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.toString(GridDhtTxLocal.java:650) ~[ignite-core-2.7.5.jar:2.7.5]
at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_222]
at java.lang.StringBuilder.append(StringBuilder.java:131) ~[?:1.8.0_222]
at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:942) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:796) ~[ignite-core-2.7.5.jar:2.7.5]….
As I found in sources, it was error during handling of other error. So this error masks real problem.
See org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter lines 934-944:
catch (Throwable ex) {
// We are about to initiate transaction rollback when tx has started to committing.
// Need to remove version from committed list.
cctx.tm().removeCommittedTx(this);
boolean isNodeStopping = X.hasCause(ex, NodeStoppingException.class);
boolean hasInvalidEnvironmentIssue = X.hasCause(ex, InvalidEnvironmentException.class);
IgniteCheckedException err0 = new IgniteTxHeuristicCheckedException("Failed to locally write to cache " +
"(all transaction entries will be invalidated, however there was a window when " +
"entries for this transaction were visible to others): " + this, ex);
Exception occurs during IgniteTxHeuristicCheckedException creation. So we got unhandled error and server halt instead of IgniteTxHeuristicCheckedException and rollback logic.
And the main issue: we lost any information about origin problem.
Andrey.
Re: Exception during exception handling
Posted by Denis Magda <dm...@apache.org>.
Andrey,
Would you mind sending a pull-request if you have a clear understanding of
how this needs to be fixed?
-
Denis
On Wed, Sep 25, 2019 at 2:06 PM Andrey Davydov <an...@gmail.com>
wrote:
> There are two root causes.
>
>
>
> The first one is deserialization of binary object for “toString” when it
> is impossible to deserialize (like IGNITE-12178). I saw stackowerflow
> discussion about fails on debug logging while googling current problem.
>
>
>
> The second, but it may be more important, it is unsafe exception
> handling, information about origin error that cause transaction fail should
> not be lost anyway. It seems “try finaly” block should be implemented
> inside “catch” section. Because we have chance to get exception during
> exception handling anyway.
>
>
>
> Andrey.
>
>
>
> *От: *Denis Magda <dm...@apache.org>
> *Отправлено: *25 сентября 2019 г. в 22:36
> *Кому: *user@ignite.apache.org; Ilya Kasnacheev <il...@apache.org>
> *Тема: *Re: Exception during exception handling
>
>
>
> Hello Andrey and thanks for reporting!
>
>
>
> This reminds me of this issue that has a similar stack trace:
>
> https://issues.apache.org/jira/browse/IGNITE-12178
>
>
>
> Ilya, looks like the root cause is absolutely the same, doesn't it?
>
>
> -
>
> Denis
>
>
>
>
>
> On Wed, Sep 25, 2019 at 10:02 AM Andrey Davydov <an...@gmail.com>
> wrote:
>
> In ignite 2.7.5 I got following exception:
>
>
>
> org.apache.ignite.IgniteException: Failed to create string representation
> of binary object.
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1022)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:762)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:710)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.toString(IgniteTxLocalAdapter.java:1645)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.toString(GridDhtTxLocalAdapter.java:944)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.toString(GridDhtTxLocal.java:650)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_222]
>
> at java.lang.StringBuilder.append(StringBuilder.java:131)
> ~[?:1.8.0_222]
>
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:942)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:796)
> ~[ignite-core-2.7.5.jar:2.7.5]….
>
>
>
> As I found in sources, it was error during handling of other error. So
> this error masks real problem.
>
> See
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter
> lines 934-944:
>
>
>
> catch (Throwable ex) {
>
> // We are about to initiate transaction rollback
> when tx has started to committing.
>
> // Need to remove version from committed list.
>
> cctx.tm().removeCommittedTx(this);
>
>
>
> boolean isNodeStopping = X.hasCause(ex,
> NodeStoppingException.class);
>
> boolean hasInvalidEnvironmentIssue =
> X.hasCause(ex, InvalidEnvironmentException.class);
>
>
>
> IgniteCheckedException err0 = new
> IgniteTxHeuristicCheckedException("Failed to locally write to cache " +
>
> "(all transaction entries will be invalidated,
> however there was a window when " +
>
> "entries for this transaction were visible to
> others): " + this, ex);
>
>
>
> Exception occurs during IgniteTxHeuristicCheckedException creation. So we
> got unhandled error and server halt instead of IgniteTxHeuristicCheckedException
> and rollback logic.
>
>
>
> And the main issue: we lost any information about origin problem.
>
>
>
> Andrey.
>
>
>
>
>
RE: Exception during exception handling
Posted by Andrey Davydov <an...@gmail.com>.
There are two root causes.
The first one is deserialization of binary object for “toString” when it is impossible to deserialize (like IGNITE-12178). I saw stackowerflow discussion about fails on debug logging while googling current problem.
The second, but it may be more important, it is unsafe exception handling, information about origin error that cause transaction fail should not be lost anyway. It seems “try finaly” block should be implemented inside “catch” section. Because we have chance to get exception during exception handling anyway.
Andrey.
От: Denis Magda
Отправлено: 25 сентября 2019 г. в 22:36
Кому: user@ignite.apache.org; Ilya Kasnacheev
Тема: Re: Exception during exception handling
Hello Andrey and thanks for reporting!
This reminds me of this issue that has a similar stack trace:
https://issues.apache.org/jira/browse/IGNITE-12178
Ilya, looks like the root cause is absolutely the same, doesn't it?
-
Denis
On Wed, Sep 25, 2019 at 10:02 AM Andrey Davydov <an...@gmail.com> wrote:
In ignite 2.7.5 I got following exception:
org.apache.ignite.IgniteException: Failed to create string representation of binary object.
at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1022) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:762) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:710) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.toString(IgniteTxLocalAdapter.java:1645) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.toString(GridDhtTxLocalAdapter.java:944) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.toString(GridDhtTxLocal.java:650) ~[ignite-core-2.7.5.jar:2.7.5]
at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_222]
at java.lang.StringBuilder.append(StringBuilder.java:131) ~[?:1.8.0_222]
at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:942) ~[ignite-core-2.7.5.jar:2.7.5]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:796) ~[ignite-core-2.7.5.jar:2.7.5]….
As I found in sources, it was error during handling of other error. So this error masks real problem.
See org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter lines 934-944:
catch (Throwable ex) {
// We are about to initiate transaction rollback when tx has started to committing.
// Need to remove version from committed list.
cctx.tm().removeCommittedTx(this);
boolean isNodeStopping = X.hasCause(ex, NodeStoppingException.class);
boolean hasInvalidEnvironmentIssue = X.hasCause(ex, InvalidEnvironmentException.class);
IgniteCheckedException err0 = new IgniteTxHeuristicCheckedException("Failed to locally write to cache " +
"(all transaction entries will be invalidated, however there was a window when " +
"entries for this transaction were visible to others): " + this, ex);
Exception occurs during IgniteTxHeuristicCheckedException creation. So we got unhandled error and server halt instead of IgniteTxHeuristicCheckedException and rollback logic.
And the main issue: we lost any information about origin problem.
Andrey.
Re: Exception during exception handling
Posted by Denis Magda <dm...@apache.org>.
Hello Andrey and thanks for reporting!
This reminds me of this issue that has a similar stack trace:
https://issues.apache.org/jira/browse/IGNITE-12178
Ilya, looks like the root cause is absolutely the same, doesn't it?
-
Denis
On Wed, Sep 25, 2019 at 10:02 AM Andrey Davydov <an...@gmail.com>
wrote:
> In ignite 2.7.5 I got following exception:
>
>
>
> org.apache.ignite.IgniteException: Failed to create string representation
> of binary object.
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1022)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:762)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:710)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.toString(IgniteTxLocalAdapter.java:1645)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.toString(GridDhtTxLocalAdapter.java:944)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.toString(GridDhtTxLocal.java:650)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_222]
>
> at java.lang.StringBuilder.append(StringBuilder.java:131)
> ~[?:1.8.0_222]
>
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:942)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:796)
> ~[ignite-core-2.7.5.jar:2.7.5]….
>
>
>
> As I found in sources, it was error during handling of other error. So
> this error masks real problem.
>
> See
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter
> lines 934-944:
>
>
>
> catch (Throwable ex) {
>
> // We are about to initiate transaction rollback
> when tx has started to committing.
>
> // Need to remove version from committed list.
>
> cctx.tm().removeCommittedTx(this);
>
>
>
> boolean isNodeStopping = X.hasCause(ex,
> NodeStoppingException.class);
>
> boolean hasInvalidEnvironmentIssue =
> X.hasCause(ex, InvalidEnvironmentException.class);
>
>
>
> IgniteCheckedException err0 = new
> IgniteTxHeuristicCheckedException("Failed to locally write to cache " +
>
> "(all transaction entries will be invalidated,
> however there was a window when " +
>
> "entries for this transaction were visible to
> others): " + this, ex);
>
>
>
> Exception occurs during IgniteTxHeuristicCheckedException creation. So we
> got unhandled error and server halt instead of IgniteTxHeuristicCheckedException
> and rollback logic.
>
>
>
> And the main issue: we lost any information about origin problem.
>
>
>
> Andrey.
>
>
>