You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Ashutosh Mestry <am...@hortonworks.com> on 2017/09/12 17:18:56 UTC
Review Request 62251: ATLAS-2129: Loss of Imported Data In case of
Abrupt Atlas Shutdown
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62251/
-----------------------------------------------------------
Review request for atlas and Madhan Neethiraj.
Bugs: ATLAS-2129
https://issues.apache.org/jira/browse/ATLAS-2129
Repository: atlas
Description
-------
**Background**
Existing implementation adds a _GraphTransaction_ annotation on the _bulkImport_ method. This means that the transaction is open during entire import process.
Also, existing implementation of _GraphTransaction_ does not handle nested transactions in the best way.
**Implementation**
This has an approach for dealing with nested transactions.
Here are some highlights:
- Maintains _ThreadLocal_ count of invocations. Each invocation increments the count, success in execution reduces the count. Commit is done when count reaches ZERO.
- For rollback: Based on some research on transactions, there are 2 types viz. open and closed.
* _Closed_: What we currently have, rollback is called when any transaction within the nesting encounters an error.
* _Open_: Continues processing, and lets the current method deal with exceptions. The txn is closed when nesting reaches outer level.
- I have added _GraphTransactionInterceptorTest_. This exercises capabilities of the _GraphTransasctionInterceptor_.
Update to Import: _bulkImport_ method no longer is marked with _GraphTransaction_. This addresses the problem reported in the bug.
Diffs
-----
common/src/main/java/org/apache/atlas/annotation/GraphTransaction.java 7120166
common/src/main/java/org/apache/atlas/annotation/NestingType.java PRE-CREATION
repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java c6a4bbe
repository/src/test/java/org/apache/atlas/GraphTransactionInterceptorTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/62251/diff/1/
Testing
-------
**Unit tests**
- New unit tests added to address problem with nested transactions, to test the new _GraphTransactionInterceptor_ functionality.
- Executed existing unit tests.
**Functional tests**
Import functional tests.
Thanks,
Ashutosh Mestry
Re: Review Request 62251: ATLAS-2129: Loss of Imported Data In case of
Abrupt Atlas Shutdown
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62251/#review185222
-----------------------------------------------------------
common/src/main/java/org/apache/atlas/annotation/GraphTransaction.java
Lines 28 (patched)
<https://reviews.apache.org/r/62251/#comment261512>
This is supported only from JDK 1.8. Won't work in branch-0.8.
repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java
Lines 95 (patched)
<https://reviews.apache.org/r/62251/#comment261511>
Why set to '0' here? Wouldn't it cause the count to go negative as outer methods complete invokation? And eventually endup leaving uncommitted transaction when the outer most method completes? And this might impact subsequent transactions in this thread as well.
It will be best if we can avoid getting into nested transactions scenario.
- Madhan Neethiraj
On Sept. 12, 2017, 5:18 p.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62251/
> -----------------------------------------------------------
>
> (Updated Sept. 12, 2017, 5:18 p.m.)
>
>
> Review request for atlas and Madhan Neethiraj.
>
>
> Bugs: ATLAS-2129
> https://issues.apache.org/jira/browse/ATLAS-2129
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Background**
> Existing implementation adds a _GraphTransaction_ annotation on the _bulkImport_ method. This means that the transaction is open during entire import process.
>
> Also, existing implementation of _GraphTransaction_ does not handle nested transactions in the best way.
>
> **Implementation**
> This has an approach for dealing with nested transactions.
>
> Here are some highlights:
> - Maintains _ThreadLocal_ count of invocations. Each invocation increments the count, success in execution reduces the count. Commit is done when count reaches ZERO.
> - For rollback: Based on some research on transactions, there are 2 types viz. open and closed.
> * _Closed_: What we currently have, rollback is called when any transaction within the nesting encounters an error.
> * _Open_: Continues processing, and lets the current method deal with exceptions. The txn is closed when nesting reaches outer level.
> - I have added _GraphTransactionInterceptorTest_. This exercises capabilities of the _GraphTransasctionInterceptor_.
>
> Update to Import: _bulkImport_ method no longer is marked with _GraphTransaction_. This addresses the problem reported in the bug.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/atlas/annotation/GraphTransaction.java 7120166
> common/src/main/java/org/apache/atlas/annotation/NestingType.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java c6a4bbe
> repository/src/test/java/org/apache/atlas/GraphTransactionInterceptorTest.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/62251/diff/1/
>
>
> Testing
> -------
>
> **Unit tests**
> - New unit tests added to address problem with nested transactions, to test the new _GraphTransactionInterceptor_ functionality.
> - Executed existing unit tests.
>
> **Functional tests**
> Import functional tests.
>
>
> Thanks,
>
> Ashutosh Mestry
>
>