You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Ashutosh Mestry <am...@hortonworks.com> on 2017/09/12 17:18:56 UTC

Review Request 62251: ATLAS-2129: Loss of Imported Data In case of Abrupt Atlas Shutdown

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62251/
-----------------------------------------------------------

Review request for atlas and Madhan Neethiraj.


Bugs: ATLAS-2129
    https://issues.apache.org/jira/browse/ATLAS-2129


Repository: atlas


Description
-------

**Background**
Existing implementation adds a _GraphTransaction_ annotation on the _bulkImport_ method. This means that the transaction is open during entire import process.

Also, existing implementation of _GraphTransaction_ does not handle nested transactions in the best way.

**Implementation**
This has an approach for dealing with nested transactions. 

Here are some highlights:
 - Maintains _ThreadLocal_ count of invocations. Each invocation increments the count, success in execution reduces the count. Commit is done when count reaches ZERO.
 - For rollback: Based on some research on transactions, there are 2 types viz. open and closed.
    * _Closed_: What we currently have, rollback is called when any transaction within the nesting encounters an error.
    * _Open_: Continues processing, and lets the current method deal with exceptions. The txn is closed when nesting reaches outer level.
 - I have added _GraphTransactionInterceptorTest_. This exercises capabilities of the _GraphTransasctionInterceptor_.

Update to Import: _bulkImport_ method no longer is marked with _GraphTransaction_. This addresses the problem reported in the bug.


Diffs
-----

  common/src/main/java/org/apache/atlas/annotation/GraphTransaction.java 7120166 
  common/src/main/java/org/apache/atlas/annotation/NestingType.java PRE-CREATION 
  repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java c6a4bbe 
  repository/src/test/java/org/apache/atlas/GraphTransactionInterceptorTest.java PRE-CREATION 


Diff: https://reviews.apache.org/r/62251/diff/1/


Testing
-------

**Unit tests**
- New unit tests added to address problem with nested transactions, to test the new _GraphTransactionInterceptor_ functionality.
- Executed existing unit tests.

**Functional tests**
Import functional tests.


Thanks,

Ashutosh Mestry


Re: Review Request 62251: ATLAS-2129: Loss of Imported Data In case of Abrupt Atlas Shutdown

Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62251/#review185222
-----------------------------------------------------------




common/src/main/java/org/apache/atlas/annotation/GraphTransaction.java
Lines 28 (patched)
<https://reviews.apache.org/r/62251/#comment261512>

    This is supported only from JDK 1.8. Won't work in branch-0.8.



repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java
Lines 95 (patched)
<https://reviews.apache.org/r/62251/#comment261511>

    Why set to '0' here? Wouldn't it cause the count to go negative as outer methods complete invokation? And eventually endup leaving uncommitted transaction when the outer most method completes? And this might impact subsequent transactions in this thread as well.
    
    It will be best if we can avoid getting into nested transactions scenario.


- Madhan Neethiraj


On Sept. 12, 2017, 5:18 p.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62251/
> -----------------------------------------------------------
> 
> (Updated Sept. 12, 2017, 5:18 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2129
>     https://issues.apache.org/jira/browse/ATLAS-2129
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Background**
> Existing implementation adds a _GraphTransaction_ annotation on the _bulkImport_ method. This means that the transaction is open during entire import process.
> 
> Also, existing implementation of _GraphTransaction_ does not handle nested transactions in the best way.
> 
> **Implementation**
> This has an approach for dealing with nested transactions. 
> 
> Here are some highlights:
>  - Maintains _ThreadLocal_ count of invocations. Each invocation increments the count, success in execution reduces the count. Commit is done when count reaches ZERO.
>  - For rollback: Based on some research on transactions, there are 2 types viz. open and closed.
>     * _Closed_: What we currently have, rollback is called when any transaction within the nesting encounters an error.
>     * _Open_: Continues processing, and lets the current method deal with exceptions. The txn is closed when nesting reaches outer level.
>  - I have added _GraphTransactionInterceptorTest_. This exercises capabilities of the _GraphTransasctionInterceptor_.
> 
> Update to Import: _bulkImport_ method no longer is marked with _GraphTransaction_. This addresses the problem reported in the bug.
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/atlas/annotation/GraphTransaction.java 7120166 
>   common/src/main/java/org/apache/atlas/annotation/NestingType.java PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java c6a4bbe 
>   repository/src/test/java/org/apache/atlas/GraphTransactionInterceptorTest.java PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62251/diff/1/
> 
> 
> Testing
> -------
> 
> **Unit tests**
> - New unit tests added to address problem with nested transactions, to test the new _GraphTransactionInterceptor_ functionality.
> - Executed existing unit tests.
> 
> **Functional tests**
> Import functional tests.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>