[jira] [Commented] (TRAFODION-34) Support region splitting and re-balancing with transactions active

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TRAFODION-34?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14964529#comment-14964529 ] 

ASF GitHub Bot commented on TRAFODION-34:
-----------------------------------------

GitHub user bucaojit opened a pull request:

    https://github.com/apache/incubator-trafodion/pull/137

    [TRAFODION-34] Support region splitting and re-balancing with transactions active

    Functionality being checked in disabled.
    
    To enable the flush/read functionality, enter the following property into hbase-site.xml:
      <property>
        <name>hbase.transaction.split.delay.noflush</name>
        <value>false</value>
      </property>


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/bucaojit/incubator-trafodion jira_34

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-trafodion/pull/137.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #137
    
----
commit 65d0f1cfe43cb51077227277e5b6321fd69aa442
Author: Oliver Bucaojit <ol...@esgyn.com>
Date:   2015-10-20T04:24:54Z

    [TRAFODION-34]Support region splitting/balancing
    
    With transactions active. Functionality being checked in disabled.

commit 815fc9bb4a861a3917b2dd912c67da6ffdc3eb8f
Author: Oliver Bucaojit <ol...@esgyn.com>
Date:   2015-10-20T04:55:57Z

    Merge branch 'master' of github.com:apache/incubator-trafodion into jira_34
    
    Conflicts:
    	core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/coprocessor/transactional/TrxRegionEndpoint.java

----


> Support region splitting and re-balancing with transactions active
> ------------------------------------------------------------------
>
>                 Key: TRAFODION-34
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-34
>             Project: Apache Trafodion
>          Issue Type: New Feature
>          Components: dtm
>            Reporter: Oliver Bucaojit
>            Assignee: Oliver Bucaojit
>   Original Estimate: 504h
>  Remaining Estimate: 504h
>
> Transactional state is not persisted over a region rebalance or split.  This JIRA will cover the rebalance aspect separate from the split handling.  The implementation of the two features would be similar, split handling being more complex.
> The transactional state on the server-side is held in-memory for a region in an endpoint coprocessor.  When a region is rebalanced, this state is lost when the region comes back online.  Some of the information needed to continue the transaction is the list of transaction states by ID, committed transactions by ID, pending transactions, etc.  
> One idea that I have been testing out is persisting this transactional information by serializing it as a Google protobuf and then writing this out to disk on region preClose().  On postOpen() of the region, this transactional state will be read and the information replayed to rebuild the lists and states.
> Another detail that I would like to add is to have the operation delay while transactions are pending and when there are transactional scanners being used.  This would allow pending transactions to complete before the region is taken down.  For the scanner, I think this would make things less complicated as opposed to having to rebuild the scanner state or know which was the last row that the user received from the scanner.  
> In an earlier version, I tested having a simple delay loop that checked for a period of all active transactions to be complete before continuing.  This was causing problems for long-running tests as there would be no quiet periods and the region would not continue with the operation and eventually run out of memory. So having it only delay on pending transactions vs active transactions will allow the delay loop to find a time to continue the split/balance operation.
> More design details are in the included blueprint link.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)