[jira] [Commented] (HBASE-15935) Have a separate Walker task running concurrently with Generator

["Joseph (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316885#comment-15316885 ] 

Joseph commented on HBASE-15935:
--------------------------------

In general, the idea is to keep track of the linked-list loops that have been fully generated and flushed inside of a HBase Table, "Flushed Table". When a circular linked list enters the "Flushed Table" we can pass the circular linked list into a Walker job that will begin to validate that circular linked list concurrently with Generation and/or Verify.

As of now I am considering a few possibilities for how this is going to be implemented:

1. Have Walk-Generation be a special mode inside of Loop. This will provide us a little more flexibility in communicating between the Generation-Verify cycle and the Walk (eg: we could pause or terminate Walker at certain times depending on whether Generation or Verification is running). This is the version I have currently implemented, but my usage of Threads just seems a little out of place in the IntegrationTestBigLinkedList code.
2. Modify Walker to have a special option, where it will only grab new nodes from the "Flushed Table".
3. Another interesting option that I found was the concurrency option in the master GoraCI project that would use a similar idea of "Flushed Table" to run Verifications concurrently with Generation. I could also add this in as an optional Loop mode. 
See: https://github.com/keith-turner/goraci/commit/2bae337

Do you guys have any suggestions or comments on which of these approaches I should take? 

> Have a separate Walker task running concurrently with Generator   
> ------------------------------------------------------------------
>
>                 Key: HBASE-15935
>                 URL: https://issues.apache.org/jira/browse/HBASE-15935
>             Project: HBase
>          Issue Type: Sub-task
>          Components: integration tests
>            Reporter: Joseph
>            Assignee: Joseph
>            Priority: Minor
>
> Adding support for concurrent generation and verification. Will largely be copying over the changes from 
> https://github.com/keith-turner/goraci/commit/9ae4323c3e3ae6bbc4746bcc313a97893c0368c9
> into the HBase version of the test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)