[GitHub] [accumulo] keith-turner opened a new issue, #3358: When tablet is loaded read tablet metadata after setting location

["keith-turner (via GitHub)" <gi...@apache.org>]

keith-turner opened a new issue, #3358:
URL: https://github.com/apache/accumulo/issues/3358

   This is follow on work for #3350. In #3350 a new refresh column was added to indicate a hosted tablet needs to reload its metadata.  The bulk import fate code (and probably compactions in the future) only set the refresh column (using a conditional mutation) if the tablet has current of future location set.
   
   When a tablet loads it currently reads its metadata and then sets the location.  This order can lead to race conditions where the bulk import code does not set a refresh column and the tablet does not read the files the bulk import code wrote.  If the order is reversed and the bulk makes updates with conditional mutations that check the tablet location, then that will avoid race conditions.
   
   For example consider the following scenarios.
   
    1. Bulk code reads metadata for table T1 and sees it has no location
    2. Tablet T1 sets its location in the metadata table
    3. Bulk code attempts to add files to tablet T1 with a condition that the location is absent.  This fails because the location is not absent
    4. Tablet T1 reads it metadata and gets its set of files
    5. The bulk code reread metadata for tablet T1 and see the location L1
    6. The bulk code attempts to add files to the tablet T1 and a refresh column with a condition that the tablet location is L1
    7. The bulk sends an async rpc to the tablet asking it to refresh
    8. Tablet refreshes its metadata and deletes the refresh column
   
   In the scenario above the tablet was made aware of the new bulk files.  If the order of the tablet settings it location and reading its metadata above were reversed then the tablet would have been aware of the concurrent bulk load.  Also the bulk load would not have been aware of the concurrent tablet load.
   
   Reading the metadata after setting the location ensures the following.
   
    * Will see all changes that were made before the location was set
    * Will be notified of any changes during and after the location was set  The during part assumes other changes are also using conditional mutations.
   
   When making this changes in the tablet code need to document how important the order is.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] When tablet is loaded read tablet metadata after setting location [accumulo]

["dlmarion (via GitHub)" <gi...@apache.org>]

dlmarion closed issue #3358: When tablet is loaded read tablet metadata after setting location
URL: https://github.com/apache/accumulo/issues/3358


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org