[jira] [Work logged] (TRAFODION-3263) Disable LOB locking and refactor order of LOB iud expression evaluation

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TRAFODION-3263?focusedWorklogId=270406&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-270406 ]

ASF GitHub Bot logged work on TRAFODION-3263:
---------------------------------------------

                Author: ASF GitHub Bot
            Created on: 01/Jul/19 18:29
            Start Date: 01/Jul/19 18:29
    Worklog Time Spent: 10m 
      Work Description: sandhyasun commented on pull request #1845: [TRAFODION-3263] and other misc fixes for lob locking and refactoring 
URL: https://github.com/apache/trafodion/pull/1845
 
 
   These fixes are cherry picked from another branch and all were reviewed earlier.
   1. Fixes a core that was caused since ExLOB::update was missing a chunk of code very similar to insert . This code path is taken when updating a NULL entry. Added a testcase for it in TEST130.
   2. Regressions on Jenkins have been getting an InterruptException. Made a change to call "close" only once instead of twice. Also added a retry to take care of a small window that can cause issues when there are multiple LOB updates/inserts going on parallelly.
   3. SQL_EXEC_SwitchContext was not allowing switching back to default context. In the case of Udr ResultSets we make an exception with this change.
   In general all work will be done in the connections that are set up. So setContext in Udr ResultSet will still not allow switching to the default context. Only resetContext() will allow it. This is a day-one bug.
   
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 270406)
    Time Spent: 1.5h  (was: 1h 20m)

> Disable LOB locking and refactor order of  LOB iud expression evaluation
> ------------------------------------------------------------------------
>
>                 Key: TRAFODION-3263
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-3263
>             Project: Apache Trafodion
>          Issue Type: Improvement
>          Components: sql-general
>    Affects Versions: 2.2.0
>            Reporter: Sandhya Sundaresan
>            Assignee: Sandhya Sundaresan
>            Priority: Major
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
>  The change to use JNI to do HDFS writes improved the interface by returning more useful infomration to the caller. In TRAFODION-2946, we ddescribe the need for LOB locking because of a condition where multiple threads writing to the same LOB column could interleave and cause  problems. TWith the new JNI interface and HDFS write will now return the offset where the data was written. So we can use this return offset to store in the descriptor tables. Prior to this while using the libhdfs API, we would not get back the "written offset".
>  
> So the order of operations before this change  used to be :
>  # Get the EOD for the LOB data file in HDFS
>  # Store this offset into the LOB descriptor tables so we know where to retrieve the data from during a read. 
>  # call hdfsWrite to write to the LOB data file. And hope that the offset where the hdfsWrite writes is the same as the EOD calculated in 1. hdfs being an "append only"file system, this is usually how it works. But if another process comes in and does an insert into the LOB column between 2 and 3, then we have an incorrect offset stored int he descriptor tables. Hence we added a Lob Lock to make steps 1,2 and 3 atomic as part of Trafodion-2946 to address this issue.
> The order of operations with this change is as follows :
>  # Call JNI hdfs Write API to write the lob data to hdfs. 
>  # Use return data offset from JNI hdfswrite API in 1. as the offset to store in the LOB descriptor tables. 
>  # If there are multiple chunks to write, do it in a loop and append to the first chunk. This way each chunk can be anywhere in hdfs and not necessarily continguous. But we are guaranteed that whatever we wrote will be stored in our interna lLOB descriptor files.
>  # If any failure or TM error occurs while writing to the LOB descriptor tables,  transaction gets rolled back and the chunk of hdfs data written becomes "dead data". It doesn't harm the next operation. 
>  # GC check is now done before an update or insert. Earlier it was done as part of the ::allocateDesc operation to get the EOD of the file. 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)