[jira] [Commented] (TRAFODION-2949) LOB: Address performance issues related to inserts/extracts in tables containing LOB columns

["Sandhya Sundaresan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TRAFODION-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448874#comment-16448874 ] 

Sandhya Sundaresan commented on TRAFODION-2949:
-----------------------------------------------

For extract operations, it was found that significant time was being spent on  one step i the extract process that gets the lob handle and neeeded LOB location so we can extract the LOB data from the right location. This was  because a CLI call was being made to retrieve the LOB location from the LOB metadata tables. The initial design was that the lob location could potentially change from one LOB to another. But currently we support only one location - designed by the CQD LOB_STORAGE_FILE_DIR.  So we coudloptimize by eliminating the CLI call and getting the CQD value instead at compiletime and passing that to the runtime extract work method. 

 

> LOB: Address performance issues related to inserts/extracts in tables containing LOB columns
> --------------------------------------------------------------------------------------------
>
>                 Key: TRAFODION-2949
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-2949
>             Project: Apache Trafodion
>          Issue Type: Improvement
>            Reporter: Sandhya Sundaresan
>            Assignee: Sandhya Sundaresan
>            Priority: Major
>
> # Investigate bottlenecks in the insert and update code paths.
> A couple of bottlenecks identified initially were hdfsWrite and the LOBCLi call.
>  # Investigate if lib hdfs write operation is signifincantly slower than a direct JNI call. If so  consider refactoring code being done for hive inserts and reuse for LOB writes as well. Also investigate use of libhdfs3 , a lightweight interface by Pivotal for Hawk ,that doesn’t use JNI at all.
>  # Investigate if LOB Cli calls that do prepare and execute of statements to insert/update the LOB descriptor tables are being cached and reused when multiple LOB inserts are being done in the same session.
>  # Investigate external LOBs as a solution for faster inserts/extracts for usage compared to internal LOBs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)