[jira] [Commented] (TRAFODION-3110) Refactor LOB access to use the new implementation of HdfsClient

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TRAFODION-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16514530#comment-16514530 ] 

ASF GitHub Bot commented on TRAFODION-3110:
-------------------------------------------

GitHub user selvaganesang opened a pull request:

    https://github.com/apache/trafodion/pull/1612

    [TRAFODION-3110] Refactor LOB access to use the new implementation of…

    … HdfsClient
    
    This feature is enabled by default. To disable, set a variable USE_LIBHDFS=1 in
    $TRAF_HOME/etc/ms.env and restart the trafodion cluster.
    
    This feature includes the following:
    1. Uses single FSDataInputStream for each LOB column in a query as
       opposed to the opening the hdfs file for every row.
    2. Uses FSDataOutputStream to write the lob data but closes it
       immediately to allow concurrent writes to the hdfs file. HDFS supports
       a single writer at a time. Need to conform if multiple writes can
       be done without the need for RMS lock feature.
    3. Improved error messaging that displays the java exception stack to the
       end user.
    4. LOB worker threads are no longer created

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/selvaganesang/trafodion trafodion_2917_1

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/trafodion/pull/1612.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1612
    
----
commit 52f074af0e866fe82ea4f37eaf90cdc880871851
Author: selvaganesang <se...@...>
Date:   2018-06-15T22:53:04Z

    [TRAFODION-3110] Refactor LOB access to use the new implementation of HdfsClient
    
    This feature is enabled by default. To disable, set a variable USE_LIBHDFS=1 in
    $TRAF_HOME/etc/ms.env and restart the trafodion cluster.
    
    This feature includes the following:
    1. Uses single FSDataInputStream for each LOB column in a query as
       opposed to the opening the hdfs file for every row.
    2. Uses FSDataOutputStream to write the lob data but closes it
       immediately to allow concurrent writes to the hdfs file. HDFS supports
       a single writer at a time. Need to conform if multiple writes can
       be done without the need for RMS lock feature.
    3. Improved error messaging that displays the java exception stack to the
       end user.
    4. LOB worker threads are no longer created

----


> Refactor LOB access to use the new implementation of HdfsClient
> ---------------------------------------------------------------
>
>                 Key: TRAFODION-3110
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-3110
>             Project: Apache Trafodion
>          Issue Type: Sub-task
>          Components: sql-exe
>            Reporter: Selvaganesan Govindarajan
>            Assignee: Selvaganesan Govindarajan
>            Priority: Major
>
> By making LOB to use the new implementation of HdfsClient, the use of libHdfs is avoided in yet another feature. This also simplifies the hdfs access related code in this feature.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)