[jira] [Commented] (KNOX-421) Optimize File Upload to Webhdfs

["Larry McCay (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KNOX-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14124520#comment-14124520 ] 

Larry McCay commented on KNOX-421:
----------------------------------

By extending the HttpClientDispatch for an HdfsDispatch, we can override the creation of the HttpEntity in createRequestEntity and just return null.  Jetty automatically sends a 100 continue when the request.getInputStream is invoked thus triggering the file transfer. In order to create the HttpEntity we need to get the inputstream. Since the input stream isn't needed unit redirected to the data node we can avoid creating the entity.

The client will eventually get a 100 continue from webhdfs when it is redirected to the DN and send it then.


> Optimize File Upload to Webhdfs
> -------------------------------
>
>                 Key: KNOX-421
>                 URL: https://issues.apache.org/jira/browse/KNOX-421
>             Project: Apache Knox
>          Issue Type: Bug
>          Components: Server
>    Affects Versions: 0.4.0, 0.5.0
>            Reporter: Larry McCay
>            Assignee: Larry McCay
>             Fix For: 0.5.0
>
>
> We need to not send "100 continue" to a client that sends an Expect 100 continue until after the 307 redirect to the datanode. Otherwise, we end up penalizing performance by sending it more than once.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)