Streaming with Chunks

[dan evans <de...@voxeo.com>]

In httpcore-4.1.1,

I'd like to take advantage of the line by line facility of SocketInputBuffer to read a continuous stream of lines from a server as follows:

conn = new DefaultHttpClientConnection();
Socket socket = new Socket(host.getHostName(), host.getPort());
conn.bind(socket, params);
inbuffer = new SocketInputBuffer(socket, BUFSIZE, params);
lineBuf = new CharArrayBuffer(BUFSIZE);
...
while (streaming)
{
  lineBuf.clear();
  if (inbuffer.readLine(lineBuf) < 0)
  {
    streaming = false;
    continue;
  }
  System.out.println(lineBuf.toString());
}

This works as long as the stream is not chunked.  But if the server sends a Transfer-encoding: chunked stream, then the chunk metadata appears in the stream.  The CharArrayBuffer/SocketInputBuffer API appears to be at a lower level than ChunkedInputStream.  Do I need to managed the (de)chunking myself in this case, or have I missed the way to take advantage of ChunkedInputStream?

Dan
--



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

Re: Streaming with Chunks

[Oleg Kalnichevski <ol...@apache.org>]

On Wed, 2011-06-29 at 14:33 -0400, dan evans wrote:
> In httpcore-4.1.1,
> 
> I'd like to take advantage of the line by line facility of SocketInputBuffer to read a continuous stream of lines from a server as follows:
> 
> conn = new DefaultHttpClientConnection();
> Socket socket = new Socket(host.getHostName(), host.getPort());
> conn.bind(socket, params);
> inbuffer = new SocketInputBuffer(socket, BUFSIZE, params);
> lineBuf = new CharArrayBuffer(BUFSIZE);
> ...
> while (streaming)
> {
>   lineBuf.clear();
>   if (inbuffer.readLine(lineBuf) < 0)
>   {
>     streaming = false;
>     continue;
>   }
>   System.out.println(lineBuf.toString());
> }
> 
> This works as long as the stream is not chunked.  But if the server sends a Transfer-encoding: chunked stream, then the chunk metadata appears in the stream.  The CharArrayBuffer/SocketInputBuffer API appears to be at a lower level than ChunkedInputStream.  Do I need to managed the (de)chunking myself in this case, or have I missed the way to take advantage of ChunkedInputStream?
> 
> Dan
> --
> 

SocketInputBuffer is intended to facilitate efficiently processing of
data streams that consist of sequences of CRLF delimited lines and
binary blobs. It makes no attempts of translating or decoding the data
it reads. One would need to use a decoder such as ChunkedInputStream to
decode raw data. 

Hope this helps

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org