You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Claus Stadler (Jira)" <ji...@apache.org> on 2021/05/10 00:44:00 UTC

[jira] [Created] (VFS-805) HTTP seek always exhausts response

Claus Stadler created VFS-805:
---------------------------------

             Summary: HTTP seek always exhausts response
                 Key: VFS-805
                 URL: https://issues.apache.org/jira/browse/VFS-805
             Project: Commons VFS
          Issue Type: Bug
    Affects Versions: 2.8.0
            Reporter: Claus Stadler


Seeking on an HTTP resource always downloads ALL  content if a Content-Length header is present. The problem is that seeking closes the current input stream which eventually ends up in ContentLengthInputStream.close() of the (ancient) http client library.
See the example below.

My use case is to perform binary search on sorted datasets on the Web (RDF data in sorted ntriple syntax) - the binary search works locally and *in principle* works on HTTP resources abstract with VFS2, but the seek implementation that downloads *ALL* data (in my case several GBs) unfortunately defeats the purpose :(

{code:java}
// org.apache.commons.httpclient.ContentLengthInputStream (commons-httpclient-3.1)
    public void close() throws IOException {
        if (!closed) {
            try {
                ChunkedInputStream.exhaustInputStream(this);
            } finally {
                // close after above so that we don't throw an exception trying
                // to read after closed!
                closed = true;
            }
        }
    }{code}


{code:java}
	public static void main(String[] args) throws Exception {
		String url = "http://localhost/large-file-2gb.txt";
		FileSystemManager fsManager = VFS.getManager();
		
		try (FileObject file = fsManager.resolveFile(url)) {	
			try (RandomAccessContent r = file.getContent().getRandomAccessContent(RandomAccessMode.READ)) {
				
				StopWatch sw1 = StopWatch.createStarted();
				r.seek(20);
				System.out.println("Initial seek: " + sw1.getTime(TimeUnit.MILLISECONDS));

				StopWatch sw2 = StopWatch.createStarted();
				byte[] bytes = new byte[100];
				r.readFully(bytes);
				System.out.println("Read: " + sw2.getTime(TimeUnit.MILLISECONDS));
				
				StopWatch sw3 = StopWatch.createStarted();
				r.seek(100);
				System.out.println("Subsequent seek: " + sw3.getTime(TimeUnit.MILLISECONDS));
			}
		}
		System.out.println("Done");
	}
/*
Output (times in milliseconds):

Initial seek: 0
Read: 4
Subsequent seek: 2538
Done
*/
{code}






--
This message was sent by Atlassian Jira
(v8.3.4#803005)