Revised HTTP Framework in ARQ 2.10.2

[Rob Vesse <rv...@yarcdata.com>]

Hi All

This email is to make you aware that myself and Andy have been working on a revised HTTP framework for inclusion in ARQ 2.10.2, this work is now in principal finished and integrated into the current 2.10.2-SNAPSHOT builds so we'd like to request that interested users test the new functionality.

What has changed?

In current and previous versions of ARQ each part of the API that used HTTP rolled its own code and own authentication support.  This meant that some APIs provided more features than others, e.g. some had authentication while some did not.  In 2.10.2 all HTTP operations are routed through HttpOp which provides a single consistent API for making HTTP requests that includes appropriate logging and improved authentication support.

The actual APIs affected are as follows:

  *   QueryEngineHTTP I.e. the QueryExecution returned by QueryExecutionFactory.sparqlService() calls
  *   UpdateProcessRemoteBase I.e. the base class of UpdateProcessor implementations returned by UpdateExecutionFactory.createRemote() and createRemoteForm() calls
  *   DatasetGraphAccessorHTTP I.e. the underlying DatasetGraphAccessor behind DatasetAccessors returned by DatasetAccessorFactory.createHTTP() calls

What does this mean for my code?

In terms of existing usage of the affected APIs nothing has changed, all APIs remain backwards compatible.

A new authentication API has been introduced which allows for more complex and varied authentication mechanisms to be used, please see the pre-release documentation at http://jena.staging.apache.org/documentation/query/http-auth.html for details of the new API.  Some additional overloads of the relevant factory methods are provided to specify authenticators at creation time.

Another benefit of this change to you is that internally we now consistently use Apache HttpClient which makes it much easier to debug HTTP communications since setting log level to DEBUG will cause detailed HTTP traces to be printed to the logs.

One possible performance change you may notice is that queries no longer preemptively authenticate which may add additional round trips to query requests due to properly carrying out the HTTP Challenge Response process.  This change was made because the old functionality reduced security since your credentials may be exposed to servers that don't request them and it only functioned for HTTP Basic Auth.

The new authentication API supports a much wider range of authentication mechanisms hence the removal of the old functionality.  If performance is a concern to you there is a PreemptiveBasicAuthenticator which may be wrapped around another authenticator and provides preemptive HTTP basic authentication.

What can I do to help?

If you currently use HTTP authentication with ARQ we'd appreciate you testing with the latest SNAPSHOTs to verify that this new framework does not cause any regressions.  If you are able to test any of the new features introduced by these changes e.g. supports for Forms based authentication and report on those that would be much appreciated.

Thanks,

Rob