You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@jena.apache.org by "A. Soroka" <aj...@virginia.edu> on 2016/11/08 16:59:58 UTC

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?

---
A. Soroka
The University of Virginia Library

> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
> 
> Author: ajs6f
> Date: Tue Nov  8 16:53:48 2016
> New Revision: 1768736
> 
> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
> Log:
> Updates for HTTP behavior in Jena 3.1.1
> 
> Modified:
>    jena/site/trunk/content/documentation/query/http-auth.mdtext
>    jena/site/trunk/content/documentation/query/service.mdtext
> 
> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
> ==============================================================================
> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>            specific language governing permissions and limitations
>            under the License.
> 
> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for 
> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for 
> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
> 
> -## Applying Authentication
> +## HTTP Authentication before ARQ 3.1.0
> +
> +### Applying Authentication
> 
> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>         ...
>     }
> 
> -### Authenticators
> +#### Authenticators
> 
> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
> 
> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried 
> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
> 
> -#### SimpleAuthenticator
> +##### SimpleAuthenticator
> 
> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
> any service.
> @@ -56,7 +58,7 @@ any service.
> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>  Required) then credentials will not actually be submitted.
> 
> -#### ScopedAuthenticator
> +##### ScopedAuthenticator
> 
> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different 
> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are 
> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be 
> used in favor of those for `http://example.org`
> 
> -#### ServiceAuthenticator
> +##### ServiceAuthenticator
> 
> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the 
> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on 
> configuration for this.
> 
> -#### FormsAuthenticator
> +##### FormsAuthenticator
> 
> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this 
> @@ -104,7 +106,7 @@ that maps each service to an associated
> 
> Currently forms based login that require more than just a username and password are not supported.
> 
> -#### PreemptiveBasicAuthenticator
> +##### PreemptiveBasicAuthenticator
> 
> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers 
> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
> @@ -121,20 +123,12 @@ Also be aware that basic authentication
> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
> challenge response sequences.
> 
> -#### DelegatingAuthenticator
> +##### DelegatingAuthenticator
> 
> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and 
> match the types of authentication needed.
> 
> -### Debugging Authentication
> -
> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
> -
> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed 
> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
> -
> -### The Default Authenticator
> +#### The Default Authenticator
> 
> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default 
> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the 
> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
> 
> Note that the default authenticator may be disabled by setting it to `null`.
> 
> +## HTTP Authentication after ARQ 3.1.0
> +
> +### Applying Authentication
> +
> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the 
> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
> +
> +#### Examples of authentication
> +
> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
> +
> +##### Simple authentication using username and password
> +
> +First we build an authenticating client:
> +
> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
> +    HttpClient httpclient = HttpClients.custom()
> +        .setDefaultCredentialsProvider(credsProvider)
> +        .build();
> +    HttpOp.setDefaultHttpClient(httpclient);
> +
> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
> +
> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
> +    final String host = "http://example.com/sparql";
> +    final int port = 80;
> +    final String realm = "aRealm";
> +    final String schemeName = "DIGEST";
> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
> +    credsProvider.setCredentials(authscope, scopedCredentials);
> +    HttpClient httpclient = HttpClients.custom()
> +        .setDefaultCredentialsProvider(credsProvider)
> +        .build();
> +    HttpOp.setDefaultHttpClient(httpclient);
> +	
> +##### Authenticating via a form
> +
> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
> +
> +    // we'll use this context to maintain our HTTP "conversation"
> +    HttpClientContext httpContext = new HttpClientContext();
> +
> +    // first we use a method on HttpOp to log in and get our cookie
> +    Params params = new Params();
> +    params.addParam("username", "Bob Wu");
> +    params.addParam("password", "my password");
> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
> +
> +    // now our cookie is stored in httpContext
> +    CookieStore cookieStore = httpContext.getCookieStore();
> +
> +    // lastly we build a client that uses that cookie
> +    HttpClient httpclient = HttpClients.custom()
> +	    .setDefaultCookieStore(cookieStore)
> +		.build();
> +    HttpOp.setDefaultHttpClient(httpclient);
> +
> +## Other concerns
> +
> +### Debugging Authentication
> +
> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
> +
> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed 
> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
> +
> +### Authenticating to a SPARQL federated service
> +
> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
> +
>   [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>   [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>   [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
> @@ -161,4 +231,7 @@ Note that the default authenticator may
>   [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>   [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>   [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
> -  [14]: http://hc.apache.org
> \ No newline at end of file
> +  [14]: http://hc.apache.org
> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
> \ No newline at end of file
> 
> Modified: jena/site/trunk/content/documentation/query/service.mdtext
> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
> ==============================================================================
> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
> without regard to how selective the pattern is. So the order of the
> query will affect the speed of execution. Because it involves HTTP
> operations, asking the query in the right order matters a lot.
> -Don't ask for the whole of a bookstore just to find book whose
> +Don't ask for the whole of a bookstore just to find a book whose
> title comes from a local RDF file - ask the bookshop a query with
> the title already bound from earlier in the query.
> 
> ## Controlling `SERVICE` requests.
> 
> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
> -The values for configuration can be set in the global context (accessed via 
> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via 
> `ARQ.getContext()`) or in the per-query execution context.
> 
> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
> 
> -### Summary
> +### Configuration for ARQ through version 3.1.0
> 
> Symbol | Usage
> ------ | -----
> @@ -71,7 +70,7 @@ Symbol | Usage
> `srv:queryAuthPwd` |  Basic authentication
> `srv:queryContext` | Per-endpoint configuration
> 
> -### `srv:queryTimeout`
> +#### `srv:queryTimeout`
> 
> Set the connect and read timeouts for the query.
> 
> @@ -86,21 +85,21 @@ read timout = 0
> 
> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
> 
> -### `srv:queryGzip`
> +#### `srv:queryGzip`
> 
> Sets the allow Gzip flag.
> 
> Boolean: True indicates that gzip compressed data is acceptable.
> false
> 
> -### `srv:queryDeflate`
> +#### `srv:queryDeflate`
> 
> Sets the allow Deflate flag.
> 
> Boolean: True indicates that deflate compression is acceptable
> False
> 
> -### `srv:queryAuthUser`
> +#### `srv:queryAuthUser`
> 
> Sets the user id for basic auth.
> 
> @@ -108,7 +107,7 @@ String: The user id to log in with
> 
> If null or null length no user id is sent.
> 
> -### `srv:queryAuthPwd`
> +#### `srv:queryAuthPwd`
> 
> Sets the password for basic auth.
> 
> @@ -116,13 +115,43 @@ String: The password to log in with.
> 
> If null or null length no password is sent.
> 
> -### srv:serviceContext
> +#### `srv:serviceContext`
> Provides a mechanism to override system context settings on a per URI basis.
> 
> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
> 
> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
> 
> +### Configuration for ARQ after version 3.1.0
> 
> +Symbol | Usage | Default
> +------ | ----- | -------
> +`srv:queryTimeout` | Set timeouts | none
> +`srv:queryCompression` | Enable use of deflation and GZip | true
> +`srv:queryClient` | Enable use of a specific client | none
> +`srv:queryContext` | Per-endpoint configuration | none
> +
> +#### `srv:queryTimeout`
> +
> +As documented above.
> +
> +
> +#### `srv:queryCompression`
> +
> +Sets the flag for use of deflation and GZip.
> +
> +Boolean: True indicates that gzip compressed data is acceptable.
> +
> +#### `srv:queryClient`
> +
> +Enable use of a specific client
> +
> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
> +
> +#### `srv:serviceContext`
> +
> +As documented above.
> 
> [ARQ documentation index](index.html)
> +
> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
> 
>

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by "A. Soroka" <aj...@virginia.edu>.

How bizarre! I tried non-HTTPS and it behaves differently in Firefox and Chrome. Firefox shows me the proxied CMS site, Chrome shows me the Jena staging site.

{sigh}

More importantly than my browser woes, interested parties can find the doc changes here:

http://jena.staging.apache.org/documentation/query/http-auth.html
http://jena.staging.apache.org/documentation/query/http-auth.html#http-authentication-after-arq-310
http://jena.staging.apache.org/documentation/query/service.html
http://jena.staging.apache.org/documentation/query/service.html#configuration-for-arq-after-version-310

---
A. Soroka
The University of Virginia Library

> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org> wrote:
> 
> 
> 
> On 08/11/16 16:59, A. Soroka wrote:
>> This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?
>> 
> 
> It does not for me.
> 
> Try http://jena.staging.apache.org/  (not https)
> 
> PDF attached, cc'ed to you in the hope it get through.
> 
> Comments:
> 
> 1/ I'd put the current (3.1.1) text first and the previous second so the current is more visible.
> 
> Links at the end of the intro to "current" and "previous", or in the intro as this difference is mentioned.
> 
> 2/ Title tweaking:
> 
> "HTTP Authentication after Jena 3.1.0" ->
> "HTTP Authentication from Jena 3.1.1"
> 
> "HTTP Authentication before Jena 3.1.0" =>
> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
> 
> (so the range includes 3.1.0 !)
> 
> Mentioning Jena 2.x is not necessary IMO - the additional detail adds confusion for current users and 3.x upgrading users (the majority).
> 
> 3/
> "Simple authentication using username and password"
> 
> "Authenticating via a form"
> 
> The <h5> don't show up as different on teh screen for me so maybe bump <h4> "Examples of authentication" up a level to <h3> and move sub <5> to <h4> .
> 
> Maybe drop <h3> "Applying Authentication" (section title immediately after a section title) and have the paragraph there straight away.
> 
>    Andy
> 
>> ---
>> A. Soroka
>> The University of Virginia Library
>> 
>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>> 
>>> Author: ajs6f
>>> Date: Tue Nov  8 16:53:48 2016
>>> New Revision: 1768736
>>> 
>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>>> Log:
>>> Updates for HTTP behavior in Jena 3.1.1
>>> 
>>> Modified:
>>>   jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>   jena/site/trunk/content/documentation/query/service.mdtext
>>> 
>>> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>> ==============================================================================
>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>>           specific language governing permissions and limitations
>>>           under the License.
>>> 
>>> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for
>>> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for
>>> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
>>> 
>>> -## Applying Authentication
>>> +## HTTP Authentication before ARQ 3.1.0
>>> +
>>> +### Applying Authentication
>>> 
>>> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
>>> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>>        ...
>>>    }
>>> 
>>> -### Authenticators
>>> +#### Authenticators
>>> 
>>> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
>>> 
>>> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried
>>> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
>>> 
>>> -#### SimpleAuthenticator
>>> +##### SimpleAuthenticator
>>> 
>>> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
>>> any service.
>>> @@ -56,7 +58,7 @@ any service.
>>> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>> Required) then credentials will not actually be submitted.
>>> 
>>> -#### ScopedAuthenticator
>>> +##### ScopedAuthenticator
>>> 
>>> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different
>>> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are
>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>>> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be
>>> used in favor of those for `http://example.org`
>>> 
>>> -#### ServiceAuthenticator
>>> +##### ServiceAuthenticator
>>> 
>>> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the
>>> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on
>>> configuration for this.
>>> 
>>> -#### FormsAuthenticator
>>> +##### FormsAuthenticator
>>> 
>>> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
>>> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this
>>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>> 
>>> Currently forms based login that require more than just a username and password are not supported.
>>> 
>>> -#### PreemptiveBasicAuthenticator
>>> +##### PreemptiveBasicAuthenticator
>>> 
>>> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers
>>> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>>> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
>>> challenge response sequences.
>>> 
>>> -#### DelegatingAuthenticator
>>> +##### DelegatingAuthenticator
>>> 
>>> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and
>>> match the types of authentication needed.
>>> 
>>> -### Debugging Authentication
>>> -
>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>> -
>>> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>> -
>>> -### The Default Authenticator
>>> +#### The Default Authenticator
>>> 
>>> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default
>>> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the
>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>> 
>>> Note that the default authenticator may be disabled by setting it to `null`.
>>> 
>>> +## HTTP Authentication after ARQ 3.1.0
>>> +
>>> +### Applying Authentication
>>> +
>>> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the
>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
>>> +
>>> +#### Examples of authentication
>>> +
>>> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
>>> +
>>> +##### Simple authentication using username and password
>>> +
>>> +First we build an authenticating client:
>>> +
>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>>> +    HttpClient httpclient = HttpClients.custom()
>>> +        .setDefaultCredentialsProvider(credsProvider)
>>> +        .build();
>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>> +
>>> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
>>> +
>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
>>> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>> +    final String host = "http://example.com/sparql";
>>> +    final int port = 80;
>>> +    final String realm = "aRealm";
>>> +    final String schemeName = "DIGEST";
>>> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>>> +    HttpClient httpclient = HttpClients.custom()
>>> +        .setDefaultCredentialsProvider(credsProvider)
>>> +        .build();
>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>> +	
>>> +##### Authenticating via a form
>>> +
>>> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
>>> +
>>> +    // we'll use this context to maintain our HTTP "conversation"
>>> +    HttpClientContext httpContext = new HttpClientContext();
>>> +
>>> +    // first we use a method on HttpOp to log in and get our cookie
>>> +    Params params = new Params();
>>> +    params.addParam("username", "Bob Wu");
>>> +    params.addParam("password", "my password");
>>> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
>>> +
>>> +    // now our cookie is stored in httpContext
>>> +    CookieStore cookieStore = httpContext.getCookieStore();
>>> +
>>> +    // lastly we build a client that uses that cookie
>>> +    HttpClient httpclient = HttpClients.custom()
>>> +	    .setDefaultCookieStore(cookieStore)
>>> +		.build();
>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>> +
>>> +## Other concerns
>>> +
>>> +### Debugging Authentication
>>> +
>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>> +
>>> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>> +
>>> +### Authenticating to a SPARQL federated service
>>> +
>>> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
>>> +
>>>  [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>>>  [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>>  [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>>  [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>>>  [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>>  [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
>>> -  [14]: http://hc.apache.org
>>> \ No newline at end of file
>>> +  [14]: http://hc.apache.org
>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>>> \ No newline at end of file
>>> 
>>> Modified: jena/site/trunk/content/documentation/query/service.mdtext
>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>> ==============================================================================
>>> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
>>> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>>> without regard to how selective the pattern is. So the order of the
>>> query will affect the speed of execution. Because it involves HTTP
>>> operations, asking the query in the right order matters a lot.
>>> -Don't ask for the whole of a bookstore just to find book whose
>>> +Don't ask for the whole of a bookstore just to find a book whose
>>> title comes from a local RDF file - ask the bookshop a query with
>>> the title already bound from earlier in the query.
>>> 
>>> ## Controlling `SERVICE` requests.
>>> 
>>> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
>>> -The values for configuration can be set in the global context (accessed via
>>> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via
>>> `ARQ.getContext()`) or in the per-query execution context.
>>> 
>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
>>> 
>>> -### Summary
>>> +### Configuration for ARQ through version 3.1.0
>>> 
>>> Symbol | Usage
>>> ------ | -----
>>> @@ -71,7 +70,7 @@ Symbol | Usage
>>> `srv:queryAuthPwd` |  Basic authentication
>>> `srv:queryContext` | Per-endpoint configuration
>>> 
>>> -### `srv:queryTimeout`
>>> +#### `srv:queryTimeout`
>>> 
>>> Set the connect and read timeouts for the query.
>>> 
>>> @@ -86,21 +85,21 @@ read timout = 0
>>> 
>>> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
>>> 
>>> -### `srv:queryGzip`
>>> +#### `srv:queryGzip`
>>> 
>>> Sets the allow Gzip flag.
>>> 
>>> Boolean: True indicates that gzip compressed data is acceptable.
>>> false
>>> 
>>> -### `srv:queryDeflate`
>>> +#### `srv:queryDeflate`
>>> 
>>> Sets the allow Deflate flag.
>>> 
>>> Boolean: True indicates that deflate compression is acceptable
>>> False
>>> 
>>> -### `srv:queryAuthUser`
>>> +#### `srv:queryAuthUser`
>>> 
>>> Sets the user id for basic auth.
>>> 
>>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>> 
>>> If null or null length no user id is sent.
>>> 
>>> -### `srv:queryAuthPwd`
>>> +#### `srv:queryAuthPwd`
>>> 
>>> Sets the password for basic auth.
>>> 
>>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>> 
>>> If null or null length no password is sent.
>>> 
>>> -### srv:serviceContext
>>> +#### `srv:serviceContext`
>>> Provides a mechanism to override system context settings on a per URI basis.
>>> 
>>> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
>>> 
>>> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
>>> 
>>> +### Configuration for ARQ after version 3.1.0
>>> 
>>> +Symbol | Usage | Default
>>> +------ | ----- | -------
>>> +`srv:queryTimeout` | Set timeouts | none
>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>>> +`srv:queryClient` | Enable use of a specific client | none
>>> +`srv:queryContext` | Per-endpoint configuration | none
>>> +
>>> +#### `srv:queryTimeout`
>>> +
>>> +As documented above.
>>> +
>>> +
>>> +#### `srv:queryCompression`
>>> +
>>> +Sets the flag for use of deflation and GZip.
>>> +
>>> +Boolean: True indicates that gzip compressed data is acceptable.
>>> +
>>> +#### `srv:queryClient`
>>> +
>>> +Enable use of a specific client
>>> +
>>> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
>>> +
>>> +#### `srv:serviceContext`
>>> +
>>> +As documented above.
>>> 
>>> [ARQ documentation index](index.html)
>>> +
>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>> 
>>> 
>>

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by "A. Soroka" <aj...@virginia.edu>.

No, I wouldn't want to take "extra" action there, especially because it might take options away from the end user. Better to give an example in docs of how to reuse the client _if that's what the user wants_.

---
A. Soroka
The University of Virginia Library

> On Nov 9, 2016, at 11:46 AM, Stian Soiland-Reyes <st...@apache.org> wrote:
> 
> Yeah, as the client is set static inside JSON-LD Java it might not
> good for be Jena  HttpOps to be "helpful" and propagate its client -
> at least not without a flag.  But we can link to it from our
> documentation.
> 
> On 9 November 2016 at 16:39, A. Soroka <aj...@virginia.edu> wrote:
>> Hm, that's a really good point. I don't understand what's happening in JSON-LD very well, but I think you must be correct to say that it wouldn't take up a supplied client, because it wouldn't be using state from Jena. I'll check this a bit in the code and add a note as you suggest.
>> 
>> ---
>> A. Soroka
>> The University of Virginia Library
>> 
>>> On Nov 9, 2016, at 11:37 AM, Stian Soiland-Reyes <st...@apache.org> wrote:
>>> 
>>> I think the new text looks good, quite easy to understand.
>>> 
>>> Could you add a paragraph about when the configured client would be
>>> used?   It might not be clear when this HttpClient would be accessed
>>> or not.  For instance I assume it would be used for remote SPARQL
>>> queries or loading of HTTP URLs from RDFDataMgr -- but may not be
>>> propagated through to JSON-LD Java's @context loading - which has a
>>> similar httpclient setting and documentation on how to configure
>>> caching [1]
>>> 
>>> [1] https://github.com/jsonld-java/jsonld-java#controlling-network-traffic
>>> 
>>> 
>>> 
>>> 
>>> On 9 November 2016 at 15:42, A. Soroka <aj...@virginia.edu> wrote:
>>>> Done. I'll wait to hear from other folks before pulling a trigger on (re)publishing the site.
>>>> 
>>>> ---
>>>> A. Soroka
>>>> The University of Virginia Library
>>>> 
>>>>> On Nov 9, 2016, at 6:30 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>> 
>>>>> Great -
>>>>> 
>>>>> One (!) other thing:
>>>>> 
>>>>> A section specifically calling out migrating SPARQL remote calls: QueryExecutionFactory.sparqlService and QueryEngineHTTP.
>>>>> 
>>>>> On the latter, older code may still be directly using QueryEngineHTTP.setBasicAuthentication
>>>>> 
>>>>>     Andy
>>>>> 
>>>>> On 08/11/16 17:58, A. Soroka wrote:
>>>>>> I've made those changes-- should be restaging now.
>>>>>> 
>>>>>> ---
>>>>>> A. Soroka
>>>>>> The University of Virginia Library
>>>>>> 
>>>>>>> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 08/11/16 16:59, A. Soroka wrote:
>>>>>>>> This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?
>>>>>>>> 
>>>>>>> 
>>>>>>> It does not for me.
>>>>>>> 
>>>>>>> Try http://jena.staging.apache.org/  (not https)
>>>>>>> 
>>>>>>> PDF attached, cc'ed to you in the hope it get through.
>>>>>>> 
>>>>>>> Comments:
>>>>>>> 
>>>>>>> 1/ I'd put the current (3.1.1) text first and the previous second so the current is more visible.
>>>>>>> 
>>>>>>> Links at the end of the intro to "current" and "previous", or in the intro as this difference is mentioned.
>>>>>>> 
>>>>>>> 2/ Title tweaking:
>>>>>>> 
>>>>>>> "HTTP Authentication after Jena 3.1.0" ->
>>>>>>> "HTTP Authentication from Jena 3.1.1"
>>>>>>> 
>>>>>>> "HTTP Authentication before Jena 3.1.0" =>
>>>>>>> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
>>>>>>> 
>>>>>>> (so the range includes 3.1.0 !)
>>>>>>> 
>>>>>>> Mentioning Jena 2.x is not necessary IMO - the additional detail adds confusion for current users and 3.x upgrading users (the majority).
>>>>>>> 
>>>>>>> 3/
>>>>>>> "Simple authentication using username and password"
>>>>>>> 
>>>>>>> "Authenticating via a form"
>>>>>>> 
>>>>>>> The <h5> don't show up as different on teh screen for me so maybe bump <h4> "Examples of authentication" up a level to <h3> and move sub <5> to <h4> .
>>>>>>> 
>>>>>>> Maybe drop <h3> "Applying Authentication" (section title immediately after a section title) and have the paragraph there straight away.
>>>>>>> 
>>>>>>> Andy
>>>>>>> 
>>>>>>>> ---
>>>>>>>> A. Soroka
>>>>>>>> The University of Virginia Library
>>>>>>>> 
>>>>>>>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>>>>>>>> 
>>>>>>>>> Author: ajs6f
>>>>>>>>> Date: Tue Nov  8 16:53:48 2016
>>>>>>>>> New Revision: 1768736
>>>>>>>>> 
>>>>>>>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>>>>>>>>> Log:
>>>>>>>>> Updates for HTTP behavior in Jena 3.1.1
>>>>>>>>> 
>>>>>>>>> Modified:
>>>>>>>>> jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>>>>> jena/site/trunk/content/documentation/query/service.mdtext
>>>>>>>>> 
>>>>>>>>> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>>>>>> ==============================================================================
>>>>>>>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
>>>>>>>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
>>>>>>>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>>>>>>>>        specific language governing permissions and limitations
>>>>>>>>>        under the License.
>>>>>>>>> 
>>>>>>>>> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for
>>>>>>>>> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
>>>>>>>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for
>>>>>>>>> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
>>>>>>>>> 
>>>>>>>>> -## Applying Authentication
>>>>>>>>> +## HTTP Authentication before ARQ 3.1.0
>>>>>>>>> +
>>>>>>>>> +### Applying Authentication
>>>>>>>>> 
>>>>>>>>> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
>>>>>>>>> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>>>>>>>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>>>>>>>>     ...
>>>>>>>>> }
>>>>>>>>> 
>>>>>>>>> -### Authenticators
>>>>>>>>> +#### Authenticators
>>>>>>>>> 
>>>>>>>>> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
>>>>>>>>> 
>>>>>>>>> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried
>>>>>>>>> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
>>>>>>>>> 
>>>>>>>>> -#### SimpleAuthenticator
>>>>>>>>> +##### SimpleAuthenticator
>>>>>>>>> 
>>>>>>>>> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
>>>>>>>>> any service.
>>>>>>>>> @@ -56,7 +58,7 @@ any service.
>>>>>>>>> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>>>>>>>> Required) then credentials will not actually be submitted.
>>>>>>>>> 
>>>>>>>>> -#### ScopedAuthenticator
>>>>>>>>> +##### ScopedAuthenticator
>>>>>>>>> 
>>>>>>>>> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different
>>>>>>>>> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are
>>>>>>>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>>>>>>>>> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be
>>>>>>>>> used in favor of those for `http://example.org`
>>>>>>>>> 
>>>>>>>>> -#### ServiceAuthenticator
>>>>>>>>> +##### ServiceAuthenticator
>>>>>>>>> 
>>>>>>>>> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the
>>>>>>>>> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on
>>>>>>>>> configuration for this.
>>>>>>>>> 
>>>>>>>>> -#### FormsAuthenticator
>>>>>>>>> +##### FormsAuthenticator
>>>>>>>>> 
>>>>>>>>> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
>>>>>>>>> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this
>>>>>>>>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>>>>>>>> 
>>>>>>>>> Currently forms based login that require more than just a username and password are not supported.
>>>>>>>>> 
>>>>>>>>> -#### PreemptiveBasicAuthenticator
>>>>>>>>> +##### PreemptiveBasicAuthenticator
>>>>>>>>> 
>>>>>>>>> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers
>>>>>>>>> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
>>>>>>>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>>>>>>>>> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
>>>>>>>>> challenge response sequences.
>>>>>>>>> 
>>>>>>>>> -#### DelegatingAuthenticator
>>>>>>>>> +##### DelegatingAuthenticator
>>>>>>>>> 
>>>>>>>>> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and
>>>>>>>>> match the types of authentication needed.
>>>>>>>>> 
>>>>>>>>> -### Debugging Authentication
>>>>>>>>> -
>>>>>>>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>>>>>> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>>>>>> -
>>>>>>>>> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>>>>>> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>>>>>> -
>>>>>>>>> -### The Default Authenticator
>>>>>>>>> +#### The Default Authenticator
>>>>>>>>> 
>>>>>>>>> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default
>>>>>>>>> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the
>>>>>>>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>>>>>>>> 
>>>>>>>>> Note that the default authenticator may be disabled by setting it to `null`.
>>>>>>>>> 
>>>>>>>>> +## HTTP Authentication after ARQ 3.1.0
>>>>>>>>> +
>>>>>>>>> +### Applying Authentication
>>>>>>>>> +
>>>>>>>>> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the
>>>>>>>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
>>>>>>>>> +
>>>>>>>>> +#### Examples of authentication
>>>>>>>>> +
>>>>>>>>> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
>>>>>>>>> +
>>>>>>>>> +##### Simple authentication using username and password
>>>>>>>>> +
>>>>>>>>> +First we build an authenticating client:
>>>>>>>>> +
>>>>>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>>>>>> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>>>>> +        .build();
>>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>>> +
>>>>>>>>> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
>>>>>>>>> +
>>>>>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>>>>>> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
>>>>>>>>> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>>>> +    final String host = "http://example.com/sparql";
>>>>>>>>> +    final int port = 80;
>>>>>>>>> +    final String realm = "aRealm";
>>>>>>>>> +    final String schemeName = "DIGEST";
>>>>>>>>> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
>>>>>>>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>>>>> +        .build();
>>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>>> +
>>>>>>>>> +##### Authenticating via a form
>>>>>>>>> +
>>>>>>>>> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
>>>>>>>>> +
>>>>>>>>> +    // we'll use this context to maintain our HTTP "conversation"
>>>>>>>>> +    HttpClientContext httpContext = new HttpClientContext();
>>>>>>>>> +
>>>>>>>>> +    // first we use a method on HttpOp to log in and get our cookie
>>>>>>>>> +    Params params = new Params();
>>>>>>>>> +    params.addParam("username", "Bob Wu");
>>>>>>>>> +    params.addParam("password", "my password");
>>>>>>>>> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
>>>>>>>>> +
>>>>>>>>> +    // now our cookie is stored in httpContext
>>>>>>>>> +    CookieStore cookieStore = httpContext.getCookieStore();
>>>>>>>>> +
>>>>>>>>> +    // lastly we build a client that uses that cookie
>>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>>> +     .setDefaultCookieStore(cookieStore)
>>>>>>>>> +         .build();
>>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>>> +
>>>>>>>>> +## Other concerns
>>>>>>>>> +
>>>>>>>>> +### Debugging Authentication
>>>>>>>>> +
>>>>>>>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>>>>>> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>>>>>> +
>>>>>>>>> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>>>>>> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>>>>>> +
>>>>>>>>> +### Authenticating to a SPARQL federated service
>>>>>>>>> +
>>>>>>>>> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
>>>>>>>>> +
>>>>>>>>> [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>>>>>>>>> [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>>>>>>>> [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
>>>>>>>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>>>>>>>> [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>>>>>>>>> [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>>>>>>>> [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
>>>>>>>>> -  [14]: http://hc.apache.org
>>>>>>>>> \ No newline at end of file
>>>>>>>>> +  [14]: http://hc.apache.org
>>>>>>>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
>>>>>>>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>>>>>>>>> \ No newline at end of file
>>>>>>>>> 
>>>>>>>>> Modified: jena/site/trunk/content/documentation/query/service.mdtext
>>>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>>>>>> ==============================================================================
>>>>>>>>> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
>>>>>>>>> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
>>>>>>>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>>>>>>>>> without regard to how selective the pattern is. So the order of the
>>>>>>>>> query will affect the speed of execution. Because it involves HTTP
>>>>>>>>> operations, asking the query in the right order matters a lot.
>>>>>>>>> -Don't ask for the whole of a bookstore just to find book whose
>>>>>>>>> +Don't ask for the whole of a bookstore just to find a book whose
>>>>>>>>> title comes from a local RDF file - ask the bookshop a query with
>>>>>>>>> the title already bound from earlier in the query.
>>>>>>>>> 
>>>>>>>>> ## Controlling `SERVICE` requests.
>>>>>>>>> 
>>>>>>>>> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
>>>>>>>>> -The values for configuration can be set in the global context (accessed via
>>>>>>>>> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via
>>>>>>>>> `ARQ.getContext()`) or in the per-query execution context.
>>>>>>>>> 
>>>>>>>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
>>>>>>>>> 
>>>>>>>>> -### Summary
>>>>>>>>> +### Configuration for ARQ through version 3.1.0
>>>>>>>>> 
>>>>>>>>> Symbol | Usage
>>>>>>>>> ------ | -----
>>>>>>>>> @@ -71,7 +70,7 @@ Symbol | Usage
>>>>>>>>> `srv:queryAuthPwd` |  Basic authentication
>>>>>>>>> `srv:queryContext` | Per-endpoint configuration
>>>>>>>>> 
>>>>>>>>> -### `srv:queryTimeout`
>>>>>>>>> +#### `srv:queryTimeout`
>>>>>>>>> 
>>>>>>>>> Set the connect and read timeouts for the query.
>>>>>>>>> 
>>>>>>>>> @@ -86,21 +85,21 @@ read timout = 0
>>>>>>>>> 
>>>>>>>>> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
>>>>>>>>> 
>>>>>>>>> -### `srv:queryGzip`
>>>>>>>>> +#### `srv:queryGzip`
>>>>>>>>> 
>>>>>>>>> Sets the allow Gzip flag.
>>>>>>>>> 
>>>>>>>>> Boolean: True indicates that gzip compressed data is acceptable.
>>>>>>>>> false
>>>>>>>>> 
>>>>>>>>> -### `srv:queryDeflate`
>>>>>>>>> +#### `srv:queryDeflate`
>>>>>>>>> 
>>>>>>>>> Sets the allow Deflate flag.
>>>>>>>>> 
>>>>>>>>> Boolean: True indicates that deflate compression is acceptable
>>>>>>>>> False
>>>>>>>>> 
>>>>>>>>> -### `srv:queryAuthUser`
>>>>>>>>> +#### `srv:queryAuthUser`
>>>>>>>>> 
>>>>>>>>> Sets the user id for basic auth.
>>>>>>>>> 
>>>>>>>>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>>>>>>>> 
>>>>>>>>> If null or null length no user id is sent.
>>>>>>>>> 
>>>>>>>>> -### `srv:queryAuthPwd`
>>>>>>>>> +#### `srv:queryAuthPwd`
>>>>>>>>> 
>>>>>>>>> Sets the password for basic auth.
>>>>>>>>> 
>>>>>>>>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>>>>>>>> 
>>>>>>>>> If null or null length no password is sent.
>>>>>>>>> 
>>>>>>>>> -### srv:serviceContext
>>>>>>>>> +#### `srv:serviceContext`
>>>>>>>>> Provides a mechanism to override system context settings on a per URI basis.
>>>>>>>>> 
>>>>>>>>> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
>>>>>>>>> 
>>>>>>>>> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
>>>>>>>>> 
>>>>>>>>> +### Configuration for ARQ after version 3.1.0
>>>>>>>>> 
>>>>>>>>> +Symbol | Usage | Default
>>>>>>>>> +------ | ----- | -------
>>>>>>>>> +`srv:queryTimeout` | Set timeouts | none
>>>>>>>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>>>>>>>>> +`srv:queryClient` | Enable use of a specific client | none
>>>>>>>>> +`srv:queryContext` | Per-endpoint configuration | none
>>>>>>>>> +
>>>>>>>>> +#### `srv:queryTimeout`
>>>>>>>>> +
>>>>>>>>> +As documented above.
>>>>>>>>> +
>>>>>>>>> +
>>>>>>>>> +#### `srv:queryCompression`
>>>>>>>>> +
>>>>>>>>> +Sets the flag for use of deflation and GZip.
>>>>>>>>> +
>>>>>>>>> +Boolean: True indicates that gzip compressed data is acceptable.
>>>>>>>>> +
>>>>>>>>> +#### `srv:queryClient`
>>>>>>>>> +
>>>>>>>>> +Enable use of a specific client
>>>>>>>>> +
>>>>>>>>> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
>>>>>>>>> +
>>>>>>>>> +#### `srv:serviceContext`
>>>>>>>>> +
>>>>>>>>> +As documented above.
>>>>>>>>> 
>>>>>>>>> [ARQ documentation index](index.html)
>>>>>>>>> +
>>>>>>>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Stian Soiland-Reyes
>>> http://orcid.org/0000-0001-9842-9718
>> 
> 
> 
> 
> -- 
> Stian Soiland-Reyes
> http://orcid.org/0000-0001-9842-9718

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by "A. Soroka" <aj...@virginia.edu>.

Yes, so I guess the question becomes whether that is appropriately documented as part of "HTTP auth stuff" or as part of "JSON-specific stuff". I'm rather inclined to the latter.

---
A. Soroka
The University of Virginia Library

> On Nov 14, 2016, at 7:03 PM, Stian Soiland-Reyes <st...@apache.org> wrote:
> 
> No, I've only used the default caching mechanism, (as I wrote it ;-) - I
> guess to do it properly we would then need to pass this through options or
> the Documents loader to use.. :-/
> 
> On 9 Nov 2016 5:39 pm, "A. Soroka" <aj...@virginia.edu> wrote:
> 
>> Hm, now I'm not sure whether I can give a good example of managing the
>> JSON-LD HTTP client within Jena. oaj.riot.lang.JsonLDReader doesn't seem to
>> allow users to get at the DocumentLoader instance in use, and that's where
>> the JSON-LD docs show an override to use a specific client. I don't see any
>> static state of the kind you describe in JSON-LD, Stian-- can you give me a
>> pointer?
>> 
>> ---
>> A. Soroka
>> The University of Virginia Library
>> 
>>> On Nov 9, 2016, at 11:46 AM, Stian Soiland-Reyes <st...@apache.org>
>> wrote:
>>> 
>>> Yeah, as the client is set static inside JSON-LD Java it might not
>>> good for be Jena  HttpOps to be "helpful" and propagate its client -
>>> at least not without a flag.  But we can link to it from our
>>> documentation.
>>> 
>>> On 9 November 2016 at 16:39, A. Soroka <aj...@virginia.edu> wrote:
>>>> Hm, that's a really good point. I don't understand what's happening in
>> JSON-LD very well, but I think you must be correct to say that it wouldn't
>> take up a supplied client, because it wouldn't be using state from Jena.
>> I'll check this a bit in the code and add a note as you suggest.
>>>> 
>>>> ---
>>>> A. Soroka
>>>> The University of Virginia Library
>>>> 
>>>>> On Nov 9, 2016, at 11:37 AM, Stian Soiland-Reyes <st...@apache.org>
>> wrote:
>>>>> 
>>>>> I think the new text looks good, quite easy to understand.
>>>>> 
>>>>> Could you add a paragraph about when the configured client would be
>>>>> used?   It might not be clear when this HttpClient would be accessed
>>>>> or not.  For instance I assume it would be used for remote SPARQL
>>>>> queries or loading of HTTP URLs from RDFDataMgr -- but may not be
>>>>> propagated through to JSON-LD Java's @context loading - which has a
>>>>> similar httpclient setting and documentation on how to configure
>>>>> caching [1]
>>>>> 
>>>>> [1] https://github.com/jsonld-java/jsonld-java#controlling-
>> network-traffic
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> On 9 November 2016 at 15:42, A. Soroka <aj...@virginia.edu> wrote:
>>>>>> Done. I'll wait to hear from other folks before pulling a trigger on
>> (re)publishing the site.
>>>>>> 
>>>>>> ---
>>>>>> A. Soroka
>>>>>> The University of Virginia Library
>>>>>> 
>>>>>>> On Nov 9, 2016, at 6:30 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>>>> 
>>>>>>> Great -
>>>>>>> 
>>>>>>> One (!) other thing:
>>>>>>> 
>>>>>>> A section specifically calling out migrating SPARQL remote calls:
>> QueryExecutionFactory.sparqlService and QueryEngineHTTP.
>>>>>>> 
>>>>>>> On the latter, older code may still be directly using
>> QueryEngineHTTP.setBasicAuthentication
>>>>>>> 
>>>>>>>    Andy
>>>>>>> 
>>>>>>> On 08/11/16 17:58, A. Soroka wrote:
>>>>>>>> I've made those changes-- should be restaging now.
>>>>>>>> 
>>>>>>>> ---
>>>>>>>> A. Soroka
>>>>>>>> The University of Virginia Library
>>>>>>>> 
>>>>>>>>> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org>
>> wrote:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 08/11/16 16:59, A. Soroka wrote:
>>>>>>>>>> This commit includes the new docs for HTTP behavior in Jena
>> 3.1.1. I can't find any way to see a view of this on the staging site--
>> https://jena.staging.apache.org/ just seems to proxy
>> https://cms.apache.org/, for some reason?
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> It does not for me.
>>>>>>>>> 
>>>>>>>>> Try http://jena.staging.apache.org/  (not https)
>>>>>>>>> 
>>>>>>>>> PDF attached, cc'ed to you in the hope it get through.
>>>>>>>>> 
>>>>>>>>> Comments:
>>>>>>>>> 
>>>>>>>>> 1/ I'd put the current (3.1.1) text first and the previous second
>> so the current is more visible.
>>>>>>>>> 
>>>>>>>>> Links at the end of the intro to "current" and "previous", or in
>> the intro as this difference is mentioned.
>>>>>>>>> 
>>>>>>>>> 2/ Title tweaking:
>>>>>>>>> 
>>>>>>>>> "HTTP Authentication after Jena 3.1.0" ->
>>>>>>>>> "HTTP Authentication from Jena 3.1.1"
>>>>>>>>> 
>>>>>>>>> "HTTP Authentication before Jena 3.1.0" =>
>>>>>>>>> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
>>>>>>>>> 
>>>>>>>>> (so the range includes 3.1.0 !)
>>>>>>>>> 
>>>>>>>>> Mentioning Jena 2.x is not necessary IMO - the additional detail
>> adds confusion for current users and 3.x upgrading users (the majority).
>>>>>>>>> 
>>>>>>>>> 3/
>>>>>>>>> "Simple authentication using username and password"
>>>>>>>>> 
>>>>>>>>> "Authenticating via a form"
>>>>>>>>> 
>>>>>>>>> The <h5> don't show up as different on teh screen for me so maybe
>> bump <h4> "Examples of authentication" up a level to <h3> and move sub <5>
>> to <h4> .
>>>>>>>>> 
>>>>>>>>> Maybe drop <h3> "Applying Authentication" (section title
>> immediately after a section title) and have the paragraph there straight
>> away.
>>>>>>>>> 
>>>>>>>>> Andy
>>>>>>>>> 
>>>>>>>>>> ---
>>>>>>>>>> A. Soroka
>>>>>>>>>> The University of Virginia Library
>>>>>>>>>> 
>>>>>>>>>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Author: ajs6f
>>>>>>>>>>> Date: Tue Nov  8 16:53:48 2016
>>>>>>>>>>> New Revision: 1768736
>>>>>>>>>>> 
>>>>>>>>>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>>>>>>>>>>> Log:
>>>>>>>>>>> Updates for HTTP behavior in Jena 3.1.1
>>>>>>>>>>> 
>>>>>>>>>>> Modified:
>>>>>>>>>>> jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>>>>>>> jena/site/trunk/content/documentation/query/service.mdtext
>>>>>>>>>>> 
>>>>>>>>>>> Modified: jena/site/trunk/content/documentation/query/http-auth.
>> mdtext
>>>>>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/
>> documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&
>> r2=1768736&view=diff
>>>>>>>>>>> ============================================================
>> ==================
>>>>>>>>>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext
>> (original)
>>>>>>>>>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext
>> Tue Nov  8 16:53:48 2016
>>>>>>>>>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>>>>>>>>>>       specific language governing permissions and limitations
>>>>>>>>>>>       under the License.
>>>>>>>>>>> 
>>>>>>>>>>> -As of ARQ 2.11.0 there is a new unified HTTP operation
>> framework that provides a uniform mechanism for
>>>>>>>>>>> -HTTP authentication that also allows ARQ to support a broader
>> range of authentication mechanisms than were previously possible.
>>>>>>>>>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific
>> unified HTTP operation framework that provides a uniform mechanism for
>>>>>>>>>>> +HTTP authentication that also allows ARQ to support a broader
>> range of authentication mechanisms than were previously possible. After ARQ
>> 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same
>> end. This documentation is therefore devided into two sections. The first
>> explains the older Jena-specific functionality, and the second explains how
>> to use HTTP Commons code to the same ends.
>>>>>>>>>>> 
>>>>>>>>>>> -## Applying Authentication
>>>>>>>>>>> +## HTTP Authentication before ARQ 3.1.0
>>>>>>>>>>> +
>>>>>>>>>>> +### Applying Authentication
>>>>>>>>>>> 
>>>>>>>>>>> APIs that support authentication typically provide two methods
>> for providing authenticators, a `setAuthentication(String username, char[]
>> password)` method
>>>>>>>>>>> which merely configures a `SimpleAuthenticator`.  There will
>> also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>>>>>>>>>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>>>>>>>>>>    ...
>>>>>>>>>>> }
>>>>>>>>>>> 
>>>>>>>>>>> -### Authenticators
>>>>>>>>>>> +#### Authenticators
>>>>>>>>>>> 
>>>>>>>>>>> Authentication mechanisms are provided by [HttpAuthenticator][1]
>> implementations of which a number are provided built into ARQ.
>>>>>>>>>>> 
>>>>>>>>>>> This API provides the authenticator with access to the
>> `HttpClient`, `HttpContext` and target `URI` of the request that is about
>> to be carried
>>>>>>>>>>> out.  This allows for authenticators to add credentials to
>> requests on a per-request basis and/or to use different mechanisms and
>> credentials for different services.
>>>>>>>>>>> 
>>>>>>>>>>> -#### SimpleAuthenticator
>>>>>>>>>>> +##### SimpleAuthenticator
>>>>>>>>>>> 
>>>>>>>>>>> The [simple authenticator][2] is as the name suggests the
>> simplest implementation.  It takes a single set of credentials which is
>> applied to
>>>>>>>>>>> any service.
>>>>>>>>>>> @@ -56,7 +58,7 @@ any service.
>>>>>>>>>>> Authentication however is not preemptive so unless the remote
>> service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>>>>>>>>>> Required) then credentials will not actually be submitted.
>>>>>>>>>>> 
>>>>>>>>>>> -#### ScopedAuthenticator
>>>>>>>>>>> +##### ScopedAuthenticator
>>>>>>>>>>> 
>>>>>>>>>>> The [scoped authenticator][3] is an authenticator which maps
>> credentials to different service URIs.  This allows you to specify different
>>>>>>>>>>> credentials for different services as appropriate.  Similarly to
>> the simple authenticator this is not preemptive authentication so
>> credentials are
>>>>>>>>>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>>>>>>>>>>> e.g. `http://example.org/some/path`.  However if you had also
>> defined credentials for `http://example.org/some/path` then these would be
>>>>>>>>>>> used in favor of those for `http://example.org`
>>>>>>>>>>> 
>>>>>>>>>>> -#### ServiceAuthenticator
>>>>>>>>>>> +##### ServiceAuthenticator
>>>>>>>>>>> 
>>>>>>>>>>> The [service authenticator][4] is an authenticator which uses
>> information encoded in the ARQ context and basically provides access to the
>>>>>>>>>>> existing credential provision mechanisms provided for the
>> `SERVICE` clause, see [Basic Federated Query][5] for more information on
>>>>>>>>>>> configuration for this.
>>>>>>>>>>> 
>>>>>>>>>>> -#### FormsAuthenticator
>>>>>>>>>>> +##### FormsAuthenticator
>>>>>>>>>>> 
>>>>>>>>>>> The [forms authenticator][6] is an authenticator usable with
>> services that require form based logins and use session cookies to verify
>> login state.
>>>>>>>>>>> This is intended for use with services that don't support HTTP's
>> built-in authentication mechanisms for whatever reason.  One example of this
>>>>>>>>>>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>>>>>>>>>> 
>>>>>>>>>>> Currently forms based login that require more than just a
>> username and password are not supported.
>>>>>>>>>>> 
>>>>>>>>>>> -#### PreemptiveBasicAuthenticator
>>>>>>>>>>> +##### PreemptiveBasicAuthenticator
>>>>>>>>>>> 
>>>>>>>>>>> This [authenticator][8] is a decorator over another
>> authenticator that enables preemptive basic authentication, this **only**
>> works for servers
>>>>>>>>>>> that support basic authentication and so will cause
>> authentication failures when any other authentication scheme is required.
>> You should **only**
>>>>>>>>>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>>>>>>>>>>> many servers will use more secure schemes like Digest
>> authentication which **cannot** be done preemptively as they require more
>> complex
>>>>>>>>>>> challenge response sequences.
>>>>>>>>>>> 
>>>>>>>>>>> -#### DelegatingAuthenticator
>>>>>>>>>>> +##### DelegatingAuthenticator
>>>>>>>>>>> 
>>>>>>>>>>> The [delegating authenticator][12] allows for mapping different
>> authenticators to different services, this is useful when you need to mix
>> and
>>>>>>>>>>> match the types of authentication needed.
>>>>>>>>>>> 
>>>>>>>>>>> -### Debugging Authentication
>>>>>>>>>>> -
>>>>>>>>>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations
>> and this provides detailed logging information that can be used for
>> debugging.  To
>>>>>>>>>>> -see this information you need to configure your logging
>> framework to set the `org.apache.http` package to either `DEBUG` or `TRACE`
>> level.
>>>>>>>>>>> -
>>>>>>>>>>> -The `DEBUG` level will give you general diagnostic information
>> about requests and responses while the `TRACE` level will give you detailed
>>>>>>>>>>> -HTTP traces i.e. allow you to see the exact HTTP requests and
>> responses which can be extremely useful for debugging authentication
>> problems.
>>>>>>>>>>> -
>>>>>>>>>>> -### The Default Authenticator
>>>>>>>>>>> +#### The Default Authenticator
>>>>>>>>>>> 
>>>>>>>>>>> Since it may not always be possible/practical to configure
>> authenticators on a per-request basis the API includes a means to specify a
>> default
>>>>>>>>>>> authenticator that is used when no authenticator is explicitly
>> specified.  This may be configured via the
>>>>>>>>>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>>>>>>>>>> 
>>>>>>>>>>> Note that the default authenticator may be disabled by setting
>> it to `null`.
>>>>>>>>>>> 
>>>>>>>>>>> +## HTTP Authentication after ARQ 3.1.0
>>>>>>>>>>> +
>>>>>>>>>>> +### Applying Authentication
>>>>>>>>>>> +
>>>>>>>>>>> +APIs that support authentication typically provide methods for
>> providing an [HttpClient] for use with the given instance of that API
>> class. `HttpClient` is [extremely flexible][16] and can handle most
>> scenarios very well. Since it may not always be possible/practical to
>> configure authenticators on a per-request basis the API includes a means to
>> specify a default client that is used when no other client is explicitly
>> specified.  This may be configured via the
>>>>>>>>>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the
>> [HttpOp][13] class. This allows for static-scoped configuration of HTTP
>> behavior.
>>>>>>>>>>> +
>>>>>>>>>>> +#### Examples of authentication
>>>>>>>>>>> +
>>>>>>>>>>> +This section includes a series of examples showing how to use
>> HTTP Commons classes to perform authenticated work. Most of them take
>> advantage of `HttpOp.setDefaultHttpClient` as described above.
>>>>>>>>>>> +
>>>>>>>>>>> +##### Simple authentication using username and password
>>>>>>>>>>> +
>>>>>>>>>>> +First we build an authenticating client:
>>>>>>>>>>> +
>>>>>>>>>>> +    CredentialsProvider credsProvider = new
>> BasicCredentialsProvider();
>>>>>>>>>>> +    Credentials credentials = new UsernamePasswordCredentials("user",
>> "passwd");
>>>>>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>>>>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>>>>>>> +        .build();
>>>>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>>>>> +
>>>>>>>>>>> +Notice that we gave no scope for use with the credentials
>> (`AuthScope.ANY`). We can make further use of that parameter if we want to
>> assign a scope for some credentials:
>>>>>>>>>>> +
>>>>>>>>>>> +    CredentialsProvider credsProvider = new
>> BasicCredentialsProvider();
>>>>>>>>>>> +    Credentials unscopedCredentials = new
>> UsernamePasswordCredentials("user", "passwd");
>>>>>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY,
>> unscopedCredentials);
>>>>>>>>>>> +    Credentials scopedCredentials = new
>> UsernamePasswordCredentials("user", "passwd");
>>>>>>>>>>> +    final String host = "http://example.com/sparql";
>>>>>>>>>>> +    final int port = 80;
>>>>>>>>>>> +    final String realm = "aRealm";
>>>>>>>>>>> +    final String schemeName = "DIGEST";
>>>>>>>>>>> +    AuthScope authscope = new AuthScope(host, port, realm,
>> schemeName);
>>>>>>>>>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>>>>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>>>>>>> +        .build();
>>>>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>>>>> +
>>>>>>>>>>> +##### Authenticating via a form
>>>>>>>>>>> +
>>>>>>>>>>> +For this case we introduce an [HttpClientContext][17], which we
>> can use to retrieve the cookie we get from logging into a form. We then use
>> the cookie to authenticate elsewhere.
>>>>>>>>>>> +
>>>>>>>>>>> +    // we'll use this context to maintain our HTTP
>> "conversation"
>>>>>>>>>>> +    HttpClientContext httpContext = new HttpClientContext();
>>>>>>>>>>> +
>>>>>>>>>>> +    // first we use a method on HttpOp to log in and get our
>> cookie
>>>>>>>>>>> +    Params params = new Params();
>>>>>>>>>>> +    params.addParam("username", "Bob Wu");
>>>>>>>>>>> +    params.addParam("password", "my password");
>>>>>>>>>>> +    HttpOp.execHttpPostForm("http://example.com/loginform",
>> params , null, null, null, httpContext);
>>>>>>>>>>> +
>>>>>>>>>>> +    // now our cookie is stored in httpContext
>>>>>>>>>>> +    CookieStore cookieStore = httpContext.getCookieStore();
>>>>>>>>>>> +
>>>>>>>>>>> +    // lastly we build a client that uses that cookie
>>>>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>>>>> +     .setDefaultCookieStore(cookieStore)
>>>>>>>>>>> +         .build();
>>>>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>>>>> +
>>>>>>>>>>> +## Other concerns
>>>>>>>>>>> +
>>>>>>>>>>> +### Debugging Authentication
>>>>>>>>>>> +
>>>>>>>>>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations
>> and this provides detailed logging information that can be used for
>> debugging.  To
>>>>>>>>>>> +see this information you need to configure your logging
>> framework to set the `org.apache.http` package to either `DEBUG` or `TRACE`
>> level.
>>>>>>>>>>> +
>>>>>>>>>>> +The `DEBUG` level will give you general diagnostic information
>> about requests and responses while the `TRACE` level will give you detailed
>>>>>>>>>>> +HTTP traces i.e. allow you to see the exact HTTP requests and
>> responses which can be extremely useful for debugging authentication
>> problems.
>>>>>>>>>>> +
>>>>>>>>>>> +### Authenticating to a SPARQL federated service
>>>>>>>>>>> +
>>>>>>>>>>> +ARQ allows the user to configure HTTP behavior to use on a
>> per-`SERVICE` basis, including authentication behavior such as is described
>> above. This works via the ARQ context. See [Basic Federated Query][5] for
>> more information on configuring this functionality.
>>>>>>>>>>> +
>>>>>>>>>>> [1]: http://jena.apache.org/documentation/javadoc/arq/org/
>> apache/jena/atlas/web/auth/HttpAuthenticator.html
>>>>>>>>>>> [2]: http://jena.apache.org/documentation/javadoc/arq/org/
>> apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>>>>>>>>>> [3]: http://jena.apache.org/documentation/javadoc/arq/org/
>> apache/jena/atlas/web/auth/ScopedAuthenticator.html
>>>>>>>>>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>>>>>>>>>> [11]: http://jena.apache.org/documentation/javadoc/arq/org/
>> apache/jena/web/DatasetGraphAccessorHTTP.html
>>>>>>>>>>> [12]: http://jena.apache.org/documentation/javadoc/arq/org/
>> apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>>>>>>>>>> [13]: http://jena.apache.org/documentation/javadoc/arq/org/
>> apache/jena/riot/web/HttpOp.html
>>>>>>>>>>> -  [14]: http://hc.apache.org
>>>>>>>>>>> \ No newline at end of file
>>>>>>>>>>> +  [14]: http://hc.apache.org
>>>>>>>>>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/
>> httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>>>>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/
>> examples.html
>>>>>>>>>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/
>> httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>>>>>>>>>>> \ No newline at end of file
>>>>>>>>>>> 
>>>>>>>>>>> Modified: jena/site/trunk/content/documentation/query/service.
>> mdtext
>>>>>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/
>> documentation/query/service.mdtext?rev=1768736&r1=1768735&
>> r2=1768736&view=diff
>>>>>>>>>>> ============================================================
>> ==================
>>>>>>>>>>> --- jena/site/trunk/content/documentation/query/service.mdtext
>> (original)
>>>>>>>>>>> +++ jena/site/trunk/content/documentation/query/service.mdtext
>> Tue Nov  8 16:53:48 2016
>>>>>>>>>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>>>>>>>>>>> without regard to how selective the pattern is. So the order of
>> the
>>>>>>>>>>> query will affect the speed of execution. Because it involves
>> HTTP
>>>>>>>>>>> operations, asking the query in the right order matters a lot.
>>>>>>>>>>> -Don't ask for the whole of a bookstore just to find book whose
>>>>>>>>>>> +Don't ask for the whole of a bookstore just to find a book whose
>>>>>>>>>>> title comes from a local RDF file - ask the bookshop a query with
>>>>>>>>>>> the title already bound from earlier in the query.
>>>>>>>>>>> 
>>>>>>>>>>> ## Controlling `SERVICE` requests.
>>>>>>>>>>> 
>>>>>>>>>>> -The `SERVICE` operation in a SPARQL query may be configured via
>> the Context.
>>>>>>>>>>> -The values for configuration can be set in the global context
>> (accessed via
>>>>>>>>>>> +The `SERVICE` operation in a SPARQL query may be configured via
>> the Context. The values for configuration can be set in the global context
>> (accessed via
>>>>>>>>>>> `ARQ.getContext()`) or in the per-query execution context.
>>>>>>>>>>> 
>>>>>>>>>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#
>>> `.
>>>>>>>>>>> 
>>>>>>>>>>> -### Summary
>>>>>>>>>>> +### Configuration for ARQ through version 3.1.0
>>>>>>>>>>> 
>>>>>>>>>>> Symbol | Usage
>>>>>>>>>>> ------ | -----
>>>>>>>>>>> @@ -71,7 +70,7 @@ Symbol | Usage
>>>>>>>>>>> `srv:queryAuthPwd` |  Basic authentication
>>>>>>>>>>> `srv:queryContext` | Per-endpoint configuration
>>>>>>>>>>> 
>>>>>>>>>>> -### `srv:queryTimeout`
>>>>>>>>>>> +#### `srv:queryTimeout`
>>>>>>>>>>> 
>>>>>>>>>>> Set the connect and read timeouts for the query.
>>>>>>>>>>> 
>>>>>>>>>>> @@ -86,21 +85,21 @@ read timout = 0
>>>>>>>>>>> 
>>>>>>>>>>> Values of 0 indicate no timeout and service operation will wait
>> until the remote server responds.
>>>>>>>>>>> 
>>>>>>>>>>> -### `srv:queryGzip`
>>>>>>>>>>> +#### `srv:queryGzip`
>>>>>>>>>>> 
>>>>>>>>>>> Sets the allow Gzip flag.
>>>>>>>>>>> 
>>>>>>>>>>> Boolean: True indicates that gzip compressed data is acceptable.
>>>>>>>>>>> false
>>>>>>>>>>> 
>>>>>>>>>>> -### `srv:queryDeflate`
>>>>>>>>>>> +#### `srv:queryDeflate`
>>>>>>>>>>> 
>>>>>>>>>>> Sets the allow Deflate flag.
>>>>>>>>>>> 
>>>>>>>>>>> Boolean: True indicates that deflate compression is acceptable
>>>>>>>>>>> False
>>>>>>>>>>> 
>>>>>>>>>>> -### `srv:queryAuthUser`
>>>>>>>>>>> +#### `srv:queryAuthUser`
>>>>>>>>>>> 
>>>>>>>>>>> Sets the user id for basic auth.
>>>>>>>>>>> 
>>>>>>>>>>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>>>>>>>>>> 
>>>>>>>>>>> If null or null length no user id is sent.
>>>>>>>>>>> 
>>>>>>>>>>> -### `srv:queryAuthPwd`
>>>>>>>>>>> +#### `srv:queryAuthPwd`
>>>>>>>>>>> 
>>>>>>>>>>> Sets the password for basic auth.
>>>>>>>>>>> 
>>>>>>>>>>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>>>>>>>>>> 
>>>>>>>>>>> If null or null length no password is sent.
>>>>>>>>>>> 
>>>>>>>>>>> -### srv:serviceContext
>>>>>>>>>>> +#### `srv:serviceContext`
>>>>>>>>>>> Provides a mechanism to override system context settings on a
>> per URI basis.
>>>>>>>>>>> 
>>>>>>>>>>> The value is a `Map<String,Context>` where the map key is the
>> URI of the service endpoint, and the `Context` is a set of values to
>> override the default values.
>>>>>>>>>>> 
>>>>>>>>>>> If a context is provided for the URI the system context is
>> copied and the URI specific values are then copied in.  This ensures that
>> any URI specific settings will be used.
>>>>>>>>>>> 
>>>>>>>>>>> +### Configuration for ARQ after version 3.1.0
>>>>>>>>>>> 
>>>>>>>>>>> +Symbol | Usage | Default
>>>>>>>>>>> +------ | ----- | -------
>>>>>>>>>>> +`srv:queryTimeout` | Set timeouts | none
>>>>>>>>>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>>>>>>>>>>> +`srv:queryClient` | Enable use of a specific client | none
>>>>>>>>>>> +`srv:queryContext` | Per-endpoint configuration | none
>>>>>>>>>>> +
>>>>>>>>>>> +#### `srv:queryTimeout`
>>>>>>>>>>> +
>>>>>>>>>>> +As documented above.
>>>>>>>>>>> +
>>>>>>>>>>> +
>>>>>>>>>>> +#### `srv:queryCompression`
>>>>>>>>>>> +
>>>>>>>>>>> +Sets the flag for use of deflation and GZip.
>>>>>>>>>>> +
>>>>>>>>>>> +Boolean: True indicates that gzip compressed data is acceptable.
>>>>>>>>>>> +
>>>>>>>>>>> +#### `srv:queryClient`
>>>>>>>>>>> +
>>>>>>>>>>> +Enable use of a specific client
>>>>>>>>>>> +
>>>>>>>>>>> +Provides a slot for a specific [HttpClient][1] for use with a
>> specific `SERVICE`
>>>>>>>>>>> +
>>>>>>>>>>> +#### `srv:serviceContext`
>>>>>>>>>>> +
>>>>>>>>>>> +As documented above.
>>>>>>>>>>> 
>>>>>>>>>>> [ARQ documentation index](index.html)
>>>>>>>>>>> +
>>>>>>>>>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/
>> httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Stian Soiland-Reyes
>>>>> http://orcid.org/0000-0001-9842-9718
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Stian Soiland-Reyes
>>> http://orcid.org/0000-0001-9842-9718
>> 
>>

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by Stian Soiland-Reyes <st...@apache.org>.

No, I've only used the default caching mechanism, (as I wrote it ;-) - I
guess to do it properly we would then need to pass this through options or
the Documents loader to use.. :-/

On 9 Nov 2016 5:39 pm, "A. Soroka" <aj...@virginia.edu> wrote:

> Hm, now I'm not sure whether I can give a good example of managing the
> JSON-LD HTTP client within Jena. oaj.riot.lang.JsonLDReader doesn't seem to
> allow users to get at the DocumentLoader instance in use, and that's where
> the JSON-LD docs show an override to use a specific client. I don't see any
> static state of the kind you describe in JSON-LD, Stian-- can you give me a
> pointer?
>
> ---
> A. Soroka
> The University of Virginia Library
>
> > On Nov 9, 2016, at 11:46 AM, Stian Soiland-Reyes <st...@apache.org>
> wrote:
> >
> > Yeah, as the client is set static inside JSON-LD Java it might not
> > good for be Jena  HttpOps to be "helpful" and propagate its client -
> > at least not without a flag.  But we can link to it from our
> > documentation.
> >
> > On 9 November 2016 at 16:39, A. Soroka <aj...@virginia.edu> wrote:
> >> Hm, that's a really good point. I don't understand what's happening in
> JSON-LD very well, but I think you must be correct to say that it wouldn't
> take up a supplied client, because it wouldn't be using state from Jena.
> I'll check this a bit in the code and add a note as you suggest.
> >>
> >> ---
> >> A. Soroka
> >> The University of Virginia Library
> >>
> >>> On Nov 9, 2016, at 11:37 AM, Stian Soiland-Reyes <st...@apache.org>
> wrote:
> >>>
> >>> I think the new text looks good, quite easy to understand.
> >>>
> >>> Could you add a paragraph about when the configured client would be
> >>> used?   It might not be clear when this HttpClient would be accessed
> >>> or not.  For instance I assume it would be used for remote SPARQL
> >>> queries or loading of HTTP URLs from RDFDataMgr -- but may not be
> >>> propagated through to JSON-LD Java's @context loading - which has a
> >>> similar httpclient setting and documentation on how to configure
> >>> caching [1]
> >>>
> >>> [1] https://github.com/jsonld-java/jsonld-java#controlling-
> network-traffic
> >>>
> >>>
> >>>
> >>>
> >>> On 9 November 2016 at 15:42, A. Soroka <aj...@virginia.edu> wrote:
> >>>> Done. I'll wait to hear from other folks before pulling a trigger on
> (re)publishing the site.
> >>>>
> >>>> ---
> >>>> A. Soroka
> >>>> The University of Virginia Library
> >>>>
> >>>>> On Nov 9, 2016, at 6:30 AM, Andy Seaborne <an...@apache.org> wrote:
> >>>>>
> >>>>> Great -
> >>>>>
> >>>>> One (!) other thing:
> >>>>>
> >>>>> A section specifically calling out migrating SPARQL remote calls:
> QueryExecutionFactory.sparqlService and QueryEngineHTTP.
> >>>>>
> >>>>> On the latter, older code may still be directly using
> QueryEngineHTTP.setBasicAuthentication
> >>>>>
> >>>>>     Andy
> >>>>>
> >>>>> On 08/11/16 17:58, A. Soroka wrote:
> >>>>>> I've made those changes-- should be restaging now.
> >>>>>>
> >>>>>> ---
> >>>>>> A. Soroka
> >>>>>> The University of Virginia Library
> >>>>>>
> >>>>>>> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org>
> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On 08/11/16 16:59, A. Soroka wrote:
> >>>>>>>> This commit includes the new docs for HTTP behavior in Jena
> 3.1.1. I can't find any way to see a view of this on the staging site--
> https://jena.staging.apache.org/ just seems to proxy
> https://cms.apache.org/, for some reason?
> >>>>>>>>
> >>>>>>>
> >>>>>>> It does not for me.
> >>>>>>>
> >>>>>>> Try http://jena.staging.apache.org/  (not https)
> >>>>>>>
> >>>>>>> PDF attached, cc'ed to you in the hope it get through.
> >>>>>>>
> >>>>>>> Comments:
> >>>>>>>
> >>>>>>> 1/ I'd put the current (3.1.1) text first and the previous second
> so the current is more visible.
> >>>>>>>
> >>>>>>> Links at the end of the intro to "current" and "previous", or in
> the intro as this difference is mentioned.
> >>>>>>>
> >>>>>>> 2/ Title tweaking:
> >>>>>>>
> >>>>>>> "HTTP Authentication after Jena 3.1.0" ->
> >>>>>>> "HTTP Authentication from Jena 3.1.1"
> >>>>>>>
> >>>>>>> "HTTP Authentication before Jena 3.1.0" =>
> >>>>>>> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
> >>>>>>>
> >>>>>>> (so the range includes 3.1.0 !)
> >>>>>>>
> >>>>>>> Mentioning Jena 2.x is not necessary IMO - the additional detail
> adds confusion for current users and 3.x upgrading users (the majority).
> >>>>>>>
> >>>>>>> 3/
> >>>>>>> "Simple authentication using username and password"
> >>>>>>>
> >>>>>>> "Authenticating via a form"
> >>>>>>>
> >>>>>>> The <h5> don't show up as different on teh screen for me so maybe
> bump <h4> "Examples of authentication" up a level to <h3> and move sub <5>
> to <h4> .
> >>>>>>>
> >>>>>>> Maybe drop <h3> "Applying Authentication" (section title
> immediately after a section title) and have the paragraph there straight
> away.
> >>>>>>>
> >>>>>>> Andy
> >>>>>>>
> >>>>>>>> ---
> >>>>>>>> A. Soroka
> >>>>>>>> The University of Virginia Library
> >>>>>>>>
> >>>>>>>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
> >>>>>>>>>
> >>>>>>>>> Author: ajs6f
> >>>>>>>>> Date: Tue Nov  8 16:53:48 2016
> >>>>>>>>> New Revision: 1768736
> >>>>>>>>>
> >>>>>>>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
> >>>>>>>>> Log:
> >>>>>>>>> Updates for HTTP behavior in Jena 3.1.1
> >>>>>>>>>
> >>>>>>>>> Modified:
> >>>>>>>>> jena/site/trunk/content/documentation/query/http-auth.mdtext
> >>>>>>>>> jena/site/trunk/content/documentation/query/service.mdtext
> >>>>>>>>>
> >>>>>>>>> Modified: jena/site/trunk/content/documentation/query/http-auth.
> mdtext
> >>>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/
> documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&
> r2=1768736&view=diff
> >>>>>>>>> ============================================================
> ==================
> >>>>>>>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext
> (original)
> >>>>>>>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext
> Tue Nov  8 16:53:48 2016
> >>>>>>>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
> >>>>>>>>>        specific language governing permissions and limitations
> >>>>>>>>>        under the License.
> >>>>>>>>>
> >>>>>>>>> -As of ARQ 2.11.0 there is a new unified HTTP operation
> framework that provides a uniform mechanism for
> >>>>>>>>> -HTTP authentication that also allows ARQ to support a broader
> range of authentication mechanisms than were previously possible.
> >>>>>>>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific
> unified HTTP operation framework that provides a uniform mechanism for
> >>>>>>>>> +HTTP authentication that also allows ARQ to support a broader
> range of authentication mechanisms than were previously possible. After ARQ
> 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same
> end. This documentation is therefore devided into two sections. The first
> explains the older Jena-specific functionality, and the second explains how
> to use HTTP Commons code to the same ends.
> >>>>>>>>>
> >>>>>>>>> -## Applying Authentication
> >>>>>>>>> +## HTTP Authentication before ARQ 3.1.0
> >>>>>>>>> +
> >>>>>>>>> +### Applying Authentication
> >>>>>>>>>
> >>>>>>>>> APIs that support authentication typically provide two methods
> for providing authenticators, a `setAuthentication(String username, char[]
> password)` method
> >>>>>>>>> which merely configures a `SimpleAuthenticator`.  There will
> also be a `setAuthenticator(HttpAuthenticator authenticator)` method
> >>>>>>>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
> >>>>>>>>>     ...
> >>>>>>>>> }
> >>>>>>>>>
> >>>>>>>>> -### Authenticators
> >>>>>>>>> +#### Authenticators
> >>>>>>>>>
> >>>>>>>>> Authentication mechanisms are provided by [HttpAuthenticator][1]
> implementations of which a number are provided built into ARQ.
> >>>>>>>>>
> >>>>>>>>> This API provides the authenticator with access to the
> `HttpClient`, `HttpContext` and target `URI` of the request that is about
> to be carried
> >>>>>>>>> out.  This allows for authenticators to add credentials to
> requests on a per-request basis and/or to use different mechanisms and
> credentials for different services.
> >>>>>>>>>
> >>>>>>>>> -#### SimpleAuthenticator
> >>>>>>>>> +##### SimpleAuthenticator
> >>>>>>>>>
> >>>>>>>>> The [simple authenticator][2] is as the name suggests the
> simplest implementation.  It takes a single set of credentials which is
> applied to
> >>>>>>>>> any service.
> >>>>>>>>> @@ -56,7 +58,7 @@ any service.
> >>>>>>>>> Authentication however is not preemptive so unless the remote
> service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
> >>>>>>>>> Required) then credentials will not actually be submitted.
> >>>>>>>>>
> >>>>>>>>> -#### ScopedAuthenticator
> >>>>>>>>> +##### ScopedAuthenticator
> >>>>>>>>>
> >>>>>>>>> The [scoped authenticator][3] is an authenticator which maps
> credentials to different service URIs.  This allows you to specify different
> >>>>>>>>> credentials for different services as appropriate.  Similarly to
> the simple authenticator this is not preemptive authentication so
> credentials are
> >>>>>>>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
> >>>>>>>>> e.g. `http://example.org/some/path`.  However if you had also
> defined credentials for `http://example.org/some/path` then these would be
> >>>>>>>>> used in favor of those for `http://example.org`
> >>>>>>>>>
> >>>>>>>>> -#### ServiceAuthenticator
> >>>>>>>>> +##### ServiceAuthenticator
> >>>>>>>>>
> >>>>>>>>> The [service authenticator][4] is an authenticator which uses
> information encoded in the ARQ context and basically provides access to the
> >>>>>>>>> existing credential provision mechanisms provided for the
> `SERVICE` clause, see [Basic Federated Query][5] for more information on
> >>>>>>>>> configuration for this.
> >>>>>>>>>
> >>>>>>>>> -#### FormsAuthenticator
> >>>>>>>>> +##### FormsAuthenticator
> >>>>>>>>>
> >>>>>>>>> The [forms authenticator][6] is an authenticator usable with
> services that require form based logins and use session cookies to verify
> login state.
> >>>>>>>>> This is intended for use with services that don't support HTTP's
> built-in authentication mechanisms for whatever reason.  One example of this
> >>>>>>>>> @@ -104,7 +106,7 @@ that maps each service to an associated
> >>>>>>>>>
> >>>>>>>>> Currently forms based login that require more than just a
> username and password are not supported.
> >>>>>>>>>
> >>>>>>>>> -#### PreemptiveBasicAuthenticator
> >>>>>>>>> +##### PreemptiveBasicAuthenticator
> >>>>>>>>>
> >>>>>>>>> This [authenticator][8] is a decorator over another
> authenticator that enables preemptive basic authentication, this **only**
> works for servers
> >>>>>>>>> that support basic authentication and so will cause
> authentication failures when any other authentication scheme is required.
> You should **only**
> >>>>>>>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
> >>>>>>>>> many servers will use more secure schemes like Digest
> authentication which **cannot** be done preemptively as they require more
> complex
> >>>>>>>>> challenge response sequences.
> >>>>>>>>>
> >>>>>>>>> -#### DelegatingAuthenticator
> >>>>>>>>> +##### DelegatingAuthenticator
> >>>>>>>>>
> >>>>>>>>> The [delegating authenticator][12] allows for mapping different
> authenticators to different services, this is useful when you need to mix
> and
> >>>>>>>>> match the types of authentication needed.
> >>>>>>>>>
> >>>>>>>>> -### Debugging Authentication
> >>>>>>>>> -
> >>>>>>>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations
> and this provides detailed logging information that can be used for
> debugging.  To
> >>>>>>>>> -see this information you need to configure your logging
> framework to set the `org.apache.http` package to either `DEBUG` or `TRACE`
> level.
> >>>>>>>>> -
> >>>>>>>>> -The `DEBUG` level will give you general diagnostic information
> about requests and responses while the `TRACE` level will give you detailed
> >>>>>>>>> -HTTP traces i.e. allow you to see the exact HTTP requests and
> responses which can be extremely useful for debugging authentication
> problems.
> >>>>>>>>> -
> >>>>>>>>> -### The Default Authenticator
> >>>>>>>>> +#### The Default Authenticator
> >>>>>>>>>
> >>>>>>>>> Since it may not always be possible/practical to configure
> authenticators on a per-request basis the API includes a means to specify a
> default
> >>>>>>>>> authenticator that is used when no authenticator is explicitly
> specified.  This may be configured via the
> >>>>>>>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
> >>>>>>>>>
> >>>>>>>>> Note that the default authenticator may be disabled by setting
> it to `null`.
> >>>>>>>>>
> >>>>>>>>> +## HTTP Authentication after ARQ 3.1.0
> >>>>>>>>> +
> >>>>>>>>> +### Applying Authentication
> >>>>>>>>> +
> >>>>>>>>> +APIs that support authentication typically provide methods for
> providing an [HttpClient] for use with the given instance of that API
> class. `HttpClient` is [extremely flexible][16] and can handle most
> scenarios very well. Since it may not always be possible/practical to
> configure authenticators on a per-request basis the API includes a means to
> specify a default client that is used when no other client is explicitly
> specified.  This may be configured via the
> >>>>>>>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the
> [HttpOp][13] class. This allows for static-scoped configuration of HTTP
> behavior.
> >>>>>>>>> +
> >>>>>>>>> +#### Examples of authentication
> >>>>>>>>> +
> >>>>>>>>> +This section includes a series of examples showing how to use
> HTTP Commons classes to perform authenticated work. Most of them take
> advantage of `HttpOp.setDefaultHttpClient` as described above.
> >>>>>>>>> +
> >>>>>>>>> +##### Simple authentication using username and password
> >>>>>>>>> +
> >>>>>>>>> +First we build an authenticating client:
> >>>>>>>>> +
> >>>>>>>>> +    CredentialsProvider credsProvider = new
> BasicCredentialsProvider();
> >>>>>>>>> +    Credentials credentials = new UsernamePasswordCredentials("user",
> "passwd");
> >>>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
> >>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
> >>>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
> >>>>>>>>> +        .build();
> >>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
> >>>>>>>>> +
> >>>>>>>>> +Notice that we gave no scope for use with the credentials
> (`AuthScope.ANY`). We can make further use of that parameter if we want to
> assign a scope for some credentials:
> >>>>>>>>> +
> >>>>>>>>> +    CredentialsProvider credsProvider = new
> BasicCredentialsProvider();
> >>>>>>>>> +    Credentials unscopedCredentials = new
> UsernamePasswordCredentials("user", "passwd");
> >>>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY,
> unscopedCredentials);
> >>>>>>>>> +    Credentials scopedCredentials = new
> UsernamePasswordCredentials("user", "passwd");
> >>>>>>>>> +    final String host = "http://example.com/sparql";
> >>>>>>>>> +    final int port = 80;
> >>>>>>>>> +    final String realm = "aRealm";
> >>>>>>>>> +    final String schemeName = "DIGEST";
> >>>>>>>>> +    AuthScope authscope = new AuthScope(host, port, realm,
> schemeName);
> >>>>>>>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
> >>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
> >>>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
> >>>>>>>>> +        .build();
> >>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
> >>>>>>>>> +
> >>>>>>>>> +##### Authenticating via a form
> >>>>>>>>> +
> >>>>>>>>> +For this case we introduce an [HttpClientContext][17], which we
> can use to retrieve the cookie we get from logging into a form. We then use
> the cookie to authenticate elsewhere.
> >>>>>>>>> +
> >>>>>>>>> +    // we'll use this context to maintain our HTTP
> "conversation"
> >>>>>>>>> +    HttpClientContext httpContext = new HttpClientContext();
> >>>>>>>>> +
> >>>>>>>>> +    // first we use a method on HttpOp to log in and get our
> cookie
> >>>>>>>>> +    Params params = new Params();
> >>>>>>>>> +    params.addParam("username", "Bob Wu");
> >>>>>>>>> +    params.addParam("password", "my password");
> >>>>>>>>> +    HttpOp.execHttpPostForm("http://example.com/loginform",
> params , null, null, null, httpContext);
> >>>>>>>>> +
> >>>>>>>>> +    // now our cookie is stored in httpContext
> >>>>>>>>> +    CookieStore cookieStore = httpContext.getCookieStore();
> >>>>>>>>> +
> >>>>>>>>> +    // lastly we build a client that uses that cookie
> >>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
> >>>>>>>>> +     .setDefaultCookieStore(cookieStore)
> >>>>>>>>> +         .build();
> >>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
> >>>>>>>>> +
> >>>>>>>>> +## Other concerns
> >>>>>>>>> +
> >>>>>>>>> +### Debugging Authentication
> >>>>>>>>> +
> >>>>>>>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations
> and this provides detailed logging information that can be used for
> debugging.  To
> >>>>>>>>> +see this information you need to configure your logging
> framework to set the `org.apache.http` package to either `DEBUG` or `TRACE`
> level.
> >>>>>>>>> +
> >>>>>>>>> +The `DEBUG` level will give you general diagnostic information
> about requests and responses while the `TRACE` level will give you detailed
> >>>>>>>>> +HTTP traces i.e. allow you to see the exact HTTP requests and
> responses which can be extremely useful for debugging authentication
> problems.
> >>>>>>>>> +
> >>>>>>>>> +### Authenticating to a SPARQL federated service
> >>>>>>>>> +
> >>>>>>>>> +ARQ allows the user to configure HTTP behavior to use on a
> per-`SERVICE` basis, including authentication behavior such as is described
> above. This works via the ARQ context. See [Basic Federated Query][5] for
> more information on configuring this functionality.
> >>>>>>>>> +
> >>>>>>>>> [1]: http://jena.apache.org/documentation/javadoc/arq/org/
> apache/jena/atlas/web/auth/HttpAuthenticator.html
> >>>>>>>>> [2]: http://jena.apache.org/documentation/javadoc/arq/org/
> apache/jena/atlas/web/auth/SimpleAuthenticator.html
> >>>>>>>>> [3]: http://jena.apache.org/documentation/javadoc/arq/org/
> apache/jena/atlas/web/auth/ScopedAuthenticator.html
> >>>>>>>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
> >>>>>>>>> [11]: http://jena.apache.org/documentation/javadoc/arq/org/
> apache/jena/web/DatasetGraphAccessorHTTP.html
> >>>>>>>>> [12]: http://jena.apache.org/documentation/javadoc/arq/org/
> apache/jena/atlas/web/auth/DelegatingAuthenticator.html
> >>>>>>>>> [13]: http://jena.apache.org/documentation/javadoc/arq/org/
> apache/jena/riot/web/HttpOp.html
> >>>>>>>>> -  [14]: http://hc.apache.org
> >>>>>>>>> \ No newline at end of file
> >>>>>>>>> +  [14]: http://hc.apache.org
> >>>>>>>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/
> httpclient/apidocs/org/apache/http/client/HttpClient.html
> >>>>>>>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/
> examples.html
> >>>>>>>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/
> httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
> >>>>>>>>> \ No newline at end of file
> >>>>>>>>>
> >>>>>>>>> Modified: jena/site/trunk/content/documentation/query/service.
> mdtext
> >>>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/
> documentation/query/service.mdtext?rev=1768736&r1=1768735&
> r2=1768736&view=diff
> >>>>>>>>> ============================================================
> ==================
> >>>>>>>>> --- jena/site/trunk/content/documentation/query/service.mdtext
> (original)
> >>>>>>>>> +++ jena/site/trunk/content/documentation/query/service.mdtext
> Tue Nov  8 16:53:48 2016
> >>>>>>>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
> >>>>>>>>> without regard to how selective the pattern is. So the order of
> the
> >>>>>>>>> query will affect the speed of execution. Because it involves
> HTTP
> >>>>>>>>> operations, asking the query in the right order matters a lot.
> >>>>>>>>> -Don't ask for the whole of a bookstore just to find book whose
> >>>>>>>>> +Don't ask for the whole of a bookstore just to find a book whose
> >>>>>>>>> title comes from a local RDF file - ask the bookshop a query with
> >>>>>>>>> the title already bound from earlier in the query.
> >>>>>>>>>
> >>>>>>>>> ## Controlling `SERVICE` requests.
> >>>>>>>>>
> >>>>>>>>> -The `SERVICE` operation in a SPARQL query may be configured via
> the Context.
> >>>>>>>>> -The values for configuration can be set in the global context
> (accessed via
> >>>>>>>>> +The `SERVICE` operation in a SPARQL query may be configured via
> the Context. The values for configuration can be set in the global context
> (accessed via
> >>>>>>>>> `ARQ.getContext()`) or in the per-query execution context.
> >>>>>>>>>
> >>>>>>>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#
> >`.
> >>>>>>>>>
> >>>>>>>>> -### Summary
> >>>>>>>>> +### Configuration for ARQ through version 3.1.0
> >>>>>>>>>
> >>>>>>>>> Symbol | Usage
> >>>>>>>>> ------ | -----
> >>>>>>>>> @@ -71,7 +70,7 @@ Symbol | Usage
> >>>>>>>>> `srv:queryAuthPwd` |  Basic authentication
> >>>>>>>>> `srv:queryContext` | Per-endpoint configuration
> >>>>>>>>>
> >>>>>>>>> -### `srv:queryTimeout`
> >>>>>>>>> +#### `srv:queryTimeout`
> >>>>>>>>>
> >>>>>>>>> Set the connect and read timeouts for the query.
> >>>>>>>>>
> >>>>>>>>> @@ -86,21 +85,21 @@ read timout = 0
> >>>>>>>>>
> >>>>>>>>> Values of 0 indicate no timeout and service operation will wait
> until the remote server responds.
> >>>>>>>>>
> >>>>>>>>> -### `srv:queryGzip`
> >>>>>>>>> +#### `srv:queryGzip`
> >>>>>>>>>
> >>>>>>>>> Sets the allow Gzip flag.
> >>>>>>>>>
> >>>>>>>>> Boolean: True indicates that gzip compressed data is acceptable.
> >>>>>>>>> false
> >>>>>>>>>
> >>>>>>>>> -### `srv:queryDeflate`
> >>>>>>>>> +#### `srv:queryDeflate`
> >>>>>>>>>
> >>>>>>>>> Sets the allow Deflate flag.
> >>>>>>>>>
> >>>>>>>>> Boolean: True indicates that deflate compression is acceptable
> >>>>>>>>> False
> >>>>>>>>>
> >>>>>>>>> -### `srv:queryAuthUser`
> >>>>>>>>> +#### `srv:queryAuthUser`
> >>>>>>>>>
> >>>>>>>>> Sets the user id for basic auth.
> >>>>>>>>>
> >>>>>>>>> @@ -108,7 +107,7 @@ String: The user id to log in with
> >>>>>>>>>
> >>>>>>>>> If null or null length no user id is sent.
> >>>>>>>>>
> >>>>>>>>> -### `srv:queryAuthPwd`
> >>>>>>>>> +#### `srv:queryAuthPwd`
> >>>>>>>>>
> >>>>>>>>> Sets the password for basic auth.
> >>>>>>>>>
> >>>>>>>>> @@ -116,13 +115,43 @@ String: The password to log in with.
> >>>>>>>>>
> >>>>>>>>> If null or null length no password is sent.
> >>>>>>>>>
> >>>>>>>>> -### srv:serviceContext
> >>>>>>>>> +#### `srv:serviceContext`
> >>>>>>>>> Provides a mechanism to override system context settings on a
> per URI basis.
> >>>>>>>>>
> >>>>>>>>> The value is a `Map<String,Context>` where the map key is the
> URI of the service endpoint, and the `Context` is a set of values to
> override the default values.
> >>>>>>>>>
> >>>>>>>>> If a context is provided for the URI the system context is
> copied and the URI specific values are then copied in.  This ensures that
> any URI specific settings will be used.
> >>>>>>>>>
> >>>>>>>>> +### Configuration for ARQ after version 3.1.0
> >>>>>>>>>
> >>>>>>>>> +Symbol | Usage | Default
> >>>>>>>>> +------ | ----- | -------
> >>>>>>>>> +`srv:queryTimeout` | Set timeouts | none
> >>>>>>>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
> >>>>>>>>> +`srv:queryClient` | Enable use of a specific client | none
> >>>>>>>>> +`srv:queryContext` | Per-endpoint configuration | none
> >>>>>>>>> +
> >>>>>>>>> +#### `srv:queryTimeout`
> >>>>>>>>> +
> >>>>>>>>> +As documented above.
> >>>>>>>>> +
> >>>>>>>>> +
> >>>>>>>>> +#### `srv:queryCompression`
> >>>>>>>>> +
> >>>>>>>>> +Sets the flag for use of deflation and GZip.
> >>>>>>>>> +
> >>>>>>>>> +Boolean: True indicates that gzip compressed data is acceptable.
> >>>>>>>>> +
> >>>>>>>>> +#### `srv:queryClient`
> >>>>>>>>> +
> >>>>>>>>> +Enable use of a specific client
> >>>>>>>>> +
> >>>>>>>>> +Provides a slot for a specific [HttpClient][1] for use with a
> specific `SERVICE`
> >>>>>>>>> +
> >>>>>>>>> +#### `srv:serviceContext`
> >>>>>>>>> +
> >>>>>>>>> +As documented above.
> >>>>>>>>>
> >>>>>>>>> [ARQ documentation index](index.html)
> >>>>>>>>> +
> >>>>>>>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/
> httpclient/apidocs/org/apache/http/client/HttpClient.html
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Stian Soiland-Reyes
> >>> http://orcid.org/0000-0001-9842-9718
> >>
> >
> >
> >
> > --
> > Stian Soiland-Reyes
> > http://orcid.org/0000-0001-9842-9718
>
>

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by "A. Soroka" <aj...@virginia.edu>.

Hm, now I'm not sure whether I can give a good example of managing the JSON-LD HTTP client within Jena. oaj.riot.lang.JsonLDReader doesn't seem to allow users to get at the DocumentLoader instance in use, and that's where the JSON-LD docs show an override to use a specific client. I don't see any static state of the kind you describe in JSON-LD, Stian-- can you give me a pointer?

---
A. Soroka
The University of Virginia Library

> On Nov 9, 2016, at 11:46 AM, Stian Soiland-Reyes <st...@apache.org> wrote:
> 
> Yeah, as the client is set static inside JSON-LD Java it might not
> good for be Jena  HttpOps to be "helpful" and propagate its client -
> at least not without a flag.  But we can link to it from our
> documentation.
> 
> On 9 November 2016 at 16:39, A. Soroka <aj...@virginia.edu> wrote:
>> Hm, that's a really good point. I don't understand what's happening in JSON-LD very well, but I think you must be correct to say that it wouldn't take up a supplied client, because it wouldn't be using state from Jena. I'll check this a bit in the code and add a note as you suggest.
>> 
>> ---
>> A. Soroka
>> The University of Virginia Library
>> 
>>> On Nov 9, 2016, at 11:37 AM, Stian Soiland-Reyes <st...@apache.org> wrote:
>>> 
>>> I think the new text looks good, quite easy to understand.
>>> 
>>> Could you add a paragraph about when the configured client would be
>>> used?   It might not be clear when this HttpClient would be accessed
>>> or not.  For instance I assume it would be used for remote SPARQL
>>> queries or loading of HTTP URLs from RDFDataMgr -- but may not be
>>> propagated through to JSON-LD Java's @context loading - which has a
>>> similar httpclient setting and documentation on how to configure
>>> caching [1]
>>> 
>>> [1] https://github.com/jsonld-java/jsonld-java#controlling-network-traffic
>>> 
>>> 
>>> 
>>> 
>>> On 9 November 2016 at 15:42, A. Soroka <aj...@virginia.edu> wrote:
>>>> Done. I'll wait to hear from other folks before pulling a trigger on (re)publishing the site.
>>>> 
>>>> ---
>>>> A. Soroka
>>>> The University of Virginia Library
>>>> 
>>>>> On Nov 9, 2016, at 6:30 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>> 
>>>>> Great -
>>>>> 
>>>>> One (!) other thing:
>>>>> 
>>>>> A section specifically calling out migrating SPARQL remote calls: QueryExecutionFactory.sparqlService and QueryEngineHTTP.
>>>>> 
>>>>> On the latter, older code may still be directly using QueryEngineHTTP.setBasicAuthentication
>>>>> 
>>>>>     Andy
>>>>> 
>>>>> On 08/11/16 17:58, A. Soroka wrote:
>>>>>> I've made those changes-- should be restaging now.
>>>>>> 
>>>>>> ---
>>>>>> A. Soroka
>>>>>> The University of Virginia Library
>>>>>> 
>>>>>>> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 08/11/16 16:59, A. Soroka wrote:
>>>>>>>> This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?
>>>>>>>> 
>>>>>>> 
>>>>>>> It does not for me.
>>>>>>> 
>>>>>>> Try http://jena.staging.apache.org/  (not https)
>>>>>>> 
>>>>>>> PDF attached, cc'ed to you in the hope it get through.
>>>>>>> 
>>>>>>> Comments:
>>>>>>> 
>>>>>>> 1/ I'd put the current (3.1.1) text first and the previous second so the current is more visible.
>>>>>>> 
>>>>>>> Links at the end of the intro to "current" and "previous", or in the intro as this difference is mentioned.
>>>>>>> 
>>>>>>> 2/ Title tweaking:
>>>>>>> 
>>>>>>> "HTTP Authentication after Jena 3.1.0" ->
>>>>>>> "HTTP Authentication from Jena 3.1.1"
>>>>>>> 
>>>>>>> "HTTP Authentication before Jena 3.1.0" =>
>>>>>>> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
>>>>>>> 
>>>>>>> (so the range includes 3.1.0 !)
>>>>>>> 
>>>>>>> Mentioning Jena 2.x is not necessary IMO - the additional detail adds confusion for current users and 3.x upgrading users (the majority).
>>>>>>> 
>>>>>>> 3/
>>>>>>> "Simple authentication using username and password"
>>>>>>> 
>>>>>>> "Authenticating via a form"
>>>>>>> 
>>>>>>> The <h5> don't show up as different on teh screen for me so maybe bump <h4> "Examples of authentication" up a level to <h3> and move sub <5> to <h4> .
>>>>>>> 
>>>>>>> Maybe drop <h3> "Applying Authentication" (section title immediately after a section title) and have the paragraph there straight away.
>>>>>>> 
>>>>>>> Andy
>>>>>>> 
>>>>>>>> ---
>>>>>>>> A. Soroka
>>>>>>>> The University of Virginia Library
>>>>>>>> 
>>>>>>>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>>>>>>>> 
>>>>>>>>> Author: ajs6f
>>>>>>>>> Date: Tue Nov  8 16:53:48 2016
>>>>>>>>> New Revision: 1768736
>>>>>>>>> 
>>>>>>>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>>>>>>>>> Log:
>>>>>>>>> Updates for HTTP behavior in Jena 3.1.1
>>>>>>>>> 
>>>>>>>>> Modified:
>>>>>>>>> jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>>>>> jena/site/trunk/content/documentation/query/service.mdtext
>>>>>>>>> 
>>>>>>>>> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>>>>>> ==============================================================================
>>>>>>>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
>>>>>>>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
>>>>>>>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>>>>>>>>        specific language governing permissions and limitations
>>>>>>>>>        under the License.
>>>>>>>>> 
>>>>>>>>> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for
>>>>>>>>> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
>>>>>>>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for
>>>>>>>>> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
>>>>>>>>> 
>>>>>>>>> -## Applying Authentication
>>>>>>>>> +## HTTP Authentication before ARQ 3.1.0
>>>>>>>>> +
>>>>>>>>> +### Applying Authentication
>>>>>>>>> 
>>>>>>>>> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
>>>>>>>>> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>>>>>>>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>>>>>>>>     ...
>>>>>>>>> }
>>>>>>>>> 
>>>>>>>>> -### Authenticators
>>>>>>>>> +#### Authenticators
>>>>>>>>> 
>>>>>>>>> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
>>>>>>>>> 
>>>>>>>>> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried
>>>>>>>>> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
>>>>>>>>> 
>>>>>>>>> -#### SimpleAuthenticator
>>>>>>>>> +##### SimpleAuthenticator
>>>>>>>>> 
>>>>>>>>> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
>>>>>>>>> any service.
>>>>>>>>> @@ -56,7 +58,7 @@ any service.
>>>>>>>>> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>>>>>>>> Required) then credentials will not actually be submitted.
>>>>>>>>> 
>>>>>>>>> -#### ScopedAuthenticator
>>>>>>>>> +##### ScopedAuthenticator
>>>>>>>>> 
>>>>>>>>> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different
>>>>>>>>> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are
>>>>>>>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>>>>>>>>> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be
>>>>>>>>> used in favor of those for `http://example.org`
>>>>>>>>> 
>>>>>>>>> -#### ServiceAuthenticator
>>>>>>>>> +##### ServiceAuthenticator
>>>>>>>>> 
>>>>>>>>> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the
>>>>>>>>> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on
>>>>>>>>> configuration for this.
>>>>>>>>> 
>>>>>>>>> -#### FormsAuthenticator
>>>>>>>>> +##### FormsAuthenticator
>>>>>>>>> 
>>>>>>>>> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
>>>>>>>>> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this
>>>>>>>>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>>>>>>>> 
>>>>>>>>> Currently forms based login that require more than just a username and password are not supported.
>>>>>>>>> 
>>>>>>>>> -#### PreemptiveBasicAuthenticator
>>>>>>>>> +##### PreemptiveBasicAuthenticator
>>>>>>>>> 
>>>>>>>>> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers
>>>>>>>>> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
>>>>>>>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>>>>>>>>> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
>>>>>>>>> challenge response sequences.
>>>>>>>>> 
>>>>>>>>> -#### DelegatingAuthenticator
>>>>>>>>> +##### DelegatingAuthenticator
>>>>>>>>> 
>>>>>>>>> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and
>>>>>>>>> match the types of authentication needed.
>>>>>>>>> 
>>>>>>>>> -### Debugging Authentication
>>>>>>>>> -
>>>>>>>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>>>>>> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>>>>>> -
>>>>>>>>> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>>>>>> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>>>>>> -
>>>>>>>>> -### The Default Authenticator
>>>>>>>>> +#### The Default Authenticator
>>>>>>>>> 
>>>>>>>>> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default
>>>>>>>>> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the
>>>>>>>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>>>>>>>> 
>>>>>>>>> Note that the default authenticator may be disabled by setting it to `null`.
>>>>>>>>> 
>>>>>>>>> +## HTTP Authentication after ARQ 3.1.0
>>>>>>>>> +
>>>>>>>>> +### Applying Authentication
>>>>>>>>> +
>>>>>>>>> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the
>>>>>>>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
>>>>>>>>> +
>>>>>>>>> +#### Examples of authentication
>>>>>>>>> +
>>>>>>>>> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
>>>>>>>>> +
>>>>>>>>> +##### Simple authentication using username and password
>>>>>>>>> +
>>>>>>>>> +First we build an authenticating client:
>>>>>>>>> +
>>>>>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>>>>>> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>>>>> +        .build();
>>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>>> +
>>>>>>>>> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
>>>>>>>>> +
>>>>>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>>>>>> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
>>>>>>>>> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>>>> +    final String host = "http://example.com/sparql";
>>>>>>>>> +    final int port = 80;
>>>>>>>>> +    final String realm = "aRealm";
>>>>>>>>> +    final String schemeName = "DIGEST";
>>>>>>>>> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
>>>>>>>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>>>>> +        .build();
>>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>>> +
>>>>>>>>> +##### Authenticating via a form
>>>>>>>>> +
>>>>>>>>> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
>>>>>>>>> +
>>>>>>>>> +    // we'll use this context to maintain our HTTP "conversation"
>>>>>>>>> +    HttpClientContext httpContext = new HttpClientContext();
>>>>>>>>> +
>>>>>>>>> +    // first we use a method on HttpOp to log in and get our cookie
>>>>>>>>> +    Params params = new Params();
>>>>>>>>> +    params.addParam("username", "Bob Wu");
>>>>>>>>> +    params.addParam("password", "my password");
>>>>>>>>> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
>>>>>>>>> +
>>>>>>>>> +    // now our cookie is stored in httpContext
>>>>>>>>> +    CookieStore cookieStore = httpContext.getCookieStore();
>>>>>>>>> +
>>>>>>>>> +    // lastly we build a client that uses that cookie
>>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>>> +     .setDefaultCookieStore(cookieStore)
>>>>>>>>> +         .build();
>>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>>> +
>>>>>>>>> +## Other concerns
>>>>>>>>> +
>>>>>>>>> +### Debugging Authentication
>>>>>>>>> +
>>>>>>>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>>>>>> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>>>>>> +
>>>>>>>>> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>>>>>> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>>>>>> +
>>>>>>>>> +### Authenticating to a SPARQL federated service
>>>>>>>>> +
>>>>>>>>> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
>>>>>>>>> +
>>>>>>>>> [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>>>>>>>>> [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>>>>>>>> [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
>>>>>>>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>>>>>>>> [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>>>>>>>>> [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>>>>>>>> [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
>>>>>>>>> -  [14]: http://hc.apache.org
>>>>>>>>> \ No newline at end of file
>>>>>>>>> +  [14]: http://hc.apache.org
>>>>>>>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
>>>>>>>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>>>>>>>>> \ No newline at end of file
>>>>>>>>> 
>>>>>>>>> Modified: jena/site/trunk/content/documentation/query/service.mdtext
>>>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>>>>>> ==============================================================================
>>>>>>>>> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
>>>>>>>>> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
>>>>>>>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>>>>>>>>> without regard to how selective the pattern is. So the order of the
>>>>>>>>> query will affect the speed of execution. Because it involves HTTP
>>>>>>>>> operations, asking the query in the right order matters a lot.
>>>>>>>>> -Don't ask for the whole of a bookstore just to find book whose
>>>>>>>>> +Don't ask for the whole of a bookstore just to find a book whose
>>>>>>>>> title comes from a local RDF file - ask the bookshop a query with
>>>>>>>>> the title already bound from earlier in the query.
>>>>>>>>> 
>>>>>>>>> ## Controlling `SERVICE` requests.
>>>>>>>>> 
>>>>>>>>> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
>>>>>>>>> -The values for configuration can be set in the global context (accessed via
>>>>>>>>> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via
>>>>>>>>> `ARQ.getContext()`) or in the per-query execution context.
>>>>>>>>> 
>>>>>>>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
>>>>>>>>> 
>>>>>>>>> -### Summary
>>>>>>>>> +### Configuration for ARQ through version 3.1.0
>>>>>>>>> 
>>>>>>>>> Symbol | Usage
>>>>>>>>> ------ | -----
>>>>>>>>> @@ -71,7 +70,7 @@ Symbol | Usage
>>>>>>>>> `srv:queryAuthPwd` |  Basic authentication
>>>>>>>>> `srv:queryContext` | Per-endpoint configuration
>>>>>>>>> 
>>>>>>>>> -### `srv:queryTimeout`
>>>>>>>>> +#### `srv:queryTimeout`
>>>>>>>>> 
>>>>>>>>> Set the connect and read timeouts for the query.
>>>>>>>>> 
>>>>>>>>> @@ -86,21 +85,21 @@ read timout = 0
>>>>>>>>> 
>>>>>>>>> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
>>>>>>>>> 
>>>>>>>>> -### `srv:queryGzip`
>>>>>>>>> +#### `srv:queryGzip`
>>>>>>>>> 
>>>>>>>>> Sets the allow Gzip flag.
>>>>>>>>> 
>>>>>>>>> Boolean: True indicates that gzip compressed data is acceptable.
>>>>>>>>> false
>>>>>>>>> 
>>>>>>>>> -### `srv:queryDeflate`
>>>>>>>>> +#### `srv:queryDeflate`
>>>>>>>>> 
>>>>>>>>> Sets the allow Deflate flag.
>>>>>>>>> 
>>>>>>>>> Boolean: True indicates that deflate compression is acceptable
>>>>>>>>> False
>>>>>>>>> 
>>>>>>>>> -### `srv:queryAuthUser`
>>>>>>>>> +#### `srv:queryAuthUser`
>>>>>>>>> 
>>>>>>>>> Sets the user id for basic auth.
>>>>>>>>> 
>>>>>>>>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>>>>>>>> 
>>>>>>>>> If null or null length no user id is sent.
>>>>>>>>> 
>>>>>>>>> -### `srv:queryAuthPwd`
>>>>>>>>> +#### `srv:queryAuthPwd`
>>>>>>>>> 
>>>>>>>>> Sets the password for basic auth.
>>>>>>>>> 
>>>>>>>>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>>>>>>>> 
>>>>>>>>> If null or null length no password is sent.
>>>>>>>>> 
>>>>>>>>> -### srv:serviceContext
>>>>>>>>> +#### `srv:serviceContext`
>>>>>>>>> Provides a mechanism to override system context settings on a per URI basis.
>>>>>>>>> 
>>>>>>>>> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
>>>>>>>>> 
>>>>>>>>> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
>>>>>>>>> 
>>>>>>>>> +### Configuration for ARQ after version 3.1.0
>>>>>>>>> 
>>>>>>>>> +Symbol | Usage | Default
>>>>>>>>> +------ | ----- | -------
>>>>>>>>> +`srv:queryTimeout` | Set timeouts | none
>>>>>>>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>>>>>>>>> +`srv:queryClient` | Enable use of a specific client | none
>>>>>>>>> +`srv:queryContext` | Per-endpoint configuration | none
>>>>>>>>> +
>>>>>>>>> +#### `srv:queryTimeout`
>>>>>>>>> +
>>>>>>>>> +As documented above.
>>>>>>>>> +
>>>>>>>>> +
>>>>>>>>> +#### `srv:queryCompression`
>>>>>>>>> +
>>>>>>>>> +Sets the flag for use of deflation and GZip.
>>>>>>>>> +
>>>>>>>>> +Boolean: True indicates that gzip compressed data is acceptable.
>>>>>>>>> +
>>>>>>>>> +#### `srv:queryClient`
>>>>>>>>> +
>>>>>>>>> +Enable use of a specific client
>>>>>>>>> +
>>>>>>>>> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
>>>>>>>>> +
>>>>>>>>> +#### `srv:serviceContext`
>>>>>>>>> +
>>>>>>>>> +As documented above.
>>>>>>>>> 
>>>>>>>>> [ARQ documentation index](index.html)
>>>>>>>>> +
>>>>>>>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Stian Soiland-Reyes
>>> http://orcid.org/0000-0001-9842-9718
>> 
> 
> 
> 
> -- 
> Stian Soiland-Reyes
> http://orcid.org/0000-0001-9842-9718

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by Stian Soiland-Reyes <st...@apache.org>.

Yeah, as the client is set static inside JSON-LD Java it might not
good for be Jena  HttpOps to be "helpful" and propagate its client -
at least not without a flag.  But we can link to it from our
documentation.

On 9 November 2016 at 16:39, A. Soroka <aj...@virginia.edu> wrote:
> Hm, that's a really good point. I don't understand what's happening in JSON-LD very well, but I think you must be correct to say that it wouldn't take up a supplied client, because it wouldn't be using state from Jena. I'll check this a bit in the code and add a note as you suggest.
>
> ---
> A. Soroka
> The University of Virginia Library
>
>> On Nov 9, 2016, at 11:37 AM, Stian Soiland-Reyes <st...@apache.org> wrote:
>>
>> I think the new text looks good, quite easy to understand.
>>
>> Could you add a paragraph about when the configured client would be
>> used?   It might not be clear when this HttpClient would be accessed
>> or not.  For instance I assume it would be used for remote SPARQL
>> queries or loading of HTTP URLs from RDFDataMgr -- but may not be
>> propagated through to JSON-LD Java's @context loading - which has a
>> similar httpclient setting and documentation on how to configure
>> caching [1]
>>
>> [1] https://github.com/jsonld-java/jsonld-java#controlling-network-traffic
>>
>>
>>
>>
>> On 9 November 2016 at 15:42, A. Soroka <aj...@virginia.edu> wrote:
>>> Done. I'll wait to hear from other folks before pulling a trigger on (re)publishing the site.
>>>
>>> ---
>>> A. Soroka
>>> The University of Virginia Library
>>>
>>>> On Nov 9, 2016, at 6:30 AM, Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>> Great -
>>>>
>>>> One (!) other thing:
>>>>
>>>> A section specifically calling out migrating SPARQL remote calls: QueryExecutionFactory.sparqlService and QueryEngineHTTP.
>>>>
>>>> On the latter, older code may still be directly using QueryEngineHTTP.setBasicAuthentication
>>>>
>>>>      Andy
>>>>
>>>> On 08/11/16 17:58, A. Soroka wrote:
>>>>> I've made those changes-- should be restaging now.
>>>>>
>>>>> ---
>>>>> A. Soroka
>>>>> The University of Virginia Library
>>>>>
>>>>>> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 08/11/16 16:59, A. Soroka wrote:
>>>>>>> This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?
>>>>>>>
>>>>>>
>>>>>> It does not for me.
>>>>>>
>>>>>> Try http://jena.staging.apache.org/  (not https)
>>>>>>
>>>>>> PDF attached, cc'ed to you in the hope it get through.
>>>>>>
>>>>>> Comments:
>>>>>>
>>>>>> 1/ I'd put the current (3.1.1) text first and the previous second so the current is more visible.
>>>>>>
>>>>>> Links at the end of the intro to "current" and "previous", or in the intro as this difference is mentioned.
>>>>>>
>>>>>> 2/ Title tweaking:
>>>>>>
>>>>>> "HTTP Authentication after Jena 3.1.0" ->
>>>>>> "HTTP Authentication from Jena 3.1.1"
>>>>>>
>>>>>> "HTTP Authentication before Jena 3.1.0" =>
>>>>>> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
>>>>>>
>>>>>> (so the range includes 3.1.0 !)
>>>>>>
>>>>>> Mentioning Jena 2.x is not necessary IMO - the additional detail adds confusion for current users and 3.x upgrading users (the majority).
>>>>>>
>>>>>> 3/
>>>>>> "Simple authentication using username and password"
>>>>>>
>>>>>> "Authenticating via a form"
>>>>>>
>>>>>> The <h5> don't show up as different on teh screen for me so maybe bump <h4> "Examples of authentication" up a level to <h3> and move sub <5> to <h4> .
>>>>>>
>>>>>> Maybe drop <h3> "Applying Authentication" (section title immediately after a section title) and have the paragraph there straight away.
>>>>>>
>>>>>>  Andy
>>>>>>
>>>>>>> ---
>>>>>>> A. Soroka
>>>>>>> The University of Virginia Library
>>>>>>>
>>>>>>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>>>>>>>
>>>>>>>> Author: ajs6f
>>>>>>>> Date: Tue Nov  8 16:53:48 2016
>>>>>>>> New Revision: 1768736
>>>>>>>>
>>>>>>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>>>>>>>> Log:
>>>>>>>> Updates for HTTP behavior in Jena 3.1.1
>>>>>>>>
>>>>>>>> Modified:
>>>>>>>> jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>>>> jena/site/trunk/content/documentation/query/service.mdtext
>>>>>>>>
>>>>>>>> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>>>>> ==============================================================================
>>>>>>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
>>>>>>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
>>>>>>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>>>>>>>         specific language governing permissions and limitations
>>>>>>>>         under the License.
>>>>>>>>
>>>>>>>> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for
>>>>>>>> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
>>>>>>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for
>>>>>>>> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
>>>>>>>>
>>>>>>>> -## Applying Authentication
>>>>>>>> +## HTTP Authentication before ARQ 3.1.0
>>>>>>>> +
>>>>>>>> +### Applying Authentication
>>>>>>>>
>>>>>>>> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
>>>>>>>> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>>>>>>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>>>>>>>      ...
>>>>>>>>  }
>>>>>>>>
>>>>>>>> -### Authenticators
>>>>>>>> +#### Authenticators
>>>>>>>>
>>>>>>>> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
>>>>>>>>
>>>>>>>> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried
>>>>>>>> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
>>>>>>>>
>>>>>>>> -#### SimpleAuthenticator
>>>>>>>> +##### SimpleAuthenticator
>>>>>>>>
>>>>>>>> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
>>>>>>>> any service.
>>>>>>>> @@ -56,7 +58,7 @@ any service.
>>>>>>>> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>>>>>>> Required) then credentials will not actually be submitted.
>>>>>>>>
>>>>>>>> -#### ScopedAuthenticator
>>>>>>>> +##### ScopedAuthenticator
>>>>>>>>
>>>>>>>> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different
>>>>>>>> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are
>>>>>>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>>>>>>>> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be
>>>>>>>> used in favor of those for `http://example.org`
>>>>>>>>
>>>>>>>> -#### ServiceAuthenticator
>>>>>>>> +##### ServiceAuthenticator
>>>>>>>>
>>>>>>>> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the
>>>>>>>> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on
>>>>>>>> configuration for this.
>>>>>>>>
>>>>>>>> -#### FormsAuthenticator
>>>>>>>> +##### FormsAuthenticator
>>>>>>>>
>>>>>>>> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
>>>>>>>> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this
>>>>>>>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>>>>>>>
>>>>>>>> Currently forms based login that require more than just a username and password are not supported.
>>>>>>>>
>>>>>>>> -#### PreemptiveBasicAuthenticator
>>>>>>>> +##### PreemptiveBasicAuthenticator
>>>>>>>>
>>>>>>>> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers
>>>>>>>> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
>>>>>>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>>>>>>>> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
>>>>>>>> challenge response sequences.
>>>>>>>>
>>>>>>>> -#### DelegatingAuthenticator
>>>>>>>> +##### DelegatingAuthenticator
>>>>>>>>
>>>>>>>> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and
>>>>>>>> match the types of authentication needed.
>>>>>>>>
>>>>>>>> -### Debugging Authentication
>>>>>>>> -
>>>>>>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>>>>> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>>>>> -
>>>>>>>> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>>>>> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>>>>> -
>>>>>>>> -### The Default Authenticator
>>>>>>>> +#### The Default Authenticator
>>>>>>>>
>>>>>>>> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default
>>>>>>>> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the
>>>>>>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>>>>>>>
>>>>>>>> Note that the default authenticator may be disabled by setting it to `null`.
>>>>>>>>
>>>>>>>> +## HTTP Authentication after ARQ 3.1.0
>>>>>>>> +
>>>>>>>> +### Applying Authentication
>>>>>>>> +
>>>>>>>> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the
>>>>>>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
>>>>>>>> +
>>>>>>>> +#### Examples of authentication
>>>>>>>> +
>>>>>>>> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
>>>>>>>> +
>>>>>>>> +##### Simple authentication using username and password
>>>>>>>> +
>>>>>>>> +First we build an authenticating client:
>>>>>>>> +
>>>>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>>>>> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>>>> +        .build();
>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>> +
>>>>>>>> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
>>>>>>>> +
>>>>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>>>>> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
>>>>>>>> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>>> +    final String host = "http://example.com/sparql";
>>>>>>>> +    final int port = 80;
>>>>>>>> +    final String realm = "aRealm";
>>>>>>>> +    final String schemeName = "DIGEST";
>>>>>>>> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
>>>>>>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>>>> +        .build();
>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>> +
>>>>>>>> +##### Authenticating via a form
>>>>>>>> +
>>>>>>>> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
>>>>>>>> +
>>>>>>>> +    // we'll use this context to maintain our HTTP "conversation"
>>>>>>>> +    HttpClientContext httpContext = new HttpClientContext();
>>>>>>>> +
>>>>>>>> +    // first we use a method on HttpOp to log in and get our cookie
>>>>>>>> +    Params params = new Params();
>>>>>>>> +    params.addParam("username", "Bob Wu");
>>>>>>>> +    params.addParam("password", "my password");
>>>>>>>> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
>>>>>>>> +
>>>>>>>> +    // now our cookie is stored in httpContext
>>>>>>>> +    CookieStore cookieStore = httpContext.getCookieStore();
>>>>>>>> +
>>>>>>>> +    // lastly we build a client that uses that cookie
>>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>>> +     .setDefaultCookieStore(cookieStore)
>>>>>>>> +         .build();
>>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>>> +
>>>>>>>> +## Other concerns
>>>>>>>> +
>>>>>>>> +### Debugging Authentication
>>>>>>>> +
>>>>>>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>>>>> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>>>>> +
>>>>>>>> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>>>>> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>>>>> +
>>>>>>>> +### Authenticating to a SPARQL federated service
>>>>>>>> +
>>>>>>>> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
>>>>>>>> +
>>>>>>>> [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>>>>>>>> [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>>>>>>> [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
>>>>>>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>>>>>>> [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>>>>>>>> [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>>>>>>> [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
>>>>>>>> -  [14]: http://hc.apache.org
>>>>>>>> \ No newline at end of file
>>>>>>>> +  [14]: http://hc.apache.org
>>>>>>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
>>>>>>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>>>>>>>> \ No newline at end of file
>>>>>>>>
>>>>>>>> Modified: jena/site/trunk/content/documentation/query/service.mdtext
>>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>>>>> ==============================================================================
>>>>>>>> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
>>>>>>>> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
>>>>>>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>>>>>>>> without regard to how selective the pattern is. So the order of the
>>>>>>>> query will affect the speed of execution. Because it involves HTTP
>>>>>>>> operations, asking the query in the right order matters a lot.
>>>>>>>> -Don't ask for the whole of a bookstore just to find book whose
>>>>>>>> +Don't ask for the whole of a bookstore just to find a book whose
>>>>>>>> title comes from a local RDF file - ask the bookshop a query with
>>>>>>>> the title already bound from earlier in the query.
>>>>>>>>
>>>>>>>> ## Controlling `SERVICE` requests.
>>>>>>>>
>>>>>>>> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
>>>>>>>> -The values for configuration can be set in the global context (accessed via
>>>>>>>> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via
>>>>>>>> `ARQ.getContext()`) or in the per-query execution context.
>>>>>>>>
>>>>>>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
>>>>>>>>
>>>>>>>> -### Summary
>>>>>>>> +### Configuration for ARQ through version 3.1.0
>>>>>>>>
>>>>>>>> Symbol | Usage
>>>>>>>> ------ | -----
>>>>>>>> @@ -71,7 +70,7 @@ Symbol | Usage
>>>>>>>> `srv:queryAuthPwd` |  Basic authentication
>>>>>>>> `srv:queryContext` | Per-endpoint configuration
>>>>>>>>
>>>>>>>> -### `srv:queryTimeout`
>>>>>>>> +#### `srv:queryTimeout`
>>>>>>>>
>>>>>>>> Set the connect and read timeouts for the query.
>>>>>>>>
>>>>>>>> @@ -86,21 +85,21 @@ read timout = 0
>>>>>>>>
>>>>>>>> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
>>>>>>>>
>>>>>>>> -### `srv:queryGzip`
>>>>>>>> +#### `srv:queryGzip`
>>>>>>>>
>>>>>>>> Sets the allow Gzip flag.
>>>>>>>>
>>>>>>>> Boolean: True indicates that gzip compressed data is acceptable.
>>>>>>>> false
>>>>>>>>
>>>>>>>> -### `srv:queryDeflate`
>>>>>>>> +#### `srv:queryDeflate`
>>>>>>>>
>>>>>>>> Sets the allow Deflate flag.
>>>>>>>>
>>>>>>>> Boolean: True indicates that deflate compression is acceptable
>>>>>>>> False
>>>>>>>>
>>>>>>>> -### `srv:queryAuthUser`
>>>>>>>> +#### `srv:queryAuthUser`
>>>>>>>>
>>>>>>>> Sets the user id for basic auth.
>>>>>>>>
>>>>>>>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>>>>>>>
>>>>>>>> If null or null length no user id is sent.
>>>>>>>>
>>>>>>>> -### `srv:queryAuthPwd`
>>>>>>>> +#### `srv:queryAuthPwd`
>>>>>>>>
>>>>>>>> Sets the password for basic auth.
>>>>>>>>
>>>>>>>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>>>>>>>
>>>>>>>> If null or null length no password is sent.
>>>>>>>>
>>>>>>>> -### srv:serviceContext
>>>>>>>> +#### `srv:serviceContext`
>>>>>>>> Provides a mechanism to override system context settings on a per URI basis.
>>>>>>>>
>>>>>>>> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
>>>>>>>>
>>>>>>>> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
>>>>>>>>
>>>>>>>> +### Configuration for ARQ after version 3.1.0
>>>>>>>>
>>>>>>>> +Symbol | Usage | Default
>>>>>>>> +------ | ----- | -------
>>>>>>>> +`srv:queryTimeout` | Set timeouts | none
>>>>>>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>>>>>>>> +`srv:queryClient` | Enable use of a specific client | none
>>>>>>>> +`srv:queryContext` | Per-endpoint configuration | none
>>>>>>>> +
>>>>>>>> +#### `srv:queryTimeout`
>>>>>>>> +
>>>>>>>> +As documented above.
>>>>>>>> +
>>>>>>>> +
>>>>>>>> +#### `srv:queryCompression`
>>>>>>>> +
>>>>>>>> +Sets the flag for use of deflation and GZip.
>>>>>>>> +
>>>>>>>> +Boolean: True indicates that gzip compressed data is acceptable.
>>>>>>>> +
>>>>>>>> +#### `srv:queryClient`
>>>>>>>> +
>>>>>>>> +Enable use of a specific client
>>>>>>>> +
>>>>>>>> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
>>>>>>>> +
>>>>>>>> +#### `srv:serviceContext`
>>>>>>>> +
>>>>>>>> +As documented above.
>>>>>>>>
>>>>>>>> [ARQ documentation index](index.html)
>>>>>>>> +
>>>>>>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>
>>
>>
>>
>> --
>> Stian Soiland-Reyes
>> http://orcid.org/0000-0001-9842-9718
>



-- 
Stian Soiland-Reyes
http://orcid.org/0000-0001-9842-9718

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by "A. Soroka" <aj...@virginia.edu>.

Hm, that's a really good point. I don't understand what's happening in JSON-LD very well, but I think you must be correct to say that it wouldn't take up a supplied client, because it wouldn't be using state from Jena. I'll check this a bit in the code and add a note as you suggest.

---
A. Soroka
The University of Virginia Library

> On Nov 9, 2016, at 11:37 AM, Stian Soiland-Reyes <st...@apache.org> wrote:
> 
> I think the new text looks good, quite easy to understand.
> 
> Could you add a paragraph about when the configured client would be
> used?   It might not be clear when this HttpClient would be accessed
> or not.  For instance I assume it would be used for remote SPARQL
> queries or loading of HTTP URLs from RDFDataMgr -- but may not be
> propagated through to JSON-LD Java's @context loading - which has a
> similar httpclient setting and documentation on how to configure
> caching [1]
> 
> [1] https://github.com/jsonld-java/jsonld-java#controlling-network-traffic
> 
> 
> 
> 
> On 9 November 2016 at 15:42, A. Soroka <aj...@virginia.edu> wrote:
>> Done. I'll wait to hear from other folks before pulling a trigger on (re)publishing the site.
>> 
>> ---
>> A. Soroka
>> The University of Virginia Library
>> 
>>> On Nov 9, 2016, at 6:30 AM, Andy Seaborne <an...@apache.org> wrote:
>>> 
>>> Great -
>>> 
>>> One (!) other thing:
>>> 
>>> A section specifically calling out migrating SPARQL remote calls: QueryExecutionFactory.sparqlService and QueryEngineHTTP.
>>> 
>>> On the latter, older code may still be directly using QueryEngineHTTP.setBasicAuthentication
>>> 
>>>      Andy
>>> 
>>> On 08/11/16 17:58, A. Soroka wrote:
>>>> I've made those changes-- should be restaging now.
>>>> 
>>>> ---
>>>> A. Soroka
>>>> The University of Virginia Library
>>>> 
>>>>> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>> On 08/11/16 16:59, A. Soroka wrote:
>>>>>> This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?
>>>>>> 
>>>>> 
>>>>> It does not for me.
>>>>> 
>>>>> Try http://jena.staging.apache.org/  (not https)
>>>>> 
>>>>> PDF attached, cc'ed to you in the hope it get through.
>>>>> 
>>>>> Comments:
>>>>> 
>>>>> 1/ I'd put the current (3.1.1) text first and the previous second so the current is more visible.
>>>>> 
>>>>> Links at the end of the intro to "current" and "previous", or in the intro as this difference is mentioned.
>>>>> 
>>>>> 2/ Title tweaking:
>>>>> 
>>>>> "HTTP Authentication after Jena 3.1.0" ->
>>>>> "HTTP Authentication from Jena 3.1.1"
>>>>> 
>>>>> "HTTP Authentication before Jena 3.1.0" =>
>>>>> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
>>>>> 
>>>>> (so the range includes 3.1.0 !)
>>>>> 
>>>>> Mentioning Jena 2.x is not necessary IMO - the additional detail adds confusion for current users and 3.x upgrading users (the majority).
>>>>> 
>>>>> 3/
>>>>> "Simple authentication using username and password"
>>>>> 
>>>>> "Authenticating via a form"
>>>>> 
>>>>> The <h5> don't show up as different on teh screen for me so maybe bump <h4> "Examples of authentication" up a level to <h3> and move sub <5> to <h4> .
>>>>> 
>>>>> Maybe drop <h3> "Applying Authentication" (section title immediately after a section title) and have the paragraph there straight away.
>>>>> 
>>>>>  Andy
>>>>> 
>>>>>> ---
>>>>>> A. Soroka
>>>>>> The University of Virginia Library
>>>>>> 
>>>>>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>>>>>> 
>>>>>>> Author: ajs6f
>>>>>>> Date: Tue Nov  8 16:53:48 2016
>>>>>>> New Revision: 1768736
>>>>>>> 
>>>>>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>>>>>>> Log:
>>>>>>> Updates for HTTP behavior in Jena 3.1.1
>>>>>>> 
>>>>>>> Modified:
>>>>>>> jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>>> jena/site/trunk/content/documentation/query/service.mdtext
>>>>>>> 
>>>>>>> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>>>> ==============================================================================
>>>>>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
>>>>>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
>>>>>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>>>>>>         specific language governing permissions and limitations
>>>>>>>         under the License.
>>>>>>> 
>>>>>>> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for
>>>>>>> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
>>>>>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for
>>>>>>> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
>>>>>>> 
>>>>>>> -## Applying Authentication
>>>>>>> +## HTTP Authentication before ARQ 3.1.0
>>>>>>> +
>>>>>>> +### Applying Authentication
>>>>>>> 
>>>>>>> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
>>>>>>> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>>>>>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>>>>>>      ...
>>>>>>>  }
>>>>>>> 
>>>>>>> -### Authenticators
>>>>>>> +#### Authenticators
>>>>>>> 
>>>>>>> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
>>>>>>> 
>>>>>>> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried
>>>>>>> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
>>>>>>> 
>>>>>>> -#### SimpleAuthenticator
>>>>>>> +##### SimpleAuthenticator
>>>>>>> 
>>>>>>> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
>>>>>>> any service.
>>>>>>> @@ -56,7 +58,7 @@ any service.
>>>>>>> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>>>>>> Required) then credentials will not actually be submitted.
>>>>>>> 
>>>>>>> -#### ScopedAuthenticator
>>>>>>> +##### ScopedAuthenticator
>>>>>>> 
>>>>>>> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different
>>>>>>> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are
>>>>>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>>>>>>> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be
>>>>>>> used in favor of those for `http://example.org`
>>>>>>> 
>>>>>>> -#### ServiceAuthenticator
>>>>>>> +##### ServiceAuthenticator
>>>>>>> 
>>>>>>> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the
>>>>>>> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on
>>>>>>> configuration for this.
>>>>>>> 
>>>>>>> -#### FormsAuthenticator
>>>>>>> +##### FormsAuthenticator
>>>>>>> 
>>>>>>> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
>>>>>>> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this
>>>>>>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>>>>>> 
>>>>>>> Currently forms based login that require more than just a username and password are not supported.
>>>>>>> 
>>>>>>> -#### PreemptiveBasicAuthenticator
>>>>>>> +##### PreemptiveBasicAuthenticator
>>>>>>> 
>>>>>>> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers
>>>>>>> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
>>>>>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>>>>>>> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
>>>>>>> challenge response sequences.
>>>>>>> 
>>>>>>> -#### DelegatingAuthenticator
>>>>>>> +##### DelegatingAuthenticator
>>>>>>> 
>>>>>>> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and
>>>>>>> match the types of authentication needed.
>>>>>>> 
>>>>>>> -### Debugging Authentication
>>>>>>> -
>>>>>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>>>> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>>>> -
>>>>>>> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>>>> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>>>> -
>>>>>>> -### The Default Authenticator
>>>>>>> +#### The Default Authenticator
>>>>>>> 
>>>>>>> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default
>>>>>>> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the
>>>>>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>>>>>> 
>>>>>>> Note that the default authenticator may be disabled by setting it to `null`.
>>>>>>> 
>>>>>>> +## HTTP Authentication after ARQ 3.1.0
>>>>>>> +
>>>>>>> +### Applying Authentication
>>>>>>> +
>>>>>>> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the
>>>>>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
>>>>>>> +
>>>>>>> +#### Examples of authentication
>>>>>>> +
>>>>>>> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
>>>>>>> +
>>>>>>> +##### Simple authentication using username and password
>>>>>>> +
>>>>>>> +First we build an authenticating client:
>>>>>>> +
>>>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>>>> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>>> +        .build();
>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>> +
>>>>>>> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
>>>>>>> +
>>>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>>>> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
>>>>>>> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>>> +    final String host = "http://example.com/sparql";
>>>>>>> +    final int port = 80;
>>>>>>> +    final String realm = "aRealm";
>>>>>>> +    final String schemeName = "DIGEST";
>>>>>>> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
>>>>>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>>> +        .build();
>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>> +
>>>>>>> +##### Authenticating via a form
>>>>>>> +
>>>>>>> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
>>>>>>> +
>>>>>>> +    // we'll use this context to maintain our HTTP "conversation"
>>>>>>> +    HttpClientContext httpContext = new HttpClientContext();
>>>>>>> +
>>>>>>> +    // first we use a method on HttpOp to log in and get our cookie
>>>>>>> +    Params params = new Params();
>>>>>>> +    params.addParam("username", "Bob Wu");
>>>>>>> +    params.addParam("password", "my password");
>>>>>>> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
>>>>>>> +
>>>>>>> +    // now our cookie is stored in httpContext
>>>>>>> +    CookieStore cookieStore = httpContext.getCookieStore();
>>>>>>> +
>>>>>>> +    // lastly we build a client that uses that cookie
>>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>>> +     .setDefaultCookieStore(cookieStore)
>>>>>>> +         .build();
>>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>>> +
>>>>>>> +## Other concerns
>>>>>>> +
>>>>>>> +### Debugging Authentication
>>>>>>> +
>>>>>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>>>> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>>>> +
>>>>>>> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>>>> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>>>> +
>>>>>>> +### Authenticating to a SPARQL federated service
>>>>>>> +
>>>>>>> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
>>>>>>> +
>>>>>>> [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>>>>>>> [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>>>>>> [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
>>>>>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>>>>>> [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>>>>>>> [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>>>>>> [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
>>>>>>> -  [14]: http://hc.apache.org
>>>>>>> \ No newline at end of file
>>>>>>> +  [14]: http://hc.apache.org
>>>>>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
>>>>>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>>>>>>> \ No newline at end of file
>>>>>>> 
>>>>>>> Modified: jena/site/trunk/content/documentation/query/service.mdtext
>>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>>>> ==============================================================================
>>>>>>> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
>>>>>>> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
>>>>>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>>>>>>> without regard to how selective the pattern is. So the order of the
>>>>>>> query will affect the speed of execution. Because it involves HTTP
>>>>>>> operations, asking the query in the right order matters a lot.
>>>>>>> -Don't ask for the whole of a bookstore just to find book whose
>>>>>>> +Don't ask for the whole of a bookstore just to find a book whose
>>>>>>> title comes from a local RDF file - ask the bookshop a query with
>>>>>>> the title already bound from earlier in the query.
>>>>>>> 
>>>>>>> ## Controlling `SERVICE` requests.
>>>>>>> 
>>>>>>> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
>>>>>>> -The values for configuration can be set in the global context (accessed via
>>>>>>> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via
>>>>>>> `ARQ.getContext()`) or in the per-query execution context.
>>>>>>> 
>>>>>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
>>>>>>> 
>>>>>>> -### Summary
>>>>>>> +### Configuration for ARQ through version 3.1.0
>>>>>>> 
>>>>>>> Symbol | Usage
>>>>>>> ------ | -----
>>>>>>> @@ -71,7 +70,7 @@ Symbol | Usage
>>>>>>> `srv:queryAuthPwd` |  Basic authentication
>>>>>>> `srv:queryContext` | Per-endpoint configuration
>>>>>>> 
>>>>>>> -### `srv:queryTimeout`
>>>>>>> +#### `srv:queryTimeout`
>>>>>>> 
>>>>>>> Set the connect and read timeouts for the query.
>>>>>>> 
>>>>>>> @@ -86,21 +85,21 @@ read timout = 0
>>>>>>> 
>>>>>>> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
>>>>>>> 
>>>>>>> -### `srv:queryGzip`
>>>>>>> +#### `srv:queryGzip`
>>>>>>> 
>>>>>>> Sets the allow Gzip flag.
>>>>>>> 
>>>>>>> Boolean: True indicates that gzip compressed data is acceptable.
>>>>>>> false
>>>>>>> 
>>>>>>> -### `srv:queryDeflate`
>>>>>>> +#### `srv:queryDeflate`
>>>>>>> 
>>>>>>> Sets the allow Deflate flag.
>>>>>>> 
>>>>>>> Boolean: True indicates that deflate compression is acceptable
>>>>>>> False
>>>>>>> 
>>>>>>> -### `srv:queryAuthUser`
>>>>>>> +#### `srv:queryAuthUser`
>>>>>>> 
>>>>>>> Sets the user id for basic auth.
>>>>>>> 
>>>>>>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>>>>>> 
>>>>>>> If null or null length no user id is sent.
>>>>>>> 
>>>>>>> -### `srv:queryAuthPwd`
>>>>>>> +#### `srv:queryAuthPwd`
>>>>>>> 
>>>>>>> Sets the password for basic auth.
>>>>>>> 
>>>>>>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>>>>>> 
>>>>>>> If null or null length no password is sent.
>>>>>>> 
>>>>>>> -### srv:serviceContext
>>>>>>> +#### `srv:serviceContext`
>>>>>>> Provides a mechanism to override system context settings on a per URI basis.
>>>>>>> 
>>>>>>> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
>>>>>>> 
>>>>>>> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
>>>>>>> 
>>>>>>> +### Configuration for ARQ after version 3.1.0
>>>>>>> 
>>>>>>> +Symbol | Usage | Default
>>>>>>> +------ | ----- | -------
>>>>>>> +`srv:queryTimeout` | Set timeouts | none
>>>>>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>>>>>>> +`srv:queryClient` | Enable use of a specific client | none
>>>>>>> +`srv:queryContext` | Per-endpoint configuration | none
>>>>>>> +
>>>>>>> +#### `srv:queryTimeout`
>>>>>>> +
>>>>>>> +As documented above.
>>>>>>> +
>>>>>>> +
>>>>>>> +#### `srv:queryCompression`
>>>>>>> +
>>>>>>> +Sets the flag for use of deflation and GZip.
>>>>>>> +
>>>>>>> +Boolean: True indicates that gzip compressed data is acceptable.
>>>>>>> +
>>>>>>> +#### `srv:queryClient`
>>>>>>> +
>>>>>>> +Enable use of a specific client
>>>>>>> +
>>>>>>> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
>>>>>>> +
>>>>>>> +#### `srv:serviceContext`
>>>>>>> +
>>>>>>> +As documented above.
>>>>>>> 
>>>>>>> [ARQ documentation index](index.html)
>>>>>>> +
>>>>>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>> 
> 
> 
> 
> -- 
> Stian Soiland-Reyes
> http://orcid.org/0000-0001-9842-9718

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by Stian Soiland-Reyes <st...@apache.org>.

I think the new text looks good, quite easy to understand.

Could you add a paragraph about when the configured client would be
used?   It might not be clear when this HttpClient would be accessed
or not.  For instance I assume it would be used for remote SPARQL
queries or loading of HTTP URLs from RDFDataMgr -- but may not be
propagated through to JSON-LD Java's @context loading - which has a
similar httpclient setting and documentation on how to configure
caching [1]

[1] https://github.com/jsonld-java/jsonld-java#controlling-network-traffic




On 9 November 2016 at 15:42, A. Soroka <aj...@virginia.edu> wrote:
> Done. I'll wait to hear from other folks before pulling a trigger on (re)publishing the site.
>
> ---
> A. Soroka
> The University of Virginia Library
>
>> On Nov 9, 2016, at 6:30 AM, Andy Seaborne <an...@apache.org> wrote:
>>
>> Great -
>>
>> One (!) other thing:
>>
>> A section specifically calling out migrating SPARQL remote calls: QueryExecutionFactory.sparqlService and QueryEngineHTTP.
>>
>> On the latter, older code may still be directly using QueryEngineHTTP.setBasicAuthentication
>>
>>       Andy
>>
>> On 08/11/16 17:58, A. Soroka wrote:
>>> I've made those changes-- should be restaging now.
>>>
>>> ---
>>> A. Soroka
>>> The University of Virginia Library
>>>
>>>> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>>
>>>>
>>>> On 08/11/16 16:59, A. Soroka wrote:
>>>>> This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?
>>>>>
>>>>
>>>> It does not for me.
>>>>
>>>> Try http://jena.staging.apache.org/  (not https)
>>>>
>>>> PDF attached, cc'ed to you in the hope it get through.
>>>>
>>>> Comments:
>>>>
>>>> 1/ I'd put the current (3.1.1) text first and the previous second so the current is more visible.
>>>>
>>>> Links at the end of the intro to "current" and "previous", or in the intro as this difference is mentioned.
>>>>
>>>> 2/ Title tweaking:
>>>>
>>>> "HTTP Authentication after Jena 3.1.0" ->
>>>> "HTTP Authentication from Jena 3.1.1"
>>>>
>>>> "HTTP Authentication before Jena 3.1.0" =>
>>>> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
>>>>
>>>> (so the range includes 3.1.0 !)
>>>>
>>>> Mentioning Jena 2.x is not necessary IMO - the additional detail adds confusion for current users and 3.x upgrading users (the majority).
>>>>
>>>> 3/
>>>> "Simple authentication using username and password"
>>>>
>>>> "Authenticating via a form"
>>>>
>>>> The <h5> don't show up as different on teh screen for me so maybe bump <h4> "Examples of authentication" up a level to <h3> and move sub <5> to <h4> .
>>>>
>>>> Maybe drop <h3> "Applying Authentication" (section title immediately after a section title) and have the paragraph there straight away.
>>>>
>>>>   Andy
>>>>
>>>>> ---
>>>>> A. Soroka
>>>>> The University of Virginia Library
>>>>>
>>>>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>>>>>
>>>>>> Author: ajs6f
>>>>>> Date: Tue Nov  8 16:53:48 2016
>>>>>> New Revision: 1768736
>>>>>>
>>>>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>>>>>> Log:
>>>>>> Updates for HTTP behavior in Jena 3.1.1
>>>>>>
>>>>>> Modified:
>>>>>>  jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>>  jena/site/trunk/content/documentation/query/service.mdtext
>>>>>>
>>>>>> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>>> ==============================================================================
>>>>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
>>>>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
>>>>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>>>>>          specific language governing permissions and limitations
>>>>>>          under the License.
>>>>>>
>>>>>> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for
>>>>>> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
>>>>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for
>>>>>> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
>>>>>>
>>>>>> -## Applying Authentication
>>>>>> +## HTTP Authentication before ARQ 3.1.0
>>>>>> +
>>>>>> +### Applying Authentication
>>>>>>
>>>>>> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
>>>>>> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>>>>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>>>>>       ...
>>>>>>   }
>>>>>>
>>>>>> -### Authenticators
>>>>>> +#### Authenticators
>>>>>>
>>>>>> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
>>>>>>
>>>>>> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried
>>>>>> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
>>>>>>
>>>>>> -#### SimpleAuthenticator
>>>>>> +##### SimpleAuthenticator
>>>>>>
>>>>>> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
>>>>>> any service.
>>>>>> @@ -56,7 +58,7 @@ any service.
>>>>>> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>>>>> Required) then credentials will not actually be submitted.
>>>>>>
>>>>>> -#### ScopedAuthenticator
>>>>>> +##### ScopedAuthenticator
>>>>>>
>>>>>> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different
>>>>>> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are
>>>>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>>>>>> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be
>>>>>> used in favor of those for `http://example.org`
>>>>>>
>>>>>> -#### ServiceAuthenticator
>>>>>> +##### ServiceAuthenticator
>>>>>>
>>>>>> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the
>>>>>> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on
>>>>>> configuration for this.
>>>>>>
>>>>>> -#### FormsAuthenticator
>>>>>> +##### FormsAuthenticator
>>>>>>
>>>>>> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
>>>>>> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this
>>>>>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>>>>>
>>>>>> Currently forms based login that require more than just a username and password are not supported.
>>>>>>
>>>>>> -#### PreemptiveBasicAuthenticator
>>>>>> +##### PreemptiveBasicAuthenticator
>>>>>>
>>>>>> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers
>>>>>> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
>>>>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>>>>>> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
>>>>>> challenge response sequences.
>>>>>>
>>>>>> -#### DelegatingAuthenticator
>>>>>> +##### DelegatingAuthenticator
>>>>>>
>>>>>> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and
>>>>>> match the types of authentication needed.
>>>>>>
>>>>>> -### Debugging Authentication
>>>>>> -
>>>>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>>> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>>> -
>>>>>> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>>> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>>> -
>>>>>> -### The Default Authenticator
>>>>>> +#### The Default Authenticator
>>>>>>
>>>>>> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default
>>>>>> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the
>>>>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>>>>>
>>>>>> Note that the default authenticator may be disabled by setting it to `null`.
>>>>>>
>>>>>> +## HTTP Authentication after ARQ 3.1.0
>>>>>> +
>>>>>> +### Applying Authentication
>>>>>> +
>>>>>> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the
>>>>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
>>>>>> +
>>>>>> +#### Examples of authentication
>>>>>> +
>>>>>> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
>>>>>> +
>>>>>> +##### Simple authentication using username and password
>>>>>> +
>>>>>> +First we build an authenticating client:
>>>>>> +
>>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>>> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>> +        .build();
>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>> +
>>>>>> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
>>>>>> +
>>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>>> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
>>>>>> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>>> +    final String host = "http://example.com/sparql";
>>>>>> +    final int port = 80;
>>>>>> +    final String realm = "aRealm";
>>>>>> +    final String schemeName = "DIGEST";
>>>>>> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
>>>>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>>> +        .build();
>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>> +
>>>>>> +##### Authenticating via a form
>>>>>> +
>>>>>> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
>>>>>> +
>>>>>> +    // we'll use this context to maintain our HTTP "conversation"
>>>>>> +    HttpClientContext httpContext = new HttpClientContext();
>>>>>> +
>>>>>> +    // first we use a method on HttpOp to log in and get our cookie
>>>>>> +    Params params = new Params();
>>>>>> +    params.addParam("username", "Bob Wu");
>>>>>> +    params.addParam("password", "my password");
>>>>>> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
>>>>>> +
>>>>>> +    // now our cookie is stored in httpContext
>>>>>> +    CookieStore cookieStore = httpContext.getCookieStore();
>>>>>> +
>>>>>> +    // lastly we build a client that uses that cookie
>>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>>> +     .setDefaultCookieStore(cookieStore)
>>>>>> +         .build();
>>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>>> +
>>>>>> +## Other concerns
>>>>>> +
>>>>>> +### Debugging Authentication
>>>>>> +
>>>>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>>> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>>> +
>>>>>> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>>> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>>> +
>>>>>> +### Authenticating to a SPARQL federated service
>>>>>> +
>>>>>> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
>>>>>> +
>>>>>> [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>>>>>> [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>>>>> [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
>>>>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>>>>> [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>>>>>> [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>>>>> [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
>>>>>> -  [14]: http://hc.apache.org
>>>>>> \ No newline at end of file
>>>>>> +  [14]: http://hc.apache.org
>>>>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
>>>>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>>>>>> \ No newline at end of file
>>>>>>
>>>>>> Modified: jena/site/trunk/content/documentation/query/service.mdtext
>>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>>> ==============================================================================
>>>>>> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
>>>>>> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
>>>>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>>>>>> without regard to how selective the pattern is. So the order of the
>>>>>> query will affect the speed of execution. Because it involves HTTP
>>>>>> operations, asking the query in the right order matters a lot.
>>>>>> -Don't ask for the whole of a bookstore just to find book whose
>>>>>> +Don't ask for the whole of a bookstore just to find a book whose
>>>>>> title comes from a local RDF file - ask the bookshop a query with
>>>>>> the title already bound from earlier in the query.
>>>>>>
>>>>>> ## Controlling `SERVICE` requests.
>>>>>>
>>>>>> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
>>>>>> -The values for configuration can be set in the global context (accessed via
>>>>>> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via
>>>>>> `ARQ.getContext()`) or in the per-query execution context.
>>>>>>
>>>>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
>>>>>>
>>>>>> -### Summary
>>>>>> +### Configuration for ARQ through version 3.1.0
>>>>>>
>>>>>> Symbol | Usage
>>>>>> ------ | -----
>>>>>> @@ -71,7 +70,7 @@ Symbol | Usage
>>>>>> `srv:queryAuthPwd` |  Basic authentication
>>>>>> `srv:queryContext` | Per-endpoint configuration
>>>>>>
>>>>>> -### `srv:queryTimeout`
>>>>>> +#### `srv:queryTimeout`
>>>>>>
>>>>>> Set the connect and read timeouts for the query.
>>>>>>
>>>>>> @@ -86,21 +85,21 @@ read timout = 0
>>>>>>
>>>>>> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
>>>>>>
>>>>>> -### `srv:queryGzip`
>>>>>> +#### `srv:queryGzip`
>>>>>>
>>>>>> Sets the allow Gzip flag.
>>>>>>
>>>>>> Boolean: True indicates that gzip compressed data is acceptable.
>>>>>> false
>>>>>>
>>>>>> -### `srv:queryDeflate`
>>>>>> +#### `srv:queryDeflate`
>>>>>>
>>>>>> Sets the allow Deflate flag.
>>>>>>
>>>>>> Boolean: True indicates that deflate compression is acceptable
>>>>>> False
>>>>>>
>>>>>> -### `srv:queryAuthUser`
>>>>>> +#### `srv:queryAuthUser`
>>>>>>
>>>>>> Sets the user id for basic auth.
>>>>>>
>>>>>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>>>>>
>>>>>> If null or null length no user id is sent.
>>>>>>
>>>>>> -### `srv:queryAuthPwd`
>>>>>> +#### `srv:queryAuthPwd`
>>>>>>
>>>>>> Sets the password for basic auth.
>>>>>>
>>>>>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>>>>>
>>>>>> If null or null length no password is sent.
>>>>>>
>>>>>> -### srv:serviceContext
>>>>>> +#### `srv:serviceContext`
>>>>>> Provides a mechanism to override system context settings on a per URI basis.
>>>>>>
>>>>>> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
>>>>>>
>>>>>> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
>>>>>>
>>>>>> +### Configuration for ARQ after version 3.1.0
>>>>>>
>>>>>> +Symbol | Usage | Default
>>>>>> +------ | ----- | -------
>>>>>> +`srv:queryTimeout` | Set timeouts | none
>>>>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>>>>>> +`srv:queryClient` | Enable use of a specific client | none
>>>>>> +`srv:queryContext` | Per-endpoint configuration | none
>>>>>> +
>>>>>> +#### `srv:queryTimeout`
>>>>>> +
>>>>>> +As documented above.
>>>>>> +
>>>>>> +
>>>>>> +#### `srv:queryCompression`
>>>>>> +
>>>>>> +Sets the flag for use of deflation and GZip.
>>>>>> +
>>>>>> +Boolean: True indicates that gzip compressed data is acceptable.
>>>>>> +
>>>>>> +#### `srv:queryClient`
>>>>>> +
>>>>>> +Enable use of a specific client
>>>>>> +
>>>>>> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
>>>>>> +
>>>>>> +#### `srv:serviceContext`
>>>>>> +
>>>>>> +As documented above.
>>>>>>
>>>>>> [ARQ documentation index](index.html)
>>>>>> +
>>>>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>>>
>>>>>>
>>>>>
>>>
>



-- 
Stian Soiland-Reyes
http://orcid.org/0000-0001-9842-9718

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by "A. Soroka" <aj...@virginia.edu>.

Done. I'll wait to hear from other folks before pulling a trigger on (re)publishing the site.

---
A. Soroka
The University of Virginia Library

> On Nov 9, 2016, at 6:30 AM, Andy Seaborne <an...@apache.org> wrote:
> 
> Great -
> 
> One (!) other thing:
> 
> A section specifically calling out migrating SPARQL remote calls: QueryExecutionFactory.sparqlService and QueryEngineHTTP.
> 
> On the latter, older code may still be directly using QueryEngineHTTP.setBasicAuthentication
> 
> 	Andy
> 
> On 08/11/16 17:58, A. Soroka wrote:
>> I've made those changes-- should be restaging now.
>> 
>> ---
>> A. Soroka
>> The University of Virginia Library
>> 
>>> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org> wrote:
>>> 
>>> 
>>> 
>>> On 08/11/16 16:59, A. Soroka wrote:
>>>> This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?
>>>> 
>>> 
>>> It does not for me.
>>> 
>>> Try http://jena.staging.apache.org/  (not https)
>>> 
>>> PDF attached, cc'ed to you in the hope it get through.
>>> 
>>> Comments:
>>> 
>>> 1/ I'd put the current (3.1.1) text first and the previous second so the current is more visible.
>>> 
>>> Links at the end of the intro to "current" and "previous", or in the intro as this difference is mentioned.
>>> 
>>> 2/ Title tweaking:
>>> 
>>> "HTTP Authentication after Jena 3.1.0" ->
>>> "HTTP Authentication from Jena 3.1.1"
>>> 
>>> "HTTP Authentication before Jena 3.1.0" =>
>>> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
>>> 
>>> (so the range includes 3.1.0 !)
>>> 
>>> Mentioning Jena 2.x is not necessary IMO - the additional detail adds confusion for current users and 3.x upgrading users (the majority).
>>> 
>>> 3/
>>> "Simple authentication using username and password"
>>> 
>>> "Authenticating via a form"
>>> 
>>> The <h5> don't show up as different on teh screen for me so maybe bump <h4> "Examples of authentication" up a level to <h3> and move sub <5> to <h4> .
>>> 
>>> Maybe drop <h3> "Applying Authentication" (section title immediately after a section title) and have the paragraph there straight away.
>>> 
>>>   Andy
>>> 
>>>> ---
>>>> A. Soroka
>>>> The University of Virginia Library
>>>> 
>>>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>>>> 
>>>>> Author: ajs6f
>>>>> Date: Tue Nov  8 16:53:48 2016
>>>>> New Revision: 1768736
>>>>> 
>>>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>>>>> Log:
>>>>> Updates for HTTP behavior in Jena 3.1.1
>>>>> 
>>>>> Modified:
>>>>>  jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>>  jena/site/trunk/content/documentation/query/service.mdtext
>>>>> 
>>>>> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>> ==============================================================================
>>>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
>>>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
>>>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>>>>          specific language governing permissions and limitations
>>>>>          under the License.
>>>>> 
>>>>> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for
>>>>> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
>>>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for
>>>>> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
>>>>> 
>>>>> -## Applying Authentication
>>>>> +## HTTP Authentication before ARQ 3.1.0
>>>>> +
>>>>> +### Applying Authentication
>>>>> 
>>>>> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
>>>>> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>>>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>>>>       ...
>>>>>   }
>>>>> 
>>>>> -### Authenticators
>>>>> +#### Authenticators
>>>>> 
>>>>> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
>>>>> 
>>>>> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried
>>>>> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
>>>>> 
>>>>> -#### SimpleAuthenticator
>>>>> +##### SimpleAuthenticator
>>>>> 
>>>>> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
>>>>> any service.
>>>>> @@ -56,7 +58,7 @@ any service.
>>>>> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>>>> Required) then credentials will not actually be submitted.
>>>>> 
>>>>> -#### ScopedAuthenticator
>>>>> +##### ScopedAuthenticator
>>>>> 
>>>>> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different
>>>>> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are
>>>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>>>>> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be
>>>>> used in favor of those for `http://example.org`
>>>>> 
>>>>> -#### ServiceAuthenticator
>>>>> +##### ServiceAuthenticator
>>>>> 
>>>>> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the
>>>>> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on
>>>>> configuration for this.
>>>>> 
>>>>> -#### FormsAuthenticator
>>>>> +##### FormsAuthenticator
>>>>> 
>>>>> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
>>>>> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this
>>>>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>>>> 
>>>>> Currently forms based login that require more than just a username and password are not supported.
>>>>> 
>>>>> -#### PreemptiveBasicAuthenticator
>>>>> +##### PreemptiveBasicAuthenticator
>>>>> 
>>>>> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers
>>>>> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
>>>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>>>>> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
>>>>> challenge response sequences.
>>>>> 
>>>>> -#### DelegatingAuthenticator
>>>>> +##### DelegatingAuthenticator
>>>>> 
>>>>> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and
>>>>> match the types of authentication needed.
>>>>> 
>>>>> -### Debugging Authentication
>>>>> -
>>>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>> -
>>>>> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>> -
>>>>> -### The Default Authenticator
>>>>> +#### The Default Authenticator
>>>>> 
>>>>> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default
>>>>> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the
>>>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>>>> 
>>>>> Note that the default authenticator may be disabled by setting it to `null`.
>>>>> 
>>>>> +## HTTP Authentication after ARQ 3.1.0
>>>>> +
>>>>> +### Applying Authentication
>>>>> +
>>>>> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the
>>>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
>>>>> +
>>>>> +#### Examples of authentication
>>>>> +
>>>>> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
>>>>> +
>>>>> +##### Simple authentication using username and password
>>>>> +
>>>>> +First we build an authenticating client:
>>>>> +
>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
>>>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>> +        .build();
>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>> +
>>>>> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
>>>>> +
>>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>>> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
>>>>> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>>> +    final String host = "http://example.com/sparql";
>>>>> +    final int port = 80;
>>>>> +    final String realm = "aRealm";
>>>>> +    final String schemeName = "DIGEST";
>>>>> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
>>>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>>> +        .build();
>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>> +	
>>>>> +##### Authenticating via a form
>>>>> +
>>>>> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
>>>>> +
>>>>> +    // we'll use this context to maintain our HTTP "conversation"
>>>>> +    HttpClientContext httpContext = new HttpClientContext();
>>>>> +
>>>>> +    // first we use a method on HttpOp to log in and get our cookie
>>>>> +    Params params = new Params();
>>>>> +    params.addParam("username", "Bob Wu");
>>>>> +    params.addParam("password", "my password");
>>>>> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
>>>>> +
>>>>> +    // now our cookie is stored in httpContext
>>>>> +    CookieStore cookieStore = httpContext.getCookieStore();
>>>>> +
>>>>> +    // lastly we build a client that uses that cookie
>>>>> +    HttpClient httpclient = HttpClients.custom()
>>>>> +	    .setDefaultCookieStore(cookieStore)
>>>>> +		.build();
>>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>>> +
>>>>> +## Other concerns
>>>>> +
>>>>> +### Debugging Authentication
>>>>> +
>>>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>>> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>>> +
>>>>> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>>> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>>> +
>>>>> +### Authenticating to a SPARQL federated service
>>>>> +
>>>>> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
>>>>> +
>>>>> [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>>>>> [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>>>> [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
>>>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>>>> [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>>>>> [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>>>> [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
>>>>> -  [14]: http://hc.apache.org
>>>>> \ No newline at end of file
>>>>> +  [14]: http://hc.apache.org
>>>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
>>>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>>>>> \ No newline at end of file
>>>>> 
>>>>> Modified: jena/site/trunk/content/documentation/query/service.mdtext
>>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>>> ==============================================================================
>>>>> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
>>>>> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
>>>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>>>>> without regard to how selective the pattern is. So the order of the
>>>>> query will affect the speed of execution. Because it involves HTTP
>>>>> operations, asking the query in the right order matters a lot.
>>>>> -Don't ask for the whole of a bookstore just to find book whose
>>>>> +Don't ask for the whole of a bookstore just to find a book whose
>>>>> title comes from a local RDF file - ask the bookshop a query with
>>>>> the title already bound from earlier in the query.
>>>>> 
>>>>> ## Controlling `SERVICE` requests.
>>>>> 
>>>>> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
>>>>> -The values for configuration can be set in the global context (accessed via
>>>>> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via
>>>>> `ARQ.getContext()`) or in the per-query execution context.
>>>>> 
>>>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
>>>>> 
>>>>> -### Summary
>>>>> +### Configuration for ARQ through version 3.1.0
>>>>> 
>>>>> Symbol | Usage
>>>>> ------ | -----
>>>>> @@ -71,7 +70,7 @@ Symbol | Usage
>>>>> `srv:queryAuthPwd` |  Basic authentication
>>>>> `srv:queryContext` | Per-endpoint configuration
>>>>> 
>>>>> -### `srv:queryTimeout`
>>>>> +#### `srv:queryTimeout`
>>>>> 
>>>>> Set the connect and read timeouts for the query.
>>>>> 
>>>>> @@ -86,21 +85,21 @@ read timout = 0
>>>>> 
>>>>> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
>>>>> 
>>>>> -### `srv:queryGzip`
>>>>> +#### `srv:queryGzip`
>>>>> 
>>>>> Sets the allow Gzip flag.
>>>>> 
>>>>> Boolean: True indicates that gzip compressed data is acceptable.
>>>>> false
>>>>> 
>>>>> -### `srv:queryDeflate`
>>>>> +#### `srv:queryDeflate`
>>>>> 
>>>>> Sets the allow Deflate flag.
>>>>> 
>>>>> Boolean: True indicates that deflate compression is acceptable
>>>>> False
>>>>> 
>>>>> -### `srv:queryAuthUser`
>>>>> +#### `srv:queryAuthUser`
>>>>> 
>>>>> Sets the user id for basic auth.
>>>>> 
>>>>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>>>> 
>>>>> If null or null length no user id is sent.
>>>>> 
>>>>> -### `srv:queryAuthPwd`
>>>>> +#### `srv:queryAuthPwd`
>>>>> 
>>>>> Sets the password for basic auth.
>>>>> 
>>>>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>>>> 
>>>>> If null or null length no password is sent.
>>>>> 
>>>>> -### srv:serviceContext
>>>>> +#### `srv:serviceContext`
>>>>> Provides a mechanism to override system context settings on a per URI basis.
>>>>> 
>>>>> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
>>>>> 
>>>>> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
>>>>> 
>>>>> +### Configuration for ARQ after version 3.1.0
>>>>> 
>>>>> +Symbol | Usage | Default
>>>>> +------ | ----- | -------
>>>>> +`srv:queryTimeout` | Set timeouts | none
>>>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>>>>> +`srv:queryClient` | Enable use of a specific client | none
>>>>> +`srv:queryContext` | Per-endpoint configuration | none
>>>>> +
>>>>> +#### `srv:queryTimeout`
>>>>> +
>>>>> +As documented above.
>>>>> +
>>>>> +
>>>>> +#### `srv:queryCompression`
>>>>> +
>>>>> +Sets the flag for use of deflation and GZip.
>>>>> +
>>>>> +Boolean: True indicates that gzip compressed data is acceptable.
>>>>> +
>>>>> +#### `srv:queryClient`
>>>>> +
>>>>> +Enable use of a specific client
>>>>> +
>>>>> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
>>>>> +
>>>>> +#### `srv:serviceContext`
>>>>> +
>>>>> +As documented above.
>>>>> 
>>>>> [ARQ documentation index](index.html)
>>>>> +
>>>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>> 
>>>>> 
>>>> 
>>

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by Andy Seaborne <an...@apache.org>.

Great -

One (!) other thing:

A section specifically calling out migrating SPARQL remote calls: 
QueryExecutionFactory.sparqlService and QueryEngineHTTP.

On the latter, older code may still be directly using 
QueryEngineHTTP.setBasicAuthentication

	Andy

On 08/11/16 17:58, A. Soroka wrote:
> I've made those changes-- should be restaging now.
>
> ---
> A. Soroka
> The University of Virginia Library
>
>> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org> wrote:
>>
>>
>>
>> On 08/11/16 16:59, A. Soroka wrote:
>>> This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?
>>>
>>
>> It does not for me.
>>
>> Try http://jena.staging.apache.org/  (not https)
>>
>> PDF attached, cc'ed to you in the hope it get through.
>>
>> Comments:
>>
>> 1/ I'd put the current (3.1.1) text first and the previous second so the current is more visible.
>>
>> Links at the end of the intro to "current" and "previous", or in the intro as this difference is mentioned.
>>
>> 2/ Title tweaking:
>>
>> "HTTP Authentication after Jena 3.1.0" ->
>> "HTTP Authentication from Jena 3.1.1"
>>
>> "HTTP Authentication before Jena 3.1.0" =>
>> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
>>
>> (so the range includes 3.1.0 !)
>>
>> Mentioning Jena 2.x is not necessary IMO - the additional detail adds confusion for current users and 3.x upgrading users (the majority).
>>
>> 3/
>> "Simple authentication using username and password"
>>
>> "Authenticating via a form"
>>
>> The <h5> don't show up as different on teh screen for me so maybe bump <h4> "Examples of authentication" up a level to <h3> and move sub <5> to <h4> .
>>
>> Maybe drop <h3> "Applying Authentication" (section title immediately after a section title) and have the paragraph there straight away.
>>
>>    Andy
>>
>>> ---
>>> A. Soroka
>>> The University of Virginia Library
>>>
>>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>>>
>>>> Author: ajs6f
>>>> Date: Tue Nov  8 16:53:48 2016
>>>> New Revision: 1768736
>>>>
>>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>>>> Log:
>>>> Updates for HTTP behavior in Jena 3.1.1
>>>>
>>>> Modified:
>>>>   jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>>   jena/site/trunk/content/documentation/query/service.mdtext
>>>>
>>>> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>> ==============================================================================
>>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
>>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
>>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>>>           specific language governing permissions and limitations
>>>>           under the License.
>>>>
>>>> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for
>>>> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
>>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for
>>>> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
>>>>
>>>> -## Applying Authentication
>>>> +## HTTP Authentication before ARQ 3.1.0
>>>> +
>>>> +### Applying Authentication
>>>>
>>>> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
>>>> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>>>        ...
>>>>    }
>>>>
>>>> -### Authenticators
>>>> +#### Authenticators
>>>>
>>>> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
>>>>
>>>> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried
>>>> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
>>>>
>>>> -#### SimpleAuthenticator
>>>> +##### SimpleAuthenticator
>>>>
>>>> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
>>>> any service.
>>>> @@ -56,7 +58,7 @@ any service.
>>>> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>>> Required) then credentials will not actually be submitted.
>>>>
>>>> -#### ScopedAuthenticator
>>>> +##### ScopedAuthenticator
>>>>
>>>> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different
>>>> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are
>>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>>>> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be
>>>> used in favor of those for `http://example.org`
>>>>
>>>> -#### ServiceAuthenticator
>>>> +##### ServiceAuthenticator
>>>>
>>>> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the
>>>> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on
>>>> configuration for this.
>>>>
>>>> -#### FormsAuthenticator
>>>> +##### FormsAuthenticator
>>>>
>>>> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
>>>> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this
>>>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>>>
>>>> Currently forms based login that require more than just a username and password are not supported.
>>>>
>>>> -#### PreemptiveBasicAuthenticator
>>>> +##### PreemptiveBasicAuthenticator
>>>>
>>>> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers
>>>> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
>>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>>>> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
>>>> challenge response sequences.
>>>>
>>>> -#### DelegatingAuthenticator
>>>> +##### DelegatingAuthenticator
>>>>
>>>> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and
>>>> match the types of authentication needed.
>>>>
>>>> -### Debugging Authentication
>>>> -
>>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>> -
>>>> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>> -
>>>> -### The Default Authenticator
>>>> +#### The Default Authenticator
>>>>
>>>> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default
>>>> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the
>>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>>>
>>>> Note that the default authenticator may be disabled by setting it to `null`.
>>>>
>>>> +## HTTP Authentication after ARQ 3.1.0
>>>> +
>>>> +### Applying Authentication
>>>> +
>>>> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the
>>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
>>>> +
>>>> +#### Examples of authentication
>>>> +
>>>> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
>>>> +
>>>> +##### Simple authentication using username and password
>>>> +
>>>> +First we build an authenticating client:
>>>> +
>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
>>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>>>> +    HttpClient httpclient = HttpClients.custom()
>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>> +        .build();
>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>> +
>>>> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
>>>> +
>>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>>> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
>>>> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>>> +    final String host = "http://example.com/sparql";
>>>> +    final int port = 80;
>>>> +    final String realm = "aRealm";
>>>> +    final String schemeName = "DIGEST";
>>>> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
>>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>>>> +    HttpClient httpclient = HttpClients.custom()
>>>> +        .setDefaultCredentialsProvider(credsProvider)
>>>> +        .build();
>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>> +	
>>>> +##### Authenticating via a form
>>>> +
>>>> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
>>>> +
>>>> +    // we'll use this context to maintain our HTTP "conversation"
>>>> +    HttpClientContext httpContext = new HttpClientContext();
>>>> +
>>>> +    // first we use a method on HttpOp to log in and get our cookie
>>>> +    Params params = new Params();
>>>> +    params.addParam("username", "Bob Wu");
>>>> +    params.addParam("password", "my password");
>>>> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
>>>> +
>>>> +    // now our cookie is stored in httpContext
>>>> +    CookieStore cookieStore = httpContext.getCookieStore();
>>>> +
>>>> +    // lastly we build a client that uses that cookie
>>>> +    HttpClient httpclient = HttpClients.custom()
>>>> +	    .setDefaultCookieStore(cookieStore)
>>>> +		.build();
>>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>>> +
>>>> +## Other concerns
>>>> +
>>>> +### Debugging Authentication
>>>> +
>>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>>> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>>> +
>>>> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>>> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>>> +
>>>> +### Authenticating to a SPARQL federated service
>>>> +
>>>> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
>>>> +
>>>>  [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>>>>  [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>>>  [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
>>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>>>  [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>>>>  [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>>>  [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
>>>> -  [14]: http://hc.apache.org
>>>> \ No newline at end of file
>>>> +  [14]: http://hc.apache.org
>>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
>>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>>>> \ No newline at end of file
>>>>
>>>> Modified: jena/site/trunk/content/documentation/query/service.mdtext
>>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>>> ==============================================================================
>>>> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
>>>> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
>>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>>>> without regard to how selective the pattern is. So the order of the
>>>> query will affect the speed of execution. Because it involves HTTP
>>>> operations, asking the query in the right order matters a lot.
>>>> -Don't ask for the whole of a bookstore just to find book whose
>>>> +Don't ask for the whole of a bookstore just to find a book whose
>>>> title comes from a local RDF file - ask the bookshop a query with
>>>> the title already bound from earlier in the query.
>>>>
>>>> ## Controlling `SERVICE` requests.
>>>>
>>>> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
>>>> -The values for configuration can be set in the global context (accessed via
>>>> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via
>>>> `ARQ.getContext()`) or in the per-query execution context.
>>>>
>>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
>>>>
>>>> -### Summary
>>>> +### Configuration for ARQ through version 3.1.0
>>>>
>>>> Symbol | Usage
>>>> ------ | -----
>>>> @@ -71,7 +70,7 @@ Symbol | Usage
>>>> `srv:queryAuthPwd` |  Basic authentication
>>>> `srv:queryContext` | Per-endpoint configuration
>>>>
>>>> -### `srv:queryTimeout`
>>>> +#### `srv:queryTimeout`
>>>>
>>>> Set the connect and read timeouts for the query.
>>>>
>>>> @@ -86,21 +85,21 @@ read timout = 0
>>>>
>>>> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
>>>>
>>>> -### `srv:queryGzip`
>>>> +#### `srv:queryGzip`
>>>>
>>>> Sets the allow Gzip flag.
>>>>
>>>> Boolean: True indicates that gzip compressed data is acceptable.
>>>> false
>>>>
>>>> -### `srv:queryDeflate`
>>>> +#### `srv:queryDeflate`
>>>>
>>>> Sets the allow Deflate flag.
>>>>
>>>> Boolean: True indicates that deflate compression is acceptable
>>>> False
>>>>
>>>> -### `srv:queryAuthUser`
>>>> +#### `srv:queryAuthUser`
>>>>
>>>> Sets the user id for basic auth.
>>>>
>>>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>>>
>>>> If null or null length no user id is sent.
>>>>
>>>> -### `srv:queryAuthPwd`
>>>> +#### `srv:queryAuthPwd`
>>>>
>>>> Sets the password for basic auth.
>>>>
>>>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>>>
>>>> If null or null length no password is sent.
>>>>
>>>> -### srv:serviceContext
>>>> +#### `srv:serviceContext`
>>>> Provides a mechanism to override system context settings on a per URI basis.
>>>>
>>>> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
>>>>
>>>> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
>>>>
>>>> +### Configuration for ARQ after version 3.1.0
>>>>
>>>> +Symbol | Usage | Default
>>>> +------ | ----- | -------
>>>> +`srv:queryTimeout` | Set timeouts | none
>>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>>>> +`srv:queryClient` | Enable use of a specific client | none
>>>> +`srv:queryContext` | Per-endpoint configuration | none
>>>> +
>>>> +#### `srv:queryTimeout`
>>>> +
>>>> +As documented above.
>>>> +
>>>> +
>>>> +#### `srv:queryCompression`
>>>> +
>>>> +Sets the flag for use of deflation and GZip.
>>>> +
>>>> +Boolean: True indicates that gzip compressed data is acceptable.
>>>> +
>>>> +#### `srv:queryClient`
>>>> +
>>>> +Enable use of a specific client
>>>> +
>>>> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
>>>> +
>>>> +#### `srv:serviceContext`
>>>> +
>>>> +As documented above.
>>>>
>>>> [ARQ documentation index](index.html)
>>>> +
>>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>>>
>>>>
>>>
>

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by "A. Soroka" <aj...@virginia.edu>.

I've made those changes-- should be restaging now.

---
A. Soroka
The University of Virginia Library

> On Nov 8, 2016, at 12:40 PM, Andy Seaborne <an...@apache.org> wrote:
> 
> 
> 
> On 08/11/16 16:59, A. Soroka wrote:
>> This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?
>> 
> 
> It does not for me.
> 
> Try http://jena.staging.apache.org/  (not https)
> 
> PDF attached, cc'ed to you in the hope it get through.
> 
> Comments:
> 
> 1/ I'd put the current (3.1.1) text first and the previous second so the current is more visible.
> 
> Links at the end of the intro to "current" and "previous", or in the intro as this difference is mentioned.
> 
> 2/ Title tweaking:
> 
> "HTTP Authentication after Jena 3.1.0" ->
> "HTTP Authentication from Jena 3.1.1"
> 
> "HTTP Authentication before Jena 3.1.0" =>
> "HTTP Authentication from Jena 3.0.0 to 3.1.0"
> 
> (so the range includes 3.1.0 !)
> 
> Mentioning Jena 2.x is not necessary IMO - the additional detail adds confusion for current users and 3.x upgrading users (the majority).
> 
> 3/
> "Simple authentication using username and password"
> 
> "Authenticating via a form"
> 
> The <h5> don't show up as different on teh screen for me so maybe bump <h4> "Examples of authentication" up a level to <h3> and move sub <5> to <h4> .
> 
> Maybe drop <h3> "Applying Authentication" (section title immediately after a section title) and have the paragraph there straight away.
> 
>    Andy
> 
>> ---
>> A. Soroka
>> The University of Virginia Library
>> 
>>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>> 
>>> Author: ajs6f
>>> Date: Tue Nov  8 16:53:48 2016
>>> New Revision: 1768736
>>> 
>>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>>> Log:
>>> Updates for HTTP behavior in Jena 3.1.1
>>> 
>>> Modified:
>>>   jena/site/trunk/content/documentation/query/http-auth.mdtext
>>>   jena/site/trunk/content/documentation/query/service.mdtext
>>> 
>>> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>> ==============================================================================
>>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
>>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
>>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>>           specific language governing permissions and limitations
>>>           under the License.
>>> 
>>> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for
>>> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
>>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for
>>> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
>>> 
>>> -## Applying Authentication
>>> +## HTTP Authentication before ARQ 3.1.0
>>> +
>>> +### Applying Authentication
>>> 
>>> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
>>> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>>        ...
>>>    }
>>> 
>>> -### Authenticators
>>> +#### Authenticators
>>> 
>>> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
>>> 
>>> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried
>>> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
>>> 
>>> -#### SimpleAuthenticator
>>> +##### SimpleAuthenticator
>>> 
>>> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
>>> any service.
>>> @@ -56,7 +58,7 @@ any service.
>>> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>> Required) then credentials will not actually be submitted.
>>> 
>>> -#### ScopedAuthenticator
>>> +##### ScopedAuthenticator
>>> 
>>> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different
>>> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are
>>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>>> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be
>>> used in favor of those for `http://example.org`
>>> 
>>> -#### ServiceAuthenticator
>>> +##### ServiceAuthenticator
>>> 
>>> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the
>>> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on
>>> configuration for this.
>>> 
>>> -#### FormsAuthenticator
>>> +##### FormsAuthenticator
>>> 
>>> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
>>> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this
>>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>> 
>>> Currently forms based login that require more than just a username and password are not supported.
>>> 
>>> -#### PreemptiveBasicAuthenticator
>>> +##### PreemptiveBasicAuthenticator
>>> 
>>> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers
>>> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
>>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>>> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
>>> challenge response sequences.
>>> 
>>> -#### DelegatingAuthenticator
>>> +##### DelegatingAuthenticator
>>> 
>>> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and
>>> match the types of authentication needed.
>>> 
>>> -### Debugging Authentication
>>> -
>>> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>> -
>>> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>> -
>>> -### The Default Authenticator
>>> +#### The Default Authenticator
>>> 
>>> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default
>>> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the
>>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>> 
>>> Note that the default authenticator may be disabled by setting it to `null`.
>>> 
>>> +## HTTP Authentication after ARQ 3.1.0
>>> +
>>> +### Applying Authentication
>>> +
>>> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the
>>> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
>>> +
>>> +#### Examples of authentication
>>> +
>>> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
>>> +
>>> +##### Simple authentication using username and password
>>> +
>>> +First we build an authenticating client:
>>> +
>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
>>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>>> +    HttpClient httpclient = HttpClients.custom()
>>> +        .setDefaultCredentialsProvider(credsProvider)
>>> +        .build();
>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>> +
>>> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
>>> +
>>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>>> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
>>> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>>> +    final String host = "http://example.com/sparql";
>>> +    final int port = 80;
>>> +    final String realm = "aRealm";
>>> +    final String schemeName = "DIGEST";
>>> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
>>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>>> +    HttpClient httpclient = HttpClients.custom()
>>> +        .setDefaultCredentialsProvider(credsProvider)
>>> +        .build();
>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>> +	
>>> +##### Authenticating via a form
>>> +
>>> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
>>> +
>>> +    // we'll use this context to maintain our HTTP "conversation"
>>> +    HttpClientContext httpContext = new HttpClientContext();
>>> +
>>> +    // first we use a method on HttpOp to log in and get our cookie
>>> +    Params params = new Params();
>>> +    params.addParam("username", "Bob Wu");
>>> +    params.addParam("password", "my password");
>>> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
>>> +
>>> +    // now our cookie is stored in httpContext
>>> +    CookieStore cookieStore = httpContext.getCookieStore();
>>> +
>>> +    // lastly we build a client that uses that cookie
>>> +    HttpClient httpclient = HttpClients.custom()
>>> +	    .setDefaultCookieStore(cookieStore)
>>> +		.build();
>>> +    HttpOp.setDefaultHttpClient(httpclient);
>>> +
>>> +## Other concerns
>>> +
>>> +### Debugging Authentication
>>> +
>>> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>>> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>>> +
>>> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>>> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>>> +
>>> +### Authenticating to a SPARQL federated service
>>> +
>>> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
>>> +
>>>  [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>>>  [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>>  [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
>>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>>  [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>>>  [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>>  [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
>>> -  [14]: http://hc.apache.org
>>> \ No newline at end of file
>>> +  [14]: http://hc.apache.org
>>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
>>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>>> \ No newline at end of file
>>> 
>>> Modified: jena/site/trunk/content/documentation/query/service.mdtext
>>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>>> ==============================================================================
>>> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
>>> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
>>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>>> without regard to how selective the pattern is. So the order of the
>>> query will affect the speed of execution. Because it involves HTTP
>>> operations, asking the query in the right order matters a lot.
>>> -Don't ask for the whole of a bookstore just to find book whose
>>> +Don't ask for the whole of a bookstore just to find a book whose
>>> title comes from a local RDF file - ask the bookshop a query with
>>> the title already bound from earlier in the query.
>>> 
>>> ## Controlling `SERVICE` requests.
>>> 
>>> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
>>> -The values for configuration can be set in the global context (accessed via
>>> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via
>>> `ARQ.getContext()`) or in the per-query execution context.
>>> 
>>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
>>> 
>>> -### Summary
>>> +### Configuration for ARQ through version 3.1.0
>>> 
>>> Symbol | Usage
>>> ------ | -----
>>> @@ -71,7 +70,7 @@ Symbol | Usage
>>> `srv:queryAuthPwd` |  Basic authentication
>>> `srv:queryContext` | Per-endpoint configuration
>>> 
>>> -### `srv:queryTimeout`
>>> +#### `srv:queryTimeout`
>>> 
>>> Set the connect and read timeouts for the query.
>>> 
>>> @@ -86,21 +85,21 @@ read timout = 0
>>> 
>>> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
>>> 
>>> -### `srv:queryGzip`
>>> +#### `srv:queryGzip`
>>> 
>>> Sets the allow Gzip flag.
>>> 
>>> Boolean: True indicates that gzip compressed data is acceptable.
>>> false
>>> 
>>> -### `srv:queryDeflate`
>>> +#### `srv:queryDeflate`
>>> 
>>> Sets the allow Deflate flag.
>>> 
>>> Boolean: True indicates that deflate compression is acceptable
>>> False
>>> 
>>> -### `srv:queryAuthUser`
>>> +#### `srv:queryAuthUser`
>>> 
>>> Sets the user id for basic auth.
>>> 
>>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>> 
>>> If null or null length no user id is sent.
>>> 
>>> -### `srv:queryAuthPwd`
>>> +#### `srv:queryAuthPwd`
>>> 
>>> Sets the password for basic auth.
>>> 
>>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>> 
>>> If null or null length no password is sent.
>>> 
>>> -### srv:serviceContext
>>> +#### `srv:serviceContext`
>>> Provides a mechanism to override system context settings on a per URI basis.
>>> 
>>> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
>>> 
>>> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
>>> 
>>> +### Configuration for ARQ after version 3.1.0
>>> 
>>> +Symbol | Usage | Default
>>> +------ | ----- | -------
>>> +`srv:queryTimeout` | Set timeouts | none
>>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>>> +`srv:queryClient` | Enable use of a specific client | none
>>> +`srv:queryContext` | Per-endpoint configuration | none
>>> +
>>> +#### `srv:queryTimeout`
>>> +
>>> +As documented above.
>>> +
>>> +
>>> +#### `srv:queryCompression`
>>> +
>>> +Sets the flag for use of deflation and GZip.
>>> +
>>> +Boolean: True indicates that gzip compressed data is acceptable.
>>> +
>>> +#### `srv:queryClient`
>>> +
>>> +Enable use of a specific client
>>> +
>>> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
>>> +
>>> +#### `srv:serviceContext`
>>> +
>>> +As documented above.
>>> 
>>> [ARQ documentation index](index.html)
>>> +
>>> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>> 
>>> 
>>

Re: svn commit: r1768736 - in /jena/site/trunk/content/documentation/query: http-auth.mdtext service.mdtext

Posted by Andy Seaborne <an...@apache.org>.


On 08/11/16 16:59, A. Soroka wrote:
> This commit includes the new docs for HTTP behavior in Jena 3.1.1. I can't find any way to see a view of this on the staging site-- https://jena.staging.apache.org/ just seems to proxy https://cms.apache.org/, for some reason?
>

It does not for me.

Try http://jena.staging.apache.org/  (not https)

PDF attached, cc'ed to you in the hope it get through.

Comments:

1/ I'd put the current (3.1.1) text first and the previous second so the 
current is more visible.

Links at the end of the intro to "current" and "previous", or in the 
intro as this difference is mentioned.

2/ Title tweaking:

"HTTP Authentication after Jena 3.1.0" ->
"HTTP Authentication from Jena 3.1.1"

"HTTP Authentication before Jena 3.1.0" =>
"HTTP Authentication from Jena 3.0.0 to 3.1.0"

(so the range includes 3.1.0 !)

Mentioning Jena 2.x is not necessary IMO - the additional detail adds 
confusion for current users and 3.x upgrading users (the majority).

3/
"Simple authentication using username and password"

"Authenticating via a form"

The <h5> don't show up as different on teh screen for me so maybe bump 
<h4> "Examples of authentication" up a level to <h3> and move sub <5> to 
<h4> .

Maybe drop <h3> "Applying Authentication" (section title immediately 
after a section title) and have the paragraph there straight away.

     Andy

> ---
> A. Soroka
> The University of Virginia Library
>
>> On Nov 8, 2016, at 11:53 AM, ajs6f@apache.org wrote:
>>
>> Author: ajs6f
>> Date: Tue Nov  8 16:53:48 2016
>> New Revision: 1768736
>>
>> URL: http://svn.apache.org/viewvc?rev=1768736&view=rev
>> Log:
>> Updates for HTTP behavior in Jena 3.1.1
>>
>> Modified:
>>    jena/site/trunk/content/documentation/query/http-auth.mdtext
>>    jena/site/trunk/content/documentation/query/service.mdtext
>>
>> Modified: jena/site/trunk/content/documentation/query/http-auth.mdtext
>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/http-auth.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>> ==============================================================================
>> --- jena/site/trunk/content/documentation/query/http-auth.mdtext (original)
>> +++ jena/site/trunk/content/documentation/query/http-auth.mdtext Tue Nov  8 16:53:48 2016
>> @@ -16,10 +16,12 @@ Notice:    Licensed to the Apache Softwa
>>            specific language governing permissions and limitations
>>            under the License.
>>
>> -As of ARQ 2.11.0 there is a new unified HTTP operation framework that provides a uniform mechanism for
>> -HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible.
>> +From ARQ 2.11.0 through ARQ 3.1.0 there is a Jena-specific unified HTTP operation framework that provides a uniform mechanism for
>> +HTTP authentication that also allows ARQ to support a broader range of authentication mechanisms than were previously possible. After ARQ 3.1.0, Jena exposes the underlying HTTP Commons functionality to the same end. This documentation is therefore devided into two sections. The first explains the older Jena-specific functionality, and the second explains how to use HTTP Commons code to the same ends.
>>
>> -## Applying Authentication
>> +## HTTP Authentication before ARQ 3.1.0
>> +
>> +### Applying Authentication
>>
>> APIs that support authentication typically provide two methods for providing authenticators, a `setAuthentication(String username, char[] password)` method
>> which merely configures a `SimpleAuthenticator`.  There will also be a `setAuthenticator(HttpAuthenticator authenticator)` method
>> @@ -41,14 +43,14 @@ avoids the needs to cast and manually se
>>         ...
>>     }
>>
>> -### Authenticators
>> +#### Authenticators
>>
>> Authentication mechanisms are provided by [HttpAuthenticator][1] implementations of which a number are provided built into ARQ.
>>
>> This API provides the authenticator with access to the `HttpClient`, `HttpContext` and target `URI` of the request that is about to be carried
>> out.  This allows for authenticators to add credentials to requests on a per-request basis and/or to use different mechanisms and credentials for different services.
>>
>> -#### SimpleAuthenticator
>> +##### SimpleAuthenticator
>>
>> The [simple authenticator][2] is as the name suggests the simplest implementation.  It takes a single set of credentials which is applied to
>> any service.
>> @@ -56,7 +58,7 @@ any service.
>> Authentication however is not preemptive so unless the remote service sends a HTTP challenge (401 Unauthorized or 407 Proxy Authorization
>>  Required) then credentials will not actually be submitted.
>>
>> -#### ScopedAuthenticator
>> +##### ScopedAuthenticator
>>
>> The [scoped authenticator][3] is an authenticator which maps credentials to different service URIs.  This allows you to specify different
>> credentials for different services as appropriate.  Similarly to the simple authenticator this is not preemptive authentication so credentials are
>> @@ -67,13 +69,13 @@ if you define credentialsfor `http://exa
>> e.g. `http://example.org/some/path`.  However if you had also defined credentials for `http://example.org/some/path` then these would be
>> used in favor of those for `http://example.org`
>>
>> -#### ServiceAuthenticator
>> +##### ServiceAuthenticator
>>
>> The [service authenticator][4] is an authenticator which uses information encoded in the ARQ context and basically provides access to the
>> existing credential provision mechanisms provided for the `SERVICE` clause, see [Basic Federated Query][5] for more information on
>> configuration for this.
>>
>> -#### FormsAuthenticator
>> +##### FormsAuthenticator
>>
>> The [forms authenticator][6] is an authenticator usable with services that require form based logins and use session cookies to verify login state.
>> This is intended for use with services that don't support HTTP's built-in authentication mechanisms for whatever reason.  One example of this
>> @@ -104,7 +106,7 @@ that maps each service to an associated
>>
>> Currently forms based login that require more than just a username and password are not supported.
>>
>> -#### PreemptiveBasicAuthenticator
>> +##### PreemptiveBasicAuthenticator
>>
>> This [authenticator][8] is a decorator over another authenticator that enables preemptive basic authentication, this **only** works for servers
>> that support basic authentication and so will cause authentication failures when any other authentication scheme is required.  You should **only**
>> @@ -121,20 +123,12 @@ Also be aware that basic authentication
>> many servers will use more secure schemes like Digest authentication which **cannot** be done preemptively as they require more complex
>> challenge response sequences.
>>
>> -#### DelegatingAuthenticator
>> +##### DelegatingAuthenticator
>>
>> The [delegating authenticator][12] allows for mapping different authenticators to different services, this is useful when you need to mix and
>> match the types of authentication needed.
>>
>> -### Debugging Authentication
>> -
>> -ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>> -see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>> -
>> -The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>> -HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>> -
>> -### The Default Authenticator
>> +#### The Default Authenticator
>>
>> Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default
>> authenticator that is used when no authenticator is explicitly specified.  This may be configured via the
>> @@ -148,6 +142,82 @@ provided that it is using ARQs APIs to m
>>
>> Note that the default authenticator may be disabled by setting it to `null`.
>>
>> +## HTTP Authentication after ARQ 3.1.0
>> +
>> +### Applying Authentication
>> +
>> +APIs that support authentication typically provide methods for providing an [HttpClient] for use with the given instance of that API class. `HttpClient` is [extremely flexible][16] and can handle most scenarios very well. Since it may not always be possible/practical to configure authenticators on a per-request basis the API includes a means to specify a default client that is used when no other client is explicitly specified.  This may be configured via the
>> +`setDefaultHttpClient(HttpClient httpClient)` method of the [HttpOp][13] class. This allows for static-scoped configuration of HTTP behavior.
>> +
>> +#### Examples of authentication
>> +
>> +This section includes a series of examples showing how to use HTTP Commons classes to perform authenticated work. Most of them take advantage of `HttpOp.setDefaultHttpClient` as described above.
>> +
>> +##### Simple authentication using username and password
>> +
>> +First we build an authenticating client:
>> +
>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>> +    Credentials credentials = new UsernamePasswordCredentials("user", "passwd");
>> +    credsProvider.setCredentials(AuthScope.ANY, credentials);
>> +    HttpClient httpclient = HttpClients.custom()
>> +        .setDefaultCredentialsProvider(credsProvider)
>> +        .build();
>> +    HttpOp.setDefaultHttpClient(httpclient);
>> +
>> +Notice that we gave no scope for use with the credentials (`AuthScope.ANY`). We can make further use of that parameter if we want to assign a scope for some credentials:
>> +
>> +    CredentialsProvider credsProvider = new BasicCredentialsProvider();
>> +    Credentials unscopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>> +    credsProvider.setCredentials(AuthScope.ANY, unscopedCredentials);
>> +    Credentials scopedCredentials = new UsernamePasswordCredentials("user", "passwd");
>> +    final String host = "http://example.com/sparql";
>> +    final int port = 80;
>> +    final String realm = "aRealm";
>> +    final String schemeName = "DIGEST";
>> +    AuthScope authscope = new AuthScope(host, port, realm, schemeName);
>> +    credsProvider.setCredentials(authscope, scopedCredentials);
>> +    HttpClient httpclient = HttpClients.custom()
>> +        .setDefaultCredentialsProvider(credsProvider)
>> +        .build();
>> +    HttpOp.setDefaultHttpClient(httpclient);
>> +	
>> +##### Authenticating via a form
>> +
>> +For this case we introduce an [HttpClientContext][17], which we can use to retrieve the cookie we get from logging into a form. We then use the cookie to authenticate elsewhere.
>> +
>> +    // we'll use this context to maintain our HTTP "conversation"
>> +    HttpClientContext httpContext = new HttpClientContext();
>> +
>> +    // first we use a method on HttpOp to log in and get our cookie
>> +    Params params = new Params();
>> +    params.addParam("username", "Bob Wu");
>> +    params.addParam("password", "my password");
>> +    HttpOp.execHttpPostForm("http://example.com/loginform", params , null, null, null, httpContext);
>> +
>> +    // now our cookie is stored in httpContext
>> +    CookieStore cookieStore = httpContext.getCookieStore();
>> +
>> +    // lastly we build a client that uses that cookie
>> +    HttpClient httpclient = HttpClients.custom()
>> +	    .setDefaultCookieStore(cookieStore)
>> +		.build();
>> +    HttpOp.setDefaultHttpClient(httpclient);
>> +
>> +## Other concerns
>> +
>> +### Debugging Authentication
>> +
>> +ARQ uses [Apache Http Client][14] for all its HTTP operations and this provides detailed logging information that can be used for debugging.  To
>> +see this information you need to configure your logging framework to set the `org.apache.http` package to either `DEBUG` or `TRACE` level.
>> +
>> +The `DEBUG` level will give you general diagnostic information about requests and responses while the `TRACE` level will give you detailed
>> +HTTP traces i.e. allow you to see the exact HTTP requests and responses which can be extremely useful for debugging authentication problems.
>> +
>> +### Authenticating to a SPARQL federated service
>> +
>> +ARQ allows the user to configure HTTP behavior to use on a per-`SERVICE` basis, including authentication behavior such as is described above. This works via the ARQ context. See [Basic Federated Query][5] for more information on configuring this functionality.
>> +
>>   [1]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/HttpAuthenticator.html
>>   [2]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/SimpleAuthenticator.html
>>   [3]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/ScopedAuthenticator.html
>> @@ -161,4 +231,7 @@ Note that the default authenticator may
>>   [11]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/web/DatasetGraphAccessorHTTP.html
>>   [12]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/atlas/web/auth/DelegatingAuthenticator.html
>>   [13]: http://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/web/HttpOp.html
>> -  [14]: http://hc.apache.org
>> \ No newline at end of file
>> +  [14]: http://hc.apache.org
>> +  [15]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>> +  [16]: https://hc.apache.org/httpcomponents-client-ga/examples.html
>> +  [17]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/protocol/HttpClientContext.html
>> \ No newline at end of file
>>
>> Modified: jena/site/trunk/content/documentation/query/service.mdtext
>> URL: http://svn.apache.org/viewvc/jena/site/trunk/content/documentation/query/service.mdtext?rev=1768736&r1=1768735&r2=1768736&view=diff
>> ==============================================================================
>> --- jena/site/trunk/content/documentation/query/service.mdtext (original)
>> +++ jena/site/trunk/content/documentation/query/service.mdtext Tue Nov  8 16:53:48 2016
>> @@ -48,19 +48,18 @@ distributed query evaluation. The algebr
>> without regard to how selective the pattern is. So the order of the
>> query will affect the speed of execution. Because it involves HTTP
>> operations, asking the query in the right order matters a lot.
>> -Don't ask for the whole of a bookstore just to find book whose
>> +Don't ask for the whole of a bookstore just to find a book whose
>> title comes from a local RDF file - ask the bookshop a query with
>> the title already bound from earlier in the query.
>>
>> ## Controlling `SERVICE` requests.
>>
>> -The `SERVICE` operation in a SPARQL query may be configured via the Context.
>> -The values for configuration can be set in the global context (accessed via
>> +The `SERVICE` operation in a SPARQL query may be configured via the Context. The values for configuration can be set in the global context (accessed via
>> `ARQ.getContext()`) or in the per-query execution context.
>>
>> The prefix  `srv:` is the IRI `<http://jena.hpl.hp.com/Service#>`.
>>
>> -### Summary
>> +### Configuration for ARQ through version 3.1.0
>>
>> Symbol | Usage
>> ------ | -----
>> @@ -71,7 +70,7 @@ Symbol | Usage
>> `srv:queryAuthPwd` |  Basic authentication
>> `srv:queryContext` | Per-endpoint configuration
>>
>> -### `srv:queryTimeout`
>> +#### `srv:queryTimeout`
>>
>> Set the connect and read timeouts for the query.
>>
>> @@ -86,21 +85,21 @@ read timout = 0
>>
>> Values of 0 indicate no timeout and service operation will wait until the remote server responds.
>>
>> -### `srv:queryGzip`
>> +#### `srv:queryGzip`
>>
>> Sets the allow Gzip flag.
>>
>> Boolean: True indicates that gzip compressed data is acceptable.
>> false
>>
>> -### `srv:queryDeflate`
>> +#### `srv:queryDeflate`
>>
>> Sets the allow Deflate flag.
>>
>> Boolean: True indicates that deflate compression is acceptable
>> False
>>
>> -### `srv:queryAuthUser`
>> +#### `srv:queryAuthUser`
>>
>> Sets the user id for basic auth.
>>
>> @@ -108,7 +107,7 @@ String: The user id to log in with
>>
>> If null or null length no user id is sent.
>>
>> -### `srv:queryAuthPwd`
>> +#### `srv:queryAuthPwd`
>>
>> Sets the password for basic auth.
>>
>> @@ -116,13 +115,43 @@ String: The password to log in with.
>>
>> If null or null length no password is sent.
>>
>> -### srv:serviceContext
>> +#### `srv:serviceContext`
>> Provides a mechanism to override system context settings on a per URI basis.
>>
>> The value is a `Map<String,Context>` where the map key is the URI of the service endpoint, and the `Context` is a set of values to override the default values.
>>
>> If a context is provided for the URI the system context is copied and the URI specific values are then copied in.  This ensures that any URI specific settings will be used.
>>
>> +### Configuration for ARQ after version 3.1.0
>>
>> +Symbol | Usage | Default
>> +------ | ----- | -------
>> +`srv:queryTimeout` | Set timeouts | none
>> +`srv:queryCompression` | Enable use of deflation and GZip | true
>> +`srv:queryClient` | Enable use of a specific client | none
>> +`srv:queryContext` | Per-endpoint configuration | none
>> +
>> +#### `srv:queryTimeout`
>> +
>> +As documented above.
>> +
>> +
>> +#### `srv:queryCompression`
>> +
>> +Sets the flag for use of deflation and GZip.
>> +
>> +Boolean: True indicates that gzip compressed data is acceptable.
>> +
>> +#### `srv:queryClient`
>> +
>> +Enable use of a specific client
>> +
>> +Provides a slot for a specific [HttpClient][1] for use with a specific `SERVICE`
>> +
>> +#### `srv:serviceContext`
>> +
>> +As documented above.
>>
>> [ARQ documentation index](index.html)
>> +
>> +[1]: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/HttpClient.html
>>
>>
>