Using httpclient on the server to intercept and modify HTTP response

[Chris Brown <br...@reflexe.fr>]

Hello,

I've created an application that performs term highlighting as a service for
search engines, but I need to add support for retrieving documents from
NTLM-protected servers (as well as BASIC-auth-protected servers, and servers
with no protection whatsoever).  Currently, it works with the basic JDK HTTP
support, so I can't do this.

I've had a go at solving this myself, but I'm a bit stuck with the HTTP
client.

Here's the basic approach that I use at present:

- search engine results contain a link, passing the URL of the document to
highlight, and the terms to highlight. the user clicks on this link, sending
the request to the webapp (running on tomcat), which provides the
highlighting service.

- the highlighting software connects to the original server, and gets the
response containing the original document.

- the highlighting software modifies the stream on-the-fly and forwards the
modified stream to the client.

The current limitation that's forcing a move away from the basic JDK APIs is
that the second step above is blocked.  I could add basic authentification
easily enough with standard APIs, but not much else.

So, here are the specific problems (I hope someone has the time to help me
out here):

1./ when connecting to something with the HTTPClient API, how do I determine
dynamically which type of credentials I should use, if any?

2./ given that I'm forwarding requests from many clients via one host, are
there any problems or pitfalls with multiple connections from that host
using different credentials for each client?  is there any caching for
example that might mean that the first NT login is used for some/all
subsequent requests that should in fact be using the credentials for a
different client?

3./ as I'll need to ask the client to provide credentials, I can either use
a form or a BASIC authentification request.  If I limit the request to
"username" and "password" fields, can I safely request that the client's
username contains "DOMAIN\username" (and then decode it), as I'm not clear
about where I should obtain domain names for?

Thanks, much appreciated,
Chris B