Control Character Woes :)

[Eric Anderson <er...@afaik.us>]

I have a problem that I need a bit of help in solving. We are working 
with a company that is providing web services to export data out of a 
hosted application we are using. Their web services are built using 
.NET. We are writing an application to export that data and import it 
into another hosted application.

Our problem is that occasionally some data will contain a control 
character encoded as an character reference that is invalid in XML (for 
example a form feed character). It is my understanding that certain 
control characters are not valid in XML 1.0 and Microsoft's XML 
generator should not be producing them. From what I understand Microsoft 
has even fixed this in recent versions of their XML library but our 
problem is the company we are interacting with will do absolutely 
nothing to resolve the problem. Their solution if for us to switch to 
.NET. This is not possible for us because there is already a significant 
about of code written under Sun's Java. So we are getting these invalid 
XML soap messages whether we like it or not and need to figure out a way 
to deal with it.

My thought is since SOAP is built on HTTP perhaps we could put a HTTP 
proxy between our application and their web service to remove these 
invalid control characters. My questions are the following:

1) Does anybody know of any existing solutions out there already to deal 
with this problem (perhaps an existing proxy to sanitize the XML).

2) Am I correct in my understanding that these control characters are 
not allowed in XML (even though they are encoded as character 
references) and that this is really a bug on their side that we are 
having to work around?

3) Are there any other ways to fix this that might be easier (cheaper)?

Our current system involves us generating java source code from the wsdl 
file via wsdl2java included with Axis. So we never really interact with 
the Java's XML parser or Java's HTTP libraries directly. Is there anyway 
we could put some sort of input stream filter on the HTTP input stream?

Also just for completeness I have copy and pasted the error message we 
are receiving below to ensure that I am understanding the problem correctly:

org.xml.sax.SAXParseException: Character reference "&#xC" is an invalid 
XML character

I appreciate any help you guys an give.

Eric