Odd problem with SolrJ - some kind of issue with authentication

[Shawn Heisey <ap...@elyograg.org>]

I am encountering an exception with SolrJ.  You can find it at the 
following URL:

https://paste.elyograg.org/view/dd2685a8


The code at line 161 in my code is:

       req.process(client, collection);

The request is an UpdateRequest which has a batch of 1000 
SolrInputDocument objects added.  It also has commitWithin set with a 5 
minute value.

The client is CloudHttp2SolrClient.  That client is created with 3 ZK 
server:port values, a ZK chroot, and a custom Http2SolrClient which has 
had a username and password added for authentication added.

My program successfully indexed 25 million documents before this 
exception was thrown, and indexes continuously for about an hour with no 
problem.  There are a total of 30 million documents to index.

SolrJ 9.2.1 comes with the 10.0.13 version of Jetty client.  I forced an 
upgrade to 10.0.15 in my build.gradle and there was no change in behavior.

The Solr servers are running Solr 9.1.1, with TLS and authentication 
enabled.  I have already suggested they upgrade to 9.2.1, they are 
resisting this.

Any ideas about what might be going wrong?  To me this looks like a 
possible bug in Jetty, but I am not going to assume anything right now.

Solr 9.1.1 uses Jetty 9, and 9.2.1 is on Jetty 10.  How likely is it 
that a Solr upgrade (which includes a Jetty upgrade) would fix this?

Thanks,
Shawn