You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Vince McMahon <si...@gmail.com> on 2023/12/06 07:56:37 UTC

how to fix full processing disrupted in solr 8.11

Hi,

I did a full-import of an entity city and should have Indexed 1,510,000
documents as in the source.  The UI has the status of "Indexing completed.
Added/Updated: 1510000 documents. Deleted 0 documents. (Duration: 33s)"

After a while, the source got new documents and I am expecting a second
additional 160,000 of changed documents.

However, when I query the count, I've got only 10000.  The log has
error that I can't find answer from google, which is at the end of this
email.  I am guessing something disrupted the Indexing.  What is it and how
to correct it?

query count of core p4a
{

  "responseHeader":{
    "status":0,
    "QTime":0,
    "params":{
      "q":"*:*",
      "indent":"true",
      "q.op":"OR",
      "rows":"0",
      "_":"1701848047557"}},
  "response":{"numFound":10000,"start":0,"numFoundExact":true,"docs":[]
  }}


The config is the following

<dataConfig>
  <dataSource type="JdbcDataSource"
              driver="org.postgresql.Driver"
              url="jdbc:postgresql://localhost/world"
              user="solrpull"
              password="WorkingPassword"/>
  <document>
    <entity name            ="findDelta"
            query           ="select id FROM city WHERE update_at >
TO_TIMESTAMP('${dih.last_index_time}', 'YYYY-MM-DD,  HH24:MI:SS') -
INTERVAL '180 day' "
            rootEntity      ="false"
    >
    </entity>
    <entity name            ="city"
            pk              ="id"
            query           ="select id, name, update_at FROM city
WHERE ( '${dataimporter.request.clean}' = 'true'
            OR update_at > TO_TIMESTAMP('${dih.last_index_time}',
'YYYY-MM-DD,  HH24:MI:SS') - INTERVAL '180 day' )"
    >
      <field column="id" name="id"/>
      <field column="name" name="name"/>
      <field column="update_at" name="update_at"/>
    </entity>
  </document></dataConfig>


The 2nd time Raw Status-Output is

{
  "responseHeader": {
    "status": 0,
    "QTime": 0
  },
  "initArgs": [
    "defaults",
    [
      "config",
      "db-data-config.xml"
    ]
  ],
  "command": "status",
  "status": "idle",
  "importResponse": "",
  "statusMessages": {
    "Total Requests made to DataSource": "2",
    "Total Rows Fetched": "753718",
    "Total Documents Processed": "376859",
    "Total Documents Skipped": "0",
    "Full Dump Started": "2023-12-06 07:36:34",
    "": "Indexing completed. Added/Updated: 376859 documents. Deleted
0 documents.",
    "Committed": "2023-12-06 07:36:44",
    "Time taken": "0:0:10.57"
  }
}

The log has the error that I can't find from google.


2023-12-06 07:36:34.157 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.DataImporter Loading DIH Configuration: db-data-config.xml
2023-12-06 07:36:34.158 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.c.DIHConfiguration id is a required field in SolrSchema .
But not found in DataConfig
2023-12-06 07:36:34.159 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.DataImporter Data Configuration loaded successfully
2023-12-06 07:36:34.159 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.DataImporter Starting Full Import
2023-12-06 07:36:34.163 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.SimplePropertiesWriter Read dataimport.properties
2023-12-06 07:36:34.167 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.JdbcDataSource Creating a connection for entity findDelta
with URL: jdbc:postgresql://localhost/world
2023-12-06 07:36:34.200 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.JdbcDataSource Time taken for getConnection(): 33
2023-12-06 07:36:34.871 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.JdbcDataSource Creating a connection for entity city with
URL: jdbc:postgresql://localhost/world
2023-12-06 07:36:34.898 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.JdbcDataSource Time taken for getConnection(): 27
2023-12-06 07:36:36.231 INFO  (qtp548482954-66) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={core=p4a&debug=true&indent=on&commit=true&name=dataimport&clean=false&rows=9999999&wt=json&command=full-import&_=1701847947387&verbose=true}
status=0 QTime=0
2023-12-06 07:36:36.234 INFO  (qtp548482954-86) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:38.414 INFO  (qtp548482954-86) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:39.927 INFO  (qtp548482954-66) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:40.546 INFO  (qtp548482954-86) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:41.123 INFO  (qtp548482954-66) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:41.623 INFO  (qtp548482954-86) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:41.778 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.DocBuilder Import completed successfully
2023-12-06 07:36:42.109 INFO  (qtp548482954-66) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:42.538 INFO  (qtp548482954-86) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:42.980 INFO  (qtp548482954-66) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:43.368 INFO  (qtp548482954-86) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:43.762 INFO  (qtp548482954-66) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:44.133 INFO  (qtp548482954-86) [   x:p4a]
o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
params={indent=on&wt=json&command=status&_=1701847947387} status=0
QTime=0
2023-12-06 07:36:44.181 INFO
(searcherExecutor-45-thread-1-processing-x:p4a) [   x:p4a]
o.a.s.c.QuerySenderListener QuerySenderListener done.
2023-12-06 07:36:44.183 INFO
(searcherExecutor-45-thread-1-processing-x:p4a) [   x:p4a]
o.a.s.c.SolrCore [p4a]  Registered new searcher autowarm time: 0 ms
2023-12-06 07:36:44.219 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.SimplePropertiesWriter Read dataimport.properties
2023-12-06 07:36:44.220 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.SimplePropertiesWriter Wrote last indexed time to
dataimport.properties
2023-12-06 07:36:44.220 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.h.d.DocBuilder Time taken = 0:0:10.57
2023-12-06 07:36:44.220 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.u.p.LogUpdateProcessorFactory [p4a]  webapp=/solr
path=/dataimport
params={core=p4a&debug=true&indent=on&commit=true&name=dataimport&clean=false&rows=9999999&wt=json&command=full-import&_=1701847947387&verbose=true}{add=[8904
(1784517172815462400), 670 (1784517172823851008), 6054
(1784517172823851009), 4999 (1784517172823851010), 3956
(1784517172823851011), 8054 (1784517172823851012), 510
(1784517172823851013), 357 (1784517172823851014), 9655
(1784517172823851015), 3990 (1784517172824899584), ... (376859
adds)],commit=} 0 10065
2023-12-06 07:36:44.234 ERROR (qtp548482954-31) [   x:p4a]
o.a.s.s.HttpSolrCall java.lang.RuntimeException:
org.eclipse.jetty.io.EofException => java.lang.RuntimeException:
org.eclipse.jetty.io.EofException
    at org.apache.solr.common.MapWriter$EntryWriter.putNoEx(MapWriter.java:103)
java.lang.RuntimeException: org.eclipse.jetty.io.EofException
    at org.apache.solr.common.MapWriter$EntryWriter.putNoEx(MapWriter.java:103)
~[?:?]
    at org.apache.solr.common.MapWriter$EntryWriter.lambda$getBiConsumer$0(MapWriter.java:161)
~[?:?]
    at org.apache.solr.common.SolrInputDocument.lambda$writeMap$0(SolrInputDocument.java:59)
~[?:?]
    at java.util.LinkedHashMap.forEach(Unknown Source) ~[?:?]
    at org.apache.solr.common.SolrInputDocument.writeMap(SolrInputDocument.java:61)
~[?:?]
    at org.apache.solr.common.util.JsonTextWriter.writeMap(JsonTextWriter.java:164)
~[?:?]
    at org.apache.solr.common.util.TextWriter.writeMap(TextWriter.java:216)
~[?:?]
    at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:69)
~[?:?]
    at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
~[?:?]
    at org.apache.solr.common.util.JsonTextWriter.writeJsonIter(JsonTextWriter.java:194)
~[?:?]
    at org.apache.solr.common.util.JsonTextWriter.writeArray(JsonTextWriter.java:277)
~[?:?]
    at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:78)
~[?:?]
    at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
~[?:?]
    at org.apache.solr.common.util.JsonTextWriter.writeNamedListAsMapWithDups(JsonTextWriter.java:387)
~[?:?]
    at org.apache.solr.common.util.JsonTextWriter.writeNamedList(JsonTextWriter.java:293)
~[?:?]
    at org.apache.solr.response.JSONWriter.writeResponse(JSONWriter.java:73)
~[?:?]
    at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:66)
~[?:?]
    at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
~[?:?]
    at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:887)
~[?:?]
    at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:580) ~[?:?]
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427)
~[?:?]
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:357)
~[?:?]
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)
~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
~[jetty-security-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
~[jetty-rewrite-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.Server.handle(Server.java:516)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: org.eclipse.jetty.io.EofException
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:381)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:831)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:555)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:927)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:999)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor$GzipBufferCB.process(GzipHttpOutputInterceptor.java:416)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor.gzip(GzipHttpOutputInterceptor.java:139)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor.write(GzipHttpOutputInterceptor.java:121)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:285)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:269)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:1054)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.apache.solr.servlet.ServletOutputStreamWrapper.write(ServletOutputStreamWrapper.java:126)
~[?:?]
    at org.apache.solr.response.QueryResponseWriterUtil$1.write(QueryResponseWriterUtil.java:54)
~[?:?]
    at java.io.OutputStream.write(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.write(Unknown Source) ~[?:?]
    at java.io.OutputStreamWriter.write(Unknown Source) ~[?:?]
    at org.apache.solr.common.util.FastWriter.flush(FastWriter.java:140) ~[?:?]
    at org.apache.solr.common.util.FastWriter.write(FastWriter.java:125) ~[?:?]
    at java.io.Writer.write(Unknown Source) ~[?:?]
    at org.apache.solr.response.JSONWriter._writeStr(JSONWriter.java:219) ~[?:?]
    at org.apache.solr.common.util.JsonTextWriter.writeStr(JsonTextWriter.java:133)
~[?:?]
    at org.apache.solr.common.util.JsonTextWriter.writeDate(JsonTextWriter.java:232)
~[?:?]
    at org.apache.solr.common.util.TextWriter.writeDate(TextWriter.java:163)
~[?:?]
    at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:61)
~[?:?]
    at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
~[?:?]
    at org.apache.solr.common.util.JsonTextWriter$2.put(JsonTextWriter.java:176)
~[?:?]
    at org.apache.solr.common.MapWriter$EntryWriter.putNoEx(MapWriter.java:101)
~[?:?]
    ... 55 more
Caused by: java.io.IOException: An established connection was aborted
by the software in your host machine
    at sun.nio.ch.SocketDispatcher.writev0(Native Method) ~[?:?]
    at sun.nio.ch.SocketDispatcher.writev(Unknown Source) ~[?:?]
    at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
    at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
    at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[?:?]
    at java.nio.channels.SocketChannel.write(Unknown Source) ~[?:?]
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:273)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:381)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:831)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:555)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:927)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:999)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor$GzipBufferCB.process(GzipHttpOutputInterceptor.java:416)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor.gzip(GzipHttpOutputInterceptor.java:139)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor.write(GzipHttpOutputInterceptor.java:121)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:285)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:269)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:1054)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.apache.solr.servlet.ServletOutputStreamWrapper.write(ServletOutputStreamWrapper.java:126)
~[?:?]
    at org.apache.solr.response.QueryResponseWriterUtil$1.write(QueryResponseWriterUtil.java:54)
~[?:?]
    at java.io.OutputStream.write(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.write(Unknown Source) ~[?:?]
    at java.io.OutputStreamWriter.write(Unknown Source) ~[?:?]
    at org.apache.solr.common.util.FastWriter.flush(FastWriter.java:140) ~[?:?]
    at org.apache.solr.common.util.FastWriter.write(FastWriter.java:125) ~[?:?]
    at java.io.Writer.write(Unknown Source) ~[?:?]
    at org.apache.solr.response.JSONWriter._writeStr(JSONWriter.java:219) ~[?:?]
    at org.apache.solr.common.util.JsonTextWriter.writeStr(JsonTextWriter.java:133)
~[?:?]
    at org.apache.solr.common.util.JsonTextWriter.writeDate(JsonTextWriter.java:232)
~[?:?]
    at org.apache.solr.common.util.TextWriter.writeDate(TextWriter.java:163)
~[?:?]
    at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:61)
~[?:?]
    at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
~[?:?]
    at org.apache.solr.common.util.JsonTextWriter$2.put(JsonTextWriter.java:176)
~[?:?]
    at org.apache.solr.common.MapWriter$EntryWriter.putNoEx(MapWriter.java:101)
~[?:?]
    ... 55 more
2023-12-06 07:36:44.235 INFO  (qtp548482954-31) [   x:p4a]
o.a.s.s.HttpSolrCall Unable to write response, client closed
connection or we are shutting down =>
org.eclipse.jetty.io.EofException: Closed
    at org.eclipse.jetty.server.HttpOutput.checkWritable(HttpOutput.java:771)
org.eclipse.jetty.io.EofException: Closed
    at org.eclipse.jetty.server.HttpOutput.checkWritable(HttpOutput.java:771)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:1007)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.apache.solr.servlet.ServletOutputStreamWrapper.write(ServletOutputStreamWrapper.java:126)
~[?:?]
    at org.apache.solr.response.QueryResponseWriterUtil$1.write(QueryResponseWriterUtil.java:54)
~[?:?]
    at java.io.OutputStream.write(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
    at sun.nio.cs.StreamEncoder.write(Unknown Source) ~[?:?]
    at java.io.OutputStreamWriter.write(Unknown Source) ~[?:?]
    at org.apache.solr.common.util.FastWriter.flush(FastWriter.java:140) ~[?:?]
    at org.apache.solr.common.util.FastWriter.flushBuffer(FastWriter.java:154)
~[?:?]
    at org.apache.solr.response.TextResponseWriter.close(TextResponseWriter.java:83)
~[?:?]
    at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:68)
~[?:?]
    at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
~[?:?]
    at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:887)
~[?:?]
    at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:759)
~[?:?]
    at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:589) ~[?:?]
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427)
~[?:?]
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:357)
~[?:?]
    at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)
~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
~[jetty-security-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
~[jetty-rewrite-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.Server.handle(Server.java:516)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at java.lang.Thread.run(Unknown Source) [?:?]

Re: how to fix full processing disrupted in solr 8.11

Posted by Vince McMahon <si...@gmail.com>.
I think I've gone pass this.  The "client" is not blowing up anymore.
There are something else I'll start a new thread.

On Wed, Dec 6, 2023 at 3:36 PM Vince McMahon <si...@gmail.com>
wrote:

> so, I've incrased the SOLR_JAVA_MEM by 8 times.  That memory increase is
> from solr status command.  But, no luck.  the Indexing process using
> full-import with Clean and Commit option says:
> *Indexing completed. Added/Updated: 915000 documents. Deleted 0 documents.
> (Duration: 20s)*
>
> The solr count query says, 5000 rows.  How to get the rest of the 915000
> indexed?
>
> {
>   "responseHeader":{
>     "status":0,
>     "QTime":0,
>     "params":{
>       "q":"*:*",
>       "indent":"true",
>       "q.op":"OR",
>       "rows":"0",
>       "_":"1701894264264"}},
>   "response":{"numFound":5000,"start":0,"numFoundExact":true,"docs":[]
>   }}
>
>
> the solr_gc.log says these:
>
> [2023-12-06T15:17:50.106-0500][0.012s] Heap region size: 4M
> [2023-12-06T15:17:50.132-0500][0.047s] Using G1
> [2023-12-06T15:17:50.132-0500][0.047s] Heap address: 0x0000000600000000, size: 8192 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
> [2023-12-06T15:17:51.119-0500][1.034s] GC(0) Pause Young (Concurrent Start) (Metadata GC Threshold)
> [2023-12-06T15:17:51.119-0500][1.034s] GC(0) Using 15 workers of 15 for evacuation
> [2023-12-06T15:17:51.134-0500][1.042s] GC(0)   Pre Evacuate Collection Set: 0.0ms
> [2023-12-06T15:17:51.134-0500][1.042s] GC(0)   Evacuate Collection Set: 6.8ms
> [2023-12-06T15:17:51.134-0500][1.042s] GC(0)   Post Evacuate Collection Set: 0.7ms
> [2023-12-06T15:17:51.134-0500][1.042s] GC(0)   Other: 0.6ms
> [2023-12-06T15:17:51.134-0500][1.044s] GC(0) Eden regions: 17->0(98)
> [2023-12-06T15:17:51.134-0500][1.044s] GC(0) Survivor regions: 0->4(13)
> [2023-12-06T15:17:51.134-0500][1.044s] GC(0) Old regions: 0->0
> [2023-12-06T15:17:51.134-0500][1.044s] GC(0) Humongous regions: 0->0
> [2023-12-06T15:17:51.134-0500][1.044s] GC(0) Metaspace: 20679K->20679K(1069056K)
> [2023-12-06T15:17:51.134-0500][1.044s] GC(0) Pause Young (Concurrent Start) (Metadata GC Threshold) 64M->14M(8192M) 10.371ms
> [2023-12-06T15:17:51.134-0500][1.044s] GC(0) User=0.22s Sys=0.00s Real=0.02s
> [2023-12-06T15:17:51.134-0500][1.044s] GC(1) Concurrent Cycle
> [2023-12-06T15:17:51.134-0500][1.044s] GC(1) Concurrent Clear Claimed Marks
> [2023-12-06T15:17:51.134-0500][1.044s] GC(1) Concurrent Clear Claimed Marks 0.019ms
> [2023-12-06T15:17:51.134-0500][1.044s] GC(1) Concurrent Scan Root Regions
> [2023-12-06T15:17:51.134-0500][1.046s] GC(1) Concurrent Scan Root Regions 1.610ms
> [2023-12-06T15:17:51.134-0500][1.046s] GC(1) Concurrent Mark (1.046s)
> [2023-12-06T15:17:51.134-0500][1.046s] GC(1) Concurrent Mark From Roots
> [2023-12-06T15:17:51.134-0500][1.046s] GC(1) Using 4 workers of 4 for marking
> [2023-12-06T15:17:51.134-0500][1.048s] GC(1) Concurrent Mark From Roots 1.961ms
> [2023-12-06T15:17:51.134-0500][1.048s] GC(1) Concurrent Preclean
> [2023-12-06T15:17:51.134-0500][1.048s] GC(1) Concurrent Preclean 0.091ms
> [2023-12-06T15:17:51.134-0500][1.048s] GC(1) Concurrent Mark (1.046s, 1.048s) 2.129ms
> [2023-12-06T15:17:51.134-0500][1.048s] GC(1) Pause Remark
> [2023-12-06T15:17:51.134-0500][1.049s] GC(1) Cleaned string and symbol table, strings: 10128 processed, 0 removed, symbols: 62420 processed, 29 removed
> [2023-12-06T15:17:51.134-0500][1.049s] GC(1) Pause Remark 18M->18M(8192M) 0.857ms
> [2023-12-06T15:17:51.134-0500][1.049s] GC(1) User=0.00s Sys=0.00s Real=0.00s
> [2023-12-06T15:17:51.134-0500][1.049s] GC(1) Concurrent Rebuild Remembered Sets
> [2023-12-06T15:17:51.134-0500][1.049s] GC(1) Concurrent Rebuild Remembered Sets 0.190ms
> [2023-12-06T15:17:51.134-0500][1.050s] GC(1) Pause Cleanup
> [2023-12-06T15:17:51.134-0500][1.050s] GC(1) Pause Cleanup 18M->18M(8192M) 0.107ms
> [2023-12-06T15:17:51.134-0500][1.050s] GC(1) User=0.00s Sys=0.00s Real=0.00s
> [2023-12-06T15:17:51.134-0500][1.050s] GC(1) Concurrent Cleanup for Next Mark
> [2023-12-06T15:17:51.134-0500][1.054s] GC(1) Concurrent Cleanup for Next Mark 4.341ms
> [2023-12-06T15:17:51.134-0500][1.054s] GC(1) Concurrent Cycle 9.947ms
> [2023-12-06T15:17:52.361-0500][2.281s] GC(2) Pause Young (Concurrent Start) (Metadata GC Threshold)
> [2023-12-06T15:17:52.361-0500][2.281s] GC(2) Using 15 workers of 15 for evacuation
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2)   Pre Evacuate Collection Set: 0.0ms
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2)   Evacuate Collection Set: 8.6ms
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2)   Post Evacuate Collection Set: 1.3ms
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2)   Other: 0.2ms
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2) Eden regions: 41->0(95)
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2) Survivor regions: 4->7(13)
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2) Old regions: 0->0
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2) Humongous regions: 0->0
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2) Metaspace: 35110K->35110K(1081344K)
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2) Pause Young (Concurrent Start) (Metadata GC Threshold) 178M->25M(8192M) 10.315ms
> [2023-12-06T15:17:52.376-0500][2.291s] GC(2) User=0.06s Sys=0.03s Real=0.01s
> [2023-12-06T15:17:52.376-0500][2.291s] GC(3) Concurrent Cycle
> [2023-12-06T15:17:52.376-0500][2.291s] GC(3) Concurrent Clear Claimed Marks
> [2023-12-06T15:17:52.376-0500][2.291s] GC(3) Concurrent Clear Claimed Marks 0.033ms
> [2023-12-06T15:17:52.376-0500][2.291s] GC(3) Concurrent Scan Root Regions
> [2023-12-06T15:17:52.376-0500][2.293s] GC(3) Concurrent Scan Root Regions 1.781ms
> [2023-12-06T15:17:52.376-0500][2.293s] GC(3) Concurrent Mark (2.293s)
> [2023-12-06T15:17:52.376-0500][2.293s] GC(3) Concurrent Mark From Roots
> [2023-12-06T15:17:52.376-0500][2.293s] GC(3) Using 4 workers of 4 for marking
> [2023-12-06T15:17:52.376-0500][2.296s] GC(3) Concurrent Mark From Roots 2.782ms
> [2023-12-06T15:17:52.376-0500][2.296s] GC(3) Concurrent Preclean
> [2023-12-06T15:17:52.376-0500][2.296s] GC(3) Concurrent Preclean 0.078ms
> [2023-12-06T15:17:52.376-0500][2.296s] GC(3) Concurrent Mark (2.293s, 2.296s) 2.915ms
> [2023-12-06T15:17:52.376-0500][2.296s] GC(3) Pause Remark
> [2023-12-06T15:17:52.376-0500][2.297s] GC(3) Cleaned string and symbol table, strings: 15734 processed, 0 removed, symbols: 110518 processed, 48 removed
> [2023-12-06T15:17:52.376-0500][2.297s] GC(3) Pause Remark 29M->29M(8192M) 1.060ms
> [2023-12-06T15:17:52.376-0500][2.297s] GC(3) User=0.00s Sys=0.00s Real=0.00s
> [2023-12-06T15:17:52.376-0500][2.297s] GC(3) Concurrent Rebuild Remembered Sets
> [2023-12-06T15:17:52.376-0500][2.297s] GC(3) Concurrent Rebuild Remembered Sets 0.300ms
> [2023-12-06T15:17:52.376-0500][2.298s] GC(3) Pause Cleanup
> [2023-12-06T15:17:52.376-0500][2.298s] GC(3) Pause Cleanup 29M->29M(8192M) 0.082ms
> [2023-12-06T15:17:52.376-0500][2.298s] GC(3) User=0.00s Sys=0.00s Real=0.00s
> [2023-12-06T15:17:52.376-0500][2.298s] GC(3) Concurrent Cleanup for Next Mark
> [2023-12-06T15:17:52.392-0500][2.301s] GC(3) Concurrent Cleanup for Next Mark 3.149ms
> [2023-12-06T15:17:52.392-0500][2.301s] GC(3) Concurrent Cycle 9.980ms
> [2023-12-06T15:17:54.272-0500][4.191s] GC(4) Pause Young (Normal) (G1 Evacuation Pause)
> [2023-12-06T15:17:54.272-0500][4.191s] GC(4) Using 15 workers of 15 for evacuation
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4)   Pre Evacuate Collection Set: 0.0ms
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4)   Evacuate Collection Set: 11.3ms
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4)   Post Evacuate Collection Set: 1.5ms
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4)   Other: 0.2ms
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4) Eden regions: 95->0(90)
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4) Survivor regions: 7->12(13)
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4) Old regions: 0->0
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4) Humongous regions: 0->0
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4) Metaspace: 53343K->53343K(1097728K)
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4) Pause Young (Normal) (G1 Evacuation Pause) 405M->45M(8192M) 13.409ms
> [2023-12-06T15:17:54.286-0500][4.204s] GC(4) User=0.06s Sys=0.05s Real=0.01s
> [2023-12-06T15:22:19.716-0500][269.623s] GC(5) Pause Young (Concurrent Start) (Metadata GC Threshold)
> [2023-12-06T15:22:19.716-0500][269.623s] GC(5) Using 15 workers of 15 for evacuation
> [2023-12-06T15:22:19.767-0500][269.675s] GC(5)   Pre Evacuate Collection Set: 0.1ms
> [2023-12-06T15:22:19.768-0500][269.675s] GC(5)   Evacuate Collection Set: 49.8ms
> [2023-12-06T15:22:19.768-0500][269.675s] GC(5)   Post Evacuate Collection Set: 1.2ms
> [2023-12-06T15:22:19.768-0500][269.675s] GC(5)   Other: 0.4ms
> [2023-12-06T15:22:19.768-0500][269.675s] GC(5) Eden regions: 78->0(89)
> [2023-12-06T15:22:19.768-0500][269.675s] GC(5) Survivor regions: 12->13(13)
> [2023-12-06T15:22:19.768-0500][269.675s] GC(5) Old regions: 0->7
> [2023-12-06T15:22:19.768-0500][269.675s] GC(5) Humongous regions: 1->1
> [2023-12-06T15:22:19.768-0500][269.675s] GC(5) Metaspace: 58634K->58634K(1101824K)
> [2023-12-06T15:22:19.768-0500][269.675s] GC(5) Pause Young (Concurrent Start) (Metadata GC Threshold) 360M->83M(8192M) 51.727ms
> [2023-12-06T15:22:19.768-0500][269.675s] GC(5) User=0.16s Sys=0.06s Real=0.05s
> [2023-12-06T15:22:19.768-0500][269.675s] GC(6) Concurrent Cycle
> [2023-12-06T15:22:19.768-0500][269.675s] GC(6) Concurrent Clear Claimed Marks
> [2023-12-06T15:22:19.768-0500][269.675s] GC(6) Concurrent Clear Claimed Marks 0.096ms
> [2023-12-06T15:22:19.768-0500][269.675s] GC(6) Concurrent Scan Root Regions
> [2023-12-06T15:22:19.774-0500][269.681s] GC(6) Concurrent Scan Root Regions 5.973ms
> [2023-12-06T15:22:19.774-0500][269.681s] GC(6) Concurrent Mark (269.681s)
> [2023-12-06T15:22:19.774-0500][269.681s] GC(6) Concurrent Mark From Roots
> [2023-12-06T15:22:19.774-0500][269.681s] GC(6) Using 4 workers of 4 for marking
> [2023-12-06T15:22:19.785-0500][269.692s] GC(6) Concurrent Mark From Roots 11.003ms
> [2023-12-06T15:22:19.785-0500][269.693s] GC(6) Concurrent Preclean
> [2023-12-06T15:22:19.785-0500][269.693s] GC(6) Concurrent Preclean 0.155ms
> [2023-12-06T15:22:19.785-0500][269.693s] GC(6) Concurrent Mark (269.681s, 269.693s) 11.282ms
> [2023-12-06T15:22:19.785-0500][269.693s] GC(6) Pause Remark
> [2023-12-06T15:22:19.790-0500][269.698s] GC(6) Cleaned string and symbol table, strings: 33326 processed, 0 removed, symbols: 168206 processed, 723 removed
> [2023-12-06T15:22:19.791-0500][269.698s] GC(6) Pause Remark 86M->86M(8192M) 5.372ms
> [2023-12-06T15:22:19.791-0500][269.698s] GC(6) User=0.05s Sys=0.02s Real=0.01s
> [2023-12-06T15:22:19.791-0500][269.698s] GC(6) Concurrent Rebuild Remembered Sets
> [2023-12-06T15:22:19.796-0500][269.704s] GC(6) Concurrent Rebuild Remembered Sets 5.249ms
> [2023-12-06T15:22:19.796-0500][269.704s] GC(6) Pause Cleanup
> [2023-12-06T15:22:19.797-0500][269.704s] GC(6) Pause Cleanup 86M->86M(8192M) 0.620ms
> [2023-12-06T15:22:19.797-0500][269.704s] GC(6) User=0.00s Sys=0.00s Real=0.00s
> [2023-12-06T15:22:19.797-0500][269.705s] GC(6) Concurrent Cleanup for Next Mark
> [2023-12-06T15:22:19.803-0500][269.710s] GC(6) Concurrent Cleanup for Next Mark 5.585ms
> [2023-12-06T15:22:19.803-0500][269.710s] GC(6) Concurrent Cycle 34.946ms
> [2023-12-06T15:23:01.472-0500][311.391s] GC(7) Pause Young (Normal) (G1 Evacuation Pause)
> [2023-12-06T15:23:01.472-0500][311.391s] GC(7) Using 15 workers of 15 for evacuation
> [2023-12-06T15:23:01.488-0500][311.406s] GC(7)   Pre Evacuate Collection Set: 0.0ms
> [2023-12-06T15:23:01.488-0500][311.406s] GC(7)   Evacuate Collection Set: 14.6ms
> [2023-12-06T15:23:01.488-0500][311.406s] GC(7)   Post Evacuate Collection Set: 0.8ms
> [2023-12-06T15:23:01.488-0500][311.406s] GC(7)   Other: 0.2ms
> [2023-12-06T15:23:01.488-0500][311.406s] GC(7) Eden regions: 89->0(97)
> [2023-12-06T15:23:01.488-0500][311.406s] GC(7) Survivor regions: 13->5(13)
> [2023-12-06T15:23:01.488-0500][311.406s] GC(7) Old regions: 7->20
> [2023-12-06T15:23:01.488-0500][311.407s] GC(7) Humongous regions: 1->1
> [2023-12-06T15:23:01.488-0500][311.407s] GC(7) Metaspace: 62409K->62409K(1105920K)
> [2023-12-06T15:23:01.488-0500][311.407s] GC(7) Pause Young (Normal) (G1 Evacuation Pause) 439M->98M(8192M) 15.837ms
> [2023-12-06T15:23:01.488-0500][311.407s] GC(7) User=0.11s Sys=0.02s Real=0.02s
> [2023-12-06T15:23:02.175-0500][312.086s] GC(8) Pause Young (Normal) (G1 Evacuation Pause)
> [2023-12-06T15:23:02.175-0500][312.086s] GC(8) Using 15 workers of 15 for evacuation
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8)   Pre Evacuate Collection Set: 0.0ms
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8)   Evacuate Collection Set: 2.6ms
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8)   Post Evacuate Collection Set: 0.6ms
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8)   Other: 0.2ms
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8) Eden regions: 97->0(168)
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8) Survivor regions: 5->7(13)
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8) Old regions: 20->20
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8) Humongous regions: 1->1
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8) Metaspace: 62459K->62459K(1105920K)
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8) Pause Young (Normal) (G1 Evacuation Pause) 486M->107M(8192M) 3.651ms
> [2023-12-06T15:23:02.175-0500][312.090s] GC(8) User=0.00s Sys=0.00s Real=0.00s
> [2023-12-06T15:23:03.377-0500][313.300s] GC(9) Pause Young (Normal) (G1 Evacuation Pause)
> [2023-12-06T15:23:03.377-0500][313.300s] GC(9) Using 15 workers of 15 for evacuation
> [2023-12-06T15:23:03.393-0500][313.308s] GC(9)   Pre Evacuate Collection Set: 0.0ms
> [2023-12-06T15:23:03.393-0500][313.308s] GC(9)   Evacuate Collection Set: 7.5ms
> [2023-12-06T15:23:03.393-0500][313.308s] GC(9)   Post Evacuate Collection Set: 0.7ms
> [2023-12-06T15:23:03.393-0500][313.308s] GC(9)   Other: 0.2ms
> [2023-12-06T15:23:03.393-0500][313.308s] GC(9) Eden regions: 168->0(409)
> [2023-12-06T15:23:03.393-0500][313.308s] GC(9) Survivor regions: 7->10(22)
> [2023-12-06T15:23:03.393-0500][313.308s] GC(9) Old regions: 20->20
> [2023-12-06T15:23:03.393-0500][313.308s] GC(9) Humongous regions: 5->5
> [2023-12-06T15:23:03.393-0500][313.308s] GC(9) Metaspace: 62488K->62488K(1105920K)
> [2023-12-06T15:23:03.393-0500][313.309s] GC(9) Pause Young (Normal) (G1 Evacuation Pause) 795M->135M(8192M) 8.645ms
> [2023-12-06T15:23:03.393-0500][313.309s] GC(9) User=0.11s Sys=0.02s Real=0.02s
> [2023-12-06T15:23:06.400-0500][316.311s] GC(10) Pause Young (Normal) (G1 Evacuation Pause)
> [2023-12-06T15:23:06.400-0500][316.311s] GC(10) Using 15 workers of 15 for evacuation
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10)   Pre Evacuate Collection Set: 0.0ms
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10)   Evacuate Collection Set: 13.4ms
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10)   Post Evacuate Collection Set: 0.8ms
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10)   Other: 0.3ms
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10) Eden regions: 409->0(415)
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10) Survivor regions: 10->22(53)
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10) Old regions: 20->20
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10) Humongous regions: 13->9
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10) Metaspace: 62492K->62492K(1105920K)
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10) Pause Young (Normal) (G1 Evacuation Pause) 1803M->198M(8192M) 14.647ms
> [2023-12-06T15:23:06.416-0500][316.325s] GC(10) User=0.11s Sys=0.02s Real=0.02s
> [2023-12-06T15:23:12.186-0500][322.099s] GC(11) Pause Young (Normal) (G1 Evacuation Pause)
> [2023-12-06T15:23:12.186-0500][322.099s] GC(11) Using 15 workers of 15 for evacuation
> [2023-12-06T15:23:12.194-0500][322.104s] GC(11)   Pre Evacuate Collection Set: 0.0ms
> [2023-12-06T15:23:12.194-0500][322.104s] GC(11)   Evacuate Collection Set: 4.0ms
> [2023-12-06T15:23:12.194-0500][322.104s] GC(11)   Post Evacuate Collection Set: 0.8ms
> [2023-12-06T15:23:12.194-0500][322.105s] GC(11)   Other: 0.2ms
> [2023-12-06T15:23:12.194-0500][322.105s] GC(11) Eden regions: 415->0(1220)
> [2023-12-06T15:23:12.194-0500][322.105s] GC(11) Survivor regions: 22->8(55)
> [2023-12-06T15:23:12.194-0500][322.105s] GC(11) Old regions: 20->20
> [2023-12-06T15:23:12.194-0500][322.105s] GC(11) Humongous regions: 9->1
> [2023-12-06T15:23:12.194-0500][322.105s] GC(11) Metaspace: 63800K->63800K(1107968K)
> [2023-12-06T15:23:12.194-0500][322.105s] GC(11) Pause Young (Normal) (G1 Evacuation Pause) 1858M->109M(8192M) 5.204ms
> [2023-12-06T15:23:12.194-0500][322.105s] GC(11) User=0.00s Sys=0.00s Real=0.01s
> [2023-12-06T15:29:25.357-0500][695.278s] GC(12) Pause Young (Normal) (G1 Evacuation Pause)
> [2023-12-06T15:29:25.357-0500][695.278s] GC(12) Using 15 workers of 15 for evacuation
> [2023-12-06T15:29:25.364-0500][695.286s] GC(12)   Pre Evacuate Collection Set: 0.0ms
> [2023-12-06T15:29:25.364-0500][695.286s] GC(12)   Evacuate Collection Set: 6.6ms
> [2023-12-06T15:29:25.364-0500][695.286s] GC(12)   Post Evacuate Collection Set: 1.2ms
> [2023-12-06T15:29:25.364-0500][695.286s] GC(12)   Other: 0.4ms
> [2023-12-06T15:29:25.364-0500][695.286s] GC(12) Eden regions: 1220->0(1227)
> [2023-12-06T15:29:25.364-0500][695.286s] GC(12) Survivor regions: 8->1(154)
> [2023-12-06T15:29:25.364-0500][695.286s] GC(12) Old regions: 20->20
> [2023-12-06T15:29:25.364-0500][695.286s] GC(12) Humongous regions: 13->1
> [2023-12-06T15:29:25.364-0500][695.287s] GC(12) Metaspace: 64809K->64809K(1107968K)
> [2023-12-06T15:29:25.364-0500][695.287s] GC(12) Pause Young (Normal) (G1 Evacuation Pause) 5037M->84M(8192M) 8.522ms
> [2023-12-06T15:29:25.364-0500][695.287s] GC(12) User=0.00s Sys=0.00s Real=0.01s
>
>
>
> On Wed, Dec 6, 2023 at 2:51 PM Vince McMahon <
> sippingonesandzeros@gmail.com> wrote:
>
>> Gus, that is possible.  How best to fix it?
>>
>> On Wed, Dec 6, 2023 at 12:05 PM Gus Heck <gu...@gmail.com> wrote:
>>
>>> Looks like your client crashed out while trying to receive the response
>>> perhaps?
>>>
>>> Caused by: java.io.IOException: An established connection was aborted
>>> by the software in your host machine
>>>     at sun.nio.ch.SocketDispatcher.writev0(Native Method) ~[?:?]
>>>     at sun.nio.ch.SocketDispatcher.writev(Unknown Source) ~[?:?]
>>>     at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
>>>     at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
>>>     at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[?:?]
>>>     at java.nio.channels.SocketChannel.write(Unknown Source) ~[?:?]
>>>     at org.eclipse.jetty.io
>>> .ChannelEndPoint.flush(ChannelEndPoint.java:273)
>>> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>>>     at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
>>> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>>>
>>> On Wed, Dec 6, 2023 at 11:29 AM Vince McMahon <
>>> sippingonesandzeros@gmail.com>
>>> wrote:
>>>
>>> > Thanks for the suggestion.
>>> >
>>> > 10 rows work for full-Import and Solr count query confirm 10 imported.
>>> :)
>>> >
>>> > I've tried with debug=false and no improvement on larger number of
>>> > rows, such as 755000.   I reduced the source side from 1 million rows
>>> to
>>> > just 755000 to test volume; borrowing the idea of "10 rows". :)
>>> >
>>> > BTW, is there a place to specify the BatchSize for full-import?
>>> > full-import doesn't seem to do 20,000 at a time regardless of the
>>> batchSize
>>> > specified in JDBC driver.
>>> >
>>> > The /dataimport in solrconfig.xml block is
>>> >
>>> >     <requestDispatcher>
>>> >         <httpCaching never304="true" />
>>> >     </requestDispatcher>
>>> > *    <requestHandler name="/dataimport"
>>> class="solr.DataImportHandler">*
>>> > *        <lst name="defaults">*
>>> > *            <str name="config">db-data-config.xml</str>*
>>> > *        </lst>*
>>> > *    </requestHandler>*
>>> >     <requestHandler name="/select" class="solr.SearchHandler">
>>> >         <lst name="defaults">
>>> >             <str name="echoParams">explicit</str>
>>> >             <int name="rows">10</int>
>>> >         </lst>
>>> >     </requestHandler>
>>> >     <requestHandler name="/query" class="solr.SearchHandler">
>>> >         <lst name="defaults">
>>> >             <str name="echoParams">explicit</str>
>>> >             <str name="wt">json</str>
>>> >             <str name="indent">true</str>
>>> >         </lst>
>>> >     </requestHandler>
>>> >
>>> >
>>> > >
>>> >
>>>
>>>
>>> --
>>> http://www.needhamsoftware.com (work)
>>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
>>>
>>

Re: how to fix full processing disrupted in solr 8.11

Posted by Vince McMahon <si...@gmail.com>.
so, I've incrased the SOLR_JAVA_MEM by 8 times.  That memory increase is
from solr status command.  But, no luck.  the Indexing process using
full-import with Clean and Commit option says:
*Indexing completed. Added/Updated: 915000 documents. Deleted 0 documents.
(Duration: 20s)*

The solr count query says, 5000 rows.  How to get the rest of the 915000
indexed?

{
  "responseHeader":{
    "status":0,
    "QTime":0,
    "params":{
      "q":"*:*",
      "indent":"true",
      "q.op":"OR",
      "rows":"0",
      "_":"1701894264264"}},
  "response":{"numFound":5000,"start":0,"numFoundExact":true,"docs":[]
  }}


the solr_gc.log says these:

[2023-12-06T15:17:50.106-0500][0.012s] Heap region size: 4M
[2023-12-06T15:17:50.132-0500][0.047s] Using G1
[2023-12-06T15:17:50.132-0500][0.047s] Heap address:
0x0000000600000000, size: 8192 MB, Compressed Oops mode: Zero based,
Oop shift amount: 3
[2023-12-06T15:17:51.119-0500][1.034s] GC(0) Pause Young (Concurrent
Start) (Metadata GC Threshold)
[2023-12-06T15:17:51.119-0500][1.034s] GC(0) Using 15 workers of 15
for evacuation
[2023-12-06T15:17:51.134-0500][1.042s] GC(0)   Pre Evacuate Collection
Set: 0.0ms
[2023-12-06T15:17:51.134-0500][1.042s] GC(0)   Evacuate Collection Set: 6.8ms
[2023-12-06T15:17:51.134-0500][1.042s] GC(0)   Post Evacuate
Collection Set: 0.7ms
[2023-12-06T15:17:51.134-0500][1.042s] GC(0)   Other: 0.6ms
[2023-12-06T15:17:51.134-0500][1.044s] GC(0) Eden regions: 17->0(98)
[2023-12-06T15:17:51.134-0500][1.044s] GC(0) Survivor regions: 0->4(13)
[2023-12-06T15:17:51.134-0500][1.044s] GC(0) Old regions: 0->0
[2023-12-06T15:17:51.134-0500][1.044s] GC(0) Humongous regions: 0->0
[2023-12-06T15:17:51.134-0500][1.044s] GC(0) Metaspace: 20679K->20679K(1069056K)
[2023-12-06T15:17:51.134-0500][1.044s] GC(0) Pause Young (Concurrent
Start) (Metadata GC Threshold) 64M->14M(8192M) 10.371ms
[2023-12-06T15:17:51.134-0500][1.044s] GC(0) User=0.22s Sys=0.00s Real=0.02s
[2023-12-06T15:17:51.134-0500][1.044s] GC(1) Concurrent Cycle
[2023-12-06T15:17:51.134-0500][1.044s] GC(1) Concurrent Clear Claimed Marks
[2023-12-06T15:17:51.134-0500][1.044s] GC(1) Concurrent Clear Claimed
Marks 0.019ms
[2023-12-06T15:17:51.134-0500][1.044s] GC(1) Concurrent Scan Root Regions
[2023-12-06T15:17:51.134-0500][1.046s] GC(1) Concurrent Scan Root
Regions 1.610ms
[2023-12-06T15:17:51.134-0500][1.046s] GC(1) Concurrent Mark (1.046s)
[2023-12-06T15:17:51.134-0500][1.046s] GC(1) Concurrent Mark From Roots
[2023-12-06T15:17:51.134-0500][1.046s] GC(1) Using 4 workers of 4 for marking
[2023-12-06T15:17:51.134-0500][1.048s] GC(1) Concurrent Mark From Roots 1.961ms
[2023-12-06T15:17:51.134-0500][1.048s] GC(1) Concurrent Preclean
[2023-12-06T15:17:51.134-0500][1.048s] GC(1) Concurrent Preclean 0.091ms
[2023-12-06T15:17:51.134-0500][1.048s] GC(1) Concurrent Mark (1.046s,
1.048s) 2.129ms
[2023-12-06T15:17:51.134-0500][1.048s] GC(1) Pause Remark
[2023-12-06T15:17:51.134-0500][1.049s] GC(1) Cleaned string and symbol
table, strings: 10128 processed, 0 removed, symbols: 62420 processed,
29 removed
[2023-12-06T15:17:51.134-0500][1.049s] GC(1) Pause Remark
18M->18M(8192M) 0.857ms
[2023-12-06T15:17:51.134-0500][1.049s] GC(1) User=0.00s Sys=0.00s Real=0.00s
[2023-12-06T15:17:51.134-0500][1.049s] GC(1) Concurrent Rebuild Remembered Sets
[2023-12-06T15:17:51.134-0500][1.049s] GC(1) Concurrent Rebuild
Remembered Sets 0.190ms
[2023-12-06T15:17:51.134-0500][1.050s] GC(1) Pause Cleanup
[2023-12-06T15:17:51.134-0500][1.050s] GC(1) Pause Cleanup
18M->18M(8192M) 0.107ms
[2023-12-06T15:17:51.134-0500][1.050s] GC(1) User=0.00s Sys=0.00s Real=0.00s
[2023-12-06T15:17:51.134-0500][1.050s] GC(1) Concurrent Cleanup for Next Mark
[2023-12-06T15:17:51.134-0500][1.054s] GC(1) Concurrent Cleanup for
Next Mark 4.341ms
[2023-12-06T15:17:51.134-0500][1.054s] GC(1) Concurrent Cycle 9.947ms
[2023-12-06T15:17:52.361-0500][2.281s] GC(2) Pause Young (Concurrent
Start) (Metadata GC Threshold)
[2023-12-06T15:17:52.361-0500][2.281s] GC(2) Using 15 workers of 15
for evacuation
[2023-12-06T15:17:52.376-0500][2.291s] GC(2)   Pre Evacuate Collection
Set: 0.0ms
[2023-12-06T15:17:52.376-0500][2.291s] GC(2)   Evacuate Collection Set: 8.6ms
[2023-12-06T15:17:52.376-0500][2.291s] GC(2)   Post Evacuate
Collection Set: 1.3ms
[2023-12-06T15:17:52.376-0500][2.291s] GC(2)   Other: 0.2ms
[2023-12-06T15:17:52.376-0500][2.291s] GC(2) Eden regions: 41->0(95)
[2023-12-06T15:17:52.376-0500][2.291s] GC(2) Survivor regions: 4->7(13)
[2023-12-06T15:17:52.376-0500][2.291s] GC(2) Old regions: 0->0
[2023-12-06T15:17:52.376-0500][2.291s] GC(2) Humongous regions: 0->0
[2023-12-06T15:17:52.376-0500][2.291s] GC(2) Metaspace: 35110K->35110K(1081344K)
[2023-12-06T15:17:52.376-0500][2.291s] GC(2) Pause Young (Concurrent
Start) (Metadata GC Threshold) 178M->25M(8192M) 10.315ms
[2023-12-06T15:17:52.376-0500][2.291s] GC(2) User=0.06s Sys=0.03s Real=0.01s
[2023-12-06T15:17:52.376-0500][2.291s] GC(3) Concurrent Cycle
[2023-12-06T15:17:52.376-0500][2.291s] GC(3) Concurrent Clear Claimed Marks
[2023-12-06T15:17:52.376-0500][2.291s] GC(3) Concurrent Clear Claimed
Marks 0.033ms
[2023-12-06T15:17:52.376-0500][2.291s] GC(3) Concurrent Scan Root Regions
[2023-12-06T15:17:52.376-0500][2.293s] GC(3) Concurrent Scan Root
Regions 1.781ms
[2023-12-06T15:17:52.376-0500][2.293s] GC(3) Concurrent Mark (2.293s)
[2023-12-06T15:17:52.376-0500][2.293s] GC(3) Concurrent Mark From Roots
[2023-12-06T15:17:52.376-0500][2.293s] GC(3) Using 4 workers of 4 for marking
[2023-12-06T15:17:52.376-0500][2.296s] GC(3) Concurrent Mark From Roots 2.782ms
[2023-12-06T15:17:52.376-0500][2.296s] GC(3) Concurrent Preclean
[2023-12-06T15:17:52.376-0500][2.296s] GC(3) Concurrent Preclean 0.078ms
[2023-12-06T15:17:52.376-0500][2.296s] GC(3) Concurrent Mark (2.293s,
2.296s) 2.915ms
[2023-12-06T15:17:52.376-0500][2.296s] GC(3) Pause Remark
[2023-12-06T15:17:52.376-0500][2.297s] GC(3) Cleaned string and symbol
table, strings: 15734 processed, 0 removed, symbols: 110518 processed,
48 removed
[2023-12-06T15:17:52.376-0500][2.297s] GC(3) Pause Remark
29M->29M(8192M) 1.060ms
[2023-12-06T15:17:52.376-0500][2.297s] GC(3) User=0.00s Sys=0.00s Real=0.00s
[2023-12-06T15:17:52.376-0500][2.297s] GC(3) Concurrent Rebuild Remembered Sets
[2023-12-06T15:17:52.376-0500][2.297s] GC(3) Concurrent Rebuild
Remembered Sets 0.300ms
[2023-12-06T15:17:52.376-0500][2.298s] GC(3) Pause Cleanup
[2023-12-06T15:17:52.376-0500][2.298s] GC(3) Pause Cleanup
29M->29M(8192M) 0.082ms
[2023-12-06T15:17:52.376-0500][2.298s] GC(3) User=0.00s Sys=0.00s Real=0.00s
[2023-12-06T15:17:52.376-0500][2.298s] GC(3) Concurrent Cleanup for Next Mark
[2023-12-06T15:17:52.392-0500][2.301s] GC(3) Concurrent Cleanup for
Next Mark 3.149ms
[2023-12-06T15:17:52.392-0500][2.301s] GC(3) Concurrent Cycle 9.980ms
[2023-12-06T15:17:54.272-0500][4.191s] GC(4) Pause Young (Normal) (G1
Evacuation Pause)
[2023-12-06T15:17:54.272-0500][4.191s] GC(4) Using 15 workers of 15
for evacuation
[2023-12-06T15:17:54.286-0500][4.204s] GC(4)   Pre Evacuate Collection
Set: 0.0ms
[2023-12-06T15:17:54.286-0500][4.204s] GC(4)   Evacuate Collection Set: 11.3ms
[2023-12-06T15:17:54.286-0500][4.204s] GC(4)   Post Evacuate
Collection Set: 1.5ms
[2023-12-06T15:17:54.286-0500][4.204s] GC(4)   Other: 0.2ms
[2023-12-06T15:17:54.286-0500][4.204s] GC(4) Eden regions: 95->0(90)
[2023-12-06T15:17:54.286-0500][4.204s] GC(4) Survivor regions: 7->12(13)
[2023-12-06T15:17:54.286-0500][4.204s] GC(4) Old regions: 0->0
[2023-12-06T15:17:54.286-0500][4.204s] GC(4) Humongous regions: 0->0
[2023-12-06T15:17:54.286-0500][4.204s] GC(4) Metaspace: 53343K->53343K(1097728K)
[2023-12-06T15:17:54.286-0500][4.204s] GC(4) Pause Young (Normal) (G1
Evacuation Pause) 405M->45M(8192M) 13.409ms
[2023-12-06T15:17:54.286-0500][4.204s] GC(4) User=0.06s Sys=0.05s Real=0.01s
[2023-12-06T15:22:19.716-0500][269.623s] GC(5) Pause Young (Concurrent
Start) (Metadata GC Threshold)
[2023-12-06T15:22:19.716-0500][269.623s] GC(5) Using 15 workers of 15
for evacuation
[2023-12-06T15:22:19.767-0500][269.675s] GC(5)   Pre Evacuate
Collection Set: 0.1ms
[2023-12-06T15:22:19.768-0500][269.675s] GC(5)   Evacuate Collection Set: 49.8ms
[2023-12-06T15:22:19.768-0500][269.675s] GC(5)   Post Evacuate
Collection Set: 1.2ms
[2023-12-06T15:22:19.768-0500][269.675s] GC(5)   Other: 0.4ms
[2023-12-06T15:22:19.768-0500][269.675s] GC(5) Eden regions: 78->0(89)
[2023-12-06T15:22:19.768-0500][269.675s] GC(5) Survivor regions: 12->13(13)
[2023-12-06T15:22:19.768-0500][269.675s] GC(5) Old regions: 0->7
[2023-12-06T15:22:19.768-0500][269.675s] GC(5) Humongous regions: 1->1
[2023-12-06T15:22:19.768-0500][269.675s] GC(5) Metaspace:
58634K->58634K(1101824K)
[2023-12-06T15:22:19.768-0500][269.675s] GC(5) Pause Young (Concurrent
Start) (Metadata GC Threshold) 360M->83M(8192M) 51.727ms
[2023-12-06T15:22:19.768-0500][269.675s] GC(5) User=0.16s Sys=0.06s Real=0.05s
[2023-12-06T15:22:19.768-0500][269.675s] GC(6) Concurrent Cycle
[2023-12-06T15:22:19.768-0500][269.675s] GC(6) Concurrent Clear Claimed Marks
[2023-12-06T15:22:19.768-0500][269.675s] GC(6) Concurrent Clear
Claimed Marks 0.096ms
[2023-12-06T15:22:19.768-0500][269.675s] GC(6) Concurrent Scan Root Regions
[2023-12-06T15:22:19.774-0500][269.681s] GC(6) Concurrent Scan Root
Regions 5.973ms
[2023-12-06T15:22:19.774-0500][269.681s] GC(6) Concurrent Mark (269.681s)
[2023-12-06T15:22:19.774-0500][269.681s] GC(6) Concurrent Mark From Roots
[2023-12-06T15:22:19.774-0500][269.681s] GC(6) Using 4 workers of 4 for marking
[2023-12-06T15:22:19.785-0500][269.692s] GC(6) Concurrent Mark From
Roots 11.003ms
[2023-12-06T15:22:19.785-0500][269.693s] GC(6) Concurrent Preclean
[2023-12-06T15:22:19.785-0500][269.693s] GC(6) Concurrent Preclean 0.155ms
[2023-12-06T15:22:19.785-0500][269.693s] GC(6) Concurrent Mark
(269.681s, 269.693s) 11.282ms
[2023-12-06T15:22:19.785-0500][269.693s] GC(6) Pause Remark
[2023-12-06T15:22:19.790-0500][269.698s] GC(6) Cleaned string and
symbol table, strings: 33326 processed, 0 removed, symbols: 168206
processed, 723 removed
[2023-12-06T15:22:19.791-0500][269.698s] GC(6) Pause Remark
86M->86M(8192M) 5.372ms
[2023-12-06T15:22:19.791-0500][269.698s] GC(6) User=0.05s Sys=0.02s Real=0.01s
[2023-12-06T15:22:19.791-0500][269.698s] GC(6) Concurrent Rebuild
Remembered Sets
[2023-12-06T15:22:19.796-0500][269.704s] GC(6) Concurrent Rebuild
Remembered Sets 5.249ms
[2023-12-06T15:22:19.796-0500][269.704s] GC(6) Pause Cleanup
[2023-12-06T15:22:19.797-0500][269.704s] GC(6) Pause Cleanup
86M->86M(8192M) 0.620ms
[2023-12-06T15:22:19.797-0500][269.704s] GC(6) User=0.00s Sys=0.00s Real=0.00s
[2023-12-06T15:22:19.797-0500][269.705s] GC(6) Concurrent Cleanup for Next Mark
[2023-12-06T15:22:19.803-0500][269.710s] GC(6) Concurrent Cleanup for
Next Mark 5.585ms
[2023-12-06T15:22:19.803-0500][269.710s] GC(6) Concurrent Cycle 34.946ms
[2023-12-06T15:23:01.472-0500][311.391s] GC(7) Pause Young (Normal)
(G1 Evacuation Pause)
[2023-12-06T15:23:01.472-0500][311.391s] GC(7) Using 15 workers of 15
for evacuation
[2023-12-06T15:23:01.488-0500][311.406s] GC(7)   Pre Evacuate
Collection Set: 0.0ms
[2023-12-06T15:23:01.488-0500][311.406s] GC(7)   Evacuate Collection Set: 14.6ms
[2023-12-06T15:23:01.488-0500][311.406s] GC(7)   Post Evacuate
Collection Set: 0.8ms
[2023-12-06T15:23:01.488-0500][311.406s] GC(7)   Other: 0.2ms
[2023-12-06T15:23:01.488-0500][311.406s] GC(7) Eden regions: 89->0(97)
[2023-12-06T15:23:01.488-0500][311.406s] GC(7) Survivor regions: 13->5(13)
[2023-12-06T15:23:01.488-0500][311.406s] GC(7) Old regions: 7->20
[2023-12-06T15:23:01.488-0500][311.407s] GC(7) Humongous regions: 1->1
[2023-12-06T15:23:01.488-0500][311.407s] GC(7) Metaspace:
62409K->62409K(1105920K)
[2023-12-06T15:23:01.488-0500][311.407s] GC(7) Pause Young (Normal)
(G1 Evacuation Pause) 439M->98M(8192M) 15.837ms
[2023-12-06T15:23:01.488-0500][311.407s] GC(7) User=0.11s Sys=0.02s Real=0.02s
[2023-12-06T15:23:02.175-0500][312.086s] GC(8) Pause Young (Normal)
(G1 Evacuation Pause)
[2023-12-06T15:23:02.175-0500][312.086s] GC(8) Using 15 workers of 15
for evacuation
[2023-12-06T15:23:02.175-0500][312.090s] GC(8)   Pre Evacuate
Collection Set: 0.0ms
[2023-12-06T15:23:02.175-0500][312.090s] GC(8)   Evacuate Collection Set: 2.6ms
[2023-12-06T15:23:02.175-0500][312.090s] GC(8)   Post Evacuate
Collection Set: 0.6ms
[2023-12-06T15:23:02.175-0500][312.090s] GC(8)   Other: 0.2ms
[2023-12-06T15:23:02.175-0500][312.090s] GC(8) Eden regions: 97->0(168)
[2023-12-06T15:23:02.175-0500][312.090s] GC(8) Survivor regions: 5->7(13)
[2023-12-06T15:23:02.175-0500][312.090s] GC(8) Old regions: 20->20
[2023-12-06T15:23:02.175-0500][312.090s] GC(8) Humongous regions: 1->1
[2023-12-06T15:23:02.175-0500][312.090s] GC(8) Metaspace:
62459K->62459K(1105920K)
[2023-12-06T15:23:02.175-0500][312.090s] GC(8) Pause Young (Normal)
(G1 Evacuation Pause) 486M->107M(8192M) 3.651ms
[2023-12-06T15:23:02.175-0500][312.090s] GC(8) User=0.00s Sys=0.00s Real=0.00s
[2023-12-06T15:23:03.377-0500][313.300s] GC(9) Pause Young (Normal)
(G1 Evacuation Pause)
[2023-12-06T15:23:03.377-0500][313.300s] GC(9) Using 15 workers of 15
for evacuation
[2023-12-06T15:23:03.393-0500][313.308s] GC(9)   Pre Evacuate
Collection Set: 0.0ms
[2023-12-06T15:23:03.393-0500][313.308s] GC(9)   Evacuate Collection Set: 7.5ms
[2023-12-06T15:23:03.393-0500][313.308s] GC(9)   Post Evacuate
Collection Set: 0.7ms
[2023-12-06T15:23:03.393-0500][313.308s] GC(9)   Other: 0.2ms
[2023-12-06T15:23:03.393-0500][313.308s] GC(9) Eden regions: 168->0(409)
[2023-12-06T15:23:03.393-0500][313.308s] GC(9) Survivor regions: 7->10(22)
[2023-12-06T15:23:03.393-0500][313.308s] GC(9) Old regions: 20->20
[2023-12-06T15:23:03.393-0500][313.308s] GC(9) Humongous regions: 5->5
[2023-12-06T15:23:03.393-0500][313.308s] GC(9) Metaspace:
62488K->62488K(1105920K)
[2023-12-06T15:23:03.393-0500][313.309s] GC(9) Pause Young (Normal)
(G1 Evacuation Pause) 795M->135M(8192M) 8.645ms
[2023-12-06T15:23:03.393-0500][313.309s] GC(9) User=0.11s Sys=0.02s Real=0.02s
[2023-12-06T15:23:06.400-0500][316.311s] GC(10) Pause Young (Normal)
(G1 Evacuation Pause)
[2023-12-06T15:23:06.400-0500][316.311s] GC(10) Using 15 workers of 15
for evacuation
[2023-12-06T15:23:06.416-0500][316.325s] GC(10)   Pre Evacuate
Collection Set: 0.0ms
[2023-12-06T15:23:06.416-0500][316.325s] GC(10)   Evacuate Collection
Set: 13.4ms
[2023-12-06T15:23:06.416-0500][316.325s] GC(10)   Post Evacuate
Collection Set: 0.8ms
[2023-12-06T15:23:06.416-0500][316.325s] GC(10)   Other: 0.3ms
[2023-12-06T15:23:06.416-0500][316.325s] GC(10) Eden regions: 409->0(415)
[2023-12-06T15:23:06.416-0500][316.325s] GC(10) Survivor regions: 10->22(53)
[2023-12-06T15:23:06.416-0500][316.325s] GC(10) Old regions: 20->20
[2023-12-06T15:23:06.416-0500][316.325s] GC(10) Humongous regions: 13->9
[2023-12-06T15:23:06.416-0500][316.325s] GC(10) Metaspace:
62492K->62492K(1105920K)
[2023-12-06T15:23:06.416-0500][316.325s] GC(10) Pause Young (Normal)
(G1 Evacuation Pause) 1803M->198M(8192M) 14.647ms
[2023-12-06T15:23:06.416-0500][316.325s] GC(10) User=0.11s Sys=0.02s Real=0.02s
[2023-12-06T15:23:12.186-0500][322.099s] GC(11) Pause Young (Normal)
(G1 Evacuation Pause)
[2023-12-06T15:23:12.186-0500][322.099s] GC(11) Using 15 workers of 15
for evacuation
[2023-12-06T15:23:12.194-0500][322.104s] GC(11)   Pre Evacuate
Collection Set: 0.0ms
[2023-12-06T15:23:12.194-0500][322.104s] GC(11)   Evacuate Collection Set: 4.0ms
[2023-12-06T15:23:12.194-0500][322.104s] GC(11)   Post Evacuate
Collection Set: 0.8ms
[2023-12-06T15:23:12.194-0500][322.105s] GC(11)   Other: 0.2ms
[2023-12-06T15:23:12.194-0500][322.105s] GC(11) Eden regions: 415->0(1220)
[2023-12-06T15:23:12.194-0500][322.105s] GC(11) Survivor regions: 22->8(55)
[2023-12-06T15:23:12.194-0500][322.105s] GC(11) Old regions: 20->20
[2023-12-06T15:23:12.194-0500][322.105s] GC(11) Humongous regions: 9->1
[2023-12-06T15:23:12.194-0500][322.105s] GC(11) Metaspace:
63800K->63800K(1107968K)
[2023-12-06T15:23:12.194-0500][322.105s] GC(11) Pause Young (Normal)
(G1 Evacuation Pause) 1858M->109M(8192M) 5.204ms
[2023-12-06T15:23:12.194-0500][322.105s] GC(11) User=0.00s Sys=0.00s Real=0.01s
[2023-12-06T15:29:25.357-0500][695.278s] GC(12) Pause Young (Normal)
(G1 Evacuation Pause)
[2023-12-06T15:29:25.357-0500][695.278s] GC(12) Using 15 workers of 15
for evacuation
[2023-12-06T15:29:25.364-0500][695.286s] GC(12)   Pre Evacuate
Collection Set: 0.0ms
[2023-12-06T15:29:25.364-0500][695.286s] GC(12)   Evacuate Collection Set: 6.6ms
[2023-12-06T15:29:25.364-0500][695.286s] GC(12)   Post Evacuate
Collection Set: 1.2ms
[2023-12-06T15:29:25.364-0500][695.286s] GC(12)   Other: 0.4ms
[2023-12-06T15:29:25.364-0500][695.286s] GC(12) Eden regions: 1220->0(1227)
[2023-12-06T15:29:25.364-0500][695.286s] GC(12) Survivor regions: 8->1(154)
[2023-12-06T15:29:25.364-0500][695.286s] GC(12) Old regions: 20->20
[2023-12-06T15:29:25.364-0500][695.286s] GC(12) Humongous regions: 13->1
[2023-12-06T15:29:25.364-0500][695.287s] GC(12) Metaspace:
64809K->64809K(1107968K)
[2023-12-06T15:29:25.364-0500][695.287s] GC(12) Pause Young (Normal)
(G1 Evacuation Pause) 5037M->84M(8192M) 8.522ms
[2023-12-06T15:29:25.364-0500][695.287s] GC(12) User=0.00s Sys=0.00s Real=0.01s



On Wed, Dec 6, 2023 at 2:51 PM Vince McMahon <si...@gmail.com>
wrote:

> Gus, that is possible.  How best to fix it?
>
> On Wed, Dec 6, 2023 at 12:05 PM Gus Heck <gu...@gmail.com> wrote:
>
>> Looks like your client crashed out while trying to receive the response
>> perhaps?
>>
>> Caused by: java.io.IOException: An established connection was aborted
>> by the software in your host machine
>>     at sun.nio.ch.SocketDispatcher.writev0(Native Method) ~[?:?]
>>     at sun.nio.ch.SocketDispatcher.writev(Unknown Source) ~[?:?]
>>     at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
>>     at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
>>     at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[?:?]
>>     at java.nio.channels.SocketChannel.write(Unknown Source) ~[?:?]
>>     at org.eclipse.jetty.io
>> .ChannelEndPoint.flush(ChannelEndPoint.java:273)
>> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>>     at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
>> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>>
>> On Wed, Dec 6, 2023 at 11:29 AM Vince McMahon <
>> sippingonesandzeros@gmail.com>
>> wrote:
>>
>> > Thanks for the suggestion.
>> >
>> > 10 rows work for full-Import and Solr count query confirm 10 imported.
>> :)
>> >
>> > I've tried with debug=false and no improvement on larger number of
>> > rows, such as 755000.   I reduced the source side from 1 million rows to
>> > just 755000 to test volume; borrowing the idea of "10 rows". :)
>> >
>> > BTW, is there a place to specify the BatchSize for full-import?
>> > full-import doesn't seem to do 20,000 at a time regardless of the
>> batchSize
>> > specified in JDBC driver.
>> >
>> > The /dataimport in solrconfig.xml block is
>> >
>> >     <requestDispatcher>
>> >         <httpCaching never304="true" />
>> >     </requestDispatcher>
>> > *    <requestHandler name="/dataimport" class="solr.DataImportHandler">*
>> > *        <lst name="defaults">*
>> > *            <str name="config">db-data-config.xml</str>*
>> > *        </lst>*
>> > *    </requestHandler>*
>> >     <requestHandler name="/select" class="solr.SearchHandler">
>> >         <lst name="defaults">
>> >             <str name="echoParams">explicit</str>
>> >             <int name="rows">10</int>
>> >         </lst>
>> >     </requestHandler>
>> >     <requestHandler name="/query" class="solr.SearchHandler">
>> >         <lst name="defaults">
>> >             <str name="echoParams">explicit</str>
>> >             <str name="wt">json</str>
>> >             <str name="indent">true</str>
>> >         </lst>
>> >     </requestHandler>
>> >
>> >
>> > >
>> >
>>
>>
>> --
>> http://www.needhamsoftware.com (work)
>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
>>
>

Re: how to fix full processing disrupted in solr 8.11

Posted by Vince McMahon <si...@gmail.com>.
Gus, that is possible.  How best to fix it?

On Wed, Dec 6, 2023 at 12:05 PM Gus Heck <gu...@gmail.com> wrote:

> Looks like your client crashed out while trying to receive the response
> perhaps?
>
> Caused by: java.io.IOException: An established connection was aborted
> by the software in your host machine
>     at sun.nio.ch.SocketDispatcher.writev0(Native Method) ~[?:?]
>     at sun.nio.ch.SocketDispatcher.writev(Unknown Source) ~[?:?]
>     at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
>     at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
>     at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[?:?]
>     at java.nio.channels.SocketChannel.write(Unknown Source) ~[?:?]
>     at org.eclipse.jetty.io
> .ChannelEndPoint.flush(ChannelEndPoint.java:273)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>
> On Wed, Dec 6, 2023 at 11:29 AM Vince McMahon <
> sippingonesandzeros@gmail.com>
> wrote:
>
> > Thanks for the suggestion.
> >
> > 10 rows work for full-Import and Solr count query confirm 10 imported. :)
> >
> > I've tried with debug=false and no improvement on larger number of
> > rows, such as 755000.   I reduced the source side from 1 million rows to
> > just 755000 to test volume; borrowing the idea of "10 rows". :)
> >
> > BTW, is there a place to specify the BatchSize for full-import?
> > full-import doesn't seem to do 20,000 at a time regardless of the
> batchSize
> > specified in JDBC driver.
> >
> > The /dataimport in solrconfig.xml block is
> >
> >     <requestDispatcher>
> >         <httpCaching never304="true" />
> >     </requestDispatcher>
> > *    <requestHandler name="/dataimport" class="solr.DataImportHandler">*
> > *        <lst name="defaults">*
> > *            <str name="config">db-data-config.xml</str>*
> > *        </lst>*
> > *    </requestHandler>*
> >     <requestHandler name="/select" class="solr.SearchHandler">
> >         <lst name="defaults">
> >             <str name="echoParams">explicit</str>
> >             <int name="rows">10</int>
> >         </lst>
> >     </requestHandler>
> >     <requestHandler name="/query" class="solr.SearchHandler">
> >         <lst name="defaults">
> >             <str name="echoParams">explicit</str>
> >             <str name="wt">json</str>
> >             <str name="indent">true</str>
> >         </lst>
> >     </requestHandler>
> >
> >
> > >
> >
>
>
> --
> http://www.needhamsoftware.com (work)
> https://a.co/d/b2sZLD9 (my fantasy fiction book)
>

Re: how to fix full processing disrupted in solr 8.11

Posted by Gus Heck <gu...@gmail.com>.
Looks like your client crashed out while trying to receive the response
perhaps?

Caused by: java.io.IOException: An established connection was aborted
by the software in your host machine
    at sun.nio.ch.SocketDispatcher.writev0(Native Method) ~[?:?]
    at sun.nio.ch.SocketDispatcher.writev(Unknown Source) ~[?:?]
    at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
    at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
    at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[?:?]
    at java.nio.channels.SocketChannel.write(Unknown Source) ~[?:?]
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:273)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]

On Wed, Dec 6, 2023 at 11:29 AM Vince McMahon <si...@gmail.com>
wrote:

> Thanks for the suggestion.
>
> 10 rows work for full-Import and Solr count query confirm 10 imported. :)
>
> I've tried with debug=false and no improvement on larger number of
> rows, such as 755000.   I reduced the source side from 1 million rows to
> just 755000 to test volume; borrowing the idea of "10 rows". :)
>
> BTW, is there a place to specify the BatchSize for full-import?
> full-import doesn't seem to do 20,000 at a time regardless of the batchSize
> specified in JDBC driver.
>
> The /dataimport in solrconfig.xml block is
>
>     <requestDispatcher>
>         <httpCaching never304="true" />
>     </requestDispatcher>
> *    <requestHandler name="/dataimport" class="solr.DataImportHandler">*
> *        <lst name="defaults">*
> *            <str name="config">db-data-config.xml</str>*
> *        </lst>*
> *    </requestHandler>*
>     <requestHandler name="/select" class="solr.SearchHandler">
>         <lst name="defaults">
>             <str name="echoParams">explicit</str>
>             <int name="rows">10</int>
>         </lst>
>     </requestHandler>
>     <requestHandler name="/query" class="solr.SearchHandler">
>         <lst name="defaults">
>             <str name="echoParams">explicit</str>
>             <str name="wt">json</str>
>             <str name="indent">true</str>
>         </lst>
>     </requestHandler>
>
>
> >
>


-- 
http://www.needhamsoftware.com (work)
https://a.co/d/b2sZLD9 (my fantasy fiction book)

Re: how to fix full processing disrupted in solr 8.11

Posted by Vince McMahon <si...@gmail.com>.
Thanks for the suggestion.

10 rows work for full-Import and Solr count query confirm 10 imported. :)

I've tried with debug=false and no improvement on larger number of
rows, such as 755000.   I reduced the source side from 1 million rows to
just 755000 to test volume; borrowing the idea of "10 rows". :)

BTW, is there a place to specify the BatchSize for full-import?
full-import doesn't seem to do 20,000 at a time regardless of the batchSize
specified in JDBC driver.

The /dataimport in solrconfig.xml block is

    <requestDispatcher>
        <httpCaching never304="true" />
    </requestDispatcher>
*    <requestHandler name="/dataimport" class="solr.DataImportHandler">*
*        <lst name="defaults">*
*            <str name="config">db-data-config.xml</str>*
*        </lst>*
*    </requestHandler>*
    <requestHandler name="/select" class="solr.SearchHandler">
        <lst name="defaults">
            <str name="echoParams">explicit</str>
            <int name="rows">10</int>
        </lst>
    </requestHandler>
    <requestHandler name="/query" class="solr.SearchHandler">
        <lst name="defaults">
            <str name="echoParams">explicit</str>
            <str name="wt">json</str>
            <str name="indent">true</str>
        </lst>
    </requestHandler>


>

Re: how to fix full processing disrupted in solr 8.11

Posted by Mikhail Khludnev <mk...@apache.org>.
Hi
My blind guess is that the exception is related to the huge debug=true
output. Regarding wiping indices - have no idea. Some params might be
defined in solrconfig.xml where  /dataimport defined, it's worth to
review it. May another intervening process wipe the index? To summarize:
1. check /dataimport in solrconfig.xml
2. reduce the number or loaded rows to 10
3. and\or try debug=false

On Wed, Dec 6, 2023 at 10:58 AM Vince McMahon <si...@gmail.com>
wrote:

> Hi,
>
> I did a full-import of an entity city and should have Indexed 1,510,000
> documents as in the source.  The UI has the status of "Indexing completed.
> Added/Updated: 1510000 documents. Deleted 0 documents. (Duration: 33s)"
>
> After a while, the source got new documents and I am expecting a second
> additional 160,000 of changed documents.
>
> However, when I query the count, I've got only 10000.  The log has
> error that I can't find answer from google, which is at the end of this
> email.  I am guessing something disrupted the Indexing.  What is it and how
> to correct it?
>
> query count of core p4a
> {
>
>   "responseHeader":{
>     "status":0,
>     "QTime":0,
>     "params":{
>       "q":"*:*",
>       "indent":"true",
>       "q.op":"OR",
>       "rows":"0",
>       "_":"1701848047557"}},
>   "response":{"numFound":10000,"start":0,"numFoundExact":true,"docs":[]
>   }}
>
>
> The config is the following
>
> <dataConfig>
>   <dataSource type="JdbcDataSource"
>               driver="org.postgresql.Driver"
>               url="jdbc:postgresql://localhost/world"
>               user="solrpull"
>               password="WorkingPassword"/>
>   <document>
>     <entity name            ="findDelta"
>             query           ="select id FROM city WHERE update_at >
> TO_TIMESTAMP('${dih.last_index_time}', 'YYYY-MM-DD,  HH24:MI:SS') -
> INTERVAL '180 day' "
>             rootEntity      ="false"
>     >
>     </entity>
>     <entity name            ="city"
>             pk              ="id"
>             query           ="select id, name, update_at FROM city
> WHERE ( '${dataimporter.request.clean}' = 'true'
>             OR update_at > TO_TIMESTAMP('${dih.last_index_time}',
> 'YYYY-MM-DD,  HH24:MI:SS') - INTERVAL '180 day' )"
>     >
>       <field column="id" name="id"/>
>       <field column="name" name="name"/>
>       <field column="update_at" name="update_at"/>
>     </entity>
>   </document></dataConfig>
>
>
> The 2nd time Raw Status-Output is
>
> {
>   "responseHeader": {
>     "status": 0,
>     "QTime": 0
>   },
>   "initArgs": [
>     "defaults",
>     [
>       "config",
>       "db-data-config.xml"
>     ]
>   ],
>   "command": "status",
>   "status": "idle",
>   "importResponse": "",
>   "statusMessages": {
>     "Total Requests made to DataSource": "2",
>     "Total Rows Fetched": "753718",
>     "Total Documents Processed": "376859",
>     "Total Documents Skipped": "0",
>     "Full Dump Started": "2023-12-06 07:36:34",
>     "": "Indexing completed. Added/Updated: 376859 documents. Deleted
> 0 documents.",
>     "Committed": "2023-12-06 07:36:44",
>     "Time taken": "0:0:10.57"
>   }
> }
>
> The log has the error that I can't find from google.
>
>
> 2023-12-06 07:36:34.157 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.DataImporter Loading DIH Configuration: db-data-config.xml
> 2023-12-06 07:36:34.158 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.c.DIHConfiguration id is a required field in SolrSchema .
> But not found in DataConfig
> 2023-12-06 07:36:34.159 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.DataImporter Data Configuration loaded successfully
> 2023-12-06 07:36:34.159 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.DataImporter Starting Full Import
> 2023-12-06 07:36:34.163 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.SimplePropertiesWriter Read dataimport.properties
> 2023-12-06 07:36:34.167 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.JdbcDataSource Creating a connection for entity findDelta
> with URL: jdbc:postgresql://localhost/world
> 2023-12-06 07:36:34.200 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.JdbcDataSource Time taken for getConnection(): 33
> 2023-12-06 07:36:34.871 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.JdbcDataSource Creating a connection for entity city with
> URL: jdbc:postgresql://localhost/world
> 2023-12-06 07:36:34.898 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.JdbcDataSource Time taken for getConnection(): 27
> 2023-12-06 07:36:36.231 INFO  (qtp548482954-66) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
>
> params={core=p4a&debug=true&indent=on&commit=true&name=dataimport&clean=false&rows=9999999&wt=json&command=full-import&_=1701847947387&verbose=true}
> status=0 QTime=0
> 2023-12-06 07:36:36.234 INFO  (qtp548482954-86) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:38.414 INFO  (qtp548482954-86) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:39.927 INFO  (qtp548482954-66) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:40.546 INFO  (qtp548482954-86) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:41.123 INFO  (qtp548482954-66) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:41.623 INFO  (qtp548482954-86) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:41.778 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.DocBuilder Import completed successfully
> 2023-12-06 07:36:42.109 INFO  (qtp548482954-66) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:42.538 INFO  (qtp548482954-86) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:42.980 INFO  (qtp548482954-66) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:43.368 INFO  (qtp548482954-86) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:43.762 INFO  (qtp548482954-66) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:44.133 INFO  (qtp548482954-86) [   x:p4a]
> o.a.s.c.S.Request [p4a]  webapp=/solr path=/dataimport
> params={indent=on&wt=json&command=status&_=1701847947387} status=0
> QTime=0
> 2023-12-06 07:36:44.181 INFO
> (searcherExecutor-45-thread-1-processing-x:p4a) [   x:p4a]
> o.a.s.c.QuerySenderListener QuerySenderListener done.
> 2023-12-06 07:36:44.183 INFO
> (searcherExecutor-45-thread-1-processing-x:p4a) [   x:p4a]
> o.a.s.c.SolrCore [p4a]  Registered new searcher autowarm time: 0 ms
> 2023-12-06 07:36:44.219 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.SimplePropertiesWriter Read dataimport.properties
> 2023-12-06 07:36:44.220 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.SimplePropertiesWriter Wrote last indexed time to
> dataimport.properties
> 2023-12-06 07:36:44.220 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.h.d.DocBuilder Time taken = 0:0:10.57
> 2023-12-06 07:36:44.220 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.u.p.LogUpdateProcessorFactory [p4a]  webapp=/solr
> path=/dataimport
>
> params={core=p4a&debug=true&indent=on&commit=true&name=dataimport&clean=false&rows=9999999&wt=json&command=full-import&_=1701847947387&verbose=true}{add=[8904
> (1784517172815462400), 670 (1784517172823851008), 6054
> (1784517172823851009), 4999 (1784517172823851010), 3956
> (1784517172823851011), 8054 (1784517172823851012), 510
> (1784517172823851013), 357 (1784517172823851014), 9655
> (1784517172823851015), 3990 (1784517172824899584), ... (376859
> adds)],commit=} 0 10065
> 2023-12-06 07:36:44.234 ERROR (qtp548482954-31) [   x:p4a]
> o.a.s.s.HttpSolrCall java.lang.RuntimeException:
> org.eclipse.jetty.io.EofException => java.lang.RuntimeException:
> org.eclipse.jetty.io.EofException
>     at
> org.apache.solr.common.MapWriter$EntryWriter.putNoEx(MapWriter.java:103)
> java.lang.RuntimeException: org.eclipse.jetty.io.EofException
>     at
> org.apache.solr.common.MapWriter$EntryWriter.putNoEx(MapWriter.java:103)
> ~[?:?]
>     at
> org.apache.solr.common.MapWriter$EntryWriter.lambda$getBiConsumer$0(MapWriter.java:161)
> ~[?:?]
>     at
> org.apache.solr.common.SolrInputDocument.lambda$writeMap$0(SolrInputDocument.java:59)
> ~[?:?]
>     at java.util.LinkedHashMap.forEach(Unknown Source) ~[?:?]
>     at
> org.apache.solr.common.SolrInputDocument.writeMap(SolrInputDocument.java:61)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter.writeMap(JsonTextWriter.java:164)
> ~[?:?]
>     at org.apache.solr.common.util.TextWriter.writeMap(TextWriter.java:216)
> ~[?:?]
>     at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:69)
> ~[?:?]
>     at
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter.writeJsonIter(JsonTextWriter.java:194)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter.writeArray(JsonTextWriter.java:277)
> ~[?:?]
>     at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:78)
> ~[?:?]
>     at
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter.writeNamedListAsMapWithDups(JsonTextWriter.java:387)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter.writeNamedList(JsonTextWriter.java:293)
> ~[?:?]
>     at
> org.apache.solr.response.JSONWriter.writeResponse(JSONWriter.java:73)
> ~[?:?]
>     at
> org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:66)
> ~[?:?]
>     at
> org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
> ~[?:?]
>     at
> org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:887)
> ~[?:?]
>     at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:580)
> ~[?:?]
>     at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427)
> ~[?:?]
>     at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:357)
> ~[?:?]
>     at
> org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)
> ~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
> ~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
> ~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
> ~[jetty-security-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
> ~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
> ~[jetty-rewrite-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.Server.handle(Server.java:516)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io
> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io
> .ChannelEndPoint$1.run(ChannelEndPoint.java:104)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at java.lang.Thread.run(Unknown Source) [?:?]
> Caused by: org.eclipse.jetty.io.EofException
>     at org.eclipse.jetty.io
> .ChannelEndPoint.flush(ChannelEndPoint.java:279)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io
> .AbstractEndPoint.write(AbstractEndPoint.java:381)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:831)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:555)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:927)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:999)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor$GzipBufferCB.process(GzipHttpOutputInterceptor.java:416)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor.gzip(GzipHttpOutputInterceptor.java:139)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor.write(GzipHttpOutputInterceptor.java:121)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:285)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:269)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:1054)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.apache.solr.servlet.ServletOutputStreamWrapper.write(ServletOutputStreamWrapper.java:126)
> ~[?:?]
>     at
> org.apache.solr.response.QueryResponseWriterUtil$1.write(QueryResponseWriterUtil.java:54)
> ~[?:?]
>     at java.io.OutputStream.write(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.write(Unknown Source) ~[?:?]
>     at java.io.OutputStreamWriter.write(Unknown Source) ~[?:?]
>     at org.apache.solr.common.util.FastWriter.flush(FastWriter.java:140)
> ~[?:?]
>     at org.apache.solr.common.util.FastWriter.write(FastWriter.java:125)
> ~[?:?]
>     at java.io.Writer.write(Unknown Source) ~[?:?]
>     at org.apache.solr.response.JSONWriter._writeStr(JSONWriter.java:219)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter.writeStr(JsonTextWriter.java:133)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter.writeDate(JsonTextWriter.java:232)
> ~[?:?]
>     at
> org.apache.solr.common.util.TextWriter.writeDate(TextWriter.java:163)
> ~[?:?]
>     at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:61)
> ~[?:?]
>     at
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter$2.put(JsonTextWriter.java:176)
> ~[?:?]
>     at
> org.apache.solr.common.MapWriter$EntryWriter.putNoEx(MapWriter.java:101)
> ~[?:?]
>     ... 55 more
> Caused by: java.io.IOException: An established connection was aborted
> by the software in your host machine
>     at sun.nio.ch.SocketDispatcher.writev0(Native Method) ~[?:?]
>     at sun.nio.ch.SocketDispatcher.writev(Unknown Source) ~[?:?]
>     at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
>     at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
>     at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[?:?]
>     at java.nio.channels.SocketChannel.write(Unknown Source) ~[?:?]
>     at org.eclipse.jetty.io
> .ChannelEndPoint.flush(ChannelEndPoint.java:273)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io
> .AbstractEndPoint.write(AbstractEndPoint.java:381)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:831)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:555)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:927)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:999)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor$GzipBufferCB.process(GzipHttpOutputInterceptor.java:416)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor.gzip(GzipHttpOutputInterceptor.java:139)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor.write(GzipHttpOutputInterceptor.java:121)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:285)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:269)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:1054)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.apache.solr.servlet.ServletOutputStreamWrapper.write(ServletOutputStreamWrapper.java:126)
> ~[?:?]
>     at
> org.apache.solr.response.QueryResponseWriterUtil$1.write(QueryResponseWriterUtil.java:54)
> ~[?:?]
>     at java.io.OutputStream.write(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.write(Unknown Source) ~[?:?]
>     at java.io.OutputStreamWriter.write(Unknown Source) ~[?:?]
>     at org.apache.solr.common.util.FastWriter.flush(FastWriter.java:140)
> ~[?:?]
>     at org.apache.solr.common.util.FastWriter.write(FastWriter.java:125)
> ~[?:?]
>     at java.io.Writer.write(Unknown Source) ~[?:?]
>     at org.apache.solr.response.JSONWriter._writeStr(JSONWriter.java:219)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter.writeStr(JsonTextWriter.java:133)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter.writeDate(JsonTextWriter.java:232)
> ~[?:?]
>     at
> org.apache.solr.common.util.TextWriter.writeDate(TextWriter.java:163)
> ~[?:?]
>     at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:61)
> ~[?:?]
>     at
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
> ~[?:?]
>     at
> org.apache.solr.common.util.JsonTextWriter$2.put(JsonTextWriter.java:176)
> ~[?:?]
>     at
> org.apache.solr.common.MapWriter$EntryWriter.putNoEx(MapWriter.java:101)
> ~[?:?]
>     ... 55 more
> 2023-12-06 07:36:44.235 INFO  (qtp548482954-31) [   x:p4a]
> o.a.s.s.HttpSolrCall Unable to write response, client closed
> connection or we are shutting down =>
> org.eclipse.jetty.io.EofException: Closed
>     at
> org.eclipse.jetty.server.HttpOutput.checkWritable(HttpOutput.java:771)
> org.eclipse.jetty.io.EofException: Closed
>     at
> org.eclipse.jetty.server.HttpOutput.checkWritable(HttpOutput.java:771)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:1007)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.apache.solr.servlet.ServletOutputStreamWrapper.write(ServletOutputStreamWrapper.java:126)
> ~[?:?]
>     at
> org.apache.solr.response.QueryResponseWriterUtil$1.write(QueryResponseWriterUtil.java:54)
> ~[?:?]
>     at java.io.OutputStream.write(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) ~[?:?]
>     at sun.nio.cs.StreamEncoder.write(Unknown Source) ~[?:?]
>     at java.io.OutputStreamWriter.write(Unknown Source) ~[?:?]
>     at org.apache.solr.common.util.FastWriter.flush(FastWriter.java:140)
> ~[?:?]
>     at
> org.apache.solr.common.util.FastWriter.flushBuffer(FastWriter.java:154)
> ~[?:?]
>     at
> org.apache.solr.response.TextResponseWriter.close(TextResponseWriter.java:83)
> ~[?:?]
>     at
> org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:68)
> ~[?:?]
>     at
> org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
> ~[?:?]
>     at
> org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:887)
> ~[?:?]
>     at
> org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:759)
> ~[?:?]
>     at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:589)
> ~[?:?]
>     at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427)
> ~[?:?]
>     at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:357)
> ~[?:?]
>     at
> org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)
> ~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
> ~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
> ~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
> ~[jetty-security-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
> ~[jetty-servlet-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
> ~[jetty-rewrite-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.Server.handle(Server.java:516)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
> ~[jetty-server-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io
> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at org.eclipse.jetty.io
> .ChannelEndPoint$1.run(ChannelEndPoint.java:104)
> ~[jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
> ~[jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
>     at java.lang.Thread.run(Unknown Source) [?:?]
>


-- 
Sincerely yours
Mikhail Khludnev