You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Fiz N <fi...@gmail.com> on 2019/11/15 01:14:21 UTC
2 Async exceptions during distributed update
Hi Solr Experts,
SOLR Version : 6.6.2
OS – Linux 3.1.2
JDK – 1.8
Shard – 16 – All are active.
Xms – 16 gb
Xmx – 16 gb
Schema fields count – 91
Dynamic fields – 83.
There are multiple threads sending products to solr. tested with batch size
per thread 50 & 100, it worked fine without error and if the batch size is
1000 the following error occurs.
there are 64 cores that are sending batches concurrently.
*I am getting the following error when the batch size is 1000. Please
advise.*
2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
{collection=c:ducts, core=x:ducts_shard15_replica1,
node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15}
-
org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
2 Async exceptions during distributed update:
10.YYY.40.62:8983 failed to respond
10.YYY.40.62:8983 failed to respond
2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
{collection=c:ducts, core=x:ducts_shard7_replica1,
node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7}
-
org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
Async exception during distributed update: 10.YYY.40.81:8983 failed to
respond
2019-11-14T19:36:11,599 - ERROR
[updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
<http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
n:10.YYY.40.68:8983_solr c:ducts
s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
- {ConcurrentUpdateSolrClient.url=
http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
<http://10.148.40.68:8983/solr/products_shard11_replica2>,
collection=c:ducts, core=x:ducts_shard3_replica2,
node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3}
- error
org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to respond
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
~[httpclient-4.4.1.jar:4.4.1]
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
~[httpclient-4.4.1.jar:4.4.1]
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
~[httpcore-4.4.1.jar:4.4.1]
2019-11-14T19:36:14,567 - ERROR
[updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
<http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
n:10.YYY.40.68:8983_solr c:ducts
s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
- {ConcurrentUpdateSolrClient.url=
http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
<http://10.148.40.62:8983/solr/products_shard2_replica1>,
collection=c:ducts, core=x:ducts_shard11_replica2,
node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
- error
java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
~[?:1.8.0_232]
at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
~[?:1.8.0_232]
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
~[?:1.8.0_232]
2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
{collection=c:ducts, core=x:ducts_shard11_replica2,
node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
-
null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
Async exception during distributed update: Broken pipe (Write failed)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
Thanks
Fiz..
Fwd: 2 Async exceptions during distributed update
Posted by Fiz N <fi...@gmail.com>.
Thanks Jorn on getting back to me. Please find attached memory/cpu
utilization and complete update process chain.
Please let me know your thoughts.
On Sat, Nov 16, 2019 at 9:09 AM Jörn Franke <jo...@gmail.com> wrote:
> Can you please provide the whole update chain ? Below is the
> directoryFactory and not the update processor chain.
>
> > Am 15.11.2019 um 17:34 schrieb Fiz N <fi...@gmail.com>:
> >
> >
> > Thanks for your response.
> >
> > <directoryFactory
> name="DirectoryFactory"class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
> >
> > attaching the screenshot of physical memory and cpu.
> > Please let me know your thoughts on the below issue.
> >
> >
> >> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <jo...@gmail.com>
> wrote:
> >> Do you use a updateprocess factory? How does it look like?
> >>
> >> What is the physical memory size and CPU?
> >> What do you mean by “there are 64 cores sending concurrently?” An
> application has 64 threats that send concurrently those updates?
> >>
> >> > Am 15.11.2019 um 02:14 schrieb Fiz N <fi...@gmail.com>:
> >> >
> >> > Hi Solr Experts,
> >> >
> >> > SOLR Version : 6.6.2
> >> >
> >> > OS – Linux 3.1.2
> >> >
> >> > JDK – 1.8
> >> >
> >> >
> >> >
> >> > Shard – 16 – All are active.
> >> >
> >> > Xms – 16 gb
> >> >
> >> > Xmx – 16 gb
> >> >
> >> >
> >> >
> >> > Schema fields count – 91
> >> >
> >> > Dynamic fields – 83.
> >> >
> >> >
> >> >
> >> > There are multiple threads sending products to solr. tested with
> batch size
> >> > per thread 50 & 100, it worked fine without error and if the batch
> size is
> >> > 1000 the following error occurs.
> >> >
> >> > there are 64 cores that are sending batches concurrently.
> >> >
> >> >
> >> >
> >> > *I am getting the following error when the batch size is 1000. Please
> >> > advise.*
> >> >
> >> >
> >> >
> >> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY]
> -
> >> > {collection=c:ducts, core=x:ducts_shard15_replica1,
> >> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30,
> shard=s:shard15}
> >> > -
> >> >
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> >> > 2 Async exceptions during distributed update:
> >> >
> >> > 10.YYY.40.62:8983 failed to respond
> >> >
> >> > 10.YYY.40.62:8983 failed to respond
> >> >
> >> >
> >> >
> >> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY]
> -
> >> > {collection=c:ducts, core=x:ducts_shard7_replica1,
> >> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29,
> shard=s:shard7}
> >> > -
> >> >
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> >> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
> >> > respond
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > 2019-11-14T19:36:11,599 - ERROR
> >> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
> >> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
> >> > <http://10.148.40.68:8983/solr/products_shard11_replica2>
> r:core_node26
> >> > n:10.YYY.40.68:8983_solr c:ducts
> >> >
> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
> ]
> >> > - {ConcurrentUpdateSolrClient.url=
> >> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
> >> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
> >> > collection=c:ducts, core=x:ducts_shard3_replica2,
> >> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26,
> shard=s:shard3}
> >> > - error
> >> >
> >> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to
> respond
> >> >
> >> > at
> >> >
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
> >> > ~[httpclient-4.4.1.jar:4.4.1]
> >> >
> >> > at
> >> >
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
> >> > ~[httpclient-4.4.1.jar:4.4.1]
> >> >
> >> > at
> >> > org.apache.http.impl.io
> .AbstractMessageParser.parse(AbstractMessageParser.java:261)
> >> > ~[httpcore-4.4.1.jar:4.4.1]
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > 2019-11-14T19:36:14,567 - ERROR
> >> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
> >> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
> >> > <http://10.148.40.62:8983/solr/products_shard2_replica1>
> r:core_node25
> >> > n:10.YYY.40.68:8983_solr c:ducts
> >> >
> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
> ]
> >> > - {ConcurrentUpdateSolrClient.url=
> >> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
> >> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
> >> > collection=c:ducts, core=x:ducts_shard11_replica2,
> >> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
> shard=s:shard11}
> >> > - error
> >> >
> >> > java.net.SocketException: Broken pipe (Write failed)
> >> >
> >> > at java.net.SocketOutputStream.socketWrite0(Native Method)
> >> > ~[?:1.8.0_232]
> >> >
> >> > at
> >> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
> >> > ~[?:1.8.0_232]
> >> >
> >> > at
> java.net.SocketOutputStream.write(SocketOutputStream.java:155)
> >> > ~[?:1.8.0_232]
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159]
> -
> >> > {collection=c:ducts, core=x:ducts_shard11_replica2,
> >> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
> shard=s:shard11}
> >> > -
> >> >
> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> >> > Async exception during distributed update: Broken pipe (Write failed)
> >> >
> >> > at
> >> >
> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
> >> >
> >> > at
> >> >
> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
> >> >
> >> >
> >> >
> >> >
> >> > Thanks
> >> >
> >> > Fiz..
>
Re: 2 Async exceptions during distributed update
Posted by Fiz N <fi...@gmail.com>.
Thanks Jorn on getting back to me. Please find attached memory/cpu
utilization and complete update process chain.
Please let me know your thoughts.
On Sat, Nov 16, 2019 at 9:09 AM Jörn Franke <jo...@gmail.com> wrote:
> Can you please provide the whole update chain ? Below is the
> directoryFactory and not the update processor chain.
>
> > Am 15.11.2019 um 17:34 schrieb Fiz N <fi...@gmail.com>:
> >
> >
> > Thanks for your response.
> >
> > <directoryFactory
> name="DirectoryFactory"class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
> >
> > attaching the screenshot of physical memory and cpu.
> > Please let me know your thoughts on the below issue.
> >
> >
> >> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <jo...@gmail.com>
> wrote:
> >> Do you use a updateprocess factory? How does it look like?
> >>
> >> What is the physical memory size and CPU?
> >> What do you mean by “there are 64 cores sending concurrently?” An
> application has 64 threats that send concurrently those updates?
> >>
> >> > Am 15.11.2019 um 02:14 schrieb Fiz N <fi...@gmail.com>:
> >> >
> >> > Hi Solr Experts,
> >> >
> >> > SOLR Version : 6.6.2
> >> >
> >> > OS – Linux 3.1.2
> >> >
> >> > JDK – 1.8
> >> >
> >> >
> >> >
> >> > Shard – 16 – All are active.
> >> >
> >> > Xms – 16 gb
> >> >
> >> > Xmx – 16 gb
> >> >
> >> >
> >> >
> >> > Schema fields count – 91
> >> >
> >> > Dynamic fields – 83.
> >> >
> >> >
> >> >
> >> > There are multiple threads sending products to solr. tested with
> batch size
> >> > per thread 50 & 100, it worked fine without error and if the batch
> size is
> >> > 1000 the following error occurs.
> >> >
> >> > there are 64 cores that are sending batches concurrently.
> >> >
> >> >
> >> >
> >> > *I am getting the following error when the batch size is 1000. Please
> >> > advise.*
> >> >
> >> >
> >> >
> >> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY]
> -
> >> > {collection=c:ducts, core=x:ducts_shard15_replica1,
> >> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30,
> shard=s:shard15}
> >> > -
> >> >
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> >> > 2 Async exceptions during distributed update:
> >> >
> >> > 10.YYY.40.62:8983 failed to respond
> >> >
> >> > 10.YYY.40.62:8983 failed to respond
> >> >
> >> >
> >> >
> >> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY]
> -
> >> > {collection=c:ducts, core=x:ducts_shard7_replica1,
> >> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29,
> shard=s:shard7}
> >> > -
> >> >
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> >> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
> >> > respond
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > 2019-11-14T19:36:11,599 - ERROR
> >> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
> >> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
> >> > <http://10.148.40.68:8983/solr/products_shard11_replica2>
> r:core_node26
> >> > n:10.YYY.40.68:8983_solr c:ducts
> >> >
> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
> ]
> >> > - {ConcurrentUpdateSolrClient.url=
> >> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
> >> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
> >> > collection=c:ducts, core=x:ducts_shard3_replica2,
> >> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26,
> shard=s:shard3}
> >> > - error
> >> >
> >> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to
> respond
> >> >
> >> > at
> >> >
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
> >> > ~[httpclient-4.4.1.jar:4.4.1]
> >> >
> >> > at
> >> >
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
> >> > ~[httpclient-4.4.1.jar:4.4.1]
> >> >
> >> > at
> >> > org.apache.http.impl.io
> .AbstractMessageParser.parse(AbstractMessageParser.java:261)
> >> > ~[httpcore-4.4.1.jar:4.4.1]
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > 2019-11-14T19:36:14,567 - ERROR
> >> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
> >> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
> >> > <http://10.148.40.62:8983/solr/products_shard2_replica1>
> r:core_node25
> >> > n:10.YYY.40.68:8983_solr c:ducts
> >> >
> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
> ]
> >> > - {ConcurrentUpdateSolrClient.url=
> >> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
> >> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
> >> > collection=c:ducts, core=x:ducts_shard11_replica2,
> >> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
> shard=s:shard11}
> >> > - error
> >> >
> >> > java.net.SocketException: Broken pipe (Write failed)
> >> >
> >> > at java.net.SocketOutputStream.socketWrite0(Native Method)
> >> > ~[?:1.8.0_232]
> >> >
> >> > at
> >> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
> >> > ~[?:1.8.0_232]
> >> >
> >> > at
> java.net.SocketOutputStream.write(SocketOutputStream.java:155)
> >> > ~[?:1.8.0_232]
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159]
> -
> >> > {collection=c:ducts, core=x:ducts_shard11_replica2,
> >> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
> shard=s:shard11}
> >> > -
> >> >
> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> >> > Async exception during distributed update: Broken pipe (Write failed)
> >> >
> >> > at
> >> >
> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
> >> >
> >> > at
> >> >
> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
> >> >
> >> >
> >> >
> >> >
> >> > Thanks
> >> >
> >> > Fiz..
>
Re: 2 Async exceptions during distributed update
Posted by Jörn Franke <jo...@gmail.com>.
Can you please provide the whole update chain ? Below is the directoryFactory and not the update processor chain.
> Am 15.11.2019 um 17:34 schrieb Fiz N <fi...@gmail.com>:
>
>
> Thanks for your response.
>
> <directoryFactory name="DirectoryFactory"class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>
> attaching the screenshot of physical memory and cpu.
> Please let me know your thoughts on the below issue.
>
>
>> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <jo...@gmail.com> wrote:
>> Do you use a updateprocess factory? How does it look like?
>>
>> What is the physical memory size and CPU?
>> What do you mean by “there are 64 cores sending concurrently?” An application has 64 threats that send concurrently those updates?
>>
>> > Am 15.11.2019 um 02:14 schrieb Fiz N <fi...@gmail.com>:
>> >
>> > Hi Solr Experts,
>> >
>> > SOLR Version : 6.6.2
>> >
>> > OS – Linux 3.1.2
>> >
>> > JDK – 1.8
>> >
>> >
>> >
>> > Shard – 16 – All are active.
>> >
>> > Xms – 16 gb
>> >
>> > Xmx – 16 gb
>> >
>> >
>> >
>> > Schema fields count – 91
>> >
>> > Dynamic fields – 83.
>> >
>> >
>> >
>> > There are multiple threads sending products to solr. tested with batch size
>> > per thread 50 & 100, it worked fine without error and if the batch size is
>> > 1000 the following error occurs.
>> >
>> > there are 64 cores that are sending batches concurrently.
>> >
>> >
>> >
>> > *I am getting the following error when the batch size is 1000. Please
>> > advise.*
>> >
>> >
>> >
>> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard15_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15}
>> > -
>> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > 2 Async exceptions during distributed update:
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> >
>> >
>> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard7_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7}
>> > -
>> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
>> > respond
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:11,599 - ERROR
>> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
>> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
>> > n:10.YYY.40.68:8983_solr c:ducts
>> > s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
>> > collection=c:ducts, core=x:ducts_shard3_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3}
>> > - error
>> >
>> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to respond
>> >
>> > at
>> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> > at
>> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> > at
>> > org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
>> > ~[httpcore-4.4.1.jar:4.4.1]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:14,567 - ERROR
>> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
>> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
>> > n:10.YYY.40.68:8983_solr c:ducts
>> > s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
>> > collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
>> > - error
>> >
>> > java.net.SocketException: Broken pipe (Write failed)
>> >
>> > at java.net.SocketOutputStream.socketWrite0(Native Method)
>> > ~[?:1.8.0_232]
>> >
>> > at
>> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
>> > ~[?:1.8.0_232]
>> >
>> > at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
>> > ~[?:1.8.0_232]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
>> > {collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
>> > -
>> > null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: Broken pipe (Write failed)
>> >
>> > at
>> > org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>> >
>> > at
>> > org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>> >
>> >
>> >
>> >
>> > Thanks
>> >
>> > Fiz..
Re: 2 Async exceptions during distributed update
Posted by Jörn Franke <jo...@gmail.com>.
No screenshot attached. The Apache mailservers filter attachments. Can you
please provide an external link.
On Fri, Nov 15, 2019 at 5:34 PM Fiz N <fi...@gmail.com> wrote:
> Thanks for your response.
>
> <directoryFactory name="DirectoryFactory"class=
> "${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>
>
>
> attaching the screenshot of physical memory and cpu.
>
> Please let me know your thoughts on the below issue.
>
>
>
> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <jo...@gmail.com> wrote:
>
>> Do you use a updateprocess factory? How does it look like?
>>
>> What is the physical memory size and CPU?
>> What do you mean by “there are 64 cores sending concurrently?” An
>> application has 64 threats that send concurrently those updates?
>>
>> > Am 15.11.2019 um 02:14 schrieb Fiz N <fi...@gmail.com>:
>> >
>> > Hi Solr Experts,
>> >
>> > SOLR Version : 6.6.2
>> >
>> > OS – Linux 3.1.2
>> >
>> > JDK – 1.8
>> >
>> >
>> >
>> > Shard – 16 – All are active.
>> >
>> > Xms – 16 gb
>> >
>> > Xmx – 16 gb
>> >
>> >
>> >
>> > Schema fields count – 91
>> >
>> > Dynamic fields – 83.
>> >
>> >
>> >
>> > There are multiple threads sending products to solr. tested with batch
>> size
>> > per thread 50 & 100, it worked fine without error and if the batch size
>> is
>> > 1000 the following error occurs.
>> >
>> > there are 64 cores that are sending batches concurrently.
>> >
>> >
>> >
>> > *I am getting the following error when the batch size is 1000. Please
>> > advise.*
>> >
>> >
>> >
>> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard15_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30,
>> shard=s:shard15}
>> > -
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > 2 Async exceptions during distributed update:
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> >
>> >
>> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard7_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29,
>> shard=s:shard7}
>> > -
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
>> > respond
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:11,599 - ERROR
>> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
>> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
>> > n:10.YYY.40.68:8983_solr c:ducts
>> >
>> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
>> ]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
>> > collection=c:ducts, core=x:ducts_shard3_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26,
>> shard=s:shard3}
>> > - error
>> >
>> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to
>> respond
>> >
>> > at
>> >
>> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> > at
>> >
>> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> > at
>> > org.apache.http.impl.io
>> .AbstractMessageParser.parse(AbstractMessageParser.java:261)
>> > ~[httpcore-4.4.1.jar:4.4.1]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:14,567 - ERROR
>> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
>> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
>> > n:10.YYY.40.68:8983_solr c:ducts
>> >
>> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
>> ]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
>> > collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
>> shard=s:shard11}
>> > - error
>> >
>> > java.net.SocketException: Broken pipe (Write failed)
>> >
>> > at java.net.SocketOutputStream.socketWrite0(Native Method)
>> > ~[?:1.8.0_232]
>> >
>> > at
>> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
>> > ~[?:1.8.0_232]
>> >
>> > at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
>> > ~[?:1.8.0_232]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
>> > {collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
>> shard=s:shard11}
>> > -
>> >
>> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: Broken pipe (Write failed)
>> >
>> > at
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>> >
>> > at
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>> >
>> >
>> >
>> >
>> > Thanks
>> >
>> > Fiz..
>>
>
Re: 2 Async exceptions during distributed update
Posted by Fiz N <fi...@gmail.com>.
Hi Solr Experts,
Do you have any thoughts on the below issue ?
On Fri, Nov 15, 2019 at 11:33 AM Fiz N <fi...@gmail.com> wrote:
> Thanks for your response.
>
> <directoryFactory name="DirectoryFactory"class=
> "${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>
>
>
> attaching the screenshot of physical memory and cpu.
>
> Please let me know your thoughts on the below issue.
>
>
>
> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <jo...@gmail.com> wrote:
>
>> Do you use a updateprocess factory? How does it look like?
>>
>> What is the physical memory size and CPU?
>> What do you mean by “there are 64 cores sending concurrently?” An
>> application has 64 threats that send concurrently those updates?
>>
>> > Am 15.11.2019 um 02:14 schrieb Fiz N <fi...@gmail.com>:
>> >
>> > Hi Solr Experts,
>> >
>> > SOLR Version : 6.6.2
>> >
>> > OS – Linux 3.1.2
>> >
>> > JDK – 1.8
>> >
>> >
>> >
>> > Shard – 16 – All are active.
>> >
>> > Xms – 16 gb
>> >
>> > Xmx – 16 gb
>> >
>> >
>> >
>> > Schema fields count – 91
>> >
>> > Dynamic fields – 83.
>> >
>> >
>> >
>> > There are multiple threads sending products to solr. tested with batch
>> size
>> > per thread 50 & 100, it worked fine without error and if the batch size
>> is
>> > 1000 the following error occurs.
>> >
>> > there are 64 cores that are sending batches concurrently.
>> >
>> >
>> >
>> > *I am getting the following error when the batch size is 1000. Please
>> > advise.*
>> >
>> >
>> >
>> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard15_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30,
>> shard=s:shard15}
>> > -
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > 2 Async exceptions during distributed update:
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> >
>> >
>> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard7_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29,
>> shard=s:shard7}
>> > -
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
>> > respond
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:11,599 - ERROR
>> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
>> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
>> > n:10.YYY.40.68:8983_solr c:ducts
>> >
>> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
>> ]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
>> > collection=c:ducts, core=x:ducts_shard3_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26,
>> shard=s:shard3}
>> > - error
>> >
>> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to
>> respond
>> >
>> > at
>> >
>> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> > at
>> >
>> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> > at
>> > org.apache.http.impl.io
>> .AbstractMessageParser.parse(AbstractMessageParser.java:261)
>> > ~[httpcore-4.4.1.jar:4.4.1]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:14,567 - ERROR
>> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
>> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
>> > n:10.YYY.40.68:8983_solr c:ducts
>> >
>> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
>> ]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
>> > collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
>> shard=s:shard11}
>> > - error
>> >
>> > java.net.SocketException: Broken pipe (Write failed)
>> >
>> > at java.net.SocketOutputStream.socketWrite0(Native Method)
>> > ~[?:1.8.0_232]
>> >
>> > at
>> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
>> > ~[?:1.8.0_232]
>> >
>> > at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
>> > ~[?:1.8.0_232]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
>> > {collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
>> shard=s:shard11}
>> > -
>> >
>> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: Broken pipe (Write failed)
>> >
>> > at
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>> >
>> > at
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>> >
>> >
>> >
>> >
>> > Thanks
>> >
>> > Fiz..
>>
>
Re: 2 Async exceptions during distributed update
Posted by Fiz N <fi...@gmail.com>.
Thanks for your response.
<directoryFactory name="DirectoryFactory"class=
"${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
attaching the screenshot of physical memory and cpu.
Please let me know your thoughts on the below issue.
On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <jo...@gmail.com> wrote:
> Do you use a updateprocess factory? How does it look like?
>
> What is the physical memory size and CPU?
> What do you mean by “there are 64 cores sending concurrently?” An
> application has 64 threats that send concurrently those updates?
>
> > Am 15.11.2019 um 02:14 schrieb Fiz N <fi...@gmail.com>:
> >
> > Hi Solr Experts,
> >
> > SOLR Version : 6.6.2
> >
> > OS – Linux 3.1.2
> >
> > JDK – 1.8
> >
> >
> >
> > Shard – 16 – All are active.
> >
> > Xms – 16 gb
> >
> > Xmx – 16 gb
> >
> >
> >
> > Schema fields count – 91
> >
> > Dynamic fields – 83.
> >
> >
> >
> > There are multiple threads sending products to solr. tested with batch
> size
> > per thread 50 & 100, it worked fine without error and if the batch size
> is
> > 1000 the following error occurs.
> >
> > there are 64 cores that are sending batches concurrently.
> >
> >
> >
> > *I am getting the following error when the batch size is 1000. Please
> > advise.*
> >
> >
> >
> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
> > {collection=c:ducts, core=x:ducts_shard15_replica1,
> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30,
> shard=s:shard15}
> > -
> >
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> > 2 Async exceptions during distributed update:
> >
> > 10.YYY.40.62:8983 failed to respond
> >
> > 10.YYY.40.62:8983 failed to respond
> >
> >
> >
> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
> > {collection=c:ducts, core=x:ducts_shard7_replica1,
> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29,
> shard=s:shard7}
> > -
> >
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
> > respond
> >
> >
> >
> >
> >
> > 2019-11-14T19:36:11,599 - ERROR
> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
> > <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
> > n:10.YYY.40.68:8983_solr c:ducts
> >
> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
> ]
> > - {ConcurrentUpdateSolrClient.url=
> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
> > collection=c:ducts, core=x:ducts_shard3_replica2,
> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26,
> shard=s:shard3}
> > - error
> >
> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to
> respond
> >
> > at
> >
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
> > ~[httpclient-4.4.1.jar:4.4.1]
> >
> > at
> >
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
> > ~[httpclient-4.4.1.jar:4.4.1]
> >
> > at
> > org.apache.http.impl.io
> .AbstractMessageParser.parse(AbstractMessageParser.java:261)
> > ~[httpcore-4.4.1.jar:4.4.1]
> >
> >
> >
> >
> >
> > 2019-11-14T19:36:14,567 - ERROR
> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
> > <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
> > n:10.YYY.40.68:8983_solr c:ducts
> >
> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
> ]
> > - {ConcurrentUpdateSolrClient.url=
> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
> > collection=c:ducts, core=x:ducts_shard11_replica2,
> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
> shard=s:shard11}
> > - error
> >
> > java.net.SocketException: Broken pipe (Write failed)
> >
> > at java.net.SocketOutputStream.socketWrite0(Native Method)
> > ~[?:1.8.0_232]
> >
> > at
> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
> > ~[?:1.8.0_232]
> >
> > at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
> > ~[?:1.8.0_232]
> >
> >
> >
> >
> >
> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
> > {collection=c:ducts, core=x:ducts_shard11_replica2,
> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
> shard=s:shard11}
> > -
> >
> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> > Async exception during distributed update: Broken pipe (Write failed)
> >
> > at
> >
> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
> >
> > at
> >
> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
> >
> >
> >
> >
> > Thanks
> >
> > Fiz..
>
Re: 2 Async exceptions during distributed update
Posted by Jörn Franke <jo...@gmail.com>.
Do you use a updateprocess factory? How does it look like?
What is the physical memory size and CPU?
What do you mean by “there are 64 cores sending concurrently?” An application has 64 threats that send concurrently those updates?
> Am 15.11.2019 um 02:14 schrieb Fiz N <fi...@gmail.com>:
>
> Hi Solr Experts,
>
> SOLR Version : 6.6.2
>
> OS – Linux 3.1.2
>
> JDK – 1.8
>
>
>
> Shard – 16 – All are active.
>
> Xms – 16 gb
>
> Xmx – 16 gb
>
>
>
> Schema fields count – 91
>
> Dynamic fields – 83.
>
>
>
> There are multiple threads sending products to solr. tested with batch size
> per thread 50 & 100, it worked fine without error and if the batch size is
> 1000 the following error occurs.
>
> there are 64 cores that are sending batches concurrently.
>
>
>
> *I am getting the following error when the batch size is 1000. Please
> advise.*
>
>
>
> 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
> {collection=c:ducts, core=x:ducts_shard15_replica1,
> node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15}
> -
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> 2 Async exceptions during distributed update:
>
> 10.YYY.40.62:8983 failed to respond
>
> 10.YYY.40.62:8983 failed to respond
>
>
>
> 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
> {collection=c:ducts, core=x:ducts_shard7_replica1,
> node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7}
> -
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> Async exception during distributed update: 10.YYY.40.81:8983 failed to
> respond
>
>
>
>
>
> 2019-11-14T19:36:11,599 - ERROR
> [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
> http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
> <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
> n:10.YYY.40.68:8983_solr c:ducts
> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
> - {ConcurrentUpdateSolrClient.url=
> http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
> <http://10.148.40.68:8983/solr/products_shard11_replica2>,
> collection=c:ducts, core=x:ducts_shard3_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3}
> - error
>
> org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to respond
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
> ~[httpclient-4.4.1.jar:4.4.1]
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
> ~[httpclient-4.4.1.jar:4.4.1]
>
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
> ~[httpcore-4.4.1.jar:4.4.1]
>
>
>
>
>
> 2019-11-14T19:36:14,567 - ERROR
> [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
> http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
> <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
> n:10.YYY.40.68:8983_solr c:ducts
> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
> - {ConcurrentUpdateSolrClient.url=
> http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
> <http://10.148.40.62:8983/solr/products_shard2_replica1>,
> collection=c:ducts, core=x:ducts_shard11_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
> - error
>
> java.net.SocketException: Broken pipe (Write failed)
>
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> ~[?:1.8.0_232]
>
> at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
> ~[?:1.8.0_232]
>
> at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
> ~[?:1.8.0_232]
>
>
>
>
>
> 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
> {collection=c:ducts, core=x:ducts_shard11_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
> -
> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> Async exception during distributed update: Broken pipe (Write failed)
>
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>
>
>
>
> Thanks
>
> Fiz..