You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by rajat rastogi <ra...@hindustantimes.com> on 2017/07/19 09:28:31 UTC
Re: Solr Issue While indexing Data
Hi Eric ,
Thanks for your Reply.
I tried the solution given , but it did not work.
Please help me to narrow down the problem.
Please let me know if any more inputs are required from my end viz schema,
configs etc.
Can this problem be related to GC ?
regards
Rajat
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-Issue-While-indexing-Data-tp4339417p4346745.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Issue While indexing Data
Posted by Susheel Kumar <su...@gmail.com>.
What is you current
a) softcommit and hardcommit settings. you can share as it is from config
and how are you committing then?
b) how much is heap out of 124gb
c) how many documents are you adding that is taking long and approx how
many fields including copy fields?
Thnx
On Wed, Jul 19, 2017 at 7:32 AM, rajat rastogi <
rajat.rastogi@hindustantimes.com> wrote:
> Hi Erik,
>
> Some Logs of solr
>
>
> 2017-07-19 08:14:09.104 INFO (qtp434091818-6937) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
> params={wt=javabin&version=2}{add=[54945918f81f4e218b994b75]} 0 24362730
> 2017-07-19 08:14:09.181 DEBUG (qtp434091818-10997) [ x:cda6m]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
> add{,id=54945918f81f4e218b994b75} {wt=javabin&version=2&df=text}
> 2017-07-19 08:25:06.333 DEBUG (qtp434091818-7165) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
> {df=text&qt=edismax&wt=javabin&version=2}
> 2017-07-19 08:25:06.334 INFO (qtp434091818-7165) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
> params={wt=javabin&version=2}{add=[541011730e39384b04ba64ed]} 0 24005213
> 2017-07-19 08:25:06.454 DEBUG (qtp434091818-10958) [ x:cda6m]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
> add{,id=541011730e39384b04ba64ed} {wt=javabin&version=2&df=text}
> 2017-07-19 09:14:57.321 DEBUG (qtp434091818-10682) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
> {df=text&qt=edismax&wt=javabin&version=2}
> 2017-07-19 09:14:57.321 INFO (qtp434091818-10682) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
> params={wt=javabin&version=2}{add=[54c92064a9b16e2fafeff7a0]} 0 8594125
> 2017-07-19 09:14:57.373 DEBUG (qtp434091818-12118) [ x:cda6m]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
> add{,id=54c92064a9b16e2fafeff7a0} {wt=javabin&version=2&df=text}
> 2017-07-19 09:23:26.477 DEBUG (qtp434091818-10770) [ x:cda6m]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
> {wt=javabin&version=2&df=text}
> 2017-07-19 09:23:26.477 INFO (qtp434091818-10770) [ x:cda6m]
> o.a.s.u.p.LogUpdateProcessorFactory [cda6m] webapp=/solr path=/update
> params={wt=javabin&version=2}{add=[520f8aaf53941a4044cdbe86]} 0 8555637
> 2017-07-19 10:17:29.822 DEBUG (qtp434091818-9274) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
> {df=text&qt=edismax&wt=javabin&version=2}
> 2017-07-19 10:17:29.822 INFO (qtp434091818-9274) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
> params={wt=javabin&version=2}{add=[559623155ffdf728657573ef]} 0 19477949
> 2017-07-19 10:17:29.896 DEBUG (qtp434091818-13046) [ x:cda6m]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
> add{,id=559623155ffdf728657573ef} {wt=javabin&version=2&df=text}
> 2017-07-19 10:18:47.465 INFO (qtp434091818-7737) [ x:cda]
> o.a.s.u.DirectUpdateHandler2 end_commit_flush
> 2017-07-19 10:18:47.465 INFO (qtp434091818-10998) [ x:cda]
> o.a.s.u.DirectUpdateHandler2 start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,
> expungeDeletes=false,softCommit=false,prepareCommit=false}
> 2017-07-19 10:18:47.465 INFO (qtp434091818-10998) [ x:cda]
> o.a.s.u.SolrIndexWriter Calling setCommitData with
> IW:org.apache.solr.update.SolrIndexWriter@4843d921
> 2017-07-19 10:18:47.469 DEBUG (qtp434091818-7737) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
> {commit=true&df=text&qt=edismax}
> 2017-07-19 10:18:47.469 INFO (qtp434091818-7737) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
> params={commit=true}{commit=} 0 28038519
> 2017-07-19 10:18:48.620 INFO (qtp434091818-10998) [ x:cda]
> o.a.s.u.DirectUpdateHandler2 end_commit_flush
> 2017-07-19 10:18:48.728 DEBUG (qtp434091818-10998) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
> {df=text&qt=edismax&waitSearcher=true&commit=true&
> softCommit=false&wt=javabin&version=2}
> 2017-07-19 10:18:48.728 INFO (qtp434091818-10998) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
> params={waitSearcher=true&commit=true&softCommit=false&
> wt=javabin&version=2}{commit=}
> 0 10735236
> 2017-07-19 10:18:48.731 DEBUG (qtp434091818-13082) [ x:cda6m]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
> commit{,optimize=false,openSearcher=true,waitSearcher=true,
> expungeDeletes=false,softCommit=false,prepareCommit=false}
> {commit=true&softCommit=false&df=text&waitSearcher=true&wt=
> javabin&version=2}
> root@shineRecSolrMast:~/solr-6.4.2/server/logs# tail -f solr.log
> 2017-07-19 10:17:29.896 DEBUG (qtp434091818-13046) [ x:cda6m]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
> add{,id=559623155ffdf728657573ef} {wt=javabin&version=2&df=text}
> 2017-07-19 10:18:47.465 INFO (qtp434091818-7737) [ x:cda]
> o.a.s.u.DirectUpdateHandler2 end_commit_flush
> 2017-07-19 10:18:47.465 INFO (qtp434091818-10998) [ x:cda]
> o.a.s.u.DirectUpdateHandler2 start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,
> expungeDeletes=false,softCommit=false,prepareCommit=false}
> 2017-07-19 10:18:47.465 INFO (qtp434091818-10998) [ x:cda]
> o.a.s.u.SolrIndexWriter Calling setCommitData with
> IW:org.apache.solr.update.SolrIndexWriter@4843d921
> 2017-07-19 10:18:47.469 DEBUG (qtp434091818-7737) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
> {commit=true&df=text&qt=edismax}
> 2017-07-19 10:18:47.469 INFO (qtp434091818-7737) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
> params={commit=true}{commit=} 0 28038519
> 2017-07-19 10:18:48.620 INFO (qtp434091818-10998) [ x:cda]
> o.a.s.u.DirectUpdateHandler2 end_commit_flush
> 2017-07-19 10:18:48.728 DEBUG (qtp434091818-10998) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
> {df=text&qt=edismax&waitSearcher=true&commit=true&
> softCommit=false&wt=javabin&version=2}
> 2017-07-19 10:18:48.728 INFO (qtp434091818-10998) [ x:cda]
> o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
> params={waitSearcher=true&commit=true&softCommit=false&
> wt=javabin&version=2}{commit=}
> 0 10735236
> 2017-07-19 10:18:48.731 DEBUG (qtp434091818-13082) [ x:cda6m]
> o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
> commit{,optimize=false,openSearcher=true,waitSearcher=true,
> expungeDeletes=false,softCommit=false,prepareCommit=false}
> {commit=true&softCommit=false&df=text&waitSearcher=true&wt=
> javabin&version=2}
>
> For Better in sight
> regards
> Rajat
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/Solr-Issue-While-indexing-Data-tp4339417p4346762.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Solr Issue While indexing Data
Posted by rajat rastogi <ra...@hindustantimes.com>.
Hi Erik,
Some Logs of solr
2017-07-19 08:14:09.104 INFO (qtp434091818-6937) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
params={wt=javabin&version=2}{add=[54945918f81f4e218b994b75]} 0 24362730
2017-07-19 08:14:09.181 DEBUG (qtp434091818-10997) [ x:cda6m]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
add{,id=54945918f81f4e218b994b75} {wt=javabin&version=2&df=text}
2017-07-19 08:25:06.333 DEBUG (qtp434091818-7165) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
{df=text&qt=edismax&wt=javabin&version=2}
2017-07-19 08:25:06.334 INFO (qtp434091818-7165) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
params={wt=javabin&version=2}{add=[541011730e39384b04ba64ed]} 0 24005213
2017-07-19 08:25:06.454 DEBUG (qtp434091818-10958) [ x:cda6m]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
add{,id=541011730e39384b04ba64ed} {wt=javabin&version=2&df=text}
2017-07-19 09:14:57.321 DEBUG (qtp434091818-10682) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
{df=text&qt=edismax&wt=javabin&version=2}
2017-07-19 09:14:57.321 INFO (qtp434091818-10682) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
params={wt=javabin&version=2}{add=[54c92064a9b16e2fafeff7a0]} 0 8594125
2017-07-19 09:14:57.373 DEBUG (qtp434091818-12118) [ x:cda6m]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
add{,id=54c92064a9b16e2fafeff7a0} {wt=javabin&version=2&df=text}
2017-07-19 09:23:26.477 DEBUG (qtp434091818-10770) [ x:cda6m]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
{wt=javabin&version=2&df=text}
2017-07-19 09:23:26.477 INFO (qtp434091818-10770) [ x:cda6m]
o.a.s.u.p.LogUpdateProcessorFactory [cda6m] webapp=/solr path=/update
params={wt=javabin&version=2}{add=[520f8aaf53941a4044cdbe86]} 0 8555637
2017-07-19 10:17:29.822 DEBUG (qtp434091818-9274) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
{df=text&qt=edismax&wt=javabin&version=2}
2017-07-19 10:17:29.822 INFO (qtp434091818-9274) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
params={wt=javabin&version=2}{add=[559623155ffdf728657573ef]} 0 19477949
2017-07-19 10:17:29.896 DEBUG (qtp434091818-13046) [ x:cda6m]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
add{,id=559623155ffdf728657573ef} {wt=javabin&version=2&df=text}
2017-07-19 10:18:47.465 INFO (qtp434091818-7737) [ x:cda]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
2017-07-19 10:18:47.465 INFO (qtp434091818-10998) [ x:cda]
o.a.s.u.DirectUpdateHandler2 start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2017-07-19 10:18:47.465 INFO (qtp434091818-10998) [ x:cda]
o.a.s.u.SolrIndexWriter Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@4843d921
2017-07-19 10:18:47.469 DEBUG (qtp434091818-7737) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
{commit=true&df=text&qt=edismax}
2017-07-19 10:18:47.469 INFO (qtp434091818-7737) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
params={commit=true}{commit=} 0 28038519
2017-07-19 10:18:48.620 INFO (qtp434091818-10998) [ x:cda]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
2017-07-19 10:18:48.728 DEBUG (qtp434091818-10998) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
{df=text&qt=edismax&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}
2017-07-19 10:18:48.728 INFO (qtp434091818-10998) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=}
0 10735236
2017-07-19 10:18:48.731 DEBUG (qtp434091818-13082) [ x:cda6m]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
{commit=true&softCommit=false&df=text&waitSearcher=true&wt=javabin&version=2}
root@shineRecSolrMast:~/solr-6.4.2/server/logs# tail -f solr.log
2017-07-19 10:17:29.896 DEBUG (qtp434091818-13046) [ x:cda6m]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
add{,id=559623155ffdf728657573ef} {wt=javabin&version=2&df=text}
2017-07-19 10:18:47.465 INFO (qtp434091818-7737) [ x:cda]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
2017-07-19 10:18:47.465 INFO (qtp434091818-10998) [ x:cda]
o.a.s.u.DirectUpdateHandler2 start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2017-07-19 10:18:47.465 INFO (qtp434091818-10998) [ x:cda]
o.a.s.u.SolrIndexWriter Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@4843d921
2017-07-19 10:18:47.469 DEBUG (qtp434091818-7737) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
{commit=true&df=text&qt=edismax}
2017-07-19 10:18:47.469 INFO (qtp434091818-7737) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
params={commit=true}{commit=} 0 28038519
2017-07-19 10:18:48.620 INFO (qtp434091818-10998) [ x:cda]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
2017-07-19 10:18:48.728 DEBUG (qtp434091818-10998) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE FINISH
{df=text&qt=edismax&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}
2017-07-19 10:18:48.728 INFO (qtp434091818-10998) [ x:cda]
o.a.s.u.p.LogUpdateProcessorFactory [cda] webapp=/solr path=/update
params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=}
0 10735236
2017-07-19 10:18:48.731 DEBUG (qtp434091818-13082) [ x:cda6m]
o.a.s.u.p.LogUpdateProcessorFactory PRE_UPDATE
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
{commit=true&softCommit=false&df=text&waitSearcher=true&wt=javabin&version=2}
For Better in sight
regards
Rajat
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-Issue-While-indexing-Data-tp4339417p4346762.html
Sent from the Solr - User mailing list archive at Nabble.com.