You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Mukhopadhyay, Aratrika" <Ar...@mail.house.gov> on 2017/12/08 15:27:58 UTC
Need Help Configuring Solr To Work With Nutch
Good Morning,
I am running nutch 2.3 , hbase 0.98 and I am integrating nutch with solr 6.4. I have a successful crawl in nutch and when I see that it is indexing the content into solr. However I cannot query and get any results. Its as if Nutch isn't writing anything to solr at all. I am stuck and need someone who is familiar with solr/nutch to provide assistance. Can someone please help ?
This is what I see when I index into solr. I see no errors.
[cid:image001.png@01D3700F.37209960]
Regards,
Aratrika Mukhopadhyay
RE: FW: Need Help Configuring Solr To Work With Nutch
Posted by "Mukhopadhyay, Aratrika" <Ar...@mail.house.gov>.
Thank you Erick Erickson and Rick Leir . My issue was permission related where the solr user was not running the indexing job through Nutch and therefore was being unable to write anything to Solr. I changed the ownership of the nutch's runtime directory to solr and all is well and working. I thank you for your help. Your tip about the numDocs put me on the right track .
Aratrika
-----Original Message-----
From: Rick Leir [mailto:rleir@leirtech.com]
Sent: Saturday, December 09, 2017 10:25 AM
To: solr-user@lucene.apache.org
Subject: RE: FW: Need Help Configuring Solr To Work With Nutch
Ara
The config for soft commit would not be in schema.xml, please look in solrconfig.xml.
Look in solr.log for evidence of commits occurring. Explore the SolrAdmin console, what are the document counts?
You can post snippets from your config files here.
Cheers --Rick
On December 8, 2017 4:23:00 PM EST, "Mukhopadhyay, Aratrika" <Ar...@mail.house.gov> wrote:
>Rick ,
>Thanks for your reply. I do not see any errors or exceptions in the
>solr logs. I have read that the my schema in nutch needs to match the
>schema in solr. When I change the schema in in the config directory and
>restart solr my changes are lost. Leaving the schema alone is the only
>way I can get the indexing job to run but I cant query for the data in
>solr. Would you like me to send you specific configuration files ? I
>cant seem to get this to work.
>
>Kind regards,
>Aratrika Mukhopadhyay
>
>-----Original Message-----
>From: Rick Leir [mailto:rleir@leirtech.com]
>Sent: Friday, December 08, 2017 4:06 PM
>To: solr-user@lucene.apache.org
>Subject: Re: FW: Need Help Configuring Solr To Work With Nutch
>
>Ara
>Softcommit might be the default in Solrconfig.xml, and if not then you
>should probably make it so. Then you need to have a look in solr.log if
>things are not working as you expect.
>Cheers -- Rick
>
>On December 8, 2017 3:23:35 PM EST, "Mukhopadhyay, Aratrika"
><Ar...@mail.house.gov> wrote:
>>Erick,
>>Do I need to set the softCommit = true and prepareCommit to true in my
>
>>solrconfig ? I am still at a loss as to what is happening. Thanks
>again
>>for your help.
>>
>>Aratrika
>>
>>From: Mukhopadhyay, Aratrika
>>Sent: Friday, December 08, 2017 11:34 AM
>>To: solr-user <so...@lucene.apache.org>
>>Subject: RE: Need Help Configuring Solr To Work With Nutch
>>
>>
>>Hello Erick ,
>>
>> This is what I see in the logs :
>>
>>[cid:image001.png@01D37018.62D3CC90]
>>
>>
>>
>>I am sorry it sbeen a while since I worked with solr. I did not do
>>anything to specifically commit the changes to the core. Thanks for
>>your prompt attention to this matter.
>>
>>
>>
>>Aratrika Mukhopadhyay
>>
>>
>>
>>-----Original Message-----
>>From: Erick Erickson [mailto:erickerickson@gmail.com]
>>Sent: Friday, December 08, 2017 11:06 AM
>>To: solr-user
>><so...@lucene.apache.org>>
>>Subject: Re: Need Help Configuring Solr To Work With Nutch
>>
>>
>>
>>1> do you see update messages in the Solr logs?
>>
>>2> did you issue a commit?
>>
>>
>>
>>Best,
>>
>>Erick
>>
>>
>>
>>On Fri, Dec 8, 2017 at 7:27 AM, Mukhopadhyay, Aratrika <
>>Aratrika.Mukhopadhyay@mail.house.gov<mailto:Aratrika.Mukhopadhyay@mail.
>>house.gov>>
>>wrote:
>>
>>
>>
>>> Good Morning,
>>
>>>
>>
>>> I am running nutch 2.3 , hbase 0.98 and I am integrating
>>
>>> nutch with solr 6.4. I have a successful crawl in nutch and when I
>>see
>>
>>> that it is indexing the content into solr. However I cannot query
>and
>>get any results.
>>
>>> Its as if Nutch isn’t writing anything to solr at all. I am stuck
>and
>>
>>> need someone who is familiar with solr/nutch to provide assistance.
>>
>>> Can someone please help ?
>>
>>>
>>
>>>
>>
>>>
>>
>>> This is what I see when I index into solr. I see no errors.
>>
>>>
>>
>>>
>>
>>>
>>
>>>
>>
>>>
>>
>>>
>>
>>>
>>
>>> Regards,
>>
>>>
>>
>>> Aratrika Mukhopadhyay
>>
>>>
>
>--
>Sorry for being brief. Alternate email is rickleir at yahoo dot com
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
RE: FW: Need Help Configuring Solr To Work With Nutch
Posted by Rick Leir <rl...@leirtech.com>.
Ara
The config for soft commit would not be in schema.xml, please look in solrconfig.xml.
Look in solr.log for evidence of commits occurring. Explore the SolrAdmin console, what are the document counts?
You can post snippets from your config files here.
Cheers --Rick
On December 8, 2017 4:23:00 PM EST, "Mukhopadhyay, Aratrika" <Ar...@mail.house.gov> wrote:
>Rick ,
>Thanks for your reply. I do not see any errors or exceptions in the
>solr logs. I have read that the my schema in nutch needs to match the
>schema in solr. When I change the schema in in the config directory and
>restart solr my changes are lost. Leaving the schema alone is the only
>way I can get the indexing job to run but I cant query for the data in
>solr. Would you like me to send you specific configuration files ? I
>cant seem to get this to work.
>
>Kind regards,
>Aratrika Mukhopadhyay
>
>-----Original Message-----
>From: Rick Leir [mailto:rleir@leirtech.com]
>Sent: Friday, December 08, 2017 4:06 PM
>To: solr-user@lucene.apache.org
>Subject: Re: FW: Need Help Configuring Solr To Work With Nutch
>
>Ara
>Softcommit might be the default in Solrconfig.xml, and if not then you
>should probably make it so. Then you need to have a look in solr.log if
>things are not working as you expect.
>Cheers -- Rick
>
>On December 8, 2017 3:23:35 PM EST, "Mukhopadhyay, Aratrika"
><Ar...@mail.house.gov> wrote:
>>Erick,
>>Do I need to set the softCommit = true and prepareCommit to true in my
>
>>solrconfig ? I am still at a loss as to what is happening. Thanks
>again
>>for your help.
>>
>>Aratrika
>>
>>From: Mukhopadhyay, Aratrika
>>Sent: Friday, December 08, 2017 11:34 AM
>>To: solr-user <so...@lucene.apache.org>
>>Subject: RE: Need Help Configuring Solr To Work With Nutch
>>
>>
>>Hello Erick ,
>>
>> This is what I see in the logs :
>>
>>[cid:image001.png@01D37018.62D3CC90]
>>
>>
>>
>>I am sorry it sbeen a while since I worked with solr. I did not do
>>anything to specifically commit the changes to the core. Thanks for
>>your prompt attention to this matter.
>>
>>
>>
>>Aratrika Mukhopadhyay
>>
>>
>>
>>-----Original Message-----
>>From: Erick Erickson [mailto:erickerickson@gmail.com]
>>Sent: Friday, December 08, 2017 11:06 AM
>>To: solr-user
>><so...@lucene.apache.org>>
>>Subject: Re: Need Help Configuring Solr To Work With Nutch
>>
>>
>>
>>1> do you see update messages in the Solr logs?
>>
>>2> did you issue a commit?
>>
>>
>>
>>Best,
>>
>>Erick
>>
>>
>>
>>On Fri, Dec 8, 2017 at 7:27 AM, Mukhopadhyay, Aratrika <
>>Aratrika.Mukhopadhyay@mail.house.gov<mailto:Aratrika.Mukhopadhyay@mail.
>>house.gov>>
>>wrote:
>>
>>
>>
>>> Good Morning,
>>
>>>
>>
>>> I am running nutch 2.3 , hbase 0.98 and I am integrating
>>
>>> nutch with solr 6.4. I have a successful crawl in nutch and when I
>>see
>>
>>> that it is indexing the content into solr. However I cannot query
>and
>>get any results.
>>
>>> Its as if Nutch isn’t writing anything to solr at all. I am stuck
>and
>>
>>> need someone who is familiar with solr/nutch to provide assistance.
>>
>>> Can someone please help ?
>>
>>>
>>
>>>
>>
>>>
>>
>>> This is what I see when I index into solr. I see no errors.
>>
>>>
>>
>>>
>>
>>>
>>
>>>
>>
>>>
>>
>>>
>>
>>>
>>
>>> Regards,
>>
>>>
>>
>>> Aratrika Mukhopadhyay
>>
>>>
>
>--
>Sorry for being brief. Alternate email is rickleir at yahoo dot com
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
RE: FW: Need Help Configuring Solr To Work With Nutch
Posted by "Mukhopadhyay, Aratrika" <Ar...@mail.house.gov>.
Rick ,
Thanks for your reply. I do not see any errors or exceptions in the solr logs. I have read that the my schema in nutch needs to match the schema in solr. When I change the schema in in the config directory and restart solr my changes are lost. Leaving the schema alone is the only way I can get the indexing job to run but I cant query for the data in solr. Would you like me to send you specific configuration files ? I cant seem to get this to work.
Kind regards,
Aratrika Mukhopadhyay
-----Original Message-----
From: Rick Leir [mailto:rleir@leirtech.com]
Sent: Friday, December 08, 2017 4:06 PM
To: solr-user@lucene.apache.org
Subject: Re: FW: Need Help Configuring Solr To Work With Nutch
Ara
Softcommit might be the default in Solrconfig.xml, and if not then you should probably make it so. Then you need to have a look in solr.log if things are not working as you expect.
Cheers -- Rick
On December 8, 2017 3:23:35 PM EST, "Mukhopadhyay, Aratrika" <Ar...@mail.house.gov> wrote:
>Erick,
>Do I need to set the softCommit = true and prepareCommit to true in my
>solrconfig ? I am still at a loss as to what is happening. Thanks again
>for your help.
>
>Aratrika
>
>From: Mukhopadhyay, Aratrika
>Sent: Friday, December 08, 2017 11:34 AM
>To: solr-user <so...@lucene.apache.org>
>Subject: RE: Need Help Configuring Solr To Work With Nutch
>
>
>Hello Erick ,
>
> This is what I see in the logs :
>
>[cid:image001.png@01D37018.62D3CC90]
>
>
>
>I am sorry it sbeen a while since I worked with solr. I did not do
>anything to specifically commit the changes to the core. Thanks for
>your prompt attention to this matter.
>
>
>
>Aratrika Mukhopadhyay
>
>
>
>-----Original Message-----
>From: Erick Erickson [mailto:erickerickson@gmail.com]
>Sent: Friday, December 08, 2017 11:06 AM
>To: solr-user
><so...@lucene.apache.org>>
>Subject: Re: Need Help Configuring Solr To Work With Nutch
>
>
>
>1> do you see update messages in the Solr logs?
>
>2> did you issue a commit?
>
>
>
>Best,
>
>Erick
>
>
>
>On Fri, Dec 8, 2017 at 7:27 AM, Mukhopadhyay, Aratrika <
>Aratrika.Mukhopadhyay@mail.house.gov<mailto:Aratrika.Mukhopadhyay@mail.
>house.gov>>
>wrote:
>
>
>
>> Good Morning,
>
>>
>
>> I am running nutch 2.3 , hbase 0.98 and I am integrating
>
>> nutch with solr 6.4. I have a successful crawl in nutch and when I
>see
>
>> that it is indexing the content into solr. However I cannot query and
>get any results.
>
>> Its as if Nutch isn’t writing anything to solr at all. I am stuck and
>
>> need someone who is familiar with solr/nutch to provide assistance.
>
>> Can someone please help ?
>
>>
>
>>
>
>>
>
>> This is what I see when I index into solr. I see no errors.
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> Regards,
>
>>
>
>> Aratrika Mukhopadhyay
>
>>
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
Re: FW: Need Help Configuring Solr To Work With Nutch
Posted by Rick Leir <rl...@leirtech.com>.
Ara
Softcommit might be the default in Solrconfig.xml, and if not then you should probably make it so. Then you need to have a look in solr.log if things are not working as you expect.
Cheers -- Rick
On December 8, 2017 3:23:35 PM EST, "Mukhopadhyay, Aratrika" <Ar...@mail.house.gov> wrote:
>Erick,
>Do I need to set the softCommit = true and prepareCommit to true in my
>solrconfig ? I am still at a loss as to what is happening. Thanks again
>for your help.
>
>Aratrika
>
>From: Mukhopadhyay, Aratrika
>Sent: Friday, December 08, 2017 11:34 AM
>To: solr-user <so...@lucene.apache.org>
>Subject: RE: Need Help Configuring Solr To Work With Nutch
>
>
>Hello Erick ,
>
> This is what I see in the logs :
>
>[cid:image001.png@01D37018.62D3CC90]
>
>
>
>I am sorry it sbeen a while since I worked with solr. I did not do
>anything to specifically commit the changes to the core. Thanks for
>your prompt attention to this matter.
>
>
>
>Aratrika Mukhopadhyay
>
>
>
>-----Original Message-----
>From: Erick Erickson [mailto:erickerickson@gmail.com]
>Sent: Friday, December 08, 2017 11:06 AM
>To: solr-user
><so...@lucene.apache.org>>
>Subject: Re: Need Help Configuring Solr To Work With Nutch
>
>
>
>1> do you see update messages in the Solr logs?
>
>2> did you issue a commit?
>
>
>
>Best,
>
>Erick
>
>
>
>On Fri, Dec 8, 2017 at 7:27 AM, Mukhopadhyay, Aratrika <
>Aratrika.Mukhopadhyay@mail.house.gov<ma...@mail.house.gov>>
>wrote:
>
>
>
>> Good Morning,
>
>>
>
>> I am running nutch 2.3 , hbase 0.98 and I am integrating
>
>> nutch with solr 6.4. I have a successful crawl in nutch and when I
>see
>
>> that it is indexing the content into solr. However I cannot query and
>get any results.
>
>> Its as if Nutch isn’t writing anything to solr at all. I am stuck and
>
>> need someone who is familiar with solr/nutch to provide assistance.
>
>> Can someone please help ?
>
>>
>
>>
>
>>
>
>> This is what I see when I index into solr. I see no errors.
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> Regards,
>
>>
>
>> Aratrika Mukhopadhyay
>
>>
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
FW: Need Help Configuring Solr To Work With Nutch
Posted by "Mukhopadhyay, Aratrika" <Ar...@mail.house.gov>.
Erick,
Do I need to set the softCommit = true and prepareCommit to true in my solrconfig ? I am still at a loss as to what is happening. Thanks again for your help.
Aratrika
From: Mukhopadhyay, Aratrika
Sent: Friday, December 08, 2017 11:34 AM
To: solr-user <so...@lucene.apache.org>
Subject: RE: Need Help Configuring Solr To Work With Nutch
Hello Erick ,
This is what I see in the logs :
[cid:image001.png@01D37018.62D3CC90]
I am sorry it sbeen a while since I worked with solr. I did not do anything to specifically commit the changes to the core. Thanks for your prompt attention to this matter.
Aratrika Mukhopadhyay
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Friday, December 08, 2017 11:06 AM
To: solr-user <so...@lucene.apache.org>>
Subject: Re: Need Help Configuring Solr To Work With Nutch
1> do you see update messages in the Solr logs?
2> did you issue a commit?
Best,
Erick
On Fri, Dec 8, 2017 at 7:27 AM, Mukhopadhyay, Aratrika < Aratrika.Mukhopadhyay@mail.house.gov<ma...@mail.house.gov>> wrote:
> Good Morning,
>
> I am running nutch 2.3 , hbase 0.98 and I am integrating
> nutch with solr 6.4. I have a successful crawl in nutch and when I see
> that it is indexing the content into solr. However I cannot query and get any results.
> Its as if Nutch isn’t writing anything to solr at all. I am stuck and
> need someone who is familiar with solr/nutch to provide assistance.
> Can someone please help ?
>
>
>
> This is what I see when I index into solr. I see no errors.
>
>
>
>
>
>
>
> Regards,
>
> Aratrika Mukhopadhyay
>
RE: Need Help Configuring Solr To Work With Nutch
Posted by "Mukhopadhyay, Aratrika" <Ar...@mail.house.gov>.
Hello Erick ,
This is what I see in the logs :
[cid:image001.png@01D37018.62D3CC90]
I am sorry it sbeen a while since I worked with solr. I did not do anything to specifically commit the changes to the core. Thanks for your prompt attention to this matter.
Aratrika Mukhopadhyay
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Friday, December 08, 2017 11:06 AM
To: solr-user <so...@lucene.apache.org>
Subject: Re: Need Help Configuring Solr To Work With Nutch
1> do you see update messages in the Solr logs?
2> did you issue a commit?
Best,
Erick
On Fri, Dec 8, 2017 at 7:27 AM, Mukhopadhyay, Aratrika < Aratrika.Mukhopadhyay@mail.house.gov<ma...@mail.house.gov>> wrote:
> Good Morning,
>
> I am running nutch 2.3 , hbase 0.98 and I am integrating
> nutch with solr 6.4. I have a successful crawl in nutch and when I see
> that it is indexing the content into solr. However I cannot query and get any results.
> Its as if Nutch isn’t writing anything to solr at all. I am stuck and
> need someone who is familiar with solr/nutch to provide assistance.
> Can someone please help ?
>
>
>
> This is what I see when I index into solr. I see no errors.
>
>
>
>
>
>
>
> Regards,
>
> Aratrika Mukhopadhyay
>
Re: Need Help Configuring Solr To Work With Nutch
Posted by Erick Erickson <er...@gmail.com>.
1> do you see update messages in the Solr logs?
2> did you issue a commit?
Best,
Erick
On Fri, Dec 8, 2017 at 7:27 AM, Mukhopadhyay, Aratrika <
Aratrika.Mukhopadhyay@mail.house.gov> wrote:
> Good Morning,
>
> I am running nutch 2.3 , hbase 0.98 and I am integrating nutch
> with solr 6.4. I have a successful crawl in nutch and when I see that it is
> indexing the content into solr. However I cannot query and get any results.
> Its as if Nutch isn’t writing anything to solr at all. I am stuck and need
> someone who is familiar with solr/nutch to provide assistance. Can someone
> please help ?
>
>
>
> This is what I see when I index into solr. I see no errors.
>
>
>
>
>
>
>
> Regards,
>
> Aratrika Mukhopadhyay
>