You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by ahmed baseet <ah...@gmail.com> on 2009/04/27 12:00:05 UTC
How to post in-memory[not residing on local disks] Xml files to Solr
server for indexing?
Hi All,
I'm trying to post some files to Solr server. I've done this using the
post.jar files for posting xml files residing on my local disk[I tried
posting all those xml files from example directory]. Now I'm trying to
generate xml files on the fly, with required text to be indexed included
therein though, and want to post these files to solr. As per the examples
we've used "SimplePostTool" for posting locally resinding files but can some
one give me direction on indexing in-memory xml files[files generated on the
fly]. Actually I want to automate this process in a loop, so that I'll
extract some information and put that to xml file and push it off to Solr
for indexing.
Thanks in appreciation.
--Ahmed.
Re: How to post in-memory[not residing on local disks] Xml files to
Solr server for indexing?
Posted by ahmed baseet <ah...@gmail.com>.
Thank you very much. Now its working fine, fixed those minor classpath
issues.
Thanks,
Ahmed.
2009/4/28 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>
> the Solr distro contains all the jar files. you can take either the
> latest release (1.3) or a nightly
>
> On Tue, Apr 28, 2009 at 11:34 AM, ahmed baseet <ah...@gmail.com>
> wrote:
> > As far as I know, Maven is a build/mgmt tool for java projects quite
> similar
> > to Ant, right? No I'm not using this , then I think I don't need to worry
> > about those pom files.
> > But I'm still not able to figure out the error with classpath/jar files
> I
> > mentioned in my previous mails. Shall I try getting those jar files,
> > specifically that solr-solrj jar that contains commons-http-solr-server
> > class files? If yes then can you tell me where to get those jar files
> from,
> > on the web? Has anyone ever faced similar problems? Please help me
> fixing
> > these silly issues?
> >
> > Thanks,
> > Ahmed.
> > On Mon, Apr 27, 2009 at 6:59 PM, Shalin Shekhar Mangar <
> > shalinmangar@gmail.com> wrote:
> >
> >> On Mon, Apr 27, 2009 at 6:27 PM, ahmed baseet <ahmed.baseet@gmail.com
> >> >wrote:
> >>
> >> > Can anyone help me selecting the proper pom.xml file out of the bunch
> of
> >> > *-pom.xml.templates available.
> >> >
> >>
> >> Ahmed, are you using Maven? If not, then you do not need these pom
> files.
> >> If
> >> you are using Maven, then you need to add a dependency to solrj.
> >>
> >>
> >>
> http://wiki.apache.org/solr/Solrj#head-674dd7743df665fdd56e8eccddce16fc2de20e6e
> >>
> >> --
> >> Regards,
> >> Shalin Shekhar Mangar.
> >>
> >
>
>
>
> --
> --Noble Paul
>
Re: How to post in-memory[not residing on local disks] Xml files to
Solr server for indexing?
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
the Solr distro contains all the jar files. you can take either the
latest release (1.3) or a nightly
On Tue, Apr 28, 2009 at 11:34 AM, ahmed baseet <ah...@gmail.com> wrote:
> As far as I know, Maven is a build/mgmt tool for java projects quite similar
> to Ant, right? No I'm not using this , then I think I don't need to worry
> about those pom files.
> But I'm still not able to figure out the error with classpath/jar files I
> mentioned in my previous mails. Shall I try getting those jar files,
> specifically that solr-solrj jar that contains commons-http-solr-server
> class files? If yes then can you tell me where to get those jar files from,
> on the web? Has anyone ever faced similar problems? Please help me fixing
> these silly issues?
>
> Thanks,
> Ahmed.
> On Mon, Apr 27, 2009 at 6:59 PM, Shalin Shekhar Mangar <
> shalinmangar@gmail.com> wrote:
>
>> On Mon, Apr 27, 2009 at 6:27 PM, ahmed baseet <ahmed.baseet@gmail.com
>> >wrote:
>>
>> > Can anyone help me selecting the proper pom.xml file out of the bunch of
>> > *-pom.xml.templates available.
>> >
>>
>> Ahmed, are you using Maven? If not, then you do not need these pom files.
>> If
>> you are using Maven, then you need to add a dependency to solrj.
>>
>>
>> http://wiki.apache.org/solr/Solrj#head-674dd7743df665fdd56e8eccddce16fc2de20e6e
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>
--
--Noble Paul
Re: How to post in-memory[not residing on local disks] Xml files to
Solr server for indexing?
Posted by ahmed baseet <ah...@gmail.com>.
As far as I know, Maven is a build/mgmt tool for java projects quite similar
to Ant, right? No I'm not using this , then I think I don't need to worry
about those pom files.
But I'm still not able to figure out the error with classpath/jar files I
mentioned in my previous mails. Shall I try getting those jar files,
specifically that solr-solrj jar that contains commons-http-solr-server
class files? If yes then can you tell me where to get those jar files from,
on the web? Has anyone ever faced similar problems? Please help me fixing
these silly issues?
Thanks,
Ahmed.
On Mon, Apr 27, 2009 at 6:59 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> On Mon, Apr 27, 2009 at 6:27 PM, ahmed baseet <ahmed.baseet@gmail.com
> >wrote:
>
> > Can anyone help me selecting the proper pom.xml file out of the bunch of
> > *-pom.xml.templates available.
> >
>
> Ahmed, are you using Maven? If not, then you do not need these pom files.
> If
> you are using Maven, then you need to add a dependency to solrj.
>
>
> http://wiki.apache.org/solr/Solrj#head-674dd7743df665fdd56e8eccddce16fc2de20e6e
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
Re: How to post in-memory[not residing on local disks] Xml files to
Solr server for indexing?
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Mon, Apr 27, 2009 at 6:27 PM, ahmed baseet <ah...@gmail.com>wrote:
> Can anyone help me selecting the proper pom.xml file out of the bunch of
> *-pom.xml.templates available.
>
Ahmed, are you using Maven? If not, then you do not need these pom files. If
you are using Maven, then you need to add a dependency to solrj.
http://wiki.apache.org/solr/Solrj#head-674dd7743df665fdd56e8eccddce16fc2de20e6e
--
Regards,
Shalin Shekhar Mangar.
Re: How to post in-memory[not residing on local disks] Xml files to
Solr server for indexing?
Posted by ahmed baseet <ah...@gmail.com>.
Can anyone help me selecting the proper pom.xml file out of the bunch of
*-pom.xml.templates available.
I got the following when searched for pom.xml files,
solr-common-csv-pom.xml
solr-lucene-analyzers-pom.xml
solr-lucene-contrib-pom.xml
solr-lucene-*-pom.xml [ a lot of solr-lucene-... pom files are available,
hence shortened to avoid typing all]
solr-dataimporthandler-pom.xml
solr-common-pom.xml
solr-core-pom.xml
solr-parent-pom.xml
solr-solr-pom.xml
Thanks,
Ahmed.
On Mon, Apr 27, 2009 at 5:38 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> On Mon, Apr 27, 2009 at 4:53 PM, ahmed baseet <ahmed.baseet@gmail.com
> >wrote:
>
> >
> > To be precise it gives me the following error,
> > .....cannot find symbol:
> > symbol : class CommonsHttpSolrServer....
> >
> > I rechecked to make sure that "commons-httpclient-3.1.jar" is in the
> class
> > path. Can someone please point me what is the issue?
> >
> > I'm working on Windows and my classpath variable is this:
> >
> > .;E:\Program Files\Java\jdk1.6.0_05\bin;D:\firefox
> >
> >
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-httpclient-3.1.jar;D:\firefox
> >
> >
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\apache-solr-common.jar;D:\firefox
> >
> >
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\apache-solr-1.3.0.jar;D:\firefox
> >
> >
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\solr-solrj-1.3.0.jar;D:\firefox
> >
> >
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-io-1.3.1.jar;D:\firefox
> >
> >
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-codec-1.3.jar;D:\firefox
> >
> >
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-logging-1.0.4.jar
> >
>
> The jars look right. It is likely a problem with your classpath.
> CommonsHttpSolrServer is in the solr-solrj jar.
>
> If you are using Maven, then you'd need to change your pom.xml
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
Re: How to post in-memory[not residing on local disks] Xml files to
Solr server for indexing?
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Mon, Apr 27, 2009 at 4:53 PM, ahmed baseet <ah...@gmail.com>wrote:
>
> To be precise it gives me the following error,
> .....cannot find symbol:
> symbol : class CommonsHttpSolrServer....
>
> I rechecked to make sure that "commons-httpclient-3.1.jar" is in the class
> path. Can someone please point me what is the issue?
>
> I'm working on Windows and my classpath variable is this:
>
> .;E:\Program Files\Java\jdk1.6.0_05\bin;D:\firefox
>
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-httpclient-3.1.jar;D:\firefox
>
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\apache-solr-common.jar;D:\firefox
>
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\apache-solr-1.3.0.jar;D:\firefox
>
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\solr-solrj-1.3.0.jar;D:\firefox
>
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-io-1.3.1.jar;D:\firefox
>
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-codec-1.3.jar;D:\firefox
>
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-logging-1.0.4.jar
>
The jars look right. It is likely a problem with your classpath.
CommonsHttpSolrServer is in the solr-solrj jar.
If you are using Maven, then you'd need to change your pom.xml
--
Regards,
Shalin Shekhar Mangar.
Re: How to post in-memory[not residing on local disks] Xml files to
Solr server for indexing?
Posted by ahmed baseet <ah...@gmail.com>.
Hi,
After going through the solrj wiki I found that we've to set some
dependencies in pom.xml for using Solrj, which I haven't done yet. So I
googled to know how to do that but no help. I searched the solr directory
and found a bunch of *-pom.template files [like solr-core-pom.xml,
solr-solrj-pom.xml etc] and I'm not able to figure out which one to use. Any
help would be appreciated.
Thanks,
Ahmed.
On Mon, Apr 27, 2009 at 4:53 PM, ahmed baseet <ah...@gmail.com>wrote:
> Shalin, thanks for your quick response.
>
> Actually I'm trying to pull plaintext from html pages and trying to make
> xml files for each page. I went through the SolrJ webpage and found that the
> we've to add all the field and its contents anyway, right? but yes it makes
> adding/updating etc quite easier than using that SimplePostTool.
> I tried to use SolrJ client but it doesnot seem to be working. I added all
> the jar files mentioned in SolrJ wiki to classpath but still its giving me
> some error.
>
> To be precise it gives me the following error,
> .....cannot find symbol:
> symbol : class CommonsHttpSolrServer....
>
> I rechecked to make sure that "commons-httpclient-3.1.jar" is in the class
> path. Can someone please point me what is the issue?
>
> I'm working on Windows and my classpath variable is this:
>
> .;E:\Program Files\Java\jdk1.6.0_05\bin;D:\firefox
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-httpclient-3.1.jar;D:\firefox
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\apache-solr-common.jar;D:\firefox
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\apache-solr-1.3.0.jar;D:\firefox
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\solr-solrj-1.3.0.jar;D:\firefox
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-io-1.3.1.jar;D:\firefox
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-codec-1.3.jar;D:\firefox
> download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-logging-1.0.4.jar
>
> Thank you very much.
> Ahmed.
>
>
>
> On Mon, Apr 27, 2009 at 3:55 PM, Shalin Shekhar Mangar <
> shalinmangar@gmail.com> wrote:
>
>> On Mon, Apr 27, 2009 at 3:30 PM, ahmed baseet <ahmed.baseet@gmail.com
>> >wrote:
>>
>> > Hi All,
>> > I'm trying to post some files to Solr server. I've done this using the
>> > post.jar files for posting xml files residing on my local disk[I tried
>> > posting all those xml files from example directory]. Now I'm trying to
>> > generate xml files on the fly, with required text to be indexed included
>> > therein though, and want to post these files to solr. As per the
>> examples
>> > we've used "SimplePostTool" for posting locally resinding files but can
>> > some
>> > one give me direction on indexing in-memory xml files[files generated on
>> > the
>> > fly]. Actually I want to automate this process in a loop, so that I'll
>> > extract some information and put that to xml file and push it off to
>> Solr
>> > for indexing.
>> > Thanks in appreciation.
>> >
>>
>>
>> You can use the Solrj client to avoid building the intermediate XML
>> yourself. Extract the information, use the Solrj api to add the extracted
>> text to fields and send them to the solr server.
>>
>> http://wiki.apache.org/solr/Solrj
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>
>
Re: How to post in-memory[not residing on local disks] Xml files to
Solr server for indexing?
Posted by ahmed baseet <ah...@gmail.com>.
Shalin, thanks for your quick response.
Actually I'm trying to pull plaintext from html pages and trying to make xml
files for each page. I went through the SolrJ webpage and found that the
we've to add all the field and its contents anyway, right? but yes it makes
adding/updating etc quite easier than using that SimplePostTool.
I tried to use SolrJ client but it doesnot seem to be working. I added all
the jar files mentioned in SolrJ wiki to classpath but still its giving me
some error.
To be precise it gives me the following error,
.....cannot find symbol:
symbol : class CommonsHttpSolrServer....
I rechecked to make sure that "commons-httpclient-3.1.jar" is in the class
path. Can someone please point me what is the issue?
I'm working on Windows and my classpath variable is this:
.;E:\Program Files\Java\jdk1.6.0_05\bin;D:\firefox
download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-httpclient-3.1.jar;D:\firefox
download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\apache-solr-common.jar;D:\firefox
download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\apache-solr-1.3.0.jar;D:\firefox
download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\solr-solrj-1.3.0.jar;D:\firefox
download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-io-1.3.1.jar;D:\firefox
download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-codec-1.3.jar;D:\firefox
download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-logging-1.0.4.jar
Thank you very much.
Ahmed.
On Mon, Apr 27, 2009 at 3:55 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> On Mon, Apr 27, 2009 at 3:30 PM, ahmed baseet <ahmed.baseet@gmail.com
> >wrote:
>
> > Hi All,
> > I'm trying to post some files to Solr server. I've done this using the
> > post.jar files for posting xml files residing on my local disk[I tried
> > posting all those xml files from example directory]. Now I'm trying to
> > generate xml files on the fly, with required text to be indexed included
> > therein though, and want to post these files to solr. As per the examples
> > we've used "SimplePostTool" for posting locally resinding files but can
> > some
> > one give me direction on indexing in-memory xml files[files generated on
> > the
> > fly]. Actually I want to automate this process in a loop, so that I'll
> > extract some information and put that to xml file and push it off to Solr
> > for indexing.
> > Thanks in appreciation.
> >
>
>
> You can use the Solrj client to avoid building the intermediate XML
> yourself. Extract the information, use the Solrj api to add the extracted
> text to fields and send them to the solr server.
>
> http://wiki.apache.org/solr/Solrj
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
Re: How to post in-memory[not residing on local disks] Xml files to
Solr server for indexing?
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Mon, Apr 27, 2009 at 3:30 PM, ahmed baseet <ah...@gmail.com>wrote:
> Hi All,
> I'm trying to post some files to Solr server. I've done this using the
> post.jar files for posting xml files residing on my local disk[I tried
> posting all those xml files from example directory]. Now I'm trying to
> generate xml files on the fly, with required text to be indexed included
> therein though, and want to post these files to solr. As per the examples
> we've used "SimplePostTool" for posting locally resinding files but can
> some
> one give me direction on indexing in-memory xml files[files generated on
> the
> fly]. Actually I want to automate this process in a loop, so that I'll
> extract some information and put that to xml file and push it off to Solr
> for indexing.
> Thanks in appreciation.
>
You can use the Solrj client to avoid building the intermediate XML
yourself. Extract the information, use the Solrj api to add the extracted
text to fields and send them to the solr server.
http://wiki.apache.org/solr/Solrj
--
Regards,
Shalin Shekhar Mangar.