You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Sowmya V.B." <vb...@gmail.com> on 2011/07/07 11:38:14 UTC

(Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Hi

I am trying to add UIMA module in to Solr..and began with the readme file
given here.
https://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_1/solr/contrib/uima/README.txt

I am confused about some points in the readme file and hence the email.

2. modify your schema.xml adding the fields you want to be hold
metadata specifying proper values for type, indexed, stored and
multiValued options:

-I understood this line as: adding to my schema.xml, the new fields that
will come as a result of a UIMA pipeline. For example, in my UIMA pipeline,
post-processing, I get fields A,B,C in addition to fields X,Y,Z that I
already added to the SolrInputDocument. So, does this mean I should add
A,B,C to the schema.xml?

3. In SolrConfig.xml,

inside,

<uimaConfig>
    <runtimeParameters>

if iam not using any of those "alchemy api key..." etc, I think I can remove
those lines. However, I plan to use the openNLP tagger & tokenizer, and an
annotator I wrote for my task. Can I give my model file locations here as
runtimeParameters?

4. I did not understand what "fieldMapping" tag does. The description said:
"field mapping describes which features of which types should go in a
field"--
- For example, in this snippet from the link:

 <type name="org.apache.uima.alchemy.ts.concept.ConceptFS">
       <map feature="text" field="concept"/>
      </type>

-what does "feature" mean and what does "field" mean?


I did not understand the fieldmapping tag right and did not find any help in
previous mails. Hence, mailing the group. Sorry for the long mail!

Regards
Sowmya V.B.
----------------------------------------------------
Losing optimism is blasphemy!
http://vbsowmya.wordpress.com
----------------------------------------------------

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by "Sowmya V.B." <vb...@gmail.com>.
Can someone help me with this please?

I am not able to understand from the readme.txt file provided in the
trunk...how to plugin my own annotator in to solr.

Sowmya.

On Thu, Jul 7, 2011 at 11:38 AM, Sowmya V.B. <vb...@gmail.com> wrote:

> Hi
>
> I am trying to add UIMA module in to Solr..and began with the readme file
> given here.
>
> https://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_1/solr/contrib/uima/README.txt
>
> I am confused about some points in the readme file and hence the email.
>
> 2. modify your schema.xml adding the fields you want to be hold metadata specifying proper values for type, indexed, stored and multiValued options:
>
> -I understood this line as: adding to my schema.xml, the new fields that
> will come as a result of a UIMA pipeline. For example, in my UIMA pipeline,
> post-processing, I get fields A,B,C in addition to fields X,Y,Z that I
> already added to the SolrInputDocument. So, does this mean I should add
> A,B,C to the schema.xml?
>
> 3. In SolrConfig.xml,
>
> inside,
>
> <uimaConfig>
>     <runtimeParameters>
>
> if iam not using any of those "alchemy api key..." etc, I think I can
> remove those lines. However, I plan to use the openNLP tagger & tokenizer,
> and an annotator I wrote for my task. Can I give my model file locations
> here as runtimeParameters?
>
> 4. I did not understand what "fieldMapping" tag does. The description said:
> "field mapping describes which features of which types should go in a
> field"--
> - For example, in this snippet from the link:
>
>  <type name="org.apache.uima.alchemy.ts.concept.ConceptFS">
>        <map feature="text" field="concept"/>
>       </type>
>
> -what does "feature" mean and what does "field" mean?
>
>
> I did not understand the fieldmapping tag right and did not find any help
> in previous mails. Hence, mailing the group. Sorry for the long mail!
>
> Regards
> Sowmya V.B.
> ----------------------------------------------------
> Losing optimism is blasphemy!
> http://vbsowmya.wordpress.com
> ----------------------------------------------------
>



-- 
Sowmya V.B.
----------------------------------------------------
Losing optimism is blasphemy!
http://vbsowmya.wordpress.com
----------------------------------------------------

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by Sujit Pal <su...@comcast.net>.
Hi Sowmya,

I basically wrote an annotator and built a buffering tokenizer around it
so I could include it in a Lucene analyzer pipeline. I've blogged about
it, not sure if its good form to include links to blog posts in public
forums, but here they are, apologies in advance if this is wrong (let me
know and I won't do it again).

http://sujitpal.blogspot.com/2011/06/uima-analysis-engine-for-keyword.html 
http://sujitpal.blogspot.com/2011/06/running-uima-analysis-engine-in-lucene.html 

Of course, this is in Lucene land. I haven't worked with the SOLR-UIMA
stuff so this may not answer your question directly. But I think if you
build an Tokenizer or TokenFilter then you can declare it as an analyzer
chain in SOLR.

HTH
Sujit

On Fri, 2011-07-08 at 09:19 +0200, Sowmya V.B. wrote:
> Hi Koji
> 
> Thanks for the mail.
> 
> Thanks for all the clarifications. I am now using the version 3.3.. But,
> another query that I have about this is:
> How can I add an annotator that I wrote myself, in to Solr-UIMA?
> 
> Here is what I did before I moved to Solr:
> I wrote an annotator (which worked when I used plain vanilla lucene based
> indexer), which enriched the document with more fields (Some statistics
> about the document...all fields added were numeric fields). Those fields
> were added to the index by extending *JCasAnnotator_ImplBase* class.
> 
> But, in Solr-UIMA, I am not exactly clear on where the above setup fits in.
> I thought I would get an idea looking at the annotators that came with the
> UIMA integration of Solr, but their source was not available. So, I do not
> understand how to actually integrate my own annotator in to UIMA.
> 
> Can you please explain on how to go about this?
> 
> Sowmya.
> 
> On Fri, Jul 8, 2011 at 2:03 AM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:
> 
> > (11/07/07 18:38), Sowmya V.B. wrote:
> >
> >> Hi
> >>
> >> I am trying to add UIMA module in to Solr..and began with the readme file
> >> given here.
> >> https://svn.apache.org/repos/**asf/lucene/dev/tags/lucene_**
> >> solr_3_1/solr/contrib/uima/**README.txt<https://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_1/solr/contrib/uima/README.txt>
> >>
> >
> > I would recommend you to use Solr 3.3 rather than 3.1, as we have changed
> > some configuration
> > in solrconfig.xml for UIMA.
> >
> >
> >  2. modify your schema.xml adding the fields you want to be hold
> >> metadata specifying proper values for type, indexed, stored and
> >> multiValued options:
> >>
> >> -I understood this line as: adding to my schema.xml, the new fields that
> >> will come as a result of a UIMA pipeline. For example, in my UIMA
> >> pipeline,
> >> post-processing, I get fields A,B,C in addition to fields X,Y,Z that I
> >> already added to the SolrInputDocument. So, does this mean I should add
> >> A,B,C to the schema.xml?
> >>
> >
> > I think you got it. Have you tried it but you got some errors?
> >
> >
> >  3. In SolrConfig.xml,
> >>
> >> inside,
> >>
> >> <uimaConfig>
> >>     <runtimeParameters>
> >>
> >
> > The uimaConfig tag has been moved into update processor setting @ Solr 3.2.
> > Please see the latest README.txt.
> >
> >
> >  if iam not using any of those "alchemy api key..." etc, I think I can
> >> remove
> >> those lines. However, I plan to use the openNLP tagger&  tokenizer, and an
> >> annotator I wrote for my task. Can I give my model file locations here as
> >> runtimeParameters?
> >>
> >
> > I don't have an idea of openNLP.
> >
> >
> >  4. I did not understand what "fieldMapping" tag does. The description
> >> said:
> >> "field mapping describes which features of which types should go in a
> >> field"--
> >> - For example, in this snippet from the link:
> >>
> >>  <type name="org.apache.uima.alchemy.**ts.concept.ConceptFS">
> >>        <map feature="text" field="concept"/>
> >>       </type>
> >>
> >> -what does "feature" mean and what does "field" mean?
> >>
> >
> > This defines a map uima feature:
> >
> > http://uima.apache.org/d/**uimaj-2.3.1/references.html#**
> > ugr.ref.xml.component_**descriptor.type_system.**features<http://uima.apache.org/d/uimaj-2.3.1/references.html#ugr.ref.xml.component_descriptor.type_system.features>
> >
> > to Solr field.
> >
> > koji
> > --
> > http://www.rondhuit.com/en/
> >
> 
> 
> 


Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by "Sowmya V.B." <vb...@gmail.com>.
Hi

I've fixed the missing license issues by attaching a license.txt file to my
jar files.

Everything gets built now, with a single error.

/Users/svajjala/Downloads/apache-solr-3.3.0/solr/contrib/uima/src/main/java/org/apache/solr/uima/processor/UIMAToSolrMapper.java:62:
type org.apache.uima.cas.FeatureStructure does not take parameters

[javac]         FeatureStructure<FeatureStructure> fs = iterator.next();

Any ideas if UIMA class definitions changed dramatically? I used 2.2.0...and
the present one which came with Solr 3.3 seems to be 2.3.1.

Sowmya.

On Tue, Jul 12, 2011 at 2:13 PM, Sowmya V.B. <vb...@gmail.com> wrote:

> Hi Koji
>
> Yes, I do use SolrJ.
>
> I began recompiling the whole thing... since I thought the problem is the
> UIMA snapshot.
> Previously, I compiled files from eclipse and it worked fine.
> (Now I realize that eclipse compiled it because I added my jar files to its
> build path)
>
> I am now getting build errors when I say "ant clean dist" inside
> solr/contrib/uima, after adding all my requisite annotators and libraries
> inside it at requisite places.
> it says:
>
> There are missing LICENSE files in:
> /Users/svajjala/Downloads/apache-solr-3.3.0/solr/contrib/uima/lib Jar file
> count: 13 License Count: 6
> There may be missing NOTICE files in:
> /Users/svajjala/Downloads/apache-solr-3.3.0/solr/contrib/uima/lib.  Note,
> not all files require a NOTICE. Jar file count: 13 Notice Count: 6
>
> But the other 7 in these are the jar files that I added, for my annotators
> to work. They don't have any license or notice files. How should I go about
> this now?
>
> S
>
>
> On Tue, Jul 12, 2011 at 1:55 PM, Koji Sekiguchi <ko...@r.email.ne.jp>wrote:
>
>> Hmm, I'm bit confused. Do you really use SolrJ ?
>>
>> If so:
>>
>>
>> > If I put it inside /update, the following is the stacktrace:
>> > request:
>> > http://localhost:8080/apache-**solr-3.3.0/update/javabin?wt=**
>> javabin&version=2<http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2>
>> > org.apache.solr.common.**SolrException: Bad Request
>>
>> what did you mean by "it" and why did you put it inside /update, but still
>> tried to use /update/javabin ?
>>
>>
>> koji
>> --
>> http://www.rondhuit.com/en/
>>
>>
>> (11/07/12 16:10), Sowmya V.B. wrote:
>>
>>> Yes, I do have an '/update/javabin' request handler in SolrConfig.
>>>
>>> But, should I remove that?
>>>
>>> I tried putting the UIMA update chain inside /update/javabin instead of
>>> /update request handler..
>>> <requestHandler name="/update/javabin"
>>>                   class="solr.**BinaryUpdateRequestHandler">
>>> <lst name="defaults">
>>>     <str name="update.chain">uima</str>
>>>     </lst>
>>>   </requestHandler>
>>>
>>> .and here is the stacktrace:
>>>
>>> request:
>>> http://localhost:8080/apache-**solr-3.3.0/update/javabin?wt=**
>>> javabin&version=2<http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2>
>>> org.apache.solr.common.**SolrException: Internal Server Error
>>>
>>> Internal Server Error
>>>
>>> request:
>>> http://localhost:8080/apache-**solr-3.3.0/update/javabin?wt=**
>>> javabin&version=2<http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2>
>>>     at
>>> org.apache.solr.client.solrj.**impl.CommonsHttpSolrServer.**
>>> request(CommonsHttpSolrServer.**java:435)
>>>     at
>>> org.apache.solr.client.solrj.**impl.CommonsHttpSolrServer.**
>>> request(CommonsHttpSolrServer.**java:244)
>>>     at
>>> org.apache.solr.client.solrj.**request.AbstractUpdateRequest.**
>>> process(AbstractUpdateRequest.**java:105)
>>>     at org.apache.solr.client.solrj.**SolrServer.add(SolrServer.**
>>> java:49)
>>>     at indexerapp.ir4llSolrIndexer.**indexAll(ir4llSolrIndexer.**
>>> java:150)
>>>     at indexerapp.ir4llSolrIndexer.**main(ir4llSolrIndexer.java:57)
>>>
>>>
>>> *********
>>>
>>> If I put it inside /update, the following is the stacktrace:
>>> request:
>>> http://localhost:8080/apache-**solr-3.3.0/update/javabin?wt=**
>>> javabin&version=2<http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2>
>>> org.apache.solr.common.**SolrException: Bad Request
>>>
>>> Bad Request
>>>
>>> request:
>>> http://localhost:8080/apache-**solr-3.3.0/update/javabin?wt=**
>>> javabin&version=2<http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2>
>>>     at
>>> org.apache.solr.client.solrj.**impl.CommonsHttpSolrServer.**
>>> request(CommonsHttpSolrServer.**java:435)
>>>     at
>>> org.apache.solr.client.solrj.**impl.CommonsHttpSolrServer.**
>>> request(CommonsHttpSolrServer.**java:244)
>>>     at
>>> org.apache.solr.client.solrj.**request.AbstractUpdateRequest.**
>>> process(AbstractUpdateRequest.**java:105)
>>>     at org.apache.solr.client.solrj.**SolrServer.add(SolrServer.**
>>> java:49)
>>>     at indexerapp.ir4llSolrIndexer.**indexAll(ir4llSolrIndexer.**
>>> java:150)
>>>     at indexerapp.ir4llSolrIndexer.**main(ir4llSolrIndexer.java:57)
>>> ******
>>>
>>> But, I still don't undetstand where I can see a more detailed Log of Solr
>>> Server.
>>> On my tomcat logs (Iam running from Eclipse), (path:
>>> /Users/svajjala/Documents/**workspace/.metadata/.plugins/**
>>> org.eclipse.wst.server.core/**tmp0/logs)
>>> -I dont see anything except a single line:
>>>
>>> 134.2.129.160 - - [12/Jul/2011:09:02:16 +0200] "POST
>>> /apache-solr-3.3.0/update/**javabin?wt=javabin&version=2 HTTP/1.1" 400
>>> 1262
>>>
>>> It is difficult to understand whats going on. Can anyone tell me where I
>>> can
>>> see a more detailed log?
>>>
>>> S.
>>>
>>>
>>> On Tue, Jul 12, 2011 at 2:39 AM, Koji Sekiguchi<ko...@r.email.ne.jp>
>>>  wrote:
>>>
>>>  I don't think you have wrong setting in UIMA, but you may have the
>>>> request
>>>> handler
>>>> named "/update/javabin" in solrconfig.xml is not correct?
>>>>
>>>>
>>>> koji
>>>> --
>>>> http://www.rondhuit.com/en/
>>>>
>>>> (11/07/12 0:52), Sowmya V.B. wrote:
>>>>
>>>>  Hi
>>>>>
>>>>> I just added the fields which are added to the index by one of the
>>>>> annotators to the index, in the fieldmappings section. I am not getting
>>>>> any
>>>>> compilation errors and still see the admin interface. However, when I
>>>>> index,
>>>>> i just get a SolrException,
>>>>>
>>>>> org.apache.solr.common.****SolrException: Bad Request.
>>>>>
>>>>> On the server log, I don't see anything except for this:
>>>>> 127.0.0.1 - - [11/Jul/2011:17:44:04 +0200]  "POST
>>>>> /apache-solr-3.3.0/update/****javabin?wt=javabin&version=2 HTTP/1.1"
>>>>> 400
>>>>> 1328
>>>>>
>>>>> Here is my UpdateRequestProcessorChain in sorlconfig.xml (Just changed
>>>>> the
>>>>> original path names for privacy's sake!)
>>>>>
>>>>> ***********
>>>>>     <!-- An update request processor chain for UIMA   -->
>>>>> <updateRequestProcessorChain name="uima">
>>>>>     <processor
>>>>> class="org.apache.solr.uima.****processor.****
>>>>> UIMAUpdateRequestProcessorFact***
>>>>>
>>>>> *ory">
>>>>>       <lst name="uimaConfig">
>>>>>         <lst name="runtimeParameters">
>>>>>         <!-- Add here, the runtime parameters like Model File paths,
>>>>> list
>>>>> file paths etc. -->
>>>>>           <str
>>>>> name="****tokenizerModelFileLocation">/****Users/svajjala/Documents/**
>>>>> EnglishTok.bin.gz</str>
>>>>>           <str
>>>>> name="taggerModelFileLocation"****>/Users/svajjala/Documents/****
>>>>> tag.bin.gz</str>
>>>>>          <str name="AcceptedLanguages">****english</str>
>>>>>             <str name="****SetLanguageOnEmptyAcceptList">**
>>>>> **false</str>
>>>>>              <str
>>>>> name="GreenlineLists">/Users/****svajjala/Documents/****
>>>>> NewGreenline</str>
>>>>>        </lst>
>>>>>
>>>>>         <str
>>>>> name="analysisEngine">/Users/****svajjala/Documents/**
>>>>> ir4icallPipeline.xml</str>
>>>>>         <!-- Set to true if you want to continue indexing even if text
>>>>> processing fails.
>>>>>              Default is false. That is, Solr throws RuntimeException
>>>>> and
>>>>>              never indexed documents entirely in your session. -->
>>>>>            <bool name="ignoreErrors">false</****bool>
>>>>>         <!-- This is optional. It is used for logging when text
>>>>> processing
>>>>> fails.
>>>>>              If logField is not specified, uniqueKey will be used as
>>>>> logField.
>>>>>         <str name="logField">id</str>
>>>>>         -->
>>>>>         <lst name="analyzeFields">
>>>>>           <bool name="merge">false</bool>
>>>>>           <arr name="fields">
>>>>>             <str>text</str>
>>>>>           </arr>
>>>>>         </lst>
>>>>>
>>>>>          <fieldMapping>
>>>>>
>>>>>             <type name = "org.apache.uima.annotators.**
>>>>> vocablistClassfier">
>>>>>                 <map feature="Generic_Level5" field="Generic_Level5" />
>>>>>                 <map feature="Generic_Level6" field="Generic_Level6" />
>>>>>                 <map feature="Generic_Level7" field="Generic_Level7" />
>>>>>                 <map feature="Generic_Level8" field="Generic_Level8" />
>>>>>                 <map feature="Generic_Level9" field="Generic_Level9" />
>>>>>                 <map feature="Generic_Level10" field="Generic_Level10"
>>>>> />
>>>>>                 <map feature="Generic_Level11" field="Generic_Level11"
>>>>> />
>>>>>                 <map feature="Generic_Ratio" field="Generic_Ratio" />
>>>>>                 <map feature="Generic_****TotalWordCount"
>>>>> field="Generic_TotalWordCount" />
>>>>>             </type>
>>>>>
>>>>>         </fieldMapping>
>>>>>       </lst>
>>>>>     </processor>
>>>>>     <processor class="solr.****LogUpdateProcessorFactory" />
>>>>>     <processor class="solr.****RunUpdateProcessorFactory" />
>>>>>   </updateRequestProcessorChain>
>>>>>
>>>>>     ************
>>>>> I don't understand where exactly can I see a more detailed log of why
>>>>> its
>>>>> not getting indexed.
>>>>>
>>>>> Sowmya.
>>>>>
>>>>> On Mon, Jul 11, 2011 at 5:26 PM, Koji Sekiguchi<ko...@r.email.ne.jp>
>>>>>  wrote:
>>>>>
>>>>>  disclaimer: I'm not an expert of UIMA. I've just started using it when
>>>>>
>>>>>> Solr
>>>>>> 3.1
>>>>>> integrated UIMA!
>>>>>>
>>>>>>
>>>>>>  Thanks for the clarification. Now, I get it.
>>>>>>
>>>>>>  Should<fieldMapping>    section mention all the annotators, even if
>>>>>>> the
>>>>>>> annotators do not add any new fields?
>>>>>>>
>>>>>>>
>>>>>>>   For example, if I have a pipeline, starting from "parser",
>>>>>> "tokenizer"
>>>>>> and
>>>>>>
>>>>>>  "tagger", all of them operate on a field called "text"..which is
>>>>>>> the<html>
>>>>>>> of the document. but all these annotators do not add any new fields
>>>>>>> to
>>>>>>> the
>>>>>>> index. Should I still write fieldmappings for these annotators inside
>>>>>>> SolrConfig.?
>>>>>>>
>>>>>>>
>>>>>>>  I don't think so. Try it and if you got errors, come back to this
>>>>>> list!
>>>>>>
>>>>>>
>>>>>> koji
>>>>>> --
>>>>>> http://www.rondhuit.com/en/
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
> --
> Sowmya V.B.
> ----------------------------------------------------
> Losing optimism is blasphemy!
> http://vbsowmya.wordpress.com
> ----------------------------------------------------
>



-- 
Sowmya V.B.
----------------------------------------------------
Losing optimism is blasphemy!
http://vbsowmya.wordpress.com
----------------------------------------------------

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by "Sowmya V.B." <vb...@gmail.com>.
Hi Koji

Yes, I do use SolrJ.

I began recompiling the whole thing... since I thought the problem is the
UIMA snapshot.
Previously, I compiled files from eclipse and it worked fine.
(Now I realize that eclipse compiled it because I added my jar files to its
build path)

I am now getting build errors when I say "ant clean dist" inside
solr/contrib/uima, after adding all my requisite annotators and libraries
inside it at requisite places.
it says:

There are missing LICENSE files in:
/Users/svajjala/Downloads/apache-solr-3.3.0/solr/contrib/uima/lib Jar file
count: 13 License Count: 6
There may be missing NOTICE files in:
/Users/svajjala/Downloads/apache-solr-3.3.0/solr/contrib/uima/lib.  Note,
not all files require a NOTICE. Jar file count: 13 Notice Count: 6

But the other 7 in these are the jar files that I added, for my annotators
to work. They don't have any license or notice files. How should I go about
this now?

S

On Tue, Jul 12, 2011 at 1:55 PM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:

> Hmm, I'm bit confused. Do you really use SolrJ ?
>
> If so:
>
>
> > If I put it inside /update, the following is the stacktrace:
> > request:
> > http://localhost:8080/apache-**solr-3.3.0/update/javabin?wt=**
> javabin&version=2<http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2>
> > org.apache.solr.common.**SolrException: Bad Request
>
> what did you mean by "it" and why did you put it inside /update, but still
> tried to use /update/javabin ?
>
>
> koji
> --
> http://www.rondhuit.com/en/
>
>
> (11/07/12 16:10), Sowmya V.B. wrote:
>
>> Yes, I do have an '/update/javabin' request handler in SolrConfig.
>>
>> But, should I remove that?
>>
>> I tried putting the UIMA update chain inside /update/javabin instead of
>> /update request handler..
>> <requestHandler name="/update/javabin"
>>                   class="solr.**BinaryUpdateRequestHandler">
>> <lst name="defaults">
>>     <str name="update.chain">uima</str>
>>     </lst>
>>   </requestHandler>
>>
>> .and here is the stacktrace:
>>
>> request:
>> http://localhost:8080/apache-**solr-3.3.0/update/javabin?wt=**
>> javabin&version=2<http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2>
>> org.apache.solr.common.**SolrException: Internal Server Error
>>
>> Internal Server Error
>>
>> request:
>> http://localhost:8080/apache-**solr-3.3.0/update/javabin?wt=**
>> javabin&version=2<http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2>
>>     at
>> org.apache.solr.client.solrj.**impl.CommonsHttpSolrServer.**
>> request(CommonsHttpSolrServer.**java:435)
>>     at
>> org.apache.solr.client.solrj.**impl.CommonsHttpSolrServer.**
>> request(CommonsHttpSolrServer.**java:244)
>>     at
>> org.apache.solr.client.solrj.**request.AbstractUpdateRequest.**
>> process(AbstractUpdateRequest.**java:105)
>>     at org.apache.solr.client.solrj.**SolrServer.add(SolrServer.**
>> java:49)
>>     at indexerapp.ir4llSolrIndexer.**indexAll(ir4llSolrIndexer.**
>> java:150)
>>     at indexerapp.ir4llSolrIndexer.**main(ir4llSolrIndexer.java:57)
>>
>>
>> *********
>>
>> If I put it inside /update, the following is the stacktrace:
>> request:
>> http://localhost:8080/apache-**solr-3.3.0/update/javabin?wt=**
>> javabin&version=2<http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2>
>> org.apache.solr.common.**SolrException: Bad Request
>>
>> Bad Request
>>
>> request:
>> http://localhost:8080/apache-**solr-3.3.0/update/javabin?wt=**
>> javabin&version=2<http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2>
>>     at
>> org.apache.solr.client.solrj.**impl.CommonsHttpSolrServer.**
>> request(CommonsHttpSolrServer.**java:435)
>>     at
>> org.apache.solr.client.solrj.**impl.CommonsHttpSolrServer.**
>> request(CommonsHttpSolrServer.**java:244)
>>     at
>> org.apache.solr.client.solrj.**request.AbstractUpdateRequest.**
>> process(AbstractUpdateRequest.**java:105)
>>     at org.apache.solr.client.solrj.**SolrServer.add(SolrServer.**
>> java:49)
>>     at indexerapp.ir4llSolrIndexer.**indexAll(ir4llSolrIndexer.**
>> java:150)
>>     at indexerapp.ir4llSolrIndexer.**main(ir4llSolrIndexer.java:57)
>> ******
>>
>> But, I still don't undetstand where I can see a more detailed Log of Solr
>> Server.
>> On my tomcat logs (Iam running from Eclipse), (path:
>> /Users/svajjala/Documents/**workspace/.metadata/.plugins/**
>> org.eclipse.wst.server.core/**tmp0/logs)
>> -I dont see anything except a single line:
>>
>> 134.2.129.160 - - [12/Jul/2011:09:02:16 +0200] "POST
>> /apache-solr-3.3.0/update/**javabin?wt=javabin&version=2 HTTP/1.1" 400
>> 1262
>>
>> It is difficult to understand whats going on. Can anyone tell me where I
>> can
>> see a more detailed log?
>>
>> S.
>>
>>
>> On Tue, Jul 12, 2011 at 2:39 AM, Koji Sekiguchi<ko...@r.email.ne.jp>
>>  wrote:
>>
>>  I don't think you have wrong setting in UIMA, but you may have the
>>> request
>>> handler
>>> named "/update/javabin" in solrconfig.xml is not correct?
>>>
>>>
>>> koji
>>> --
>>> http://www.rondhuit.com/en/
>>>
>>> (11/07/12 0:52), Sowmya V.B. wrote:
>>>
>>>  Hi
>>>>
>>>> I just added the fields which are added to the index by one of the
>>>> annotators to the index, in the fieldmappings section. I am not getting
>>>> any
>>>> compilation errors and still see the admin interface. However, when I
>>>> index,
>>>> i just get a SolrException,
>>>>
>>>> org.apache.solr.common.****SolrException: Bad Request.
>>>>
>>>> On the server log, I don't see anything except for this:
>>>> 127.0.0.1 - - [11/Jul/2011:17:44:04 +0200]  "POST
>>>> /apache-solr-3.3.0/update/****javabin?wt=javabin&version=2 HTTP/1.1"
>>>> 400
>>>> 1328
>>>>
>>>> Here is my UpdateRequestProcessorChain in sorlconfig.xml (Just changed
>>>> the
>>>> original path names for privacy's sake!)
>>>>
>>>> ***********
>>>>     <!-- An update request processor chain for UIMA   -->
>>>> <updateRequestProcessorChain name="uima">
>>>>     <processor
>>>> class="org.apache.solr.uima.****processor.****
>>>> UIMAUpdateRequestProcessorFact***
>>>>
>>>> *ory">
>>>>       <lst name="uimaConfig">
>>>>         <lst name="runtimeParameters">
>>>>         <!-- Add here, the runtime parameters like Model File paths,
>>>> list
>>>> file paths etc. -->
>>>>           <str
>>>> name="****tokenizerModelFileLocation">/****Users/svajjala/Documents/**
>>>> EnglishTok.bin.gz</str>
>>>>           <str
>>>> name="taggerModelFileLocation"****>/Users/svajjala/Documents/****
>>>> tag.bin.gz</str>
>>>>          <str name="AcceptedLanguages">****english</str>
>>>>             <str name="****SetLanguageOnEmptyAcceptList">**
>>>> **false</str>
>>>>              <str
>>>> name="GreenlineLists">/Users/****svajjala/Documents/****
>>>> NewGreenline</str>
>>>>        </lst>
>>>>
>>>>         <str
>>>> name="analysisEngine">/Users/****svajjala/Documents/**
>>>> ir4icallPipeline.xml</str>
>>>>         <!-- Set to true if you want to continue indexing even if text
>>>> processing fails.
>>>>              Default is false. That is, Solr throws RuntimeException and
>>>>              never indexed documents entirely in your session. -->
>>>>            <bool name="ignoreErrors">false</****bool>
>>>>         <!-- This is optional. It is used for logging when text
>>>> processing
>>>> fails.
>>>>              If logField is not specified, uniqueKey will be used as
>>>> logField.
>>>>         <str name="logField">id</str>
>>>>         -->
>>>>         <lst name="analyzeFields">
>>>>           <bool name="merge">false</bool>
>>>>           <arr name="fields">
>>>>             <str>text</str>
>>>>           </arr>
>>>>         </lst>
>>>>
>>>>          <fieldMapping>
>>>>
>>>>             <type name = "org.apache.uima.annotators.**
>>>> vocablistClassfier">
>>>>                 <map feature="Generic_Level5" field="Generic_Level5" />
>>>>                 <map feature="Generic_Level6" field="Generic_Level6" />
>>>>                 <map feature="Generic_Level7" field="Generic_Level7" />
>>>>                 <map feature="Generic_Level8" field="Generic_Level8" />
>>>>                 <map feature="Generic_Level9" field="Generic_Level9" />
>>>>                 <map feature="Generic_Level10" field="Generic_Level10"
>>>> />
>>>>                 <map feature="Generic_Level11" field="Generic_Level11"
>>>> />
>>>>                 <map feature="Generic_Ratio" field="Generic_Ratio" />
>>>>                 <map feature="Generic_****TotalWordCount"
>>>> field="Generic_TotalWordCount" />
>>>>             </type>
>>>>
>>>>         </fieldMapping>
>>>>       </lst>
>>>>     </processor>
>>>>     <processor class="solr.****LogUpdateProcessorFactory" />
>>>>     <processor class="solr.****RunUpdateProcessorFactory" />
>>>>   </updateRequestProcessorChain>
>>>>
>>>>     ************
>>>> I don't understand where exactly can I see a more detailed log of why
>>>> its
>>>> not getting indexed.
>>>>
>>>> Sowmya.
>>>>
>>>> On Mon, Jul 11, 2011 at 5:26 PM, Koji Sekiguchi<ko...@r.email.ne.jp>
>>>>  wrote:
>>>>
>>>>  disclaimer: I'm not an expert of UIMA. I've just started using it when
>>>>
>>>>> Solr
>>>>> 3.1
>>>>> integrated UIMA!
>>>>>
>>>>>
>>>>>  Thanks for the clarification. Now, I get it.
>>>>>
>>>>>  Should<fieldMapping>    section mention all the annotators, even if
>>>>>> the
>>>>>> annotators do not add any new fields?
>>>>>>
>>>>>>
>>>>>>   For example, if I have a pipeline, starting from "parser",
>>>>> "tokenizer"
>>>>> and
>>>>>
>>>>>  "tagger", all of them operate on a field called "text"..which is
>>>>>> the<html>
>>>>>> of the document. but all these annotators do not add any new fields to
>>>>>> the
>>>>>> index. Should I still write fieldmappings for these annotators inside
>>>>>> SolrConfig.?
>>>>>>
>>>>>>
>>>>>>  I don't think so. Try it and if you got errors, come back to this
>>>>> list!
>>>>>
>>>>>
>>>>> koji
>>>>> --
>>>>> http://www.rondhuit.com/en/
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>


-- 
Sowmya V.B.
----------------------------------------------------
Losing optimism is blasphemy!
http://vbsowmya.wordpress.com
----------------------------------------------------

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
Hmm, I'm bit confused. Do you really use SolrJ ?

If so:

 > If I put it inside /update, the following is the stacktrace:
 > request:
 > http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2
 > org.apache.solr.common.SolrException: Bad Request

what did you mean by "it" and why did you put it inside /update, but still
tried to use /update/javabin ?

koji
-- 
http://www.rondhuit.com/en/


(11/07/12 16:10), Sowmya V.B. wrote:
> Yes, I do have an '/update/javabin' request handler in SolrConfig.
>
> But, should I remove that?
>
> I tried putting the UIMA update chain inside /update/javabin instead of
> /update request handler..
> <requestHandler name="/update/javabin"
>                    class="solr.BinaryUpdateRequestHandler">
> <lst name="defaults">
>      <str name="update.chain">uima</str>
>      </lst>
>    </requestHandler>
>
> .and here is the stacktrace:
>
> request:
> http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2
> org.apache.solr.common.SolrException: Internal Server Error
>
> Internal Server Error
>
> request:
> http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2
>      at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:435)
>      at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244)
>      at
> org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
>      at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:49)
>      at indexerapp.ir4llSolrIndexer.indexAll(ir4llSolrIndexer.java:150)
>      at indexerapp.ir4llSolrIndexer.main(ir4llSolrIndexer.java:57)
>
>
> *********
>
> If I put it inside /update, the following is the stacktrace:
> request:
> http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2
> org.apache.solr.common.SolrException: Bad Request
>
> Bad Request
>
> request:
> http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2
>      at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:435)
>      at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244)
>      at
> org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
>      at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:49)
>      at indexerapp.ir4llSolrIndexer.indexAll(ir4llSolrIndexer.java:150)
>      at indexerapp.ir4llSolrIndexer.main(ir4llSolrIndexer.java:57)
> ******
>
> But, I still don't undetstand where I can see a more detailed Log of Solr
> Server.
> On my tomcat logs (Iam running from Eclipse), (path:
> /Users/svajjala/Documents/workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/logs)
> -I dont see anything except a single line:
>
> 134.2.129.160 - - [12/Jul/2011:09:02:16 +0200] "POST
> /apache-solr-3.3.0/update/javabin?wt=javabin&version=2 HTTP/1.1" 400 1262
>
> It is difficult to understand whats going on. Can anyone tell me where I can
> see a more detailed log?
>
> S.
>
>
> On Tue, Jul 12, 2011 at 2:39 AM, Koji Sekiguchi<ko...@r.email.ne.jp>  wrote:
>
>> I don't think you have wrong setting in UIMA, but you may have the request
>> handler
>> named "/update/javabin" in solrconfig.xml is not correct?
>>
>>
>> koji
>> --
>> http://www.rondhuit.com/en/
>>
>> (11/07/12 0:52), Sowmya V.B. wrote:
>>
>>> Hi
>>>
>>> I just added the fields which are added to the index by one of the
>>> annotators to the index, in the fieldmappings section. I am not getting
>>> any
>>> compilation errors and still see the admin interface. However, when I
>>> index,
>>> i just get a SolrException,
>>>
>>> org.apache.solr.common.**SolrException: Bad Request.
>>>
>>> On the server log, I don't see anything except for this:
>>> 127.0.0.1 - - [11/Jul/2011:17:44:04 +0200]  "POST
>>> /apache-solr-3.3.0/update/**javabin?wt=javabin&version=2 HTTP/1.1" 400
>>> 1328
>>>
>>> Here is my UpdateRequestProcessorChain in sorlconfig.xml (Just changed the
>>> original path names for privacy's sake!)
>>>
>>> ***********
>>>      <!-- An update request processor chain for UIMA   -->
>>> <updateRequestProcessorChain name="uima">
>>>      <processor
>>> class="org.apache.solr.uima.**processor.**UIMAUpdateRequestProcessorFact*
>>> *ory">
>>>        <lst name="uimaConfig">
>>>          <lst name="runtimeParameters">
>>>          <!-- Add here, the runtime parameters like Model File paths, list
>>> file paths etc. -->
>>>            <str
>>> name="**tokenizerModelFileLocation">/**Users/svajjala/Documents/**
>>> EnglishTok.bin.gz</str>
>>>            <str
>>> name="taggerModelFileLocation"**>/Users/svajjala/Documents/**
>>> tag.bin.gz</str>
>>>           <str name="AcceptedLanguages">**english</str>
>>>              <str name="**SetLanguageOnEmptyAcceptList">**false</str>
>>>               <str
>>> name="GreenlineLists">/Users/**svajjala/Documents/**NewGreenline</str>
>>>         </lst>
>>>
>>>          <str
>>> name="analysisEngine">/Users/**svajjala/Documents/**
>>> ir4icallPipeline.xml</str>
>>>          <!-- Set to true if you want to continue indexing even if text
>>> processing fails.
>>>               Default is false. That is, Solr throws RuntimeException and
>>>               never indexed documents entirely in your session. -->
>>>             <bool name="ignoreErrors">false</**bool>
>>>          <!-- This is optional. It is used for logging when text processing
>>> fails.
>>>               If logField is not specified, uniqueKey will be used as
>>> logField.
>>>          <str name="logField">id</str>
>>>          -->
>>>          <lst name="analyzeFields">
>>>            <bool name="merge">false</bool>
>>>            <arr name="fields">
>>>              <str>text</str>
>>>            </arr>
>>>          </lst>
>>>
>>>           <fieldMapping>
>>>
>>>              <type name = "org.apache.uima.annotators.**
>>> vocablistClassfier">
>>>                  <map feature="Generic_Level5" field="Generic_Level5" />
>>>                  <map feature="Generic_Level6" field="Generic_Level6" />
>>>                  <map feature="Generic_Level7" field="Generic_Level7" />
>>>                  <map feature="Generic_Level8" field="Generic_Level8" />
>>>                  <map feature="Generic_Level9" field="Generic_Level9" />
>>>                  <map feature="Generic_Level10" field="Generic_Level10" />
>>>                  <map feature="Generic_Level11" field="Generic_Level11" />
>>>                  <map feature="Generic_Ratio" field="Generic_Ratio" />
>>>                  <map feature="Generic_**TotalWordCount"
>>> field="Generic_TotalWordCount" />
>>>              </type>
>>>
>>>          </fieldMapping>
>>>        </lst>
>>>      </processor>
>>>      <processor class="solr.**LogUpdateProcessorFactory" />
>>>      <processor class="solr.**RunUpdateProcessorFactory" />
>>>    </updateRequestProcessorChain>
>>>
>>>      ************
>>> I don't understand where exactly can I see a more detailed log of why its
>>> not getting indexed.
>>>
>>> Sowmya.
>>>
>>> On Mon, Jul 11, 2011 at 5:26 PM, Koji Sekiguchi<ko...@r.email.ne.jp>
>>>   wrote:
>>>
>>>   disclaimer: I'm not an expert of UIMA. I've just started using it when
>>>> Solr
>>>> 3.1
>>>> integrated UIMA!
>>>>
>>>>
>>>>   Thanks for the clarification. Now, I get it.
>>>>
>>>>> Should<fieldMapping>    section mention all the annotators, even if the
>>>>> annotators do not add any new fields?
>>>>>
>>>>>
>>>>   For example, if I have a pipeline, starting from "parser", "tokenizer"
>>>> and
>>>>
>>>>> "tagger", all of them operate on a field called "text"..which is
>>>>> the<html>
>>>>> of the document. but all these annotators do not add any new fields to
>>>>> the
>>>>> index. Should I still write fieldmappings for these annotators inside
>>>>> SolrConfig.?
>>>>>
>>>>>
>>>> I don't think so. Try it and if you got errors, come back to this list!
>>>>
>>>>
>>>> koji
>>>> --
>>>> http://www.rondhuit.com/en/
>>>>
>>>>
>>>
>>>
>>>
>>
>
>


Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by "Sowmya V.B." <vb...@gmail.com>.
Yes, I do have an '/update/javabin' request handler in SolrConfig.

But, should I remove that?

I tried putting the UIMA update chain inside /update/javabin instead of
/update request handler..
<requestHandler name="/update/javabin"
                  class="solr.BinaryUpdateRequestHandler">
<lst name="defaults">
    <str name="update.chain">uima</str>
    </lst>
  </requestHandler>

.and here is the stacktrace:

request:
http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2
org.apache.solr.common.SolrException: Internal Server Error

Internal Server Error

request:
http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2
    at
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:435)
    at
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244)
    at
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
    at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:49)
    at indexerapp.ir4llSolrIndexer.indexAll(ir4llSolrIndexer.java:150)
    at indexerapp.ir4llSolrIndexer.main(ir4llSolrIndexer.java:57)


*********

If I put it inside /update, the following is the stacktrace:
request:
http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2
org.apache.solr.common.SolrException: Bad Request

Bad Request

request:
http://localhost:8080/apache-solr-3.3.0/update/javabin?wt=javabin&version=2
    at
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:435)
    at
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244)
    at
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
    at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:49)
    at indexerapp.ir4llSolrIndexer.indexAll(ir4llSolrIndexer.java:150)
    at indexerapp.ir4llSolrIndexer.main(ir4llSolrIndexer.java:57)
******

But, I still don't undetstand where I can see a more detailed Log of Solr
Server.
On my tomcat logs (Iam running from Eclipse), (path:
/Users/svajjala/Documents/workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/logs)
-I dont see anything except a single line:

134.2.129.160 - - [12/Jul/2011:09:02:16 +0200] "POST
/apache-solr-3.3.0/update/javabin?wt=javabin&version=2 HTTP/1.1" 400 1262

It is difficult to understand whats going on. Can anyone tell me where I can
see a more detailed log?

S.


On Tue, Jul 12, 2011 at 2:39 AM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:

> I don't think you have wrong setting in UIMA, but you may have the request
> handler
> named "/update/javabin" in solrconfig.xml is not correct?
>
>
> koji
> --
> http://www.rondhuit.com/en/
>
> (11/07/12 0:52), Sowmya V.B. wrote:
>
>> Hi
>>
>> I just added the fields which are added to the index by one of the
>> annotators to the index, in the fieldmappings section. I am not getting
>> any
>> compilation errors and still see the admin interface. However, when I
>> index,
>> i just get a SolrException,
>>
>> org.apache.solr.common.**SolrException: Bad Request.
>>
>> On the server log, I don't see anything except for this:
>> 127.0.0.1 - - [11/Jul/2011:17:44:04 +0200]  "POST
>> /apache-solr-3.3.0/update/**javabin?wt=javabin&version=2 HTTP/1.1" 400
>> 1328
>>
>> Here is my UpdateRequestProcessorChain in sorlconfig.xml (Just changed the
>> original path names for privacy's sake!)
>>
>> ***********
>>     <!-- An update request processor chain for UIMA   -->
>> <updateRequestProcessorChain name="uima">
>>     <processor
>> class="org.apache.solr.uima.**processor.**UIMAUpdateRequestProcessorFact*
>> *ory">
>>       <lst name="uimaConfig">
>>         <lst name="runtimeParameters">
>>         <!-- Add here, the runtime parameters like Model File paths, list
>> file paths etc. -->
>>           <str
>> name="**tokenizerModelFileLocation">/**Users/svajjala/Documents/**
>> EnglishTok.bin.gz</str>
>>           <str
>> name="taggerModelFileLocation"**>/Users/svajjala/Documents/**
>> tag.bin.gz</str>
>>          <str name="AcceptedLanguages">**english</str>
>>             <str name="**SetLanguageOnEmptyAcceptList">**false</str>
>>              <str
>> name="GreenlineLists">/Users/**svajjala/Documents/**NewGreenline</str>
>>        </lst>
>>
>>         <str
>> name="analysisEngine">/Users/**svajjala/Documents/**
>> ir4icallPipeline.xml</str>
>>         <!-- Set to true if you want to continue indexing even if text
>> processing fails.
>>              Default is false. That is, Solr throws RuntimeException and
>>              never indexed documents entirely in your session. -->
>>            <bool name="ignoreErrors">false</**bool>
>>         <!-- This is optional. It is used for logging when text processing
>> fails.
>>              If logField is not specified, uniqueKey will be used as
>> logField.
>>         <str name="logField">id</str>
>>         -->
>>         <lst name="analyzeFields">
>>           <bool name="merge">false</bool>
>>           <arr name="fields">
>>             <str>text</str>
>>           </arr>
>>         </lst>
>>
>>          <fieldMapping>
>>
>>             <type name = "org.apache.uima.annotators.**
>> vocablistClassfier">
>>                 <map feature="Generic_Level5" field="Generic_Level5" />
>>                 <map feature="Generic_Level6" field="Generic_Level6" />
>>                 <map feature="Generic_Level7" field="Generic_Level7" />
>>                 <map feature="Generic_Level8" field="Generic_Level8" />
>>                 <map feature="Generic_Level9" field="Generic_Level9" />
>>                 <map feature="Generic_Level10" field="Generic_Level10" />
>>                 <map feature="Generic_Level11" field="Generic_Level11" />
>>                 <map feature="Generic_Ratio" field="Generic_Ratio" />
>>                 <map feature="Generic_**TotalWordCount"
>> field="Generic_TotalWordCount" />
>>             </type>
>>
>>         </fieldMapping>
>>       </lst>
>>     </processor>
>>     <processor class="solr.**LogUpdateProcessorFactory" />
>>     <processor class="solr.**RunUpdateProcessorFactory" />
>>   </updateRequestProcessorChain>
>>
>>     ************
>> I don't understand where exactly can I see a more detailed log of why its
>> not getting indexed.
>>
>> Sowmya.
>>
>> On Mon, Jul 11, 2011 at 5:26 PM, Koji Sekiguchi<ko...@r.email.ne.jp>
>>  wrote:
>>
>>  disclaimer: I'm not an expert of UIMA. I've just started using it when
>>> Solr
>>> 3.1
>>> integrated UIMA!
>>>
>>>
>>>  Thanks for the clarification. Now, I get it.
>>>
>>>> Should<fieldMapping>   section mention all the annotators, even if the
>>>> annotators do not add any new fields?
>>>>
>>>>
>>>  For example, if I have a pipeline, starting from "parser", "tokenizer"
>>> and
>>>
>>>> "tagger", all of them operate on a field called "text"..which is
>>>> the<html>
>>>> of the document. but all these annotators do not add any new fields to
>>>> the
>>>> index. Should I still write fieldmappings for these annotators inside
>>>> SolrConfig.?
>>>>
>>>>
>>> I don't think so. Try it and if you got errors, come back to this list!
>>>
>>>
>>> koji
>>> --
>>> http://www.rondhuit.com/en/
>>>
>>>
>>
>>
>>
>


-- 
Sowmya V.B.
----------------------------------------------------
Losing optimism is blasphemy!
http://vbsowmya.wordpress.com
----------------------------------------------------

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
I don't think you have wrong setting in UIMA, but you may have the request handler
named "/update/javabin" in solrconfig.xml is not correct?

koji
-- 
http://www.rondhuit.com/en/

(11/07/12 0:52), Sowmya V.B. wrote:
> Hi
>
> I just added the fields which are added to the index by one of the
> annotators to the index, in the fieldmappings section. I am not getting any
> compilation errors and still see the admin interface. However, when I index,
> i just get a SolrException,
>
> org.apache.solr.common.SolrException: Bad Request.
>
> On the server log, I don't see anything except for this:
> 127.0.0.1 - - [11/Jul/2011:17:44:04 +0200]  "POST
> /apache-solr-3.3.0/update/javabin?wt=javabin&version=2 HTTP/1.1" 400 1328
>
> Here is my UpdateRequestProcessorChain in sorlconfig.xml (Just changed the
> original path names for privacy's sake!)
>
> ***********
>      <!-- An update request processor chain for UIMA   -->
> <updateRequestProcessorChain name="uima">
>      <processor
> class="org.apache.solr.uima.processor.UIMAUpdateRequestProcessorFactory">
>        <lst name="uimaConfig">
>          <lst name="runtimeParameters">
>          <!-- Add here, the runtime parameters like Model File paths, list
> file paths etc. -->
>            <str
> name="tokenizerModelFileLocation">/Users/svajjala/Documents/EnglishTok.bin.gz</str>
>            <str
> name="taggerModelFileLocation">/Users/svajjala/Documents/tag.bin.gz</str>
>           <str name="AcceptedLanguages">english</str>
>              <str name="SetLanguageOnEmptyAcceptList">false</str>
>               <str
> name="GreenlineLists">/Users/svajjala/Documents/NewGreenline</str>
>         </lst>
>
>          <str
> name="analysisEngine">/Users/svajjala/Documents/ir4icallPipeline.xml</str>
>          <!-- Set to true if you want to continue indexing even if text
> processing fails.
>               Default is false. That is, Solr throws RuntimeException and
>               never indexed documents entirely in your session. -->
>             <bool name="ignoreErrors">false</bool>
>          <!-- This is optional. It is used for logging when text processing
> fails.
>               If logField is not specified, uniqueKey will be used as
> logField.
>          <str name="logField">id</str>
>          -->
>          <lst name="analyzeFields">
>            <bool name="merge">false</bool>
>            <arr name="fields">
>              <str>text</str>
>            </arr>
>          </lst>
>
>           <fieldMapping>
>
>              <type name = "org.apache.uima.annotators.vocablistClassfier">
>                  <map feature="Generic_Level5" field="Generic_Level5" />
>                  <map feature="Generic_Level6" field="Generic_Level6" />
>                  <map feature="Generic_Level7" field="Generic_Level7" />
>                  <map feature="Generic_Level8" field="Generic_Level8" />
>                  <map feature="Generic_Level9" field="Generic_Level9" />
>                  <map feature="Generic_Level10" field="Generic_Level10" />
>                  <map feature="Generic_Level11" field="Generic_Level11" />
>                  <map feature="Generic_Ratio" field="Generic_Ratio" />
>                  <map feature="Generic_TotalWordCount"
> field="Generic_TotalWordCount" />
>              </type>
>
>          </fieldMapping>
>        </lst>
>      </processor>
>      <processor class="solr.LogUpdateProcessorFactory" />
>      <processor class="solr.RunUpdateProcessorFactory" />
>    </updateRequestProcessorChain>
>
>      ************
> I don't understand where exactly can I see a more detailed log of why its
> not getting indexed.
>
> Sowmya.
>
> On Mon, Jul 11, 2011 at 5:26 PM, Koji Sekiguchi<ko...@r.email.ne.jp>  wrote:
>
>> disclaimer: I'm not an expert of UIMA. I've just started using it when Solr
>> 3.1
>> integrated UIMA!
>>
>>
>>   Thanks for the clarification. Now, I get it.
>>> Should<fieldMapping>   section mention all the annotators, even if the
>>> annotators do not add any new fields?
>>>
>>
>>   For example, if I have a pipeline, starting from "parser", "tokenizer" and
>>> "tagger", all of them operate on a field called "text"..which is the<html>
>>> of the document. but all these annotators do not add any new fields to the
>>> index. Should I still write fieldmappings for these annotators inside
>>> SolrConfig.?
>>>
>>
>> I don't think so. Try it and if you got errors, come back to this list!
>>
>>
>> koji
>> --
>> http://www.rondhuit.com/en/
>>
>
>
>


Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by "Sowmya V.B." <vb...@gmail.com>.
Hi

I just added the fields which are added to the index by one of the
annotators to the index, in the fieldmappings section. I am not getting any
compilation errors and still see the admin interface. However, when I index,
i just get a SolrException,

org.apache.solr.common.SolrException: Bad Request.

On the server log, I don't see anything except for this:
127.0.0.1 - - [11/Jul/2011:17:44:04 +0200]  "POST
/apache-solr-3.3.0/update/javabin?wt=javabin&version=2 HTTP/1.1" 400 1328

Here is my UpdateRequestProcessorChain in sorlconfig.xml (Just changed the
original path names for privacy's sake!)

***********
    <!-- An update request processor chain for UIMA   -->
<updateRequestProcessorChain name="uima">
    <processor
class="org.apache.solr.uima.processor.UIMAUpdateRequestProcessorFactory">
      <lst name="uimaConfig">
        <lst name="runtimeParameters">
        <!-- Add here, the runtime parameters like Model File paths, list
file paths etc. -->
          <str
name="tokenizerModelFileLocation">/Users/svajjala/Documents/EnglishTok.bin.gz</str>
          <str
name="taggerModelFileLocation">/Users/svajjala/Documents/tag.bin.gz</str>
         <str name="AcceptedLanguages">english</str>
            <str name="SetLanguageOnEmptyAcceptList">false</str>
             <str
name="GreenlineLists">/Users/svajjala/Documents/NewGreenline</str>
       </lst>

        <str
name="analysisEngine">/Users/svajjala/Documents/ir4icallPipeline.xml</str>
        <!-- Set to true if you want to continue indexing even if text
processing fails.
             Default is false. That is, Solr throws RuntimeException and
             never indexed documents entirely in your session. -->
           <bool name="ignoreErrors">false</bool>
        <!-- This is optional. It is used for logging when text processing
fails.
             If logField is not specified, uniqueKey will be used as
logField.
        <str name="logField">id</str>
        -->
        <lst name="analyzeFields">
          <bool name="merge">false</bool>
          <arr name="fields">
            <str>text</str>
          </arr>
        </lst>

         <fieldMapping>

            <type name = "org.apache.uima.annotators.vocablistClassfier">
                <map feature="Generic_Level5" field="Generic_Level5" />
                <map feature="Generic_Level6" field="Generic_Level6" />
                <map feature="Generic_Level7" field="Generic_Level7" />
                <map feature="Generic_Level8" field="Generic_Level8" />
                <map feature="Generic_Level9" field="Generic_Level9" />
                <map feature="Generic_Level10" field="Generic_Level10" />
                <map feature="Generic_Level11" field="Generic_Level11" />
                <map feature="Generic_Ratio" field="Generic_Ratio" />
                <map feature="Generic_TotalWordCount"
field="Generic_TotalWordCount" />
            </type>

        </fieldMapping>
      </lst>
    </processor>
    <processor class="solr.LogUpdateProcessorFactory" />
    <processor class="solr.RunUpdateProcessorFactory" />
  </updateRequestProcessorChain>

    ************
I don't understand where exactly can I see a more detailed log of why its
not getting indexed.

Sowmya.

On Mon, Jul 11, 2011 at 5:26 PM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:

> disclaimer: I'm not an expert of UIMA. I've just started using it when Solr
> 3.1
> integrated UIMA!
>
>
>  Thanks for the clarification. Now, I get it.
>> Should<fieldMapping>  section mention all the annotators, even if the
>> annotators do not add any new fields?
>>
>
>  For example, if I have a pipeline, starting from "parser", "tokenizer" and
>> "tagger", all of them operate on a field called "text"..which is the<html>
>> of the document. but all these annotators do not add any new fields to the
>> index. Should I still write fieldmappings for these annotators inside
>> SolrConfig.?
>>
>
> I don't think so. Try it and if you got errors, come back to this list!
>
>
> koji
> --
> http://www.rondhuit.com/en/
>



-- 
Sowmya V.B.
----------------------------------------------------
Losing optimism is blasphemy!
http://vbsowmya.wordpress.com
----------------------------------------------------

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
disclaimer: I'm not an expert of UIMA. I've just started using it when Solr 3.1
integrated UIMA!

> Thanks for the clarification. Now, I get it.
> Should<fieldMapping>  section mention all the annotators, even if the
> annotators do not add any new fields?

> For example, if I have a pipeline, starting from "parser", "tokenizer" and
> "tagger", all of them operate on a field called "text"..which is the<html>
> of the document. but all these annotators do not add any new fields to the
> index. Should I still write fieldmappings for these annotators inside
> SolrConfig.?

I don't think so. Try it and if you got errors, come back to this list!

koji
-- 
http://www.rondhuit.com/en/

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by "Sowmya V.B." <vb...@gmail.com>.
Koji

Thanks for the clarification. Now, I get it.
Should <fieldMapping> section mention all the annotators, even if the
annotators do not add any new fields?

For example, if I have a pipeline, starting from "parser", "tokenizer" and
"tagger", all of them operate on a field called "text"..which is the <html>
of the document. but all these annotators do not add any new fields to the
index. Should I still write fieldmappings for these annotators inside
SolrConfig.?

S

On Mon, Jul 11, 2011 at 4:35 PM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:

> Sowmya,
>
> The combination of fieldNameFeature and dynamicField can be used when
> using,
> e.g. named entity extractor that tend to produce a lot of attributes,
> organization,
> location, country, building, spot, title,... If you are going to use such
> named
> entity extractor, you don't want to define each field in schema.xml, you
> may
> want to use a dynamic field *_sm (multiValued string type) instead.
> And you want solr to map organization to organization_sm, location to
> location_sm,
> and so on. You can do it via having fieldNameFeature and dynamicField.
>
> Where "name" feature of fieldNameFeature value is used for field name in
> dynamicField.
>
>
> koji
> --
> http://www.rondhuit.com/en/
>
> (11/07/11 21:54), Sowmya V.B. wrote:
>
>> Hi Koji
>>
>> Thanks a lot for the examples. Now, I was able to compile a JAR snapshot,
>> with my own UIMA pipeline. However, despite seeing the example
>> solrconfig.xml, I am not able to figure out how to add mine.
>>
>> In the example:
>>
>>   <str name="feature">entity</str>
>>
>>             <str name="fieldNameFeature">name</**str>
>>
>>             <str name="dynamicField">*_sm</str>
>>
>> I still don't understand what "fieldnamefeature" mean, in case of dynamic
>> fields.
>>
>> For example, if the annotator takes "text" field, and gives "fieldA,
>> fieldB,
>> fieldC", how should I specify that inside this?
>>
>> I was looking on the Solr pages, and on the SolrUIMA page, (
>> http://wiki.apache.org/solr/**SolrUIMA#Using_other_UIMA_**components<http://wiki.apache.org/solr/SolrUIMA#Using_other_UIMA_components>
>> )
>> There is this example configuration, for fieldmapping specification:
>>
>> <fieldMapping>
>>     <!-- here goes the mapping between features of UIMA
>> FeatureStructures to Solr fields  -->
>>     <type name="org.apache.uima.**something.Annotation">
>>       <map feature="oneFeature" field="destination_field"/>
>>     </type>
>>     ...
>>   </fieldMapping>
>>
>>
>> Which is slightly different from the example that you used in rondhuit
>> code
>> samples.
>> So, does it mean - I can also do something like:
>> <fieldMapping>
>>     <type name = "org.apache.uima.annotators.**tagger">
>>             <map feature="text" field "text">
>>     </type>
>> <!-- Because the annotator "tagger" does not create any new fields in the
>> index. It just modifies the text field -->
>>
>>         <type name = "org.apache.uima.annotators.**stats">
>>             <map feature="FieldA" field "FieldX">
>>            <map feature="FieldB" field "FieldY">
>>             <map feature="FieldC" field "FieldZ">
>>     </type>
>> <!-- Where, Fields X,Y,Z are declared in Schema. Fields A, B, C were
>> obtained inside the "stats" annotator. -->
>>
>> </fieldMapping>
>> -if I add Fields from the annotator from within the pipeline, using
>> addFStoIndexes() method?
>>
>> Sowmya.
>>
>> On Sat, Jul 9, 2011 at 12:51 AM, Koji Sekiguchi<ko...@r.email.ne.jp>
>>  wrote:
>>
>>  Now I've pasted sample solrconfig.xml to the project top page.
>>> Can you visit and look at it again?
>>>
>>>
>>> koji
>>> --
>>> http://www.rondhuit.com/en/
>>>
>>> (11/07/09 2:29), Sowmya V.B. wrote:
>>>
>>>  Hi Koji
>>>>
>>>> Thanks. I have checked out the code and began looking at it. The code
>>>> examples gave me an idea of what to do,though I am not fully clear,
>>>> since
>>>> there are no comments there, to verify my understanding. Hence, mailing
>>>> again for clarification.
>>>>
>>>> In NamedEntity.java, you add two fields "name", "entity", to the index,
>>>> via
>>>> this processing pipeline "next"?
>>>> the methods setName() and setEntity() - add two fields "name", "entity",
>>>> to
>>>> the index?
>>>>
>>>> If so, how should I specify this in the solrconfig.xml's<****
>>>> fieldMappings>
>>>> section?
>>>>
>>>> <lst name="type">
>>>>             <str name="name">next.NamedEntity</****str>
>>>>             <lst name="mapping">
>>>>               <str name="feature">name</str>
>>>>               <str name="field">namefield</str>   (where namefield is
>>>> the field I declared in schema.xml, say)
>>>>             </lst>
>>>>           </lst>
>>>>           <lst name="type">
>>>>             <str name="name">next.NamedEntity</****str>
>>>>             <lst name="mapping">
>>>>               <str name="feature">entity</str>
>>>>               <str name="field">entityfield</str>   (where entityfield
>>>> is the field I declared in schema.xml, say)
>>>>             </lst>
>>>>           </lst>
>>>>
>>>> - Is this the right way to go? Can I declare 2 mappings which relate to
>>>> the
>>>> same class (next.NamedEntity, in this case)?
>>>>
>>>> I am sorry for repeated mails...but its a bit confusing, because there
>>>> is
>>>> no
>>>> README file.
>>>> Thankyou once again!
>>>>
>>>> Sowmya.
>>>>
>>>> On Fri, Jul 8, 2011 at 4:07 PM, Koji Sekiguchi<ko...@r.email.ne.jp>
>>>>  wrote:
>>>>
>>>>  (11/07/08 16:19), Sowmya V.B. wrote:
>>>>
>>>>>
>>>>>  Hi Koji
>>>>>
>>>>>>
>>>>>> Thanks for the mail.
>>>>>>
>>>>>> Thanks for all the clarifications. I am now using the version 3.3..
>>>>>> But,
>>>>>> another query that I have about this is:
>>>>>> How can I add an annotator that I wrote myself, in to Solr-UIMA?
>>>>>>
>>>>>> Here is what I did before I moved to Solr:
>>>>>> I wrote an annotator (which worked when I used plain vanilla lucene
>>>>>> based
>>>>>> indexer), which enriched the document with more fields (Some
>>>>>> statistics
>>>>>> about the document...all fields added were numeric fields). Those
>>>>>> fields
>>>>>> were added to the index by extending *JCasAnnotator_ImplBase* class.
>>>>>>
>>>>>> But, in Solr-UIMA, I am not exactly clear on where the above setup
>>>>>> fits
>>>>>> in.
>>>>>> I thought I would get an idea looking at the annotators that came with
>>>>>> the
>>>>>> UIMA integration of Solr, but their source was not available. So, I do
>>>>>> not
>>>>>> understand how to actually integrate my own annotator in to UIMA.
>>>>>>
>>>>>>
>>>>>>  Hi Sowmya,
>>>>>
>>>>> Please look at an example UIMA annotators that can be deployed on
>>>>> Solr-UIMA
>>>>> environment:
>>>>>
>>>>> http://code.google.com/p/******rondhuit-uima/<http://code.google.com/p/****rondhuit-uima/>
>>>>> <http://code.**google.com/p/**rondhuit-uima/<http://code.google.com/p/**rondhuit-uima/>
>>>>> >
>>>>> <http://code.**google.com/p/**rondhuit-uima/<http://google.com/p/rondhuit-uima/>
>>>>> <http://code.**google.com/p/rondhuit-uima/<http://code.google.com/p/rondhuit-uima/>
>>>>> >
>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> It comes with source code.
>>>>>
>>>>>
>>>>> koji
>>>>> --
>>>>> http://www.rondhuit.com/en/
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>


-- 
Sowmya V.B.
----------------------------------------------------
Losing optimism is blasphemy!
http://vbsowmya.wordpress.com
----------------------------------------------------

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
Sowmya,

The combination of fieldNameFeature and dynamicField can be used when using,
e.g. named entity extractor that tend to produce a lot of attributes, organization,
location, country, building, spot, title,... If you are going to use such named
entity extractor, you don't want to define each field in schema.xml, you may
want to use a dynamic field *_sm (multiValued string type) instead.
And you want solr to map organization to organization_sm, location to location_sm,
and so on. You can do it via having fieldNameFeature and dynamicField.

Where "name" feature of fieldNameFeature value is used for field name in dynamicField.

koji
-- 
http://www.rondhuit.com/en/

(11/07/11 21:54), Sowmya V.B. wrote:
> Hi Koji
>
> Thanks a lot for the examples. Now, I was able to compile a JAR snapshot,
> with my own UIMA pipeline. However, despite seeing the example
> solrconfig.xml, I am not able to figure out how to add mine.
>
> In the example:
>
>    <str name="feature">entity</str>
>
>              <str name="fieldNameFeature">name</str>
>
>              <str name="dynamicField">*_sm</str>
>
> I still don't understand what "fieldnamefeature" mean, in case of dynamic
> fields.
>
> For example, if the annotator takes "text" field, and gives "fieldA, fieldB,
> fieldC", how should I specify that inside this?
>
> I was looking on the Solr pages, and on the SolrUIMA page, (
> http://wiki.apache.org/solr/SolrUIMA#Using_other_UIMA_components)
> There is this example configuration, for fieldmapping specification:
>
> <fieldMapping>
>      <!-- here goes the mapping between features of UIMA
> FeatureStructures to Solr fields  -->
>      <type name="org.apache.uima.something.Annotation">
>        <map feature="oneFeature" field="destination_field"/>
>      </type>
>      ...
>    </fieldMapping>
>
>
> Which is slightly different from the example that you used in rondhuit code
> samples.
> So, does it mean - I can also do something like:
> <fieldMapping>
>      <type name = "org.apache.uima.annotators.tagger">
>              <map feature="text" field "text">
>      </type>
> <!-- Because the annotator "tagger" does not create any new fields in the
> index. It just modifies the text field -->
>
>          <type name = "org.apache.uima.annotators.stats">
>              <map feature="FieldA" field "FieldX">
>             <map feature="FieldB" field "FieldY">
>              <map feature="FieldC" field "FieldZ">
>      </type>
> <!-- Where, Fields X,Y,Z are declared in Schema. Fields A, B, C were
> obtained inside the "stats" annotator. -->
>
> </fieldMapping>
> -if I add Fields from the annotator from within the pipeline, using
> addFStoIndexes() method?
>
> Sowmya.
>
> On Sat, Jul 9, 2011 at 12:51 AM, Koji Sekiguchi<ko...@r.email.ne.jp>  wrote:
>
>> Now I've pasted sample solrconfig.xml to the project top page.
>> Can you visit and look at it again?
>>
>>
>> koji
>> --
>> http://www.rondhuit.com/en/
>>
>> (11/07/09 2:29), Sowmya V.B. wrote:
>>
>>> Hi Koji
>>>
>>> Thanks. I have checked out the code and began looking at it. The code
>>> examples gave me an idea of what to do,though I am not fully clear, since
>>> there are no comments there, to verify my understanding. Hence, mailing
>>> again for clarification.
>>>
>>> In NamedEntity.java, you add two fields "name", "entity", to the index,
>>> via
>>> this processing pipeline "next"?
>>> the methods setName() and setEntity() - add two fields "name", "entity",
>>> to
>>> the index?
>>>
>>> If so, how should I specify this in the solrconfig.xml's<**fieldMappings>
>>> section?
>>>
>>> <lst name="type">
>>>              <str name="name">next.NamedEntity</**str>
>>>              <lst name="mapping">
>>>                <str name="feature">name</str>
>>>                <str name="field">namefield</str>   (where namefield is
>>> the field I declared in schema.xml, say)
>>>              </lst>
>>>            </lst>
>>>            <lst name="type">
>>>              <str name="name">next.NamedEntity</**str>
>>>              <lst name="mapping">
>>>                <str name="feature">entity</str>
>>>                <str name="field">entityfield</str>   (where entityfield
>>> is the field I declared in schema.xml, say)
>>>              </lst>
>>>            </lst>
>>>
>>> - Is this the right way to go? Can I declare 2 mappings which relate to
>>> the
>>> same class (next.NamedEntity, in this case)?
>>>
>>> I am sorry for repeated mails...but its a bit confusing, because there is
>>> no
>>> README file.
>>> Thankyou once again!
>>>
>>> Sowmya.
>>>
>>> On Fri, Jul 8, 2011 at 4:07 PM, Koji Sekiguchi<ko...@r.email.ne.jp>
>>>   wrote:
>>>
>>>   (11/07/08 16:19), Sowmya V.B. wrote:
>>>>
>>>>   Hi Koji
>>>>>
>>>>> Thanks for the mail.
>>>>>
>>>>> Thanks for all the clarifications. I am now using the version 3.3.. But,
>>>>> another query that I have about this is:
>>>>> How can I add an annotator that I wrote myself, in to Solr-UIMA?
>>>>>
>>>>> Here is what I did before I moved to Solr:
>>>>> I wrote an annotator (which worked when I used plain vanilla lucene
>>>>> based
>>>>> indexer), which enriched the document with more fields (Some statistics
>>>>> about the document...all fields added were numeric fields). Those fields
>>>>> were added to the index by extending *JCasAnnotator_ImplBase* class.
>>>>>
>>>>> But, in Solr-UIMA, I am not exactly clear on where the above setup fits
>>>>> in.
>>>>> I thought I would get an idea looking at the annotators that came with
>>>>> the
>>>>> UIMA integration of Solr, but their source was not available. So, I do
>>>>> not
>>>>> understand how to actually integrate my own annotator in to UIMA.
>>>>>
>>>>>
>>>> Hi Sowmya,
>>>>
>>>> Please look at an example UIMA annotators that can be deployed on
>>>> Solr-UIMA
>>>> environment:
>>>>
>>>> http://code.google.com/p/****rondhuit-uima/<http://code.google.com/p/**rondhuit-uima/>
>>>> <http://code.**google.com/p/rondhuit-uima/<http://code.google.com/p/rondhuit-uima/>
>>>>>
>>>>
>>>>
>>>> It comes with source code.
>>>>
>>>>
>>>> koji
>>>> --
>>>> http://www.rondhuit.com/en/
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>
>


Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by "Sowmya V.B." <vb...@gmail.com>.
Hi Koji

Thanks a lot for the examples. Now, I was able to compile a JAR snapshot,
with my own UIMA pipeline. However, despite seeing the example
solrconfig.xml, I am not able to figure out how to add mine.

In the example:

  <str name="feature">entity</str>

            <str name="fieldNameFeature">name</str>

            <str name="dynamicField">*_sm</str>

I still don't understand what "fieldnamefeature" mean, in case of dynamic
fields.

For example, if the annotator takes "text" field, and gives "fieldA, fieldB,
fieldC", how should I specify that inside this?

I was looking on the Solr pages, and on the SolrUIMA page, (
http://wiki.apache.org/solr/SolrUIMA#Using_other_UIMA_components)
There is this example configuration, for fieldmapping specification:

<fieldMapping>
    <!-- here goes the mapping between features of UIMA
FeatureStructures to Solr fields  -->
    <type name="org.apache.uima.something.Annotation">
      <map feature="oneFeature" field="destination_field"/>
    </type>
    ...
  </fieldMapping>


Which is slightly different from the example that you used in rondhuit code
samples.
So, does it mean - I can also do something like:
<fieldMapping>
    <type name = "org.apache.uima.annotators.tagger">
            <map feature="text" field "text">
    </type>
<!-- Because the annotator "tagger" does not create any new fields in the
index. It just modifies the text field -->

        <type name = "org.apache.uima.annotators.stats">
            <map feature="FieldA" field "FieldX">
           <map feature="FieldB" field "FieldY">
            <map feature="FieldC" field "FieldZ">
    </type>
<!-- Where, Fields X,Y,Z are declared in Schema. Fields A, B, C were
obtained inside the "stats" annotator. -->

</fieldMapping>
-if I add Fields from the annotator from within the pipeline, using
addFStoIndexes() method?

Sowmya.

On Sat, Jul 9, 2011 at 12:51 AM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:

> Now I've pasted sample solrconfig.xml to the project top page.
> Can you visit and look at it again?
>
>
> koji
> --
> http://www.rondhuit.com/en/
>
> (11/07/09 2:29), Sowmya V.B. wrote:
>
>> Hi Koji
>>
>> Thanks. I have checked out the code and began looking at it. The code
>> examples gave me an idea of what to do,though I am not fully clear, since
>> there are no comments there, to verify my understanding. Hence, mailing
>> again for clarification.
>>
>> In NamedEntity.java, you add two fields "name", "entity", to the index,
>> via
>> this processing pipeline "next"?
>> the methods setName() and setEntity() - add two fields "name", "entity",
>> to
>> the index?
>>
>> If so, how should I specify this in the solrconfig.xml's<**fieldMappings>
>> section?
>>
>> <lst name="type">
>>             <str name="name">next.NamedEntity</**str>
>>             <lst name="mapping">
>>               <str name="feature">name</str>
>>               <str name="field">namefield</str>  (where namefield is
>> the field I declared in schema.xml, say)
>>             </lst>
>>           </lst>
>>           <lst name="type">
>>             <str name="name">next.NamedEntity</**str>
>>             <lst name="mapping">
>>               <str name="feature">entity</str>
>>               <str name="field">entityfield</str>  (where entityfield
>> is the field I declared in schema.xml, say)
>>             </lst>
>>           </lst>
>>
>> - Is this the right way to go? Can I declare 2 mappings which relate to
>> the
>> same class (next.NamedEntity, in this case)?
>>
>> I am sorry for repeated mails...but its a bit confusing, because there is
>> no
>> README file.
>> Thankyou once again!
>>
>> Sowmya.
>>
>> On Fri, Jul 8, 2011 at 4:07 PM, Koji Sekiguchi<ko...@r.email.ne.jp>
>>  wrote:
>>
>>  (11/07/08 16:19), Sowmya V.B. wrote:
>>>
>>>  Hi Koji
>>>>
>>>> Thanks for the mail.
>>>>
>>>> Thanks for all the clarifications. I am now using the version 3.3.. But,
>>>> another query that I have about this is:
>>>> How can I add an annotator that I wrote myself, in to Solr-UIMA?
>>>>
>>>> Here is what I did before I moved to Solr:
>>>> I wrote an annotator (which worked when I used plain vanilla lucene
>>>> based
>>>> indexer), which enriched the document with more fields (Some statistics
>>>> about the document...all fields added were numeric fields). Those fields
>>>> were added to the index by extending *JCasAnnotator_ImplBase* class.
>>>>
>>>> But, in Solr-UIMA, I am not exactly clear on where the above setup fits
>>>> in.
>>>> I thought I would get an idea looking at the annotators that came with
>>>> the
>>>> UIMA integration of Solr, but their source was not available. So, I do
>>>> not
>>>> understand how to actually integrate my own annotator in to UIMA.
>>>>
>>>>
>>> Hi Sowmya,
>>>
>>> Please look at an example UIMA annotators that can be deployed on
>>> Solr-UIMA
>>> environment:
>>>
>>> http://code.google.com/p/****rondhuit-uima/<http://code.google.com/p/**rondhuit-uima/>
>>> <http://code.**google.com/p/rondhuit-uima/<http://code.google.com/p/rondhuit-uima/>
>>> >
>>>
>>>
>>> It comes with source code.
>>>
>>>
>>> koji
>>> --
>>> http://www.rondhuit.com/en/
>>>
>>>
>>
>>
>>
>
>


-- 
Sowmya V.B.
----------------------------------------------------
Losing optimism is blasphemy!
http://vbsowmya.wordpress.com
----------------------------------------------------

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
Now I've pasted sample solrconfig.xml to the project top page.
Can you visit and look at it again?

koji
-- 
http://www.rondhuit.com/en/

(11/07/09 2:29), Sowmya V.B. wrote:
> Hi Koji
>
> Thanks. I have checked out the code and began looking at it. The code
> examples gave me an idea of what to do,though I am not fully clear, since
> there are no comments there, to verify my understanding. Hence, mailing
> again for clarification.
>
> In NamedEntity.java, you add two fields "name", "entity", to the index, via
> this processing pipeline "next"?
> the methods setName() and setEntity() - add two fields "name", "entity", to
> the index?
>
> If so, how should I specify this in the solrconfig.xml's<fieldMappings>
> section?
>
> <lst name="type">
>              <str name="name">next.NamedEntity</str>
>              <lst name="mapping">
>                <str name="feature">name</str>
>                <str name="field">namefield</str>  (where namefield is
> the field I declared in schema.xml, say)
>              </lst>
>            </lst>
>            <lst name="type">
>              <str name="name">next.NamedEntity</str>
>              <lst name="mapping">
>                <str name="feature">entity</str>
>                <str name="field">entityfield</str>  (where entityfield
> is the field I declared in schema.xml, say)
>              </lst>
>            </lst>
>
> - Is this the right way to go? Can I declare 2 mappings which relate to the
> same class (next.NamedEntity, in this case)?
>
> I am sorry for repeated mails...but its a bit confusing, because there is no
> README file.
> Thankyou once again!
>
> Sowmya.
>
> On Fri, Jul 8, 2011 at 4:07 PM, Koji Sekiguchi<ko...@r.email.ne.jp>  wrote:
>
>> (11/07/08 16:19), Sowmya V.B. wrote:
>>
>>> Hi Koji
>>>
>>> Thanks for the mail.
>>>
>>> Thanks for all the clarifications. I am now using the version 3.3.. But,
>>> another query that I have about this is:
>>> How can I add an annotator that I wrote myself, in to Solr-UIMA?
>>>
>>> Here is what I did before I moved to Solr:
>>> I wrote an annotator (which worked when I used plain vanilla lucene based
>>> indexer), which enriched the document with more fields (Some statistics
>>> about the document...all fields added were numeric fields). Those fields
>>> were added to the index by extending *JCasAnnotator_ImplBase* class.
>>>
>>> But, in Solr-UIMA, I am not exactly clear on where the above setup fits
>>> in.
>>> I thought I would get an idea looking at the annotators that came with the
>>> UIMA integration of Solr, but their source was not available. So, I do not
>>> understand how to actually integrate my own annotator in to UIMA.
>>>
>>
>> Hi Sowmya,
>>
>> Please look at an example UIMA annotators that can be deployed on Solr-UIMA
>> environment:
>>
>> http://code.google.com/p/**rondhuit-uima/<http://code.google.com/p/rondhuit-uima/>
>>
>> It comes with source code.
>>
>>
>> koji
>> --
>> http://www.rondhuit.com/en/
>>
>
>
>



Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by "Sowmya V.B." <vb...@gmail.com>.
Hi Koji

Thanks. I have checked out the code and began looking at it. The code
examples gave me an idea of what to do,though I am not fully clear, since
there are no comments there, to verify my understanding. Hence, mailing
again for clarification.

In NamedEntity.java, you add two fields "name", "entity", to the index, via
this processing pipeline "next"?
the methods setName() and setEntity() - add two fields "name", "entity", to
the index?

If so, how should I specify this in the solrconfig.xml's <fieldMappings>
section?

<lst name="type">
            <str name="name">next.NamedEntity</str>
            <lst name="mapping">
              <str name="feature">name</str>
              <str name="field">namefield</str> (where namefield is
the field I declared in schema.xml, say)
            </lst>
          </lst>
          <lst name="type">
            <str name="name">next.NamedEntity</str>
            <lst name="mapping">
              <str name="feature">entity</str>
              <str name="field">entityfield</str> (where entityfield
is the field I declared in schema.xml, say)
            </lst>
          </lst>

- Is this the right way to go? Can I declare 2 mappings which relate to the
same class (next.NamedEntity, in this case)?

I am sorry for repeated mails...but its a bit confusing, because there is no
README file.
Thankyou once again!

Sowmya.

On Fri, Jul 8, 2011 at 4:07 PM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:

> (11/07/08 16:19), Sowmya V.B. wrote:
>
>> Hi Koji
>>
>> Thanks for the mail.
>>
>> Thanks for all the clarifications. I am now using the version 3.3.. But,
>> another query that I have about this is:
>> How can I add an annotator that I wrote myself, in to Solr-UIMA?
>>
>> Here is what I did before I moved to Solr:
>> I wrote an annotator (which worked when I used plain vanilla lucene based
>> indexer), which enriched the document with more fields (Some statistics
>> about the document...all fields added were numeric fields). Those fields
>> were added to the index by extending *JCasAnnotator_ImplBase* class.
>>
>> But, in Solr-UIMA, I am not exactly clear on where the above setup fits
>> in.
>> I thought I would get an idea looking at the annotators that came with the
>> UIMA integration of Solr, but their source was not available. So, I do not
>> understand how to actually integrate my own annotator in to UIMA.
>>
>
> Hi Sowmya,
>
> Please look at an example UIMA annotators that can be deployed on Solr-UIMA
> environment:
>
> http://code.google.com/p/**rondhuit-uima/<http://code.google.com/p/rondhuit-uima/>
>
> It comes with source code.
>
>
> koji
> --
> http://www.rondhuit.com/en/
>



-- 
Sowmya V.B.
----------------------------------------------------
Losing optimism is blasphemy!
http://vbsowmya.wordpress.com
----------------------------------------------------

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
(11/07/08 16:19), Sowmya V.B. wrote:
> Hi Koji
>
> Thanks for the mail.
>
> Thanks for all the clarifications. I am now using the version 3.3.. But,
> another query that I have about this is:
> How can I add an annotator that I wrote myself, in to Solr-UIMA?
>
> Here is what I did before I moved to Solr:
> I wrote an annotator (which worked when I used plain vanilla lucene based
> indexer), which enriched the document with more fields (Some statistics
> about the document...all fields added were numeric fields). Those fields
> were added to the index by extending *JCasAnnotator_ImplBase* class.
>
> But, in Solr-UIMA, I am not exactly clear on where the above setup fits in.
> I thought I would get an idea looking at the annotators that came with the
> UIMA integration of Solr, but their source was not available. So, I do not
> understand how to actually integrate my own annotator in to UIMA.

Hi Sowmya,

Please look at an example UIMA annotators that can be deployed on Solr-UIMA
environment:

http://code.google.com/p/rondhuit-uima/

It comes with source code.

koji
-- 
http://www.rondhuit.com/en/

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by "Sowmya V.B." <vb...@gmail.com>.
Hi Koji

Thanks for the mail.

Thanks for all the clarifications. I am now using the version 3.3.. But,
another query that I have about this is:
How can I add an annotator that I wrote myself, in to Solr-UIMA?

Here is what I did before I moved to Solr:
I wrote an annotator (which worked when I used plain vanilla lucene based
indexer), which enriched the document with more fields (Some statistics
about the document...all fields added were numeric fields). Those fields
were added to the index by extending *JCasAnnotator_ImplBase* class.

But, in Solr-UIMA, I am not exactly clear on where the above setup fits in.
I thought I would get an idea looking at the annotators that came with the
UIMA integration of Solr, but their source was not available. So, I do not
understand how to actually integrate my own annotator in to UIMA.

Can you please explain on how to go about this?

Sowmya.

On Fri, Jul 8, 2011 at 2:03 AM, Koji Sekiguchi <ko...@r.email.ne.jp> wrote:

> (11/07/07 18:38), Sowmya V.B. wrote:
>
>> Hi
>>
>> I am trying to add UIMA module in to Solr..and began with the readme file
>> given here.
>> https://svn.apache.org/repos/**asf/lucene/dev/tags/lucene_**
>> solr_3_1/solr/contrib/uima/**README.txt<https://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_1/solr/contrib/uima/README.txt>
>>
>
> I would recommend you to use Solr 3.3 rather than 3.1, as we have changed
> some configuration
> in solrconfig.xml for UIMA.
>
>
>  2. modify your schema.xml adding the fields you want to be hold
>> metadata specifying proper values for type, indexed, stored and
>> multiValued options:
>>
>> -I understood this line as: adding to my schema.xml, the new fields that
>> will come as a result of a UIMA pipeline. For example, in my UIMA
>> pipeline,
>> post-processing, I get fields A,B,C in addition to fields X,Y,Z that I
>> already added to the SolrInputDocument. So, does this mean I should add
>> A,B,C to the schema.xml?
>>
>
> I think you got it. Have you tried it but you got some errors?
>
>
>  3. In SolrConfig.xml,
>>
>> inside,
>>
>> <uimaConfig>
>>     <runtimeParameters>
>>
>
> The uimaConfig tag has been moved into update processor setting @ Solr 3.2.
> Please see the latest README.txt.
>
>
>  if iam not using any of those "alchemy api key..." etc, I think I can
>> remove
>> those lines. However, I plan to use the openNLP tagger&  tokenizer, and an
>> annotator I wrote for my task. Can I give my model file locations here as
>> runtimeParameters?
>>
>
> I don't have an idea of openNLP.
>
>
>  4. I did not understand what "fieldMapping" tag does. The description
>> said:
>> "field mapping describes which features of which types should go in a
>> field"--
>> - For example, in this snippet from the link:
>>
>>  <type name="org.apache.uima.alchemy.**ts.concept.ConceptFS">
>>        <map feature="text" field="concept"/>
>>       </type>
>>
>> -what does "feature" mean and what does "field" mean?
>>
>
> This defines a map uima feature:
>
> http://uima.apache.org/d/**uimaj-2.3.1/references.html#**
> ugr.ref.xml.component_**descriptor.type_system.**features<http://uima.apache.org/d/uimaj-2.3.1/references.html#ugr.ref.xml.component_descriptor.type_system.features>
>
> to Solr field.
>
> koji
> --
> http://www.rondhuit.com/en/
>



-- 
Sowmya V.B.
----------------------------------------------------
Losing optimism is blasphemy!
http://vbsowmya.wordpress.com
----------------------------------------------------

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
(11/07/07 18:38), Sowmya V.B. wrote:
> Hi
>
> I am trying to add UIMA module in to Solr..and began with the readme file
> given here.
> https://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_1/solr/contrib/uima/README.txt

I would recommend you to use Solr 3.3 rather than 3.1, as we have changed some configuration
in solrconfig.xml for UIMA.

> 2. modify your schema.xml adding the fields you want to be hold
> metadata specifying proper values for type, indexed, stored and
> multiValued options:
>
> -I understood this line as: adding to my schema.xml, the new fields that
> will come as a result of a UIMA pipeline. For example, in my UIMA pipeline,
> post-processing, I get fields A,B,C in addition to fields X,Y,Z that I
> already added to the SolrInputDocument. So, does this mean I should add
> A,B,C to the schema.xml?

I think you got it. Have you tried it but you got some errors?

> 3. In SolrConfig.xml,
>
> inside,
>
> <uimaConfig>
>      <runtimeParameters>

The uimaConfig tag has been moved into update processor setting @ Solr 3.2.
Please see the latest README.txt.

> if iam not using any of those "alchemy api key..." etc, I think I can remove
> those lines. However, I plan to use the openNLP tagger&  tokenizer, and an
> annotator I wrote for my task. Can I give my model file locations here as
> runtimeParameters?

I don't have an idea of openNLP.

> 4. I did not understand what "fieldMapping" tag does. The description said:
> "field mapping describes which features of which types should go in a
> field"--
> - For example, in this snippet from the link:
>
>   <type name="org.apache.uima.alchemy.ts.concept.ConceptFS">
>         <map feature="text" field="concept"/>
>        </type>
>
> -what does "feature" mean and what does "field" mean?

This defines a map uima feature:

http://uima.apache.org/d/uimaj-2.3.1/references.html#ugr.ref.xml.component_descriptor.type_system.features

to Solr field.

koji
-- 
http://www.rondhuit.com/en/