You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@uima.apache.org by rohan rai <hi...@gmail.com> on 2008/06/16 07:18:10 UTC
UIMA Cas Consumer on hadoop
What does the given exception mean
If I run it
FSIndex documentIndex =
jCas.getAnnotationIndex(org.ziva.dedup.types.Document.type);
from within hadoop reducer, it works
But if I use this within process CAS it starts throeing exception
Any light
16 Jun, 2008 10:42:41 AM org.ziva.dedup.OnlyPhone processCas(170)
INFO: The JCAS cover class "org.ziva.dedup.types.Document_Type" could
not be loaded.
org.apache.uima.cas.CASRuntimeException: The JCAS cover class
"org.ziva.dedup.types.Document_Type" could not be loaded.
at org.apache.uima.jcas.impl.JCasImpl.getType(JCasImpl.java:417)
at org.apache.uima.jcas.impl.JCasImpl.getCasType(JCasImpl.java:436)
at org.apache.uima.jcas.impl.JCasImpl.getAnnotationIndex(JCasImpl.java:1503)
at org.ziva.dedup.OnlyPhone.processCas(OnlyPhone.java:149)
at org.ziva.dq.hadoop.DeDupHadoopMain$Reduce.reduce(DeDupHadoopMain.java:104)
at org.ziva.dq.hadoop.DeDupHadoopMain$Reduce.reduce(DeDupHadoopMain.java:1)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:333)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2084)
Re: UIMA Cas Consumer on hadoop
Posted by rohan rai <hi...@gmail.com>.
Although there has been no further discussion on the question I posed. But
here is a way out I found out. It a bit hacky but its the only the way out I
have found out.
A Cas consumer can take cas array as input to the method process...(...). So
the way is to pass this method a cas array which gets automatically called
one by one by by the UIMa framework and implement the logic accordingly
REgards
Rohan
On Mon, Jun 16, 2008 at 10:59 PM, rohan rai <hi...@gmail.com> wrote:
> Actually this is the way I agree....In normal circumstances....
> But let me add some complexity in the situation...
> Now imagine you are running UIMA over hadoop...
> You have a map reduce step....
> lets say in the reduce step u want to call a AE which in turn calls a cas
> consumer and you want to do some processing...
> Now how do you give cas as an input to the consumer...
> using classloader you read the AE descriptor
> create a cas out of somewhere...
> I have a cas string which I make it an input stream and deserialize it to
> create a cas
> Now i call the AE...
>
> This works fine...
>
> NOw the scenario in which it does not work...
> Lets say I pass a cas string to AE and furthur to consumer its calling via
> config parameter value....
> Now what i want tot starts do is read this passed string...deserialize it
> and make a cas out of it and do processing...
> Now I am able to desrialize and i am able to create cas out of ...
> I tale care of Class loading its corresponding descriptor...
> But when i try to access the index which in turn requires the type ...
> It throws exception...
>
> N I think Its because of the same reason as one of the other thread I had
> discussion...
> Now inside the consumer it start looking for type in the wrong class path
>
> Is there any way out....
>
> Regards
> Rohan
>
>
> On Mon, Jun 16, 2008 at 5:17 PM, Marshall Schor <ms...@schor.com> wrote:
>
>> rohan rai wrote:
>>
>>> The problem is similar to one I face d before....But here I have no
>>> Resource
>>> Manager to tell
>>>
>>> JCasImpl to get type from the right class path
>>>
>>>
>> The Cas object includes a field which tells JCas which class loader to
>> use; this is usually set when the Cas object is created. I would suggest
>> looking at where the Cas object (being sent to the processCas method) is
>> being created, and making sure that that creation includes the
>> ResourceManager that has the right classpath.
>>
>> -Marshall
>>
>>
>>> On Mon, Jun 16, 2008 at 10:48 AM, rohan rai <hi...@gmail.com> wrote:
>>>
>>>
>>>
>>>> What does the given exception mean
>>>> If I run it
>>>> FSIndex documentIndex =
>>>> jCas.getAnnotationIndex(org.ziva.dedup.types.Document.type);
>>>> from within hadoop reducer, it works
>>>>
>>>> But if I use this within process CAS it starts throeing exception
>>>>
>>>> Any light
>>>>
>>>>
>>>> 16 Jun, 2008 10:42:41 AM org.ziva.dedup.OnlyPhone processCas(170)
>>>> INFO: The JCAS cover class "org.ziva.dedup.types.Document_Type" could
>>>> not be loaded.
>>>> org.apache.uima.cas.CASRuntimeException: The JCAS cover class
>>>> "org.ziva.dedup.types.Document_Type" could not be loaded.
>>>>
>>>> at org.apache.uima.jcas.impl.JCasImpl.getType(JCasImpl.java:417)
>>>> at
>>>> org.apache.uima.jcas.impl.JCasImpl.getCasType(JCasImpl.java:436)
>>>> at
>>>> org.apache.uima.jcas.impl.JCasImpl.getAnnotationIndex(JCasImpl.java:1503)
>>>>
>>>> at org.ziva.dedup.OnlyPhone.processCas(OnlyPhone.java:149)
>>>> at
>>>> org.ziva.dq.hadoop.DeDupHadoopMain$Reduce.reduce(DeDupHadoopMain.java:104)
>>>> at
>>>> org.ziva.dq.hadoop.DeDupHadoopMain$Reduce.reduce(DeDupHadoopMain.java:1)
>>>>
>>>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:333)
>>>> at
>>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2084)
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>
Re: UIMA Cas Consumer on hadoop
Posted by rohan rai <hi...@gmail.com>.
Actually this is the way I agree....In normal circumstances....
But let me add some complexity in the situation...
Now imagine you are running UIMA over hadoop...
You have a map reduce step....
lets say in the reduce step u want to call a AE which in turn calls a cas
consumer and you want to do some processing...
Now how do you give cas as an input to the consumer...
using classloader you read the AE descriptor
create a cas out of somewhere...
I have a cas string which I make it an input stream and deserialize it to
create a cas
Now i call the AE...
This works fine...
NOw the scenario in which it does not work...
Lets say I pass a cas string to AE and furthur to consumer its calling via
config parameter value....
Now what i want tot starts do is read this passed string...deserialize it
and make a cas out of it and do processing...
Now I am able to desrialize and i am able to create cas out of ...
I tale care of Class loading its corresponding descriptor...
But when i try to access the index which in turn requires the type ...
It throws exception...
N I think Its because of the same reason as one of the other thread I had
discussion...
Now inside the consumer it start looking for type in the wrong class path
Is there any way out....
Regards
Rohan
On Mon, Jun 16, 2008 at 5:17 PM, Marshall Schor <ms...@schor.com> wrote:
> rohan rai wrote:
>
>> The problem is similar to one I face d before....But here I have no
>> Resource
>> Manager to tell
>>
>> JCasImpl to get type from the right class path
>>
>>
> The Cas object includes a field which tells JCas which class loader to use;
> this is usually set when the Cas object is created. I would suggest looking
> at where the Cas object (being sent to the processCas method) is being
> created, and making sure that that creation includes the ResourceManager
> that has the right classpath.
>
> -Marshall
>
>
>> On Mon, Jun 16, 2008 at 10:48 AM, rohan rai <hi...@gmail.com> wrote:
>>
>>
>>
>>> What does the given exception mean
>>> If I run it
>>> FSIndex documentIndex =
>>> jCas.getAnnotationIndex(org.ziva.dedup.types.Document.type);
>>> from within hadoop reducer, it works
>>>
>>> But if I use this within process CAS it starts throeing exception
>>>
>>> Any light
>>>
>>>
>>> 16 Jun, 2008 10:42:41 AM org.ziva.dedup.OnlyPhone processCas(170)
>>> INFO: The JCAS cover class "org.ziva.dedup.types.Document_Type" could not
>>> be loaded.
>>> org.apache.uima.cas.CASRuntimeException: The JCAS cover class
>>> "org.ziva.dedup.types.Document_Type" could not be loaded.
>>>
>>> at org.apache.uima.jcas.impl.JCasImpl.getType(JCasImpl.java:417)
>>> at
>>> org.apache.uima.jcas.impl.JCasImpl.getCasType(JCasImpl.java:436)
>>> at
>>> org.apache.uima.jcas.impl.JCasImpl.getAnnotationIndex(JCasImpl.java:1503)
>>>
>>> at org.ziva.dedup.OnlyPhone.processCas(OnlyPhone.java:149)
>>> at
>>> org.ziva.dq.hadoop.DeDupHadoopMain$Reduce.reduce(DeDupHadoopMain.java:104)
>>> at
>>> org.ziva.dq.hadoop.DeDupHadoopMain$Reduce.reduce(DeDupHadoopMain.java:1)
>>>
>>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:333)
>>> at
>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2084)
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
Re: UIMA Cas Consumer on hadoop
Posted by Marshall Schor <ms...@schor.com>.
rohan rai wrote:
> The problem is similar to one I face d before....But here I have no Resource
> Manager to tell
>
> JCasImpl to get type from the right class path
>
The Cas object includes a field which tells JCas which class loader to
use; this is usually set when the Cas object is created. I would
suggest looking at where the Cas object (being sent to the processCas
method) is being created, and making sure that that creation includes
the ResourceManager that has the right classpath.
-Marshall
>
> On Mon, Jun 16, 2008 at 10:48 AM, rohan rai <hi...@gmail.com> wrote:
>
>
>> What does the given exception mean
>> If I run it
>> FSIndex documentIndex = jCas.getAnnotationIndex(org.ziva.dedup.types.Document.type);
>> from within hadoop reducer, it works
>>
>> But if I use this within process CAS it starts throeing exception
>>
>> Any light
>>
>>
>> 16 Jun, 2008 10:42:41 AM org.ziva.dedup.OnlyPhone processCas(170)
>> INFO: The JCAS cover class "org.ziva.dedup.types.Document_Type" could not be loaded.
>> org.apache.uima.cas.CASRuntimeException: The JCAS cover class "org.ziva.dedup.types.Document_Type" could not be loaded.
>>
>> at org.apache.uima.jcas.impl.JCasImpl.getType(JCasImpl.java:417)
>> at org.apache.uima.jcas.impl.JCasImpl.getCasType(JCasImpl.java:436)
>> at org.apache.uima.jcas.impl.JCasImpl.getAnnotationIndex(JCasImpl.java:1503)
>>
>> at org.ziva.dedup.OnlyPhone.processCas(OnlyPhone.java:149)
>> at org.ziva.dq.hadoop.DeDupHadoopMain$Reduce.reduce(DeDupHadoopMain.java:104)
>> at org.ziva.dq.hadoop.DeDupHadoopMain$Reduce.reduce(DeDupHadoopMain.java:1)
>>
>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:333)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2084)
>>
>>
>>
>
>
Re: UIMA Cas Consumer on hadoop
Posted by rohan rai <hi...@gmail.com>.
The problem is similar to one I face d before....But here I have no Resource
Manager to tell
JCasImpl to get type from the right class path
On Mon, Jun 16, 2008 at 10:48 AM, rohan rai <hi...@gmail.com> wrote:
> What does the given exception mean
> If I run it
> FSIndex documentIndex = jCas.getAnnotationIndex(org.ziva.dedup.types.Document.type);
> from within hadoop reducer, it works
>
> But if I use this within process CAS it starts throeing exception
>
> Any light
>
>
> 16 Jun, 2008 10:42:41 AM org.ziva.dedup.OnlyPhone processCas(170)
> INFO: The JCAS cover class "org.ziva.dedup.types.Document_Type" could not be loaded.
> org.apache.uima.cas.CASRuntimeException: The JCAS cover class "org.ziva.dedup.types.Document_Type" could not be loaded.
>
> at org.apache.uima.jcas.impl.JCasImpl.getType(JCasImpl.java:417)
> at org.apache.uima.jcas.impl.JCasImpl.getCasType(JCasImpl.java:436)
> at org.apache.uima.jcas.impl.JCasImpl.getAnnotationIndex(JCasImpl.java:1503)
>
> at org.ziva.dedup.OnlyPhone.processCas(OnlyPhone.java:149)
> at org.ziva.dq.hadoop.DeDupHadoopMain$Reduce.reduce(DeDupHadoopMain.java:104)
> at org.ziva.dq.hadoop.DeDupHadoopMain$Reduce.reduce(DeDupHadoopMain.java:1)
>
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:333)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2084)
>
>