You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@gora.apache.org by "G. M. Shaharia Azam" <sh...@gmail.com> on 2016/12/11 19:55:36 UTC

Nutch 2.x branch MongoStore failed to initialize

Hi,
I just cloned Nutch github repo and checked out 2.x branch. I compiled that
with ant clean and ant runtime. Everything worked great.

But when i tried to inject some URL it say's the following error.


$ $NUTCH_ROOT/runtime/local/bin/nutch inject /seed

InjectorJob: starting at 2016-12-11 19:35:00
InjectorJob: Injecting urlDir: /seed
InjectorJob: org.apache.gora.util.GoraException:
java.lang.RuntimeException: java.io.IOException:
java.lang.NullPointerException
at org.apache.gora.store.DataStoreFactory.createDataStore(
DataStoreFactory.java:167)
at org.apache.gora.store.DataStoreFactory.createDataStore(
DataStoreFactory.java:135)
at org.apache.nutch.storage.StorageUtils.createWebStore(
StorageUtils.java:78)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:234)
at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:268)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:291)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:300)
Caused by: java.lang.RuntimeException: java.io.IOException:
java.lang.NullPointerException
at org.apache.gora.mongodb.store.MongoStore.initialize(MongoStore.java:131)
at org.apache.gora.store.DataStoreFactory.initializeDataStore(
DataStoreFactory.java:102)
at org.apache.gora.store.DataStoreFactory.createDataStore(
DataStoreFactory.java:161)
... 7 more
Caused by: java.io.IOException: java.lang.NullPointerException
at org.apache.gora.mongodb.store.MongoMappingBuilder.fromFile(
MongoMappingBuilder.java:123)
at org.apache.gora.mongodb.store.MongoStore.initialize(MongoStore.java:118)
... 9 more
Caused by: java.lang.NullPointerException
at org.apache.gora.mongodb.store.MongoMapping.newDocumentField(
MongoMapping.java:109)
at org.apache.gora.mongodb.store.MongoMapping.addClassField(
MongoMapping.java:169)
at org.apache.gora.mongodb.store.MongoMappingBuilder.loadPersistentClass(
MongoMappingBuilder.java:169)
at org.apache.gora.mongodb.store.MongoMappingBuilder.fromFile(
MongoMappingBuilder.java:112)
... 10 more


I have my gora.properties file OK like following

gora.datastore.autocreateschema=true
gora.datastore.default=org.apache.gora.mongodb.store.MongoStore
gora.mongodb.override_hadoop_configuration=false
gora.mongodb.mapping.file=/gora-mongodb-mapping.xml
gora.mongodb.servers=localhost:27017
gora.mongodb.db=tempnutch
gora.mongodb.login=test
gora.mongodb.secret=test
gora.mongodb.readpreference=primary
gora.mongodb.writeconcern=acknowledged

my mongodb is working well. I tested that. I am using Mongodb 3.0 version.
And also in ivy/ivy.xml i uncommented mongodb gora dependencies. So it's
any known issue or anything I can workout with any advise?

Thanks,
Shaharia

Re: Nutch 2.x branch MongoStore failed to initialize

Posted by Alfonso Nishikawa <al...@gmail.com>.

Hi, Shaharia.

Although I don't use MongoDB and I never used it, I suspect there is a bug
in Nutch mapping definition here [1]. The attribute "family" of this row
maybe should be "docfield".

I hope it is that :)

Regards,

Alfonso Nishikawa


[1] -
https://github.com/apache/nutch/blob/2.x/conf/gora-mongodb-mapping.xml#L47

2016-12-11 18:55 GMT-01:00 G. M. Shaharia Azam <sh...@gmail.com>:

> Hi,
> I just cloned Nutch github repo and checked out 2.x branch. I compiled
> that with ant clean and ant runtime. Everything worked great.
>
> But when i tried to inject some URL it say's the following error.
>
>
> $ $NUTCH_ROOT/runtime/local/bin/nutch inject /seed
>
> InjectorJob: starting at 2016-12-11 19:35:00
> InjectorJob: Injecting urlDir: /seed
> InjectorJob: org.apache.gora.util.GoraException:
> java.lang.RuntimeException: java.io.IOException:
> java.lang.NullPointerException
> at org.apache.gora.store.DataStoreFactory.createDataStore(DataS
> toreFactory.java:167)
> at org.apache.gora.store.DataStoreFactory.createDataStore(DataS
> toreFactory.java:135)
> at org.apache.nutch.storage.StorageUtils.createWebStore(Storage
> Utils.java:78)
> at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:234)
> at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:268)
> at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:291)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:300)
> Caused by: java.lang.RuntimeException: java.io.IOException:
> java.lang.NullPointerException
> at org.apache.gora.mongodb.store.MongoStore.initialize(MongoSto
> re.java:131)
> at org.apache.gora.store.DataStoreFactory.initializeDataStore(D
> ataStoreFactory.java:102)
> at org.apache.gora.store.DataStoreFactory.createDataStore(DataS
> toreFactory.java:161)
> ... 7 more
> Caused by: java.io.IOException: java.lang.NullPointerException
> at org.apache.gora.mongodb.store.MongoMappingBuilder.fromFile(M
> ongoMappingBuilder.java:123)
> at org.apache.gora.mongodb.store.MongoStore.initialize(MongoSto
> re.java:118)
> ... 9 more
> Caused by: java.lang.NullPointerException
> at org.apache.gora.mongodb.store.MongoMapping.newDocumentField(
> MongoMapping.java:109)
> at org.apache.gora.mongodb.store.MongoMapping.addClassField(Mon
> goMapping.java:169)
> at org.apache.gora.mongodb.store.MongoMappingBuilder.loadPersis
> tentClass(MongoMappingBuilder.java:169)
> at org.apache.gora.mongodb.store.MongoMappingBuilder.fromFile(M
> ongoMappingBuilder.java:112)
> ... 10 more
>
>
> I have my gora.properties file OK like following
>
> gora.datastore.autocreateschema=true
> gora.datastore.default=org.apache.gora.mongodb.store.MongoStore
> gora.mongodb.override_hadoop_configuration=false
> gora.mongodb.mapping.file=/gora-mongodb-mapping.xml
> gora.mongodb.servers=localhost:27017
> gora.mongodb.db=tempnutch
> gora.mongodb.login=test
> gora.mongodb.secret=test
> gora.mongodb.readpreference=primary
> gora.mongodb.writeconcern=acknowledged
>
> my mongodb is working well. I tested that. I am using Mongodb 3.0 version.
> And also in ivy/ivy.xml i uncommented mongodb gora dependencies. So it's
> any known issue or anything I can workout with any advise?
>
> Thanks,
> Shaharia
>