Solr + HDFS settings

[Norgorn <ls...@mail.ru>]

I'm trying to run SOLR with HDFS
 in solrconfig.xml I've written 

<directoryFactory name="DirectoryFactory" class="solr.HdfsDirectoryFactory">
	  <str name="solr.hdfs.home">hdfs://<PATH_TO_NAMENODE>/solr</str>
	  <bool name="solr.hdfs.blockcache.enabled">true</bool>
	  <int name="solr.hdfs.blockcache.slab.count">1</int>
	  <bool name="solr.hdfs.blockcache.direct.memory.allocation">true</bool>
	  <int name="solr.hdfs.blockcache.blocksperbank">16384</int>
	  <bool name="solr.hdfs.blockcache.read.enabled">true</bool>
	  <bool name="solr.hdfs.blockcache.write.enabled">true</bool>
	  <bool name="solr.hdfs.nrtcachingdirectory.enable">true</bool>
	  <int name="solr.hdfs.nrtcachingdirectory.maxmergesizemb">16</int>
	  <int name="solr.hdfs.nrtcachingdirectory.maxcachedmb">192</int>
</directoryFactory> 

But, when I'm trying to create collection, I get 
"Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol
message end-group tag did not match expected tag." 

tcpdump shows, that server answers with "Bad request", I've tried to set
path to datanode instead of namenode, but got the same result.
What am I doing wrong?

P.S. I've found, that the problem can be cause of different protobuf.jar,
I've changed that jar (and hadoop-*.jar for comparability) in my SOLR libs,
but the problem didn't change.




--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-HDFS-settings-tp4165873.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr + HDFS settings

[Norgorn <ls...@mail.ru>]

Already tried with same result (the message changed properly )



--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-HDFS-settings-tp4165873p4166089.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr + HDFS settings

[Michael Della Bitta <mi...@appinions.com>]

This doesn't answer your question, but unless something is changed, 
you're going to want to set this to false. It causes index corruption at 
the moment.

On 10/25/14 03:42, Norgorn wrote:
> 	  <bool name="solr.hdfs.blockcache.write.enabled">true</bool>

Re: Solr + HDFS settings

[Norgorn <ls...@mail.ru>]

Ok, new problem, while collection or shard creating:

Caused by: no segments* file found in
NRTCachingDirectory(HdfsDirectory@3a19dc74
lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@43507d1b;
maxCacheMB=192.0 maxMergeSizeMB=16.0): files: [HdfsDirectory@3a19dc74
lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@43507d1b-write.lock]

This problem is the same if I try to create with
"solr/admin/collections?action=CREATE&name=hdfs_test&numShards=1",
or with
"solr/admin/cores?action=CREATE&collection=hdfs_test&name=hdfs_test_sh1&shard=shard1".

Have no idea what to do with that.



--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-HDFS-settings-tp4165873p4165878.html
Sent from the Solr - User mailing list archive at Nabble.com.