You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Sidana, Mohit" <Mo...@wolterskluwer.com> on 2016/06/20 14:42:11 UTC
Encryption to Solr indexes – Using Custom Codec
Hello,
As Part of my studies I am exploring the solutions which can be used for Lucene/Solr Index encryption.
I found the patch open on Apache JIRA - Codec for index-level encryption<https://issues.apache.org/jira/browse/LUCENE-6966> (LUCENE-6966). https://issues.apache.org/jira/browse/LUCENE-6966 and I am currently trying to test this Custom codec with Solr to perform secure search over some sensitive records.
I've decided to follow the path described in Solr wiki, setting up Simple Text Codec and further tried to use Encrypted codec Source.
Here are the additional details.
I've created a basic jar file out of this source code (Build it as a jar from Eclipse using Maven Plugin).
The Solr installation I'm using to test this is the Solr 6.0.0 unzipped, and started via its embedded Jetty server and using the single core.
I've placed my jar with the codec in [My_Core\ instance Dir.]\ Lib
In:
[$SolrDir]\Solr\ My_Core \conf\solrconfig.xml
I've added the following lines:
<lib dir="./lib/" regex=".*\.jar"/>
<codecFactory class="solr.SchemaCodecFactory"/>
Then in the schema.xml file, I've declared some field and field Types that should use this codec:
<field name="City" type="strings"/>
<field name="Country" type="strings"/>
<field name="Name" type="strings"/>
<field name="Payment" type="strings"/>
<field name="Price" type="tlongs"/>
<field name="Product" type="strings"/>
<field name="State" type="strings"/>
<!-- set the per field posting format as below -->
<fieldType name="strings" class="solr.StrField" sortMissingLast="true" docValues="true" multiValued="true" postingsFormat="EncryptedLucene50"/>
<fieldType name="tlongs" class="solr.TrieLongField" positionIncrementGap="0" docValues="true" multiValued="true" precisionStep="8" postingsFormat="EncryptedLucene50"/>
I'm pretty sure I've followed all the steps described in Solr Wiki; however, when I actually try to use custom codec implementation (named "Encrypted Codec") to index some sample CSV data using simple post tool
java -Dtype=text/csv -Durl=http://localhost:8983/solr/ My_Core /update -jar post.jar Sales.csv
and I have also tried doing the same with SolrJ but I have faced the same error.
SolrClient server = new HttpSolrClient("http://localhost:8983/solr/ My_Core ");
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", "1234");
doc.addField("name", "A lovely summer holiday");
try {
server.add(doc);
server.commit();
System.out.println("Document added!");
} catch (SolrServerException | IOException e) {
e.printStackTrace();
}
}
}
I get the attached errors in Solr log.
org.apache.solr.common.SolrException: Exception writing document id b3e01ada-d0f1-4ddf-ad6a-2828bfe619a3 to the index; possible analysis error.
at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:181)
at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:68)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:335)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldNameMutatingUpdateProcessorFactory$1.processAdd(FieldNameMutatingUpdateProcessorFactory.java:74)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:936)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1091)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:714)
at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.AbstractDefaultValueUpdateProcessorFactory$DefaultValueUpdateProcessor.processAdd(AbstractDefaultValueUpdateProcessorFactory.java:93)
at org.apache.solr.handler.loader.CSVLoaderBase.doAdd(CSVLoaderBase.java:391)
at org.apache.solr.handler.loader.SingleThreadedCSVLoader.addDoc(CSVLoader.java:43)
at org.apache.solr.handler.loader.CSVLoaderBase.load(CSVLoaderBase.java:360)
at org.apache.solr.handler.loader.CSVLoader.load(CSVLoader.java:30)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:94)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:69)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2033)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:724)
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:738)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1488)
at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:282)
at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:214)
at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:169)
... 58 more
Caused by: java.lang.NoClassDefFoundError: org/apache/lucene/codecs/encrypted/DummyCipherFactory
at org.apache.lucene.codecs.encrypted.DummyEncryptedLucene60Codec$DummyEncryptedLucene50PostingsFormat.getCipherFactory(DummyEncryptedLucene60Codec.java:67)
at org.apache.lucene.codecs.encrypted.EncryptedLucene50PostingsFormat.fieldsConsumer(EncryptedLucene50PostingsFormat.java:479)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:196)
at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:107)
at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:134)
at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:423)
at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:502)
at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:614)
at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2815)
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2989)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2956)
at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:619)
at org.apache.solr.update.UpdateLog$LogReplayer.doReplay(UpdateLog.java:1453)
at org.apache.solr.update.UpdateLog$LogReplayer.run(UpdateLog.java:1256)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Caused by: java.lang.ClassNotFoundException: org.apache.lucene.codecs.encrypted.DummyCipherFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 22 more
From the output I assume that Solr is not picking up the jar with this custom codec, and I don't get why?
What I find strange is that, I have also tried to run this codec with Lucene and eclipse by writing some code for indexing and searching and same codec jar is working just fine.
System.out.println("Indexing to directory '" + indexPath + "'...");
Directory dir = FSDirectory.open(Paths.get(indexPath));
Analyzer analyzer = new StandardAnalyzer();
IndexWriterConfig iwc = new IndexWriterConfig(analyzer);
// set encrypted codec //This encrypted Codec is being used for indexing
iwc.setCodec(new DummyEncryptedLucene60Codec());
if (create) {
// Create a new index in the directory, removing any
// previously indexed documents:
iwc.setOpenMode(OpenMode.CREATE);
} else {
// Add new documents to an existing index:
iwc.setOpenMode(OpenMode.CREATE_OR_APPEND);
}
IndexWriter writer = new IndexWriter(dir, iwc);
indexDocs(writer, docDir);
writer.close();
I'm guessing that As Solr uses Java SPI options to load this codec which loads things differently, and there's something missing...?
My question is, what might I be doing wrong here or what's missing so that codec jar can be picked up by Solr?
Thanks.
Re: Encryption to Solr indexes – Using Custom Codec
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
On 21 June 2016 at 00:42, Sidana, Mohit <Mo...@wolterskluwer.com> wrote:
> As Part of my studies I am exploring the solutions which can be used for
> Lucene/Solr Index encryption.
Can't help with the actual question, but just remembered that Hitachi
was showing some Solr Encryption at the Lucene/Solr Revolution last
year. I think it is this one:
https://psg.hitachi-solutions.com/credeon/secure-full-text-search
Regards,
Alex.
----
Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/
RE: Encryption to Solr indexes – Using Custom Codec
Posted by "Sidana, Mohit" <Mo...@wolterskluwer.com>.
Hi Renaud,
I apologize for a typo. I have the jars in directory called "lib" instead of "Lib".(Mistyped while I posted).
I have also added the Encrypted codec Jar to the directory - [$SolrDir]\server\solr-webapp\webapp\WEB-INF\lib and all of a sudden it seems to be working fine now.
I don’t really understand this behavior of Solr as per Solr wiki all external jars should be in [Solr_Core\ instance Dir.]\lib and when I check logs indeed it is loading the jar from this directory.
INFO (coreLoadExecutor-6-thread-1) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 'C:\solr-6.0.0\solr-6.0.0\server\solr\demo'
1190 INFO (coreLoadExecutor-6-thread-2) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 'C:\solr-6.0.0\solr-6.0.0\server\solr\test'
1190 INFO (coreLoadExecutor-6-thread-1) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
1190 INFO (coreLoadExecutor-6-thread-2) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
1190 INFO (coreLoadExecutor-6-thread-1) [ ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: C:\solr-6.0.0\solr-6.0.0\server\solr
1190 INFO (coreLoadExecutor-6-thread-2) [ ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: C:\solr-6.0.0\solr-6.0.0\server\solr
1191 INFO (coreLoadExecutor-6-thread-2) [ ] o.a.s.c.SolrResourceLoader Adding 'file:/C:/solr-6.0.0/solr-6.0.0/server/solr/test/lib/SecureCodec-SecureCodecSNAPSHOT.jar' to classloader
1206 INFO (main) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@a67c67e{/solr,file:///C:/solr-6.0.0/solr-6.0.0/server/solr-webapp/webapp/,AVAILABLE}{C:\solr-6.0.0\solr-6.0.0\server/solr-webapp/webapp
After Indexing I get _6_EncryptedLucene50_0.doc ; _6_EncryptedLucene50_0.Pos ; _6_EncryptedLucene50_0.Tim ; _6_EncryptedLucene50_0.Tip etc. so I assume Solr is working fine and performing indexing with Encrypted codec.
build lucene/solr trunk and applied the latest patch to the Branch 6_x and after compiling all seems to be working fine now.
Thanks,
Mohit
-----Original Message-----
From: Renaud Delbru [mailto:renaud@siren.solutions]
Sent: Tuesday, June 21, 2016 11:39 AM
To: solr-user@lucene.apache.org
Subject: Re: Encryption to Solr indexes – Using Custom Codec
Hi,
maybe it is the way you created the jar ? Why not applying the patch to lucene/solr trunk and use ant jar instead to get the codecs jar created for you ?
Also, I think the directory where you put the jars should be called "lib" instead of "Lib".
you can try also to use the lib directives in your solrconfig.xml [1]
[1]
https://cwiki.apache.org/confluence/display/solr/Lib+Directives+in+SolrConfig
--
Renaud Delbru
On 20/06/16 15:42, Sidana, Mohit wrote:
> Hello,
>
> As Part of my studies I am exploring the solutions which can be used
> for Lucene/Solr Index encryption.
>
> I found the patch open on Apache JIRA- Codec for index-level
> encryption <https://issues.apache.org/jira/browse/LUCENE-6966>(LUCENE-6966).
> https://issues.apache.org/jira/browse/LUCENE-6966 and I am currently
> trying to test this Custom codec with Solr to perform secure search
> over some sensitive records.
>
> I've decided to follow the path described in Solr wiki, setting up
> Simple Text Codec and further tried to use Encrypted codec Source.
>
> *Here are the additional details.*
>
> I've created a basic jar file out of this source code (Build it as a
> jar from Eclipse using Maven Plugin).
>
> The Solr installation I'm using to test this is the Solr 6.0.0
> unzipped, and started via its embedded Jetty server and using the single core.
>
> I've placed my jar with the codec in [My_Core\ instance Dir.]\ Lib
>
> In:
>
> [$SolrDir]\Solr\ My_Core \conf\*solrconfig.xml*
>
> I've added the following lines:
>
> |<lib dir="./lib/" regex=".*\.jar"/> |||
>
> |<codecFactory class="solr.SchemaCodecFactory"/>|
>
> Then in the *schema.xml* file, I've declared some field and field
> Types that should use this codec:
>
> |<field name="City" type="strings"/>|||
>
> | <field name="Country" type="strings"/>|
>
> | <field name="Name" type="strings"/>|
>
> | <field name="Payment" type="strings"/>|
>
> | <field name="Price" type="tlongs"/>|
>
> | <field name="Product" type="strings"/>|
>
> | <field name="State" type="strings"/>|
>
> ||
>
> |<!-- set the per field posting format as below -->|
>
> |<fieldType name="strings" class="solr.StrField" sortMissingLast="true"
> docValues="true" multiValued="true"
> postingsFormat="EncryptedLucene50"/>|
>
> |<fieldType name="tlongs" class="solr.TrieLongField"
> positionIncrementGap="0" docValues="true" multiValued="true"
> precisionStep="8" postingsFormat="EncryptedLucene50"/>|
>
> I'm pretty sure I've followed all the steps described in Solr Wiki;
> however, when I actually try to use custom codec implementation (named
> "Encrypted Codec") to index some sample CSV data using simple post
> tool
>
> java -Dtype=text/csv -Durl=http://localhost:8983/solr/My_Core /update
> -jar post.jar Sales.csv
>
> and I have also tried doing the same with SolrJ but I have faced the
> same error.
>
> SolrClient _server_=
> *new*HttpSolrClient("http://localhost:8983/solr/My_Core ");
>
> SolrInputDocument doc= *new*SolrInputDocument();
>
> doc.addField("id", "1234");
>
> doc.addField("name", "A lovely summer holiday");
>
> *try*{
>
> server.add(doc);
>
> server.commit();
>
> System.*/out/*.println("Document added!");
>
> } *catch*(SolrServerException | IOException e) {
>
> e.printStackTrace();
>
> }
>
> }
>
> }
>
> I get the attached errors in Solr log.
>
> org.apache.solr.common.SolrException: Exception writing document id
> b3e01ada-d0f1-4ddf-ad6a-2828bfe619a3 to the index; possible analysis error.
>
> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler
> 2.java:181)
>
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpda
> teProcessorFactory.java:68)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(Upd
> ateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory
> $AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProces
> sorFactory.java:335)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(Upd
> ateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processA
> dd(FieldMutatingUpdateProcessor.java:117)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(Upd
> ateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processA
> dd(FieldMutatingUpdateProcessor.java:117)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(Upd
> ateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processA
> dd(FieldMutatingUpdateProcessor.java:117)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(Upd
> ateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processA
> dd(FieldMutatingUpdateProcessor.java:117)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(Upd
> ateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldNameMutatingUpdateProcessorFacto
> ry$1.processAdd(FieldNameMutatingUpdateProcessorFactory.java:74)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(Upd
> ateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processA
> dd(FieldMutatingUpdateProcessor.java:117)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(Upd
> ateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd
> (DistributedUpdateProcessor.java:936)
>
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd
> (DistributedUpdateProcessor.java:1091)
>
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd
> (DistributedUpdateProcessor.java:714)
>
> at
> org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdatePr
> ocessor.processAdd(LogUpdateProcessorFactory.java:103)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(Upd
> ateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.AbstractDefaultValueUpdateProcessorFa
> ctory$DefaultValueUpdateProcessor.processAdd(AbstractDefaultValueUpdat
> eProcessorFactory.java:93)
>
> at
> org.apache.solr.handler.loader.CSVLoaderBase.doAdd(CSVLoaderBase.java:
> 391)
>
> at
> org.apache.solr.handler.loader.SingleThreadedCSVLoader.addDoc(CSVLoade
> r.java:43)
>
> at
> org.apache.solr.handler.loader.CSVLoaderBase.load(CSVLoaderBase.java:3
> 60)
>
> at
> org.apache.solr.handler.loader.CSVLoader.load(CSVLoader.java:30)
>
> at
> org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandl
> er.java:94)
>
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Con
> tentStreamHandlerBase.java:69)
>
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandle
> rBase.java:155)
>
> at
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2033)
>
> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
>
> at
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
>
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter
> .java:229)
>
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter
> .java:184)
>
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletH
> andler.java:1668)
>
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
> 581)
>
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja
> va:143)
>
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java
> :548)
>
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandle
> r.java:226)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandle
> r.java:1160)
>
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:5
> 11)
>
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler
> .java:185)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler
> .java:1092)
>
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja
> va:141)
>
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Conte
> xtHandlerCollection.java:213)
>
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerColle
> ction.java:119)
>
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.
> java:134)
>
> at
> org.eclipse.jetty.server.Server.handle(Server.java:518)
>
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
>
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java
> :244)
>
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(Abstrac
> tConnection.java:273)
>
> at
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
>
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint
> .java:93)
>
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAn
> dRun(ExecuteProduceConsume.java:246)
>
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(Execu
> teProduceConsume.java:156)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool
> .java:654)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.
> java:572)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: org.apache.lucene.store.AlreadyClosedException: this
> IndexWriter is closed
>
> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:724)
>
> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:738)
>
> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:14
> 88)
>
> at
> org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdat
> eHandler2.java:282)
>
> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandle
> r2.java:214)
>
> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler
> 2.java:169)
>
> ... 58 more
>
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/lucene/codecs/encrypted/DummyCipherFactory
>
> at
> org.apache.lucene.codecs.encrypted.DummyEncryptedLucene60Codec$DummyEn
> cryptedLucene50PostingsFormat.getCipherFactory(DummyEncryptedLucene60C
> odec.java:67)
>
> at
> org.apache.lucene.codecs.encrypted.EncryptedLucene50PostingsFormat.fie
> ldsConsumer(EncryptedLucene50PostingsFormat.java:479)
>
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.
> write(PerFieldPostingsFormat.java:196)
>
> at
> org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.
> java:107)
>
> at
> org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChai
> n.java:134)
>
> at
> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriter
> PerThread.java:423)
>
> at
> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:5
> 02)
>
> at
> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWrite
> r.java:614)
>
> at
> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.
> java:2815)
>
> at
> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:29
> 89)
>
> at
> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2956)
>
> at
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler
> 2.java:619)
>
> at
> org.apache.solr.update.UpdateLog$LogReplayer.doReplay(UpdateLog.java:1
> 453)
>
> at
> org.apache.solr.update.UpdateLog$LogReplayer.run(UpdateLog.java:1256)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511
> )
>
> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511
> )
>
> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.la
> mbda$execute$0(ExecutorUtil.java:229)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j
> ava:1142)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
> java:617)
>
> ... 1 more
>
> Caused by: java.lang.ClassNotFoundException:
> org.apache.lucene.codecs.encrypted.DummyCipherFactory
>
> at
> java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>
> at
> java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>
> at
> java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814)
>
> at
> java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>
> ... 22 more
>
> From the output I assume that Solr is not picking up the jar with
> this custom codec, and I don't get why?
>
> What I find strange is that, I have also tried to run this codec with
> Lucene and eclipse by writing some code for indexing and searching and
> same codec jar is working just fine.
>
> System.*/out/*.println("Indexing to directory
> '"+
> indexPath+ "'...");
>
> Directory dir=
> FSDirectory./open/(Paths./get/(indexPath));
>
> Analyzer analyzer= *new*StandardAnalyzer();
>
> IndexWriterConfig iwc=
> *new*IndexWriterConfig(analyzer);
>
> // set encrypted codec //This encrypted Codec is being used for
> indexing
>
> iwc.setCodec(*new*DummyEncryptedLucene60Codec());
>
> *if*(create) {
>
> // Create a new index in the directory, removing any
>
> // previously indexed documents:
>
> iwc.setOpenMode(OpenMode.*/CREATE/*);
>
> } *else*{
>
> // Add new documents to an existing index:
>
> iwc.setOpenMode(OpenMode.*/CREATE_OR_APPEND/*);
>
> }
>
> IndexWriter writer= *new*IndexWriter(dir, iwc);
>
> /indexDocs/(writer, docDir);
>
> writer.close();
>
> I'm guessing that As Solr uses Java SPI options to load this codec
> which loads things differently, and there's something missing...?
>
> My question is, what might I be doing wrong here or what's missing so
> that codec jar can be picked up by Solr?
>
> Thanks.
>
Re: Encryption to Solr indexes – Using Custom Codec
Posted by Renaud Delbru <re...@siren.solutions>.
Hi,
maybe it is the way you created the jar ? Why not applying the patch to
lucene/solr trunk and use ant jar instead to get the codecs jar created
for you ?
Also, I think the directory where you put the jars should be called
"lib" instead of "Lib".
you can try also to use the lib directives in your solrconfig.xml [1]
[1]
https://cwiki.apache.org/confluence/display/solr/Lib+Directives+in+SolrConfig
--
Renaud Delbru
On 20/06/16 15:42, Sidana, Mohit wrote:
> Hello,
>
> As Part of my studies I am exploring the solutions which can be used for
> Lucene/Solr Index encryption.
>
> I found the patch open on Apache JIRA- Codec for index-level encryption
> <https://issues.apache.org/jira/browse/LUCENE-6966>(LUCENE-6966).
> https://issues.apache.org/jira/browse/LUCENE-6966 and I am currently
> trying to test this Custom codec with Solr to perform secure search over
> some sensitive records.
>
> I've decided to follow the path described in Solr wiki, setting up
> Simple Text Codec and further tried to use Encrypted codec Source.
>
> *Here are the additional details.*
>
> I've created a basic jar file out of this source code (Build it as a jar
> from Eclipse using Maven Plugin).
>
> The Solr installation I'm using to test this is the Solr 6.0.0 unzipped,
> and started via its embedded Jetty server and using the single core.
>
> I've placed my jar with the codec in [My_Core\ instance Dir.]\ Lib
>
> In:
>
> [$SolrDir]\Solr\ My_Core \conf\*solrconfig.xml*
>
> I've added the following lines:
>
> |<lib dir="./lib/" regex=".*\.jar"/> |||
>
> |<codecFactory class="solr.SchemaCodecFactory"/>|
>
> Then in the *schema.xml* file, I've declared some field and field Types
> that should use this codec:
>
> |<field name="City" type="strings"/>|||
>
> | <field name="Country" type="strings"/>|
>
> | <field name="Name" type="strings"/>|
>
> | <field name="Payment" type="strings"/>|
>
> | <field name="Price" type="tlongs"/>|
>
> | <field name="Product" type="strings"/>|
>
> | <field name="State" type="strings"/>|
>
> ||
>
> |<!-- set the per field posting format as below -->|
>
> |<fieldType name="strings" class="solr.StrField" sortMissingLast="true"
> docValues="true" multiValued="true" postingsFormat="EncryptedLucene50"/>|
>
> |<fieldType name="tlongs" class="solr.TrieLongField"
> positionIncrementGap="0" docValues="true" multiValued="true"
> precisionStep="8" postingsFormat="EncryptedLucene50"/>|
>
> I'm pretty sure I've followed all the steps described in Solr Wiki;
> however, when I actually try to use custom codec implementation (named
> "Encrypted Codec") to index some sample CSV data using simple post tool
>
> java -Dtype=text/csv -Durl=http://localhost:8983/solr/My_Core /update
> -jar post.jar Sales.csv
>
> and I have also tried doing the same with SolrJ but I have faced the
> same error.
>
> SolrClient _server_=
> *new*HttpSolrClient("http://localhost:8983/solr/My_Core ");
>
> SolrInputDocument doc= *new*SolrInputDocument();
>
> doc.addField("id", "1234");
>
> doc.addField("name", "A lovely summer holiday");
>
> *try*{
>
> server.add(doc);
>
> server.commit();
>
> System.*/out/*.println("Document added!");
>
> } *catch*(SolrServerException | IOException e) {
>
> e.printStackTrace();
>
> }
>
> }
>
> }
>
> I get the attached errors in Solr log.
>
> org.apache.solr.common.SolrException: Exception writing document id
> b3e01ada-d0f1-4ddf-ad6a-2828bfe619a3 to the index; possible analysis error.
>
> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:181)
>
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:68)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:335)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldNameMutatingUpdateProcessorFactory$1.processAdd(FieldNameMutatingUpdateProcessorFactory.java:74)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:936)
>
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1091)
>
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:714)
>
> at
> org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
>
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
>
> at
> org.apache.solr.update.processor.AbstractDefaultValueUpdateProcessorFactory$DefaultValueUpdateProcessor.processAdd(AbstractDefaultValueUpdateProcessorFactory.java:93)
>
> at
> org.apache.solr.handler.loader.CSVLoaderBase.doAdd(CSVLoaderBase.java:391)
>
> at
> org.apache.solr.handler.loader.SingleThreadedCSVLoader.addDoc(CSVLoader.java:43)
>
> at
> org.apache.solr.handler.loader.CSVLoaderBase.load(CSVLoaderBase.java:360)
>
> at
> org.apache.solr.handler.loader.CSVLoader.load(CSVLoader.java:30)
>
> at
> org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:94)
>
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:69)
>
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
>
> at
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2033)
>
> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
>
> at
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
>
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
>
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
>
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
>
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
>
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
>
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
>
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
>
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
>
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
>
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>
> at org.eclipse.jetty.server.Server.handle(Server.java:518)
>
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
>
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
>
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
>
> at
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
>
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
>
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
>
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: org.apache.lucene.store.AlreadyClosedException: this
> IndexWriter is closed
>
> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:724)
>
> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:738)
>
> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1488)
>
> at
> org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:282)
>
> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:214)
>
> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:169)
>
> ... 58 more
>
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/lucene/codecs/encrypted/DummyCipherFactory
>
> at
> org.apache.lucene.codecs.encrypted.DummyEncryptedLucene60Codec$DummyEncryptedLucene50PostingsFormat.getCipherFactory(DummyEncryptedLucene60Codec.java:67)
>
> at
> org.apache.lucene.codecs.encrypted.EncryptedLucene50PostingsFormat.fieldsConsumer(EncryptedLucene50PostingsFormat.java:479)
>
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:196)
>
> at
> org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:107)
>
> at
> org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:134)
>
> at
> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:423)
>
> at
> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:502)
>
> at
> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:614)
>
> at
> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2815)
>
> at
> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2989)
>
> at
> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2956)
>
> at
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:619)
>
> at
> org.apache.solr.update.UpdateLog$LogReplayer.doReplay(UpdateLog.java:1453)
>
> at
> org.apache.solr.update.UpdateLog$LogReplayer.run(UpdateLog.java:1256)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> ... 1 more
>
> Caused by: java.lang.ClassNotFoundException:
> org.apache.lucene.codecs.encrypted.DummyCipherFactory
>
> at
> java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>
> at
> java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>
> ... 22 more
>
> From the output I assume that Solr is not picking up the jar with this
> custom codec, and I don't get why?
>
> What I find strange is that, I have also tried to run this codec with
> Lucene and eclipse by writing some code for indexing and searching and
> same codec jar is working just fine.
>
> System.*/out/*.println("Indexing to directory '"+
> indexPath+ "'...");
>
> Directory dir=
> FSDirectory./open/(Paths./get/(indexPath));
>
> Analyzer analyzer= *new*StandardAnalyzer();
>
> IndexWriterConfig iwc=
> *new*IndexWriterConfig(analyzer);
>
> // set encrypted codec //This encrypted Codec is being used for indexing
>
> iwc.setCodec(*new*DummyEncryptedLucene60Codec());
>
> *if*(create) {
>
> // Create a new index in the directory, removing any
>
> // previously indexed documents:
>
> iwc.setOpenMode(OpenMode.*/CREATE/*);
>
> } *else*{
>
> // Add new documents to an existing index:
>
> iwc.setOpenMode(OpenMode.*/CREATE_OR_APPEND/*);
>
> }
>
> IndexWriter writer= *new*IndexWriter(dir, iwc);
>
> /indexDocs/(writer, docDir);
>
> writer.close();
>
> I'm guessing that As Solr uses Java SPI options to load this codec which
> loads things differently, and there's something missing...?
>
> My question is, what might I be doing wrong here or what's missing so
> that codec jar can be picked up by Solr?
>
> Thanks.
>