You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Vivek Kamble <vi...@gmail.com> on 2020/04/21 12:49:27 UTC
Lucene index not created/picked by query engine
Hi,
Im using jackrabbit oak version 1.10.8, Im trying to configure lucene
indexes, refered,https://gist.github.com/chetanmeh/c1ccc4fa588ed1af467b,
but query is not picking up luecen index, test ended up with warning,
"Traversed 1000 nodes with filter Filter, consider creating an index or
changing the query"
Debugging using explain showed that Lucene index never pickuped for query.
below is repo creation code,
final RDBOptions options =
new
RDBOptions().tablePrefix(dbDetails.get(DB_TABLE_PREFIX)).dropTablesOnClose(
false);
final DataSource ds =
RDBDataSourceFactory.forJdbcUrl(
dbDetails.get("dbURL"),
dbDetails.get("dbUser"),
dbDetails.get("dbPassword"));
final Properties properties = buildS3Properties(dbDetails);
final S3DataStore s3DataStore = buildS3DataStore(properties);
final DataStoreBlobStore dataStoreBlobStore = new
DataStoreBlobStore(s3DataStore);
final Whiteboard wb = new DefaultWhiteboard();
bapRegistration =
wb.register(
BlobAccessProvider.class,
(BlobAccessProvider) dataStoreBlobStore,
properties);
documentNodeStore =
new RDBDocumentNodeStoreBuilder()
.setBlobStore(dataStoreBlobStore)
.setBundlingDisabled(true)
.setRDBConnection(ds, options)
.build();
LuceneIndexProvider provider = new LuceneIndexProvider();
Oak oak = new Oak(documentNodeStore);
oak =
oak
.withAsyncIndexing("async", 1)
.with(new LuceneIndexEditorProvider())
.with((QueryIndexProvider) provider)
.with((Observer) provider);
repository = new Jcr(oak).with(wb).createRepository();
try {
createLuceneIndex(dbDetails);
} catch (final RepositoryException e) {
throw new CMSException(e);
}
return repository;
public static void
createLuceneIndex(final Map<String, String> dbDetails)
throws RepositoryException {
Session session = getSession(dbDetails);
try {
session.getNode("/oak:index/testLucene");
} catch (final PathNotFoundException e) {
Node lucene =
JcrUtils.getOrCreateByPath(
"/oak:index/testLucene",
"oak:Unstructured",
"oak:QueryIndexDefinition",
session,
false);
lucene.setProperty("compatVersion", 2);
lucene.setProperty("type", "lucene");
lucene.setProperty("async", "async");
Node rules = lucene.addNode("indexRules", "nt:unstructured");
Node allProps =
rules.addNode(JcrConstants.NT_FOLDER).addNode("properties",
"nt:unstructured");
Node propNode = allProps.addNode("allProps");
propNode.setProperty("name", ".*");
propNode.setProperty("isRegexp", true);
propNode.setProperty("propertyIndex", true);
propNode.setProperty("analyzed", true);
Node manuscriptNode =
rules.addNode("spr:manuscript_id").addNode("properties","nt:unstructured");
Node manuscriptNode2 =
manuscriptNode.addNode("spr:manuscript_id");
manuscriptNode2.setProperty("name","spr:manuscript_id");
manuscriptNode2.setProperty("analyzed", true);
manuscriptNode2.setProperty("isRegexp", true);
manuscriptNode2.setProperty("propertyIndex", true);
manuscriptNode2.setProperty("nodeScopeIndex", true);
manuscriptNode2.setProperty("type",
PropertyType.TYPENAME_STRING);
session.save();
session.logout();
log.info("Lucene index created");
}
}
Tried different JCR_SQL2 queries :
1) " select * from [spr:task] where contains(.*,'torgeir')"
2) " select * from [spr:task] where contains([spr:manuscript_id],'torgeir')
"
Any help would b great...
Thanks in advance.
Re: Lucene index not created/picked by query engine
Posted by Matt Ryan <ma...@apache.org>.
Hi Vivek,
Lucene indexing is a bit tricky to set up. Here is a good example on how
to do it:
-
https://svn.apache.org/repos/asf/jackrabbit/oak/trunk/oak-lucene/src/test/java/org/apache/jackrabbit/oak/composite/blueGreen/Persistence.java
This class is used in a couple of tests in the "composite" parent folder.
I referred to this code extensively when I was recently going through the
same exercise, trying to get Lucene indexing to work and be used for
queries in my own code. I was getting the same traversal warning you are
getting. Digging through this code helped me figure out what I needed to
do to make Lucene work for me.
HTH
-MR
On Tue, Apr 21, 2020 at 6:57 AM Vivek Kamble <vi...@gmail.com> wrote:
> Hi,
>
> Im using jackrabbit oak version 1.10.8, Im trying to configure lucene
> indexes, refered,https://gist.github.com/chetanmeh/c1ccc4fa588ed1af467b,
> but query is not picking up luecen index, test ended up with warning,
> "Traversed 1000 nodes with filter Filter, consider creating an index or
> changing the query"
>
>
>
> Debugging using explain showed that Lucene index never pickuped for query.
>
>
>
> below is repo creation code,
>
>
>
> final RDBOptions options =
>
> new
> RDBOptions().tablePrefix(dbDetails.get(DB_TABLE_PREFIX)).dropTablesOnClose(
>
> false);
>
> final DataSource ds =
>
> RDBDataSourceFactory.forJdbcUrl(
>
> dbDetails.get("dbURL"),
>
> dbDetails.get("dbUser"),
>
> dbDetails.get("dbPassword"));
>
> final Properties properties = buildS3Properties(dbDetails);
>
> final S3DataStore s3DataStore = buildS3DataStore(properties);
>
> final DataStoreBlobStore dataStoreBlobStore = new
> DataStoreBlobStore(s3DataStore);
>
> final Whiteboard wb = new DefaultWhiteboard();
>
> bapRegistration =
>
> wb.register(
>
> BlobAccessProvider.class,
>
> (BlobAccessProvider) dataStoreBlobStore,
>
> properties);
>
> documentNodeStore =
>
> new RDBDocumentNodeStoreBuilder()
>
> .setBlobStore(dataStoreBlobStore)
>
> .setBundlingDisabled(true)
>
> .setRDBConnection(ds, options)
>
> .build();
>
>
>
> LuceneIndexProvider provider = new LuceneIndexProvider();
>
> Oak oak = new Oak(documentNodeStore);
>
> oak =
>
> oak
>
> .withAsyncIndexing("async", 1)
>
> .with(new LuceneIndexEditorProvider())
>
> .with((QueryIndexProvider) provider)
>
> .with((Observer) provider);
>
> repository = new Jcr(oak).with(wb).createRepository();
>
>
>
>
>
> try {
>
> createLuceneIndex(dbDetails);
>
> } catch (final RepositoryException e) {
>
> throw new CMSException(e);
>
> }
>
>
>
> return repository;
>
>
>
>
>
> public static void
> createLuceneIndex(final Map<String, String> dbDetails)
>
>
> throws RepositoryException {
>
> Session session = getSession(dbDetails);
>
> try {
>
> session.getNode("/oak:index/testLucene");
>
> } catch (final PathNotFoundException e) {
>
> Node lucene =
>
> JcrUtils.getOrCreateByPath(
>
> "/oak:index/testLucene",
>
> "oak:Unstructured",
>
> "oak:QueryIndexDefinition",
>
> session,
>
> false);
>
> lucene.setProperty("compatVersion", 2);
>
> lucene.setProperty("type", "lucene");
>
> lucene.setProperty("async", "async");
>
> Node rules = lucene.addNode("indexRules", "nt:unstructured");
>
>
>
> Node allProps =
>
> rules.addNode(JcrConstants.NT_FOLDER).addNode("properties",
> "nt:unstructured");
>
> Node propNode = allProps.addNode("allProps");
>
> propNode.setProperty("name", ".*");
>
> propNode.setProperty("isRegexp", true);
>
> propNode.setProperty("propertyIndex", true);
>
> propNode.setProperty("analyzed", true);
>
>
>
>
>
> Node manuscriptNode =
>
>
> rules.addNode("spr:manuscript_id").addNode("properties","nt:unstructured");
>
> Node manuscriptNode2 =
>
> manuscriptNode.addNode("spr:manuscript_id");
>
> manuscriptNode2.setProperty("name","spr:manuscript_id");
>
> manuscriptNode2.setProperty("analyzed", true);
>
> manuscriptNode2.setProperty("isRegexp", true);
>
> manuscriptNode2.setProperty("propertyIndex", true);
>
> manuscriptNode2.setProperty("nodeScopeIndex", true);
>
> manuscriptNode2.setProperty("type",
> PropertyType.TYPENAME_STRING);
>
>
>
>
>
> session.save();
>
> session.logout();
>
> log.info("Lucene index created");
>
> }
>
> }
>
>
>
> Tried different JCR_SQL2 queries :
>
> 1) " select * from [spr:task] where contains(.*,'torgeir')"
>
> 2) " select * from [spr:task] where contains([spr:manuscript_id],'torgeir')
> "
>
>
>
> Any help would b great...
>
> Thanks in advance.
>