You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by "Charles Erwin (Created) (JIRA)" <ji...@apache.org> on 2012/04/06 20:13:22 UTC
[jira] [Created] (JCR-3287) Fulltext search breaking between 2.2
and 2.4.1
Fulltext search breaking between 2.2 and 2.4.1
----------------------------------------------
Key: JCR-3287
URL: https://issues.apache.org/jira/browse/JCR-3287
Project: Jackrabbit Content Repository
Issue Type: Bug
Components: jackrabbit-core
Affects Versions: 2.4
Environment: 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
Reporter: Charles Erwin
Upgraded Jackrabbit Dependencies from 2.2.0 to 2.4.1 and now cannot get Full Text search to work. I have created a Github project to show the repository.xml configuration and the search code to prevent it being cluttered up here: https://github.com/greymeister/jackrabbit-2.4-search-bug
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (JCR-3287) Fulltext search breaking between
2.2 and 2.4.1
Posted by Razvan Potter <ra...@gmail.com>.
I do not remember making any other configuration change. I left the
SearchIndex config in the repository xml as it came with the default
deployment.
On Wed, Apr 25, 2012 at 10:02 AM, Charles Erwin (JIRA) <ji...@apache.org>wrote:
>
> [
> https://issues.apache.org/jira/browse/JCR-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262014#comment-13262014]
>
> Charles Erwin commented on JCR-3287:
> ------------------------------------
>
> Thank you for the information Razvan. I am pretty sure that we are
> storing content with the correct mimeType, so that should not be the issue.
>
> I will try changing the dependencies for Tika as you suggest and see if I
> get different results. Did you have any other configuration changes you
> had to make for the text extraction to work? I am concerned that aside
> from the Tika issue that there is new configuration needed from 2.2.x that
> has not been updated on the Wiki.
>
> > Fulltext search breaking between 2.2 and 2.4.1
> > ----------------------------------------------
> >
> > Key: JCR-3287
> > URL: https://issues.apache.org/jira/browse/JCR-3287
> > Project: Jackrabbit Content Repository
> > Issue Type: Bug
> > Components: jackrabbit-core
> > Affects Versions: 2.4
> > Environment: 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> > java version "1.6.0_29"
> > Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
> > Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> > Reporter: Charles Erwin
> >
> > Upgraded Jackrabbit Dependencies from 2.2.0 to 2.4.1 and now cannot get
> Full Text search to work. I have created a Github project to show the
> repository.xml configuration and the search code to prevent it being
> cluttered up here:
> https://github.com/greymeister/jackrabbit-2.4-search-bug
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
[jira] [Commented] (JCR-3287) Fulltext search breaking between 2.2
and 2.4.1
Posted by "Charles Erwin (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251279#comment-13251279 ]
Charles Erwin commented on JCR-3287:
------------------------------------
I found that upgrading to 2.2.11 still works, but that I start seeing problems at 2.3.0. I'm guessing that this has to do with the Lucene 3.0 upgrade. 2.3.0 seems to search non-binary properties, but not all binary properties as in the 2.2.x series.
> Fulltext search breaking between 2.2 and 2.4.1
> ----------------------------------------------
>
> Key: JCR-3287
> URL: https://issues.apache.org/jira/browse/JCR-3287
> Project: Jackrabbit Content Repository
> Issue Type: Bug
> Components: jackrabbit-core
> Affects Versions: 2.4
> Environment: 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> Reporter: Charles Erwin
>
> Upgraded Jackrabbit Dependencies from 2.2.0 to 2.4.1 and now cannot get Full Text search to work. I have created a Github project to show the repository.xml configuration and the search code to prevent it being cluttered up here: https://github.com/greymeister/jackrabbit-2.4-search-bug
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3287) Fulltext search breaking between 2.2
and 2.4.1
Posted by "Charles Erwin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259917#comment-13259917 ]
Charles Erwin commented on JCR-3287:
------------------------------------
That could be the case, I have not changed my repository.xml for 2.4 from what it was in 2.2. What was the exclusions excerpt that Jukka posted in the work log of JCR-2885? Is that part of the indexing_configuration for Lucene? I have not specified any additional indexing configuration, so it is using whatever the default is.
I can create a few runnable Groovy files but setting up a Unit test might be tough as the indexing is asynchronous and as such I'd have to throw in some arbitrary wait period before attempting to see if the file has been indexed.
> Fulltext search breaking between 2.2 and 2.4.1
> ----------------------------------------------
>
> Key: JCR-3287
> URL: https://issues.apache.org/jira/browse/JCR-3287
> Project: Jackrabbit Content Repository
> Issue Type: Bug
> Components: jackrabbit-core
> Affects Versions: 2.4
> Environment: 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> Reporter: Charles Erwin
>
> Upgraded Jackrabbit Dependencies from 2.2.0 to 2.4.1 and now cannot get Full Text search to work. I have created a Github project to show the repository.xml configuration and the search code to prevent it being cluttered up here: https://github.com/greymeister/jackrabbit-2.4-search-bug
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3287) Fulltext search breaking between 2.2
and 2.4.1
Posted by "Alex Parvulescu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260371#comment-13260371 ]
Alex Parvulescu commented on JCR-3287:
--------------------------------------
JCR-2885 is responsible for moving the text extractors (namely tika-parsers library) out of jackrabbit-core. So you need to make sure your test project references them.
We are only talking about classpaths here. You don't need to change anything else, just make sure that tika-parsers is included.
In maven terms, this is how the dependency looks like so you should be able to add that to your project [0].
> indexing is asynchronous and as such I'd have to throw in some arbitrary wait period before attempting to see if the file has been indexed.
Yes, I'm well aware of that, we have some tricks in the Jackrabbit tests that wait for indexing to be done [1], but I'm not sure if you could hook into that for your tests.
[0] http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit-parent/pom.xml?view=markup#l372
[1] http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit-core/src/test/java/org/apache/jackrabbit/core/query/AbstractIndexingTest.java?view=markup#l61
> Fulltext search breaking between 2.2 and 2.4.1
> ----------------------------------------------
>
> Key: JCR-3287
> URL: https://issues.apache.org/jira/browse/JCR-3287
> Project: Jackrabbit Content Repository
> Issue Type: Bug
> Components: jackrabbit-core
> Affects Versions: 2.4
> Environment: 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> Reporter: Charles Erwin
>
> Upgraded Jackrabbit Dependencies from 2.2.0 to 2.4.1 and now cannot get Full Text search to work. I have created a Github project to show the repository.xml configuration and the search code to prevent it being cluttered up here: https://github.com/greymeister/jackrabbit-2.4-search-bug
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (JCR-3287) Fulltext search breaking
between 2.2 and 2.4.1
Posted by "Charles Erwin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265839#comment-13265839 ]
Charles Erwin edited comment on JCR-3287 at 5/1/12 2:43 PM:
------------------------------------------------------------
I think I have finally gotten full text search to work, given Razvan's comments. I disabled the tika-parser and tika-core dependencies from Jackrabbit 2.4.1 and manually included version 1.1 of those dependencies. This could be a defect that needs to be reviewed.
I am also guessing that the wiki/documentation just needs to include that you must add an indexing configuration and a tika-config to enable full text search on content, which I don't believe it does currently.
Now it seems that search works, but docx files do not seem to be indexed correctly. I will research that independently of this issue.
was (Author: greymeister):
I think I have finally gotten full text search to work, given Razvan's comments. I disabled the tika-parser and tika-core dependencies from Jackrabbit 2.4.1 and manually included version 1.1 of those dependencies. Now it seems that search works, but docx files do not seem to be indexed correctly. I will research that independently of this issue.
I am guessing that the wiki/documentation just needs to include that you must add an indexing configuration and a tika-config to enable full text search on content, which I don't believe it does currently.
> Fulltext search breaking between 2.2 and 2.4.1
> ----------------------------------------------
>
> Key: JCR-3287
> URL: https://issues.apache.org/jira/browse/JCR-3287
> Project: Jackrabbit Content Repository
> Issue Type: Bug
> Components: jackrabbit-core
> Affects Versions: 2.4
> Environment: 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> Reporter: Charles Erwin
> Attachments: tika_lucene_config.zip
>
>
> Upgraded Jackrabbit Dependencies from 2.2.0 to 2.4.1 and now cannot get Full Text search to work. I have created a Github project to show the repository.xml configuration and the search code to prevent it being cluttered up here: https://github.com/greymeister/jackrabbit-2.4-search-bug
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JCR-3287) Fulltext search breaking between 2.2
and 2.4.1
Posted by "Charles Erwin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Charles Erwin updated JCR-3287:
-------------------------------
Attachment: tika_lucene_config.zip
I think I have finally gotten full text search to work, given Razvan's comments. I disabled the tika-parser and tika-core dependencies from Jackrabbit 2.4.1 and manually included version 1.1 of those dependencies. Now it seems that search works, but docx files do not seem to be indexed correctly. I will research that independently of this issue.
I am guessing that the wiki/documentation just needs to include that you must add an indexing configuration and a tika-config to enable full text search on content, which I don't believe it does currently.
> Fulltext search breaking between 2.2 and 2.4.1
> ----------------------------------------------
>
> Key: JCR-3287
> URL: https://issues.apache.org/jira/browse/JCR-3287
> Project: Jackrabbit Content Repository
> Issue Type: Bug
> Components: jackrabbit-core
> Affects Versions: 2.4
> Environment: 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> Reporter: Charles Erwin
> Attachments: tika_lucene_config.zip
>
>
> Upgraded Jackrabbit Dependencies from 2.2.0 to 2.4.1 and now cannot get Full Text search to work. I have created a Github project to show the repository.xml configuration and the search code to prevent it being cluttered up here: https://github.com/greymeister/jackrabbit-2.4-search-bug
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3287) Fulltext search breaking between 2.2
and 2.4.1
Posted by "Alex Parvulescu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258129#comment-13258129 ]
Alex Parvulescu commented on JCR-3287:
--------------------------------------
Hi Charles,
Are you sure it's not JCR-2885?
(Extraction of binary properties is done by Apache Tika, not Lucene)
I don't see the classpath in the github project you've created. Can you package it as a runnable unit test?
> Fulltext search breaking between 2.2 and 2.4.1
> ----------------------------------------------
>
> Key: JCR-3287
> URL: https://issues.apache.org/jira/browse/JCR-3287
> Project: Jackrabbit Content Repository
> Issue Type: Bug
> Components: jackrabbit-core
> Affects Versions: 2.4
> Environment: 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> Reporter: Charles Erwin
>
> Upgraded Jackrabbit Dependencies from 2.2.0 to 2.4.1 and now cannot get Full Text search to work. I have created a Github project to show the repository.xml configuration and the search code to prevent it being cluttered up here: https://github.com/greymeister/jackrabbit-2.4-search-bug
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3287) Fulltext search breaking between 2.2
and 2.4.1
Posted by "Razvan Potter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261949#comment-13261949 ]
Razvan Potter commented on JCR-3287:
------------------------------------
I am currently using 2.4.1 and text extraction/search seem to work fine. I
encountered though 2 roadblocks before I got it to work; one, an issue on
my side; the other, maybe an issue with the jars distribution for JR:
1). I loaded my resource nodes and had a mimeType set incorrectly to an
empty string. Tika can't do extraction without a correct mimeType that
matches the content of the input stream.
2). One of Tika's classes was throwing a class not found error. Looking
into it I found out that the tika jars included with the JR distribution
did not include all the needed dependencies. I downloaded the tika-app jar
from Tika's website and things worked fine afterwards. Throughout this
process I replaced the tika 1.0 jars with the 1.1 jars.
Maybe this is helpful for Charles.
> Fulltext search breaking between 2.2 and 2.4.1
> ----------------------------------------------
>
> Key: JCR-3287
> URL: https://issues.apache.org/jira/browse/JCR-3287
> Project: Jackrabbit Content Repository
> Issue Type: Bug
> Components: jackrabbit-core
> Affects Versions: 2.4
> Environment: 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> Reporter: Charles Erwin
>
> Upgraded Jackrabbit Dependencies from 2.2.0 to 2.4.1 and now cannot get Full Text search to work. I have created a Github project to show the repository.xml configuration and the search code to prevent it being cluttered up here: https://github.com/greymeister/jackrabbit-2.4-search-bug
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3287) Fulltext search breaking between 2.2
and 2.4.1
Posted by "Charles Erwin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262014#comment-13262014 ]
Charles Erwin commented on JCR-3287:
------------------------------------
Thank you for the information Razvan. I am pretty sure that we are storing content with the correct mimeType, so that should not be the issue.
I will try changing the dependencies for Tika as you suggest and see if I get different results. Did you have any other configuration changes you had to make for the text extraction to work? I am concerned that aside from the Tika issue that there is new configuration needed from 2.2.x that has not been updated on the Wiki.
> Fulltext search breaking between 2.2 and 2.4.1
> ----------------------------------------------
>
> Key: JCR-3287
> URL: https://issues.apache.org/jira/browse/JCR-3287
> Project: Jackrabbit Content Repository
> Issue Type: Bug
> Components: jackrabbit-core
> Affects Versions: 2.4
> Environment: 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11D50b)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> Reporter: Charles Erwin
>
> Upgraded Jackrabbit Dependencies from 2.2.0 to 2.4.1 and now cannot get Full Text search to work. I have created a Github project to show the repository.xml configuration and the search code to prevent it being cluttered up here: https://github.com/greymeister/jackrabbit-2.4-search-bug
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira