You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@jspwiki.apache.org by Andrew Ducker <an...@thephoenixgroup.com> on 2022/04/18 10:32:32 UTC

RE: [External] Re: Search not working on unmodified files

Hi,

Thanks for the reply.

We completely cleared the WorkDir, to be on the safe side, and that didn’t help.

It has access to the Work and wikipages folders, as it can create them from scratch, and you can edit the Wiki files.

No mentions of “unable to index” in the logs.

In the jspwiki-custom.properties file there’s the following:
jspwiki.applicationName =IS Wiki
jspwiki.baseURL = <removed>
jspwiki.frontPage = Main
jspwiki.pageProvider =VersioningFileProvider
jspwiki.fileSystemProvider.pageDir =d:/JSPWiki/JSPWikiPages/Pages/wiki/
jspwiki.basicAttachmentProvider.storageDir =d:/JSPWiki/JSPWikiPages/Attachments/wiki/
jspwiki.interWikiRef.Notes = Notes:%s
jspwiki.translatorReader.inlinePattern.1 = *.jpg
jspwiki.translatorReader.inlinePattern.2 = *.png
jspwiki.translatorReader.inlinePattern.3 = http://images.com/*
jspwiki.rss.generate =true
jspwiki.rss.channelDescription = IS Wiki
mail.from =@mail.from@
mail.smtp.host =@mail.smtp.host@
jspwiki.workDir =d:/JSPWiki/WorkingDirectory/iswiki/
jspwiki.security = jaas

Lucene-specific things in the logs:
[INFO ] 2022-04-13 13:44:48.094 [main] o.a.w.s.LuceneSearchProvider - Lucene enabled, cache will be in: d:\JSPWiki\WorkingDirectory\iswiki\lucene
[WARN ] 2022-04-13 13:44:48.725 [JSPWiki Lucene Indexer] o.a.w.WikiBackgroundThread - Starting up background thread: JSPWiki Lucene Indexer.
[INFO ] 2022-04-13 13:45:48.732 [JSPWiki Lucene Indexer] o.a.w.s.LuceneSearchProvider - Files found in Lucene directory, not reindexing.

So could it be starting to index, thinking it’s already indexed (despite the whole WorkingDir being cleared out) and then stopping near-instantly?

Thanks again,

Andy

From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
Sent: 14 April 2022 16:25
To: user@jspwiki.apache.org
Subject: [External] Re: Search not working on unmodified files

Alert! This is an external email sent from juanpablo.santos@gmail.com.

Hi Andrew,

haven't had the time to look in depth at JSPWIKI-1171, so just some
questions / random thoughts:

JSPWiki has changed a lot since 2.8, but the wiki pages itself shouldn't
require any change, and should be readable by any version of JSPWiki. The
Lucene version has been upgraded several times, so it was definitely ok to
clear your Lucene folder. Also there is the refmgr.ser file and refmgr-attr
dir inside your workDir, which contain serialized information about page
references. I'm not 100% sure, but most probably the internal format has
changed since 2.8 (there was a global package rename on 2.9, when entering
the Apache incubator, and somewhat later a public API was pushed which may
have affected this format too) so you should delete those too and try to
restart your JSPWiki instance again and see if the situation persists;
perhaps deleting the entire workDir would be the safest approach. That's
what seems to be happening from the stacktrace attached at JSPWIKI-1171
(this list strips attachments, IIRC).

Also, regarding the searches, would you mind confirming that the new
JSPWiki instance has permissions to read/write either the work and
wikipages directories/files? Also are there any errors logged when starting
JSPWiki? Specifically, the Lucene indexing is done on startup (if needed)
on the background, so eventually you should get all your pages indexed or
see log errors stating something like "Unable to index page $pageName,
continuing to next".

Are you using a vanilla JSPWiki installation or do you have some
customizations? What parameters do you have on your
jspwiki-custom.properties file? I'm assuming you're deloying the war file?

best regards,
juan pablo

On Thu, Apr 14, 2022 at 4:51 PM Andrew Ducker <
andrew_ducker@thephoenixgroup.com> wrote:

> Hi!
>
>
>
> I am having a problem with an upgraded JSPWiki not indexing pages.
>
>
>
> I wasn’t sure where to report it, so added an issue on Jira:
>
> https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>
>
>
>
> I’m recreating it here, in case this is the right place to report things:
>
>
>
> Lots of pages aren't turning up in search.
>
>
>
> We cleared the lucene folder from the working directory, and it rebuilt it
> - but quite quickly for 13,000 pages. And the total size of the Lucene
> files was around 2.5MB, which seemed far too small.
>
>
>
> Doing a search failed to bring back pages that had the given word in their
> name and content.
>
>
>
> Editing one of those pages (adding a space) caused it to then appear in
> the searches.
>
>
>
> Checking the logs I found a lot of errors in the form:
>
>
>
> [ERROR] 2022-04-12 15:50:17.274 [main] o.a.w.r.DefaultReferenceManager -
> Error while trying to fetch a page name; trying to cope with the situation.
>
> org.apache.wiki.api.exceptions.ProviderException: Illegal page name
>
>
>
> (See attachment for whole exception trace)
>
>
>
> Sadly, it doesn't tell me what the illegal page names are.
>
>
>
> This wiki has been upgraded from version 2.8 - are there compatibility
> issues with page names there?
>
>
>
> Any help you can give me gratefully received!
>
>
>
> Andy
>
>
>
> --
>
>
>
>
>
> <https://www.thephoenixgroup.com/>
> *Andrew Ducker* | Senior Systems Developer
> office: +441312458823 | mobile: 07980 86 86 39 | email:
> andrew_ducker@thephoenixgroup.com
> P Please help save the environment, print only if necessary.
> [image: logobadgefinal]
>
>
>
> Confidentiality - This email is confidential.
> Not meant for you? - If you don't think this email is meant for you,
> please let us know. Do not copy or forward the information it contains, and
> delete this email from your system.
> Views expressed - Any personal views or opinions expressed in this email
> are the sender's, and do not necessarily reflect the views of the Phoenix
> Group (Phoenix Group Holdings plc and its subsidiaries).
> Monitoring - We filter and monitor emails to protect our systems and to
> keep them running smoothly.
> Emailing us - Email isn't a secure form of communication. If you want to
> send us confidential information please send it by post. However, if you do
> communicate with us by email on any subject, you are giving us permission
> to email you back.
> Phoning us - Calls may be monitored and/or recorded to protect both you
> and us and help with our training. Call charges will vary.
> Standard Life Assurance Limited is registered in Scotland (SC286833) at
> Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> the Prudential Regulation Authority and regulated by the Financial Conduct
> Authority and the Prudential Regulation Authority. www.standardlife.co.uk<http://www.standardlife.co.uk>
> Standard Life Assurance Limited and Standard Life Assets and Employee
> Services Limited are companies in the Phoenix Group (Phoenix Group Holdings
> plc and its subsidiaries). www.thephoenixgroup.com
>

Confidentiality - This email is confidential.
Not meant for you? - If you don't think this email is meant for you, please let us know. Do not copy or forward the information it contains, and delete this email from your system.
Views expressed - Any personal views or opinions expressed in this email are the sender's, and do not necessarily reflect the views of the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries).
Monitoring - We filter and monitor emails to protect our systems and to keep them running smoothly.
Emailing us - Email isn't a secure form of communication. If you want to send us confidential information please send it by post. However, if you do communicate with us by email on any subject, you are giving us permission to email you back.
Phoning us - Calls may be monitored and/or recorded to protect both you and us and help with our training. Call charges will vary.
Standard Life Assurance Limited is registered in Scotland (SC286833) at Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. www.standardlife.co.uk<http://www.standardlife.co.uk>
Standard Life Assurance Limited and Standard Life Assets and Employee Services Limited are companies in the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries). www.thephoenixgroup.com<http://www.thephoenixgroup.com>

Re: [External] Re: Search not working on unmodified files

Posted by Juan Pablo Santos Rodríguez <ju...@gmail.com>.

Hi Andrew, Jim,

all these issues should've been fixed in 2.11.3-git-07, so no more
ehcache tweaking should be needed (unless you want it to), would you
mind trying with latest master and confirm it?

thx,
juan pablo

On Fri, Apr 22, 2022 at 5:24 PM Andrew Ducker
<an...@thephoenixgroup.com> wrote:
>
> Hi,
>
> I created the ehcache file, dropped it into the same folder as jspwiki-custom.properties, stopped the service, cleared the WorkingDir, restarted the service.
>
> The good news is that it now logs all 13,000 entries!
>
> I then had to assign more memory to Tomcat, but that’s understandable.
>
> Thank you for this!
>
> Andy
>
> From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
> Sent: 21 April 2022 22:21
> To: user@jspwiki.apache.org
> Subject: Re: [External] Re: Search not working on unmodified files
>
> Alert! This is an external email sent from juanpablo.santos@gmail.com.
>
> Hi Andrew, Jim,
>
> hmm upon closer looking, seems LuceneSearchProvider requests all pages
> to the page manager, which in turn delegates
> to the caching provider, which, by default, holds up to 1.000
> elements, and that's why no more pages appear indexed on a
> full reindex. I'll push a fix for that tomorrow / this weekend.
>
> In the meantime, you can use [#1] as an example of an ehcache file.
> Please see [#2] to see the appropiate parameters
> needed to use a custom ehcache file. Would you mind testing if the
> error persist while using a custom ehcache file with
> more elements than pages available?
>
>
> best regards,
> juan pablo
>
>
> [#1] https://github.com/apache/jspwiki/blob/master/jspwiki-cache/src/main/resources/ehcache-jspwiki.xml<https://github.com/apache/jspwiki/blob/master/jspwiki-cache/src/main/resources/ehcache-jspwiki.xml>
> [#2] https://jspwiki-wiki.apache.org/Wiki.jsp?page=NewIn2.11#section-NewIn2.11-NewInJSPWiki2.11.1ReleasedOn18122021<https://jspwiki-wiki.apache.org/Wiki.jsp?page=NewIn2.11#section-NewIn2.11-NewInJSPWiki2.11.1ReleasedOn18122021>
>
> On Thu, Apr 21, 2022 at 2:38 PM Jim Willeke <ji...@willeke.com> wrote:
> >
> > I found this one:
> > https://jar-download.com/artifacts/org.apache.jspwiki/jspwiki-war/2.10.1/source-code/ehcache.xml<https://jar-download.com/artifacts/org.apache.jspwiki/jspwiki-war/2.10.1/source-code/ehcache.xml>
> >
> > --
> > -jim
> > Jim Willeke
> >
> >
> > On Thu, Apr 21, 2022 at 6:41 AM Andrew Ducker <
> > andrew_ducker@thephoenixgroup.com> wrote:
> >
> > > Is there an example ehcache file somewhere? It doesn’t get created
> > > anywhere automatically that I can see, and it’s not in the
> > > https://jspwiki-wiki.apache.org/Wiki.jsp?page=Configuration<https://jspwiki-wiki.apache.org/Wiki.jsp?page=Configuration> documentation.
> > >
> > > Cheers,
> > >
> > > Andy
> > >
> > > From: Jim Willeke [mailto:jim@willeke.com]
> > > Sent: 20 April 2022 21:35
> > > To: user@jspwiki.apache.org
> > > Subject: Re: [External] Re: Search not working on unmodified files
> > >
> > > Alert! This is an external email sent from jim@willeke.com.
> > >
> > > I have observed something similar.
> > >
> > > Active Sessions 51
> > > Uptime 0d, 18h 30m 39s
> > > Number of pages 16390FYI: I too am seeing similar issues on 2.11.0-M8.
> > >
> > > I have NOT observed the issues on 2.10.1.
> > >
> > > The instance on version 2.10.1 (https://ldapwiki.com/wiki/SystemInfo<https://ldapwiki.com/wiki/SystemInfo><
> > > https://ldapwiki.com/wiki/SystemInfo<https://ldapwiki.com/wiki/SystemInfo>>) is
> > > busy
> > > and the 2.11.0-M8 instance on has very low traffic.
> > >
> > > On the version 2.11.0-M8, Cleared the Lucene and work directory and
> > > restarted:
> > >
> > > Active Sessions 1
> > > Uptime 0d, 9h 11m 7s
> > > Number of pages 16499
> > >
> > > And one day later is shows:
> > > Active Sessions 1
> > > Uptime 1d, 10h 15m 43s
> > > Number of pages 46
> > >
> > > Then after using for a few minutes it now says:
> > > Active Sessions 1
> > > Uptime 1d, 10h 35m 52s
> > > Number of pages 16500
> > > (And yes one page was really added over the day)
> > >
> > > Where Number of pages is the [{$totalpages}]
> > >
> > > On the version 2.10.1 below is typical:
> > >
> > > Active Sessions 51
> > > Uptime 0d, 18h 30m 39s
> > > Number of pages 16390
> > >
> > > The ability to perform a search appears to be related to the
> > > [{$totalpages}] variable. That is when it shows 46 almost nothing can be
> > > found.
> > >
> > > I do see in the log:
> > > 2022-04-20 10:12:22,050 [main] WARN
> > > org.apache.wiki.providers.CachingProvider - seems
> > > Ldapwiki.jspwiki.pageCache can't hold all pages from your page repository,
> > > so we're delegating on the underlying provider instead. Please consider
> > > increasing your cache sizes on ehcache.xml to avoid this behaviour
> > >
> > > Which led to the finding the entry in
> > >
> > > https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties<https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties>
> > > <
> > > https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties<https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties>
> > > >
> > >
> > > "
> > > *# By default, JSPWiki caches will hold up to 1.000 elements, except the
> > > RSS cache, which will hold up to 250
> > > elementsjspwiki.cache.custom-config-file = jspwiki-ehcache.xml*"
> > >
> > > Both are behind nginx.
> > >
> > > --
> > > -jim
> > > Jim Willeke
> > >
> > >
> > > On Mon, Apr 18, 2022 at 3:40 PM Juan Pablo Santos Rodríguez <
> > > juanpablo.santos@gmail.com> wrote:
> > >
> > > > Hi!
> > > >
> > > > Looking at the source code, at jspwiki init, a full reindex is triggered.
> > > > The full reindex is only performed if the Lucene dir is empty. As this is
> > > > not the case, the initial full reindex isn't executed, and that's why you
> > > > are not seeing anything on searches. Pages saves do trigger a partial
> > > > reindex, though, so that explains the other side of the behaviour you're
> > > > seeing.
> > > >
> > > > What files are inside the Lucene dir? If there are Lucene index files,
> > > can
> > > > you open then with Luke to see where they came from, or the Lucene index
> > > > version, or any other information? Is there any chance other jspwiki
> > > > instance is using the same workDir?
> > > >
> > > > Last, emptying the lucene dir will enforce a full reindex next time the
> > > > background thread responsible of performing Lucene reindexes runs, but it
> > > > would be better to know why some files are getting created there outside
> > > > your jspwiki instance.
> > > >
> > > >
> > > > HTH,
> > > > juan pablo
> > > >
> > > > El lun., 18 abr. 2022 12:32, Andrew Ducker <
> > > > andrew_ducker@thephoenixgroup.com> escribió:
> > > >
> > > > > Hi,
> > > > >
> > > > > Thanks for the reply.
> > > > >
> > > > > We completely cleared the WorkDir, to be on the safe side, and that
> > > > didn’t
> > > > > help.
> > > > >
> > > > > It has access to the Work and wikipages folders, as it can create them
> > > > > from scratch, and you can edit the Wiki files.
> > > > >
> > > > > No mentions of “unable to index” in the logs.
> > > > >
> > > > > In the jspwiki-custom.properties file there’s the following:
> > > > > jspwiki.applicationName =IS Wiki
> > > > > jspwiki.baseURL = <removed>
> > > > > jspwiki.frontPage = Main
> > > > > jspwiki.pageProvider =VersioningFileProvider
> > > > > jspwiki.fileSystemProvider.pageDir =d:/JSPWiki/JSPWikiPages/Pages/wiki/
> > > > > jspwiki.basicAttachmentProvider.storageDir
> > > > > =d:/JSPWiki/JSPWikiPages/Attachments/wiki/
> > > > > jspwiki.interWikiRef.Notes = Notes:%s
> > > > > jspwiki.translatorReader.inlinePattern.1 = *.jpg
> > > > > jspwiki.translatorReader.inlinePattern.2 = *.png
> > > > > jspwiki.translatorReader.inlinePattern.3 = http://images.com/*<http://images.com/*><
> > > http://images.com/*<http://images.com/*>>
> > > > > jspwiki.rss.generate =true
> > > > > jspwiki.rss.channelDescription = IS Wiki
> > > > > mail.from =@mail.from@
> > > > > mail.smtp.host<http://mail.smtp.host><http://mail.smtp.host<http://mail.smtp.host>> =@mail.smtp.host@
> > > > > jspwiki.workDir =d:/JSPWiki/WorkingDirectory/iswiki/
> > > > > jspwiki.security = jaas
> > > > >
> > > > > Lucene-specific things in the logs:
> > > > > [INFO ] 2022-04-13 13:44:48.094 [main] o.a.w.s.LuceneSearchProvider -
> > > > > Lucene enabled, cache will be in:
> > > > d:\JSPWiki\WorkingDirectory\iswiki\lucene
> > > > > [WARN ] 2022-04-13 13:44:48.725 [JSPWiki Lucene Indexer]
> > > > > o.a.w.WikiBackgroundThread - Starting up background thread: JSPWiki
> > > > Lucene
> > > > > Indexer.
> > > > > [INFO ] 2022-04-13 13:45:48.732 [JSPWiki Lucene Indexer]
> > > > > o.a.w.s.LuceneSearchProvider - Files found in Lucene directory, not
> > > > > reindexing.
> > > > >
> > > > > So could it be starting to index, thinking it’s already indexed
> > > (despite
> > > > > the whole WorkingDir being cleared out) and then stopping
> > > near-instantly?
> > > > >
> > > > > Thanks again,
> > > > >
> > > > > Andy
> > > > >
> > > > >
> > > > > From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
> > > > > Sent: 14 April 2022 16:25
> > > > > To: user@jspwiki.apache.org
> > > > > Subject: [External] Re: Search not working on unmodified files
> > > > >
> > > > > Alert! This is an external email sent from juanpablo.santos@gmail.com.
> > > > >
> > > > > Hi Andrew,
> > > > >
> > > > > haven't had the time to look in depth at JSPWIKI-1171, so just some
> > > > > questions / random thoughts:
> > > > >
> > > > > JSPWiki has changed a lot since 2.8, but the wiki pages itself
> > > shouldn't
> > > > > require any change, and should be readable by any version of JSPWiki.
> > > The
> > > > > Lucene version has been upgraded several times, so it was definitely ok
> > > > to
> > > > > clear your Lucene folder. Also there is the refmgr.ser file and
> > > > refmgr-attr
> > > > > dir inside your workDir, which contain serialized information about
> > > page
> > > > > references. I'm not 100% sure, but most probably the internal format
> > > has
> > > > > changed since 2.8 (there was a global package rename on 2.9, when
> > > > entering
> > > > > the Apache incubator, and somewhat later a public API was pushed which
> > > > may
> > > > > have affected this format too) so you should delete those too and try
> > > to
> > > > > restart your JSPWiki instance again and see if the situation persists;
> > > > > perhaps deleting the entire workDir would be the safest approach.
> > > That's
> > > > > what seems to be happening from the stacktrace attached at JSPWIKI-1171
> > > > > (this list strips attachments, IIRC).
> > > > >
> > > > > Also, regarding the searches, would you mind confirming that the new
> > > > > JSPWiki instance has permissions to read/write either the work and
> > > > > wikipages directories/files? Also are there any errors logged when
> > > > starting
> > > > > JSPWiki? Specifically, the Lucene indexing is done on startup (if
> > > needed)
> > > > > on the background, so eventually you should get all your pages indexed
> > > or
> > > > > see log errors stating something like "Unable to index page $pageName,
> > > > > continuing to next".
> > > > >
> > > > > Are you using a vanilla JSPWiki installation or do you have some
> > > > > customizations? What parameters do you have on your
> > > > > jspwiki-custom.properties file? I'm assuming you're deloying the war
> > > > file?
> > > > >
> > > > >
> > > > > best regards,
> > > > > juan pablo
> > > > >
> > > > > On Thu, Apr 14, 2022 at 4:51 PM Andrew Ducker <
> > > > > andrew_ducker@thephoenixgroup.com> wrote:
> > > > >
> > > > > > Hi!
> > > > > >
> > > > > >
> > > > > >
> > > > > > I am having a problem with an upgraded JSPWiki not indexing pages.
> > > > > >
> > > > > >
> > > > > >
> > > > > > I wasn’t sure where to report it, so added an issue on Jira:
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171><
> > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>><
> > > > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171><
> > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>>>
> > > > > >
> > > > > >
> > > > > >
> > > > > > I’m recreating it here, in case this is the right place to report
> > > > things:
> > > > > >
> > > > > >
> > > > > >
> > > > > > Lots of pages aren't turning up in search.
> > > > > >
> > > > > >
> > > > > >
> > > > > > We cleared the lucene folder from the working directory, and it
> > > rebuilt
> > > > > it
> > > > > > - but quite quickly for 13,000 pages. And the total size of the
> > > Lucene
> > > > > > files was around 2.5MB, which seemed far too small.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Doing a search failed to bring back pages that had the given word in
> > > > > their
> > > > > > name and content.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Editing one of those pages (adding a space) caused it to then appear
> > > in
> > > > > > the searches.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Checking the logs I found a lot of errors in the form:
> > > > > >
> > > > > >
> > > > > >
> > > > > > [ERROR] 2022-04-12 15:50:17.274 [main]
> > > o.a.w.r.DefaultReferenceManager
> > > > -
> > > > > > Error while trying to fetch a page name; trying to cope with the
> > > > > situation.
> > > > > >
> > > > > > org.apache.wiki.api.exceptions.ProviderException: Illegal page name
> > > > > >
> > > > > >
> > > > > >
> > > > > > (See attachment for whole exception trace)
> > > > > >
> > > > > >
> > > > > >
> > > > > > Sadly, it doesn't tell me what the illegal page names are.
> > > > > >
> > > > > >
> > > > > >
> > > > > > This wiki has been upgraded from version 2.8 - are there
> > > compatibility
> > > > > > issues with page names there?
> > > > > >
> > > > > >
> > > > > >
> > > > > > Any help you can give me gratefully received!
> > > > > >
> > > > > >
> > > > > >
> > > > > > Andy
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > <https://www.thephoenixgroup.com/>
> > > > > > *Andrew Ducker* | Senior Systems Developer
> > > > > > office: +441312458823 | mobile: 07980 86 86 39 | email:
> > > > > > andrew_ducker@thephoenixgroup.com
> > > > > > P Please help save the environment, print only if necessary.
> > > > > > [image: logobadgefinal]
> > > > > >
> > > > > >
> > > > > >
> > > > > > Confidentiality - This email is confidential.
> > > > > > Not meant for you? - If you don't think this email is meant for you,
> > > > > > please let us know. Do not copy or forward the information it
> > > contains,
> > > > > and
> > > > > > delete this email from your system.
> > > > > > Views expressed - Any personal views or opinions expressed in this
> > > > email
> > > > > > are the sender's, and do not necessarily reflect the views of the
> > > > Phoenix
> > > > > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > > > > Monitoring - We filter and monitor emails to protect our systems and
> > > to
> > > > > > keep them running smoothly.
> > > > > > Emailing us - Email isn't a secure form of communication. If you want
> > > > to
> > > > > > send us confidential information please send it by post. However, if
> > > > you
> > > > > do
> > > > > > communicate with us by email on any subject, you are giving us
> > > > permission
> > > > > > to email you back.
> > > > > > Phoning us - Calls may be monitored and/or recorded to protect both
> > > you
> > > > > > and us and help with our training. Call charges will vary.
> > > > > > Standard Life Assurance Limited is registered in Scotland (SC286833)
> > > at
> > > > > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and
> > > authorised
> > > > by
> > > > > > the Prudential Regulation Authority and regulated by the Financial
> > > > > Conduct
> > > > > > Authority and the Prudential Regulation Authority.
> > > > > www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>><
> > > http://www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>>>
> > > > > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > > > > Services Limited are companies in the Phoenix Group (Phoenix Group
> > > > > Holdings
> > > > > > plc and its subsidiaries). www.thephoenixgroup.com
> > > > > >
> > > > >
> > > > > Confidentiality - This email is confidential.
> > > > > Not meant for you? - If you don't think this email is meant for you,
> > > > > please let us know. Do not copy or forward the information it contains,
> > > > and
> > > > > delete this email from your system.
> > > > > Views expressed - Any personal views or opinions expressed in this
> > > email
> > > > > are the sender's, and do not necessarily reflect the views of the
> > > Phoenix
> > > > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > > > Monitoring - We filter and monitor emails to protect our systems and to
> > > > > keep them running smoothly.
> > > > > Emailing us - Email isn't a secure form of communication. If you want
> > > to
> > > > > send us confidential information please send it by post. However, if
> > > you
> > > > do
> > > > > communicate with us by email on any subject, you are giving us
> > > permission
> > > > > to email you back.
> > > > > Phoning us - Calls may be monitored and/or recorded to protect both you
> > > > > and us and help with our training. Call charges will vary.
> > > > > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > > > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised
> > > by
> > > > > the Prudential Regulation Authority and regulated by the Financial
> > > > Conduct
> > > > > Authority and the Prudential Regulation Authority.
> > > > www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>><
> > > > > http://www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>>>
> > > > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > > > Services Limited are companies in the Phoenix Group (Phoenix Group
> > > > Holdings
> > > > > plc and its subsidiaries). www.thephoenixgroup.com<
> > > > > http://www.thephoenixgroup.com>
> > > > >
> > > >
> > >
> > > Confidentiality - This email is confidential.
> > > Not meant for you? - If you don't think this email is meant for you,
> > > please let us know. Do not copy or forward the information it contains, and
> > > delete this email from your system.
> > > Views expressed - Any personal views or opinions expressed in this email
> > > are the sender's, and do not necessarily reflect the views of the Phoenix
> > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > Monitoring - We filter and monitor emails to protect our systems and to
> > > keep them running smoothly.
> > > Emailing us - Email isn't a secure form of communication. If you want to
> > > send us confidential information please send it by post. However, if you do
> > > communicate with us by email on any subject, you are giving us permission
> > > to email you back.
> > > Phoning us - Calls may be monitored and/or recorded to protect both you
> > > and us and help with our training. Call charges will vary.
> > > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> > > the Prudential Regulation Authority and regulated by the Financial Conduct
> > > Authority and the Prudential Regulation Authority. www.standardlife.co.uk<http://www.standardlife.co.uk><
> > > http://www.standardlife.co.uk<http://www.standardlife.co.uk>>
> > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > Services Limited are companies in the Phoenix Group (Phoenix Group Holdings
> > > plc and its subsidiaries). www.thephoenixgroup.com<
> > > http://www.thephoenixgroup.com>
> > >
>
> Confidentiality - This email is confidential.
> Not meant for you? - If you don't think this email is meant for you, please let us know. Do not copy or forward the information it contains, and delete this email from your system.
> Views expressed - Any personal views or opinions expressed in this email are the sender's, and do not necessarily reflect the views of the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries).
> Monitoring - We filter and monitor emails to protect our systems and to keep them running smoothly.
> Emailing us - Email isn't a secure form of communication. If you want to send us confidential information please send it by post. However, if you do communicate with us by email on any subject, you are giving us permission to email you back.
> Phoning us - Calls may be monitored and/or recorded to protect both you and us and help with our training. Call charges will vary.
> Standard Life Assurance Limited is registered in Scotland (SC286833) at Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. www.standardlife.co.uk<http://www.standardlife.co.uk>
> Standard Life Assurance Limited and Standard Life Assets and Employee Services Limited are companies in the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries). www.thephoenixgroup.com<http://www.thephoenixgroup.com>

RE: [External] Re: Search not working on unmodified files

Posted by Andrew Ducker <an...@thephoenixgroup.com>.

Hi,

I created the ehcache file, dropped it into the same folder as jspwiki-custom.properties, stopped the service, cleared the WorkingDir, restarted the service.

The good news is that it now logs all 13,000 entries!

I then had to assign more memory to Tomcat, but that’s understandable.

Thank you for this!

Andy

From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
Sent: 21 April 2022 22:21
To: user@jspwiki.apache.org
Subject: Re: [External] Re: Search not working on unmodified files

Alert! This is an external email sent from juanpablo.santos@gmail.com.

Hi Andrew, Jim,

hmm upon closer looking, seems LuceneSearchProvider requests all pages
to the page manager, which in turn delegates
to the caching provider, which, by default, holds up to 1.000
elements, and that's why no more pages appear indexed on a
full reindex. I'll push a fix for that tomorrow / this weekend.

In the meantime, you can use [#1] as an example of an ehcache file.
Please see [#2] to see the appropiate parameters
needed to use a custom ehcache file. Would you mind testing if the
error persist while using a custom ehcache file with
more elements than pages available?


best regards,
juan pablo


[#1] https://github.com/apache/jspwiki/blob/master/jspwiki-cache/src/main/resources/ehcache-jspwiki.xml<https://github.com/apache/jspwiki/blob/master/jspwiki-cache/src/main/resources/ehcache-jspwiki.xml>
[#2] https://jspwiki-wiki.apache.org/Wiki.jsp?page=NewIn2.11#section-NewIn2.11-NewInJSPWiki2.11.1ReleasedOn18122021<https://jspwiki-wiki.apache.org/Wiki.jsp?page=NewIn2.11#section-NewIn2.11-NewInJSPWiki2.11.1ReleasedOn18122021>

On Thu, Apr 21, 2022 at 2:38 PM Jim Willeke <ji...@willeke.com> wrote:
>
> I found this one:
> https://jar-download.com/artifacts/org.apache.jspwiki/jspwiki-war/2.10.1/source-code/ehcache.xml<https://jar-download.com/artifacts/org.apache.jspwiki/jspwiki-war/2.10.1/source-code/ehcache.xml>
>
> --
> -jim
> Jim Willeke
>
>
> On Thu, Apr 21, 2022 at 6:41 AM Andrew Ducker <
> andrew_ducker@thephoenixgroup.com> wrote:
>
> > Is there an example ehcache file somewhere? It doesn’t get created
> > anywhere automatically that I can see, and it’s not in the
> > https://jspwiki-wiki.apache.org/Wiki.jsp?page=Configuration<https://jspwiki-wiki.apache.org/Wiki.jsp?page=Configuration> documentation.
> >
> > Cheers,
> >
> > Andy
> >
> > From: Jim Willeke [mailto:jim@willeke.com]
> > Sent: 20 April 2022 21:35
> > To: user@jspwiki.apache.org
> > Subject: Re: [External] Re: Search not working on unmodified files
> >
> > Alert! This is an external email sent from jim@willeke.com.
> >
> > I have observed something similar.
> >
> > Active Sessions 51
> > Uptime 0d, 18h 30m 39s
> > Number of pages 16390FYI: I too am seeing similar issues on 2.11.0-M8.
> >
> > I have NOT observed the issues on 2.10.1.
> >
> > The instance on version 2.10.1 (https://ldapwiki.com/wiki/SystemInfo<https://ldapwiki.com/wiki/SystemInfo><
> > https://ldapwiki.com/wiki/SystemInfo<https://ldapwiki.com/wiki/SystemInfo>>) is
> > busy
> > and the 2.11.0-M8 instance on has very low traffic.
> >
> > On the version 2.11.0-M8, Cleared the Lucene and work directory and
> > restarted:
> >
> > Active Sessions 1
> > Uptime 0d, 9h 11m 7s
> > Number of pages 16499
> >
> > And one day later is shows:
> > Active Sessions 1
> > Uptime 1d, 10h 15m 43s
> > Number of pages 46
> >
> > Then after using for a few minutes it now says:
> > Active Sessions 1
> > Uptime 1d, 10h 35m 52s
> > Number of pages 16500
> > (And yes one page was really added over the day)
> >
> > Where Number of pages is the [{$totalpages}]
> >
> > On the version 2.10.1 below is typical:
> >
> > Active Sessions 51
> > Uptime 0d, 18h 30m 39s
> > Number of pages 16390
> >
> > The ability to perform a search appears to be related to the
> > [{$totalpages}] variable. That is when it shows 46 almost nothing can be
> > found.
> >
> > I do see in the log:
> > 2022-04-20 10:12:22,050 [main] WARN
> > org.apache.wiki.providers.CachingProvider - seems
> > Ldapwiki.jspwiki.pageCache can't hold all pages from your page repository,
> > so we're delegating on the underlying provider instead. Please consider
> > increasing your cache sizes on ehcache.xml to avoid this behaviour
> >
> > Which led to the finding the entry in
> >
> > https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties<https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties>
> > <
> > https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties<https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties>
> > >
> >
> > "
> > *# By default, JSPWiki caches will hold up to 1.000 elements, except the
> > RSS cache, which will hold up to 250
> > elementsjspwiki.cache.custom-config-file = jspwiki-ehcache.xml*"
> >
> > Both are behind nginx.
> >
> > --
> > -jim
> > Jim Willeke
> >
> >
> > On Mon, Apr 18, 2022 at 3:40 PM Juan Pablo Santos Rodríguez <
> > juanpablo.santos@gmail.com> wrote:
> >
> > > Hi!
> > >
> > > Looking at the source code, at jspwiki init, a full reindex is triggered.
> > > The full reindex is only performed if the Lucene dir is empty. As this is
> > > not the case, the initial full reindex isn't executed, and that's why you
> > > are not seeing anything on searches. Pages saves do trigger a partial
> > > reindex, though, so that explains the other side of the behaviour you're
> > > seeing.
> > >
> > > What files are inside the Lucene dir? If there are Lucene index files,
> > can
> > > you open then with Luke to see where they came from, or the Lucene index
> > > version, or any other information? Is there any chance other jspwiki
> > > instance is using the same workDir?
> > >
> > > Last, emptying the lucene dir will enforce a full reindex next time the
> > > background thread responsible of performing Lucene reindexes runs, but it
> > > would be better to know why some files are getting created there outside
> > > your jspwiki instance.
> > >
> > >
> > > HTH,
> > > juan pablo
> > >
> > > El lun., 18 abr. 2022 12:32, Andrew Ducker <
> > > andrew_ducker@thephoenixgroup.com> escribió:
> > >
> > > > Hi,
> > > >
> > > > Thanks for the reply.
> > > >
> > > > We completely cleared the WorkDir, to be on the safe side, and that
> > > didn’t
> > > > help.
> > > >
> > > > It has access to the Work and wikipages folders, as it can create them
> > > > from scratch, and you can edit the Wiki files.
> > > >
> > > > No mentions of “unable to index” in the logs.
> > > >
> > > > In the jspwiki-custom.properties file there’s the following:
> > > > jspwiki.applicationName =IS Wiki
> > > > jspwiki.baseURL = <removed>
> > > > jspwiki.frontPage = Main
> > > > jspwiki.pageProvider =VersioningFileProvider
> > > > jspwiki.fileSystemProvider.pageDir =d:/JSPWiki/JSPWikiPages/Pages/wiki/
> > > > jspwiki.basicAttachmentProvider.storageDir
> > > > =d:/JSPWiki/JSPWikiPages/Attachments/wiki/
> > > > jspwiki.interWikiRef.Notes = Notes:%s
> > > > jspwiki.translatorReader.inlinePattern.1 = *.jpg
> > > > jspwiki.translatorReader.inlinePattern.2 = *.png
> > > > jspwiki.translatorReader.inlinePattern.3 = http://images.com/*<http://images.com/*><
> > http://images.com/*<http://images.com/*>>
> > > > jspwiki.rss.generate =true
> > > > jspwiki.rss.channelDescription = IS Wiki
> > > > mail.from =@mail.from@
> > > > mail.smtp.host<http://mail.smtp.host><http://mail.smtp.host<http://mail.smtp.host>> =@mail.smtp.host@
> > > > jspwiki.workDir =d:/JSPWiki/WorkingDirectory/iswiki/
> > > > jspwiki.security = jaas
> > > >
> > > > Lucene-specific things in the logs:
> > > > [INFO ] 2022-04-13 13:44:48.094 [main] o.a.w.s.LuceneSearchProvider -
> > > > Lucene enabled, cache will be in:
> > > d:\JSPWiki\WorkingDirectory\iswiki\lucene
> > > > [WARN ] 2022-04-13 13:44:48.725 [JSPWiki Lucene Indexer]
> > > > o.a.w.WikiBackgroundThread - Starting up background thread: JSPWiki
> > > Lucene
> > > > Indexer.
> > > > [INFO ] 2022-04-13 13:45:48.732 [JSPWiki Lucene Indexer]
> > > > o.a.w.s.LuceneSearchProvider - Files found in Lucene directory, not
> > > > reindexing.
> > > >
> > > > So could it be starting to index, thinking it’s already indexed
> > (despite
> > > > the whole WorkingDir being cleared out) and then stopping
> > near-instantly?
> > > >
> > > > Thanks again,
> > > >
> > > > Andy
> > > >
> > > >
> > > > From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
> > > > Sent: 14 April 2022 16:25
> > > > To: user@jspwiki.apache.org
> > > > Subject: [External] Re: Search not working on unmodified files
> > > >
> > > > Alert! This is an external email sent from juanpablo.santos@gmail.com.
> > > >
> > > > Hi Andrew,
> > > >
> > > > haven't had the time to look in depth at JSPWIKI-1171, so just some
> > > > questions / random thoughts:
> > > >
> > > > JSPWiki has changed a lot since 2.8, but the wiki pages itself
> > shouldn't
> > > > require any change, and should be readable by any version of JSPWiki.
> > The
> > > > Lucene version has been upgraded several times, so it was definitely ok
> > > to
> > > > clear your Lucene folder. Also there is the refmgr.ser file and
> > > refmgr-attr
> > > > dir inside your workDir, which contain serialized information about
> > page
> > > > references. I'm not 100% sure, but most probably the internal format
> > has
> > > > changed since 2.8 (there was a global package rename on 2.9, when
> > > entering
> > > > the Apache incubator, and somewhat later a public API was pushed which
> > > may
> > > > have affected this format too) so you should delete those too and try
> > to
> > > > restart your JSPWiki instance again and see if the situation persists;
> > > > perhaps deleting the entire workDir would be the safest approach.
> > That's
> > > > what seems to be happening from the stacktrace attached at JSPWIKI-1171
> > > > (this list strips attachments, IIRC).
> > > >
> > > > Also, regarding the searches, would you mind confirming that the new
> > > > JSPWiki instance has permissions to read/write either the work and
> > > > wikipages directories/files? Also are there any errors logged when
> > > starting
> > > > JSPWiki? Specifically, the Lucene indexing is done on startup (if
> > needed)
> > > > on the background, so eventually you should get all your pages indexed
> > or
> > > > see log errors stating something like "Unable to index page $pageName,
> > > > continuing to next".
> > > >
> > > > Are you using a vanilla JSPWiki installation or do you have some
> > > > customizations? What parameters do you have on your
> > > > jspwiki-custom.properties file? I'm assuming you're deloying the war
> > > file?
> > > >
> > > >
> > > > best regards,
> > > > juan pablo
> > > >
> > > > On Thu, Apr 14, 2022 at 4:51 PM Andrew Ducker <
> > > > andrew_ducker@thephoenixgroup.com> wrote:
> > > >
> > > > > Hi!
> > > > >
> > > > >
> > > > >
> > > > > I am having a problem with an upgraded JSPWiki not indexing pages.
> > > > >
> > > > >
> > > > >
> > > > > I wasn’t sure where to report it, so added an issue on Jira:
> > > > >
> > > > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171><
> > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>><
> > > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171><
> > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>>>
> > > > >
> > > > >
> > > > >
> > > > > I’m recreating it here, in case this is the right place to report
> > > things:
> > > > >
> > > > >
> > > > >
> > > > > Lots of pages aren't turning up in search.
> > > > >
> > > > >
> > > > >
> > > > > We cleared the lucene folder from the working directory, and it
> > rebuilt
> > > > it
> > > > > - but quite quickly for 13,000 pages. And the total size of the
> > Lucene
> > > > > files was around 2.5MB, which seemed far too small.
> > > > >
> > > > >
> > > > >
> > > > > Doing a search failed to bring back pages that had the given word in
> > > > their
> > > > > name and content.
> > > > >
> > > > >
> > > > >
> > > > > Editing one of those pages (adding a space) caused it to then appear
> > in
> > > > > the searches.
> > > > >
> > > > >
> > > > >
> > > > > Checking the logs I found a lot of errors in the form:
> > > > >
> > > > >
> > > > >
> > > > > [ERROR] 2022-04-12 15:50:17.274 [main]
> > o.a.w.r.DefaultReferenceManager
> > > -
> > > > > Error while trying to fetch a page name; trying to cope with the
> > > > situation.
> > > > >
> > > > > org.apache.wiki.api.exceptions.ProviderException: Illegal page name
> > > > >
> > > > >
> > > > >
> > > > > (See attachment for whole exception trace)
> > > > >
> > > > >
> > > > >
> > > > > Sadly, it doesn't tell me what the illegal page names are.
> > > > >
> > > > >
> > > > >
> > > > > This wiki has been upgraded from version 2.8 - are there
> > compatibility
> > > > > issues with page names there?
> > > > >
> > > > >
> > > > >
> > > > > Any help you can give me gratefully received!
> > > > >
> > > > >
> > > > >
> > > > > Andy
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > <https://www.thephoenixgroup.com/>
> > > > > *Andrew Ducker* | Senior Systems Developer
> > > > > office: +441312458823 | mobile: 07980 86 86 39 | email:
> > > > > andrew_ducker@thephoenixgroup.com
> > > > > P Please help save the environment, print only if necessary.
> > > > > [image: logobadgefinal]
> > > > >
> > > > >
> > > > >
> > > > > Confidentiality - This email is confidential.
> > > > > Not meant for you? - If you don't think this email is meant for you,
> > > > > please let us know. Do not copy or forward the information it
> > contains,
> > > > and
> > > > > delete this email from your system.
> > > > > Views expressed - Any personal views or opinions expressed in this
> > > email
> > > > > are the sender's, and do not necessarily reflect the views of the
> > > Phoenix
> > > > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > > > Monitoring - We filter and monitor emails to protect our systems and
> > to
> > > > > keep them running smoothly.
> > > > > Emailing us - Email isn't a secure form of communication. If you want
> > > to
> > > > > send us confidential information please send it by post. However, if
> > > you
> > > > do
> > > > > communicate with us by email on any subject, you are giving us
> > > permission
> > > > > to email you back.
> > > > > Phoning us - Calls may be monitored and/or recorded to protect both
> > you
> > > > > and us and help with our training. Call charges will vary.
> > > > > Standard Life Assurance Limited is registered in Scotland (SC286833)
> > at
> > > > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and
> > authorised
> > > by
> > > > > the Prudential Regulation Authority and regulated by the Financial
> > > > Conduct
> > > > > Authority and the Prudential Regulation Authority.
> > > > www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>><
> > http://www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>>>
> > > > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > > > Services Limited are companies in the Phoenix Group (Phoenix Group
> > > > Holdings
> > > > > plc and its subsidiaries). www.thephoenixgroup.com
> > > > >
> > > >
> > > > Confidentiality - This email is confidential.
> > > > Not meant for you? - If you don't think this email is meant for you,
> > > > please let us know. Do not copy or forward the information it contains,
> > > and
> > > > delete this email from your system.
> > > > Views expressed - Any personal views or opinions expressed in this
> > email
> > > > are the sender's, and do not necessarily reflect the views of the
> > Phoenix
> > > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > > Monitoring - We filter and monitor emails to protect our systems and to
> > > > keep them running smoothly.
> > > > Emailing us - Email isn't a secure form of communication. If you want
> > to
> > > > send us confidential information please send it by post. However, if
> > you
> > > do
> > > > communicate with us by email on any subject, you are giving us
> > permission
> > > > to email you back.
> > > > Phoning us - Calls may be monitored and/or recorded to protect both you
> > > > and us and help with our training. Call charges will vary.
> > > > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised
> > by
> > > > the Prudential Regulation Authority and regulated by the Financial
> > > Conduct
> > > > Authority and the Prudential Regulation Authority.
> > > www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>><
> > > > http://www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>>>
> > > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > > Services Limited are companies in the Phoenix Group (Phoenix Group
> > > Holdings
> > > > plc and its subsidiaries). www.thephoenixgroup.com<
> > > > http://www.thephoenixgroup.com>
> > > >
> > >
> >
> > Confidentiality - This email is confidential.
> > Not meant for you? - If you don't think this email is meant for you,
> > please let us know. Do not copy or forward the information it contains, and
> > delete this email from your system.
> > Views expressed - Any personal views or opinions expressed in this email
> > are the sender's, and do not necessarily reflect the views of the Phoenix
> > Group (Phoenix Group Holdings plc and its subsidiaries).
> > Monitoring - We filter and monitor emails to protect our systems and to
> > keep them running smoothly.
> > Emailing us - Email isn't a secure form of communication. If you want to
> > send us confidential information please send it by post. However, if you do
> > communicate with us by email on any subject, you are giving us permission
> > to email you back.
> > Phoning us - Calls may be monitored and/or recorded to protect both you
> > and us and help with our training. Call charges will vary.
> > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> > the Prudential Regulation Authority and regulated by the Financial Conduct
> > Authority and the Prudential Regulation Authority. www.standardlife.co.uk<http://www.standardlife.co.uk><
> > http://www.standardlife.co.uk<http://www.standardlife.co.uk>>
> > Standard Life Assurance Limited and Standard Life Assets and Employee
> > Services Limited are companies in the Phoenix Group (Phoenix Group Holdings
> > plc and its subsidiaries). www.thephoenixgroup.com<
> > http://www.thephoenixgroup.com>
> >

Confidentiality - This email is confidential.
Not meant for you? - If you don't think this email is meant for you, please let us know. Do not copy or forward the information it contains, and delete this email from your system.
Views expressed - Any personal views or opinions expressed in this email are the sender's, and do not necessarily reflect the views of the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries).
Monitoring - We filter and monitor emails to protect our systems and to keep them running smoothly.
Emailing us - Email isn't a secure form of communication. If you want to send us confidential information please send it by post. However, if you do communicate with us by email on any subject, you are giving us permission to email you back.
Phoning us - Calls may be monitored and/or recorded to protect both you and us and help with our training. Call charges will vary.
Standard Life Assurance Limited is registered in Scotland (SC286833) at Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. www.standardlife.co.uk<http://www.standardlife.co.uk>
Standard Life Assurance Limited and Standard Life Assets and Employee Services Limited are companies in the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries). www.thephoenixgroup.com<http://www.thephoenixgroup.com>

Re: [External] Re: Search not working on unmodified files

Posted by Juan Pablo Santos Rodríguez <ju...@gmail.com>.

Hi Andrew, Jim,

hmm upon closer looking, seems LuceneSearchProvider requests all pages
to the page manager, which in turn delegates
to the caching provider, which, by default, holds up to 1.000
elements, and that's why no more pages appear indexed on a
full reindex. I'll push a fix for that tomorrow / this weekend.

In the meantime, you can use [#1] as an example of an ehcache file.
Please see [#2] to see the appropiate parameters
needed to use a custom ehcache file. Would you mind testing if the
error persist while using a custom ehcache file with
more elements than pages available?


best regards,
juan pablo


[#1] https://github.com/apache/jspwiki/blob/master/jspwiki-cache/src/main/resources/ehcache-jspwiki.xml
[#2] https://jspwiki-wiki.apache.org/Wiki.jsp?page=NewIn2.11#section-NewIn2.11-NewInJSPWiki2.11.1ReleasedOn18122021

On Thu, Apr 21, 2022 at 2:38 PM Jim Willeke <ji...@willeke.com> wrote:
>
> I found this one:
> https://jar-download.com/artifacts/org.apache.jspwiki/jspwiki-war/2.10.1/source-code/ehcache.xml
>
> --
> -jim
> Jim Willeke
>
>
> On Thu, Apr 21, 2022 at 6:41 AM Andrew Ducker <
> andrew_ducker@thephoenixgroup.com> wrote:
>
> > Is there an example ehcache file somewhere?  It doesn’t get created
> > anywhere automatically that I can see, and it’s not in the
> > https://jspwiki-wiki.apache.org/Wiki.jsp?page=Configuration documentation.
> >
> > Cheers,
> >
> > Andy
> >
> > From: Jim Willeke [mailto:jim@willeke.com]
> > Sent: 20 April 2022 21:35
> > To: user@jspwiki.apache.org
> > Subject: Re: [External] Re: Search not working on unmodified files
> >
> > Alert! This is an external email sent from jim@willeke.com.
> >
> > I have observed something similar.
> >
> > Active Sessions 51
> > Uptime 0d, 18h 30m 39s
> > Number of pages 16390FYI: I too am seeing similar issues on 2.11.0-M8.
> >
> > I have NOT observed the issues on 2.10.1.
> >
> > The instance on version 2.10.1 (https://ldapwiki.com/wiki/SystemInfo<
> > https://ldapwiki.com/wiki/SystemInfo>) is
> > busy
> > and the 2.11.0-M8 instance on has very low traffic.
> >
> > On the version 2.11.0-M8, Cleared the Lucene and work directory and
> > restarted:
> >
> > Active Sessions 1
> > Uptime 0d, 9h 11m 7s
> > Number of pages 16499
> >
> > And one day later is shows:
> > Active Sessions 1
> > Uptime 1d, 10h 15m 43s
> > Number of pages 46
> >
> > Then after using for a few minutes it now says:
> > Active Sessions 1
> > Uptime 1d, 10h 35m 52s
> > Number of pages 16500
> > (And yes one page was really added over the day)
> >
> > Where Number of pages is the [{$totalpages}]
> >
> > On the version 2.10.1 below is typical:
> >
> > Active Sessions 51
> > Uptime 0d, 18h 30m 39s
> > Number of pages 16390
> >
> > The ability to perform a search appears to be related to the
> > [{$totalpages}] variable. That is when it shows 46 almost nothing can be
> > found.
> >
> > I do see in the log:
> > 2022-04-20 10:12:22,050 [main] WARN
> > org.apache.wiki.providers.CachingProvider - seems
> > Ldapwiki.jspwiki.pageCache can't hold all pages from your page repository,
> > so we're delegating on the underlying provider instead. Please consider
> > increasing your cache sizes on ehcache.xml to avoid this behaviour
> >
> > Which led to the finding the entry in
> >
> > https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties
> > <
> > https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties
> > >
> >
> > "
> > *# By default, JSPWiki caches will hold up to 1.000 elements, except the
> > RSS cache, which will hold up to 250
> > elementsjspwiki.cache.custom-config-file = jspwiki-ehcache.xml*"
> >
> > Both are behind nginx.
> >
> > --
> > -jim
> > Jim Willeke
> >
> >
> > On Mon, Apr 18, 2022 at 3:40 PM Juan Pablo Santos Rodríguez <
> > juanpablo.santos@gmail.com> wrote:
> >
> > > Hi!
> > >
> > > Looking at the source code, at jspwiki init, a full reindex is triggered.
> > > The full reindex is only performed if the Lucene dir is empty. As this is
> > > not the case, the initial full reindex isn't executed, and that's why you
> > > are not seeing anything on searches. Pages saves do trigger a partial
> > > reindex, though, so that explains the other side of the behaviour you're
> > > seeing.
> > >
> > > What files are inside the Lucene dir? If there are Lucene index files,
> > can
> > > you open then with Luke to see where they came from, or the Lucene index
> > > version, or any other information? Is there any chance other jspwiki
> > > instance is using the same workDir?
> > >
> > > Last, emptying the lucene dir will enforce a full reindex next time the
> > > background thread responsible of performing Lucene reindexes runs, but it
> > > would be better to know why some files are getting created there outside
> > > your jspwiki instance.
> > >
> > >
> > > HTH,
> > > juan pablo
> > >
> > > El lun., 18 abr. 2022 12:32, Andrew Ducker <
> > > andrew_ducker@thephoenixgroup.com> escribió:
> > >
> > > > Hi,
> > > >
> > > > Thanks for the reply.
> > > >
> > > > We completely cleared the WorkDir, to be on the safe side, and that
> > > didn’t
> > > > help.
> > > >
> > > > It has access to the Work and wikipages folders, as it can create them
> > > > from scratch, and you can edit the Wiki files.
> > > >
> > > > No mentions of “unable to index” in the logs.
> > > >
> > > > In the jspwiki-custom.properties file there’s the following:
> > > > jspwiki.applicationName =IS Wiki
> > > > jspwiki.baseURL = <removed>
> > > > jspwiki.frontPage = Main
> > > > jspwiki.pageProvider =VersioningFileProvider
> > > > jspwiki.fileSystemProvider.pageDir =d:/JSPWiki/JSPWikiPages/Pages/wiki/
> > > > jspwiki.basicAttachmentProvider.storageDir
> > > > =d:/JSPWiki/JSPWikiPages/Attachments/wiki/
> > > > jspwiki.interWikiRef.Notes = Notes:%s
> > > > jspwiki.translatorReader.inlinePattern.1 = *.jpg
> > > > jspwiki.translatorReader.inlinePattern.2 = *.png
> > > > jspwiki.translatorReader.inlinePattern.3 = http://images.com/*<
> > http://images.com/*>
> > > > jspwiki.rss.generate =true
> > > > jspwiki.rss.channelDescription = IS Wiki
> > > > mail.from =@mail.from@
> > > > mail.smtp.host<http://mail.smtp.host> =@mail.smtp.host@
> > > > jspwiki.workDir =d:/JSPWiki/WorkingDirectory/iswiki/
> > > > jspwiki.security = jaas
> > > >
> > > > Lucene-specific things in the logs:
> > > > [INFO ] 2022-04-13 13:44:48.094 [main] o.a.w.s.LuceneSearchProvider -
> > > > Lucene enabled, cache will be in:
> > > d:\JSPWiki\WorkingDirectory\iswiki\lucene
> > > > [WARN ] 2022-04-13 13:44:48.725 [JSPWiki Lucene Indexer]
> > > > o.a.w.WikiBackgroundThread - Starting up background thread: JSPWiki
> > > Lucene
> > > > Indexer.
> > > > [INFO ] 2022-04-13 13:45:48.732 [JSPWiki Lucene Indexer]
> > > > o.a.w.s.LuceneSearchProvider - Files found in Lucene directory, not
> > > > reindexing.
> > > >
> > > > So could it be starting to index, thinking it’s already indexed
> > (despite
> > > > the whole WorkingDir being cleared out) and then stopping
> > near-instantly?
> > > >
> > > > Thanks again,
> > > >
> > > > Andy
> > > >
> > > >
> > > > From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
> > > > Sent: 14 April 2022 16:25
> > > > To: user@jspwiki.apache.org
> > > > Subject: [External] Re: Search not working on unmodified files
> > > >
> > > > Alert! This is an external email sent from juanpablo.santos@gmail.com.
> > > >
> > > > Hi Andrew,
> > > >
> > > > haven't had the time to look in depth at JSPWIKI-1171, so just some
> > > > questions / random thoughts:
> > > >
> > > > JSPWiki has changed a lot since 2.8, but the wiki pages itself
> > shouldn't
> > > > require any change, and should be readable by any version of JSPWiki.
> > The
> > > > Lucene version has been upgraded several times, so it was definitely ok
> > > to
> > > > clear your Lucene folder. Also there is the refmgr.ser file and
> > > refmgr-attr
> > > > dir inside your workDir, which contain serialized information about
> > page
> > > > references. I'm not 100% sure, but most probably the internal format
> > has
> > > > changed since 2.8 (there was a global package rename on 2.9, when
> > > entering
> > > > the Apache incubator, and somewhat later a public API was pushed which
> > > may
> > > > have affected this format too) so you should delete those too and try
> > to
> > > > restart your JSPWiki instance again and see if the situation persists;
> > > > perhaps deleting the entire workDir would be the safest approach.
> > That's
> > > > what seems to be happening from the stacktrace attached at JSPWIKI-1171
> > > > (this list strips attachments, IIRC).
> > > >
> > > > Also, regarding the searches, would you mind confirming that the new
> > > > JSPWiki instance has permissions to read/write either the work and
> > > > wikipages directories/files? Also are there any errors logged when
> > > starting
> > > > JSPWiki? Specifically, the Lucene indexing is done on startup (if
> > needed)
> > > > on the background, so eventually you should get all your pages indexed
> > or
> > > > see log errors stating something like "Unable to index page $pageName,
> > > > continuing to next".
> > > >
> > > > Are you using a vanilla JSPWiki installation or do you have some
> > > > customizations? What parameters do you have on your
> > > > jspwiki-custom.properties file? I'm assuming you're deloying the war
> > > file?
> > > >
> > > >
> > > > best regards,
> > > > juan pablo
> > > >
> > > > On Thu, Apr 14, 2022 at 4:51 PM Andrew Ducker <
> > > > andrew_ducker@thephoenixgroup.com> wrote:
> > > >
> > > > > Hi!
> > > > >
> > > > >
> > > > >
> > > > > I am having a problem with an upgraded JSPWiki not indexing pages.
> > > > >
> > > > >
> > > > >
> > > > > I wasn’t sure where to report it, so added an issue on Jira:
> > > > >
> > > > > https://issues.apache.org/jira/browse/JSPWIKI-1171<
> > https://issues.apache.org/jira/browse/JSPWIKI-1171><
> > > > https://issues.apache.org/jira/browse/JSPWIKI-1171<
> > https://issues.apache.org/jira/browse/JSPWIKI-1171>>
> > > > >
> > > > >
> > > > >
> > > > > I’m recreating it here, in case this is the right place to report
> > > things:
> > > > >
> > > > >
> > > > >
> > > > > Lots of pages aren't turning up in search.
> > > > >
> > > > >
> > > > >
> > > > > We cleared the lucene folder from the working directory, and it
> > rebuilt
> > > > it
> > > > > - but quite quickly for 13,000 pages. And the total size of the
> > Lucene
> > > > > files was around 2.5MB, which seemed far too small.
> > > > >
> > > > >
> > > > >
> > > > > Doing a search failed to bring back pages that had the given word in
> > > > their
> > > > > name and content.
> > > > >
> > > > >
> > > > >
> > > > > Editing one of those pages (adding a space) caused it to then appear
> > in
> > > > > the searches.
> > > > >
> > > > >
> > > > >
> > > > > Checking the logs I found a lot of errors in the form:
> > > > >
> > > > >
> > > > >
> > > > > [ERROR] 2022-04-12 15:50:17.274 [main]
> > o.a.w.r.DefaultReferenceManager
> > > -
> > > > > Error while trying to fetch a page name; trying to cope with the
> > > > situation.
> > > > >
> > > > > org.apache.wiki.api.exceptions.ProviderException: Illegal page name
> > > > >
> > > > >
> > > > >
> > > > > (See attachment for whole exception trace)
> > > > >
> > > > >
> > > > >
> > > > > Sadly, it doesn't tell me what the illegal page names are.
> > > > >
> > > > >
> > > > >
> > > > > This wiki has been upgraded from version 2.8 - are there
> > compatibility
> > > > > issues with page names there?
> > > > >
> > > > >
> > > > >
> > > > > Any help you can give me gratefully received!
> > > > >
> > > > >
> > > > >
> > > > > Andy
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > <https://www.thephoenixgroup.com/>
> > > > > *Andrew Ducker* | Senior Systems Developer
> > > > > office: +441312458823 | mobile: 07980 86 86 39 | email:
> > > > > andrew_ducker@thephoenixgroup.com
> > > > > P Please help save the environment, print only if necessary.
> > > > > [image: logobadgefinal]
> > > > >
> > > > >
> > > > >
> > > > > Confidentiality - This email is confidential.
> > > > > Not meant for you? - If you don't think this email is meant for you,
> > > > > please let us know. Do not copy or forward the information it
> > contains,
> > > > and
> > > > > delete this email from your system.
> > > > > Views expressed - Any personal views or opinions expressed in this
> > > email
> > > > > are the sender's, and do not necessarily reflect the views of the
> > > Phoenix
> > > > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > > > Monitoring - We filter and monitor emails to protect our systems and
> > to
> > > > > keep them running smoothly.
> > > > > Emailing us - Email isn't a secure form of communication. If you want
> > > to
> > > > > send us confidential information please send it by post. However, if
> > > you
> > > > do
> > > > > communicate with us by email on any subject, you are giving us
> > > permission
> > > > > to email you back.
> > > > > Phoning us - Calls may be monitored and/or recorded to protect both
> > you
> > > > > and us and help with our training. Call charges will vary.
> > > > > Standard Life Assurance Limited is registered in Scotland (SC286833)
> > at
> > > > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and
> > authorised
> > > by
> > > > > the Prudential Regulation Authority and regulated by the Financial
> > > > Conduct
> > > > > Authority and the Prudential Regulation Authority.
> > > > www.standardlife.co.uk<http://www.standardlife.co.uk><
> > http://www.standardlife.co.uk<http://www.standardlife.co.uk>>
> > > > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > > > Services Limited are companies in the Phoenix Group (Phoenix Group
> > > > Holdings
> > > > > plc and its subsidiaries). www.thephoenixgroup.com
> > > > >
> > > >
> > > > Confidentiality - This email is confidential.
> > > > Not meant for you? - If you don't think this email is meant for you,
> > > > please let us know. Do not copy or forward the information it contains,
> > > and
> > > > delete this email from your system.
> > > > Views expressed - Any personal views or opinions expressed in this
> > email
> > > > are the sender's, and do not necessarily reflect the views of the
> > Phoenix
> > > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > > Monitoring - We filter and monitor emails to protect our systems and to
> > > > keep them running smoothly.
> > > > Emailing us - Email isn't a secure form of communication. If you want
> > to
> > > > send us confidential information please send it by post. However, if
> > you
> > > do
> > > > communicate with us by email on any subject, you are giving us
> > permission
> > > > to email you back.
> > > > Phoning us - Calls may be monitored and/or recorded to protect both you
> > > > and us and help with our training. Call charges will vary.
> > > > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised
> > by
> > > > the Prudential Regulation Authority and regulated by the Financial
> > > Conduct
> > > > Authority and the Prudential Regulation Authority.
> > > www.standardlife.co.uk<http://www.standardlife.co.uk><
> > > > http://www.standardlife.co.uk<http://www.standardlife.co.uk>>
> > > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > > Services Limited are companies in the Phoenix Group (Phoenix Group
> > > Holdings
> > > > plc and its subsidiaries). www.thephoenixgroup.com<
> > > > http://www.thephoenixgroup.com>
> > > >
> > >
> >
> > Confidentiality - This email is confidential.
> > Not meant for you? - If you don't think this email is meant for you,
> > please let us know. Do not copy or forward the information it contains, and
> > delete this email from your system.
> > Views expressed - Any personal views or opinions expressed in this email
> > are the sender's, and do not necessarily reflect the views of the Phoenix
> > Group (Phoenix Group Holdings plc and its subsidiaries).
> > Monitoring - We filter and monitor emails to protect our systems and to
> > keep them running smoothly.
> > Emailing us - Email isn't a secure form of communication. If you want to
> > send us confidential information please send it by post. However, if you do
> > communicate with us by email on any subject, you are giving us permission
> > to email you back.
> > Phoning us - Calls may be monitored and/or recorded to protect both you
> > and us and help with our training. Call charges will vary.
> > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> > the Prudential Regulation Authority and regulated by the Financial Conduct
> > Authority and the Prudential Regulation Authority. www.standardlife.co.uk<
> > http://www.standardlife.co.uk>
> > Standard Life Assurance Limited and Standard Life Assets and Employee
> > Services Limited are companies in the Phoenix Group (Phoenix Group Holdings
> > plc and its subsidiaries). www.thephoenixgroup.com<
> > http://www.thephoenixgroup.com>
> >

Re: [External] Re: Search not working on unmodified files

Posted by Jim Willeke <ji...@willeke.com>.

I found this one:
https://jar-download.com/artifacts/org.apache.jspwiki/jspwiki-war/2.10.1/source-code/ehcache.xml

--
-jim
Jim Willeke


On Thu, Apr 21, 2022 at 6:41 AM Andrew Ducker <
andrew_ducker@thephoenixgroup.com> wrote:

> Is there an example ehcache file somewhere?  It doesn’t get created
> anywhere automatically that I can see, and it’s not in the
> https://jspwiki-wiki.apache.org/Wiki.jsp?page=Configuration documentation.
>
> Cheers,
>
> Andy
>
> From: Jim Willeke [mailto:jim@willeke.com]
> Sent: 20 April 2022 21:35
> To: user@jspwiki.apache.org
> Subject: Re: [External] Re: Search not working on unmodified files
>
> Alert! This is an external email sent from jim@willeke.com.
>
> I have observed something similar.
>
> Active Sessions 51
> Uptime 0d, 18h 30m 39s
> Number of pages 16390FYI: I too am seeing similar issues on 2.11.0-M8.
>
> I have NOT observed the issues on 2.10.1.
>
> The instance on version 2.10.1 (https://ldapwiki.com/wiki/SystemInfo<
> https://ldapwiki.com/wiki/SystemInfo>) is
> busy
> and the 2.11.0-M8 instance on has very low traffic.
>
> On the version 2.11.0-M8, Cleared the Lucene and work directory and
> restarted:
>
> Active Sessions 1
> Uptime 0d, 9h 11m 7s
> Number of pages 16499
>
> And one day later is shows:
> Active Sessions 1
> Uptime 1d, 10h 15m 43s
> Number of pages 46
>
> Then after using for a few minutes it now says:
> Active Sessions 1
> Uptime 1d, 10h 35m 52s
> Number of pages 16500
> (And yes one page was really added over the day)
>
> Where Number of pages is the [{$totalpages}]
>
> On the version 2.10.1 below is typical:
>
> Active Sessions 51
> Uptime 0d, 18h 30m 39s
> Number of pages 16390
>
> The ability to perform a search appears to be related to the
> [{$totalpages}] variable. That is when it shows 46 almost nothing can be
> found.
>
> I do see in the log:
> 2022-04-20 10:12:22,050 [main] WARN
> org.apache.wiki.providers.CachingProvider - seems
> Ldapwiki.jspwiki.pageCache can't hold all pages from your page repository,
> so we're delegating on the underlying provider instead. Please consider
> increasing your cache sizes on ehcache.xml to avoid this behaviour
>
> Which led to the finding the entry in
>
> https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties
> <
> https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties
> >
>
> "
> *# By default, JSPWiki caches will hold up to 1.000 elements, except the
> RSS cache, which will hold up to 250
> elementsjspwiki.cache.custom-config-file = jspwiki-ehcache.xml*"
>
> Both are behind nginx.
>
> --
> -jim
> Jim Willeke
>
>
> On Mon, Apr 18, 2022 at 3:40 PM Juan Pablo Santos Rodríguez <
> juanpablo.santos@gmail.com> wrote:
>
> > Hi!
> >
> > Looking at the source code, at jspwiki init, a full reindex is triggered.
> > The full reindex is only performed if the Lucene dir is empty. As this is
> > not the case, the initial full reindex isn't executed, and that's why you
> > are not seeing anything on searches. Pages saves do trigger a partial
> > reindex, though, so that explains the other side of the behaviour you're
> > seeing.
> >
> > What files are inside the Lucene dir? If there are Lucene index files,
> can
> > you open then with Luke to see where they came from, or the Lucene index
> > version, or any other information? Is there any chance other jspwiki
> > instance is using the same workDir?
> >
> > Last, emptying the lucene dir will enforce a full reindex next time the
> > background thread responsible of performing Lucene reindexes runs, but it
> > would be better to know why some files are getting created there outside
> > your jspwiki instance.
> >
> >
> > HTH,
> > juan pablo
> >
> > El lun., 18 abr. 2022 12:32, Andrew Ducker <
> > andrew_ducker@thephoenixgroup.com> escribió:
> >
> > > Hi,
> > >
> > > Thanks for the reply.
> > >
> > > We completely cleared the WorkDir, to be on the safe side, and that
> > didn’t
> > > help.
> > >
> > > It has access to the Work and wikipages folders, as it can create them
> > > from scratch, and you can edit the Wiki files.
> > >
> > > No mentions of “unable to index” in the logs.
> > >
> > > In the jspwiki-custom.properties file there’s the following:
> > > jspwiki.applicationName =IS Wiki
> > > jspwiki.baseURL = <removed>
> > > jspwiki.frontPage = Main
> > > jspwiki.pageProvider =VersioningFileProvider
> > > jspwiki.fileSystemProvider.pageDir =d:/JSPWiki/JSPWikiPages/Pages/wiki/
> > > jspwiki.basicAttachmentProvider.storageDir
> > > =d:/JSPWiki/JSPWikiPages/Attachments/wiki/
> > > jspwiki.interWikiRef.Notes = Notes:%s
> > > jspwiki.translatorReader.inlinePattern.1 = *.jpg
> > > jspwiki.translatorReader.inlinePattern.2 = *.png
> > > jspwiki.translatorReader.inlinePattern.3 = http://images.com/*<
> http://images.com/*>
> > > jspwiki.rss.generate =true
> > > jspwiki.rss.channelDescription = IS Wiki
> > > mail.from =@mail.from@
> > > mail.smtp.host<http://mail.smtp.host> =@mail.smtp.host@
> > > jspwiki.workDir =d:/JSPWiki/WorkingDirectory/iswiki/
> > > jspwiki.security = jaas
> > >
> > > Lucene-specific things in the logs:
> > > [INFO ] 2022-04-13 13:44:48.094 [main] o.a.w.s.LuceneSearchProvider -
> > > Lucene enabled, cache will be in:
> > d:\JSPWiki\WorkingDirectory\iswiki\lucene
> > > [WARN ] 2022-04-13 13:44:48.725 [JSPWiki Lucene Indexer]
> > > o.a.w.WikiBackgroundThread - Starting up background thread: JSPWiki
> > Lucene
> > > Indexer.
> > > [INFO ] 2022-04-13 13:45:48.732 [JSPWiki Lucene Indexer]
> > > o.a.w.s.LuceneSearchProvider - Files found in Lucene directory, not
> > > reindexing.
> > >
> > > So could it be starting to index, thinking it’s already indexed
> (despite
> > > the whole WorkingDir being cleared out) and then stopping
> near-instantly?
> > >
> > > Thanks again,
> > >
> > > Andy
> > >
> > >
> > > From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
> > > Sent: 14 April 2022 16:25
> > > To: user@jspwiki.apache.org
> > > Subject: [External] Re: Search not working on unmodified files
> > >
> > > Alert! This is an external email sent from juanpablo.santos@gmail.com.
> > >
> > > Hi Andrew,
> > >
> > > haven't had the time to look in depth at JSPWIKI-1171, so just some
> > > questions / random thoughts:
> > >
> > > JSPWiki has changed a lot since 2.8, but the wiki pages itself
> shouldn't
> > > require any change, and should be readable by any version of JSPWiki.
> The
> > > Lucene version has been upgraded several times, so it was definitely ok
> > to
> > > clear your Lucene folder. Also there is the refmgr.ser file and
> > refmgr-attr
> > > dir inside your workDir, which contain serialized information about
> page
> > > references. I'm not 100% sure, but most probably the internal format
> has
> > > changed since 2.8 (there was a global package rename on 2.9, when
> > entering
> > > the Apache incubator, and somewhat later a public API was pushed which
> > may
> > > have affected this format too) so you should delete those too and try
> to
> > > restart your JSPWiki instance again and see if the situation persists;
> > > perhaps deleting the entire workDir would be the safest approach.
> That's
> > > what seems to be happening from the stacktrace attached at JSPWIKI-1171
> > > (this list strips attachments, IIRC).
> > >
> > > Also, regarding the searches, would you mind confirming that the new
> > > JSPWiki instance has permissions to read/write either the work and
> > > wikipages directories/files? Also are there any errors logged when
> > starting
> > > JSPWiki? Specifically, the Lucene indexing is done on startup (if
> needed)
> > > on the background, so eventually you should get all your pages indexed
> or
> > > see log errors stating something like "Unable to index page $pageName,
> > > continuing to next".
> > >
> > > Are you using a vanilla JSPWiki installation or do you have some
> > > customizations? What parameters do you have on your
> > > jspwiki-custom.properties file? I'm assuming you're deloying the war
> > file?
> > >
> > >
> > > best regards,
> > > juan pablo
> > >
> > > On Thu, Apr 14, 2022 at 4:51 PM Andrew Ducker <
> > > andrew_ducker@thephoenixgroup.com> wrote:
> > >
> > > > Hi!
> > > >
> > > >
> > > >
> > > > I am having a problem with an upgraded JSPWiki not indexing pages.
> > > >
> > > >
> > > >
> > > > I wasn’t sure where to report it, so added an issue on Jira:
> > > >
> > > > https://issues.apache.org/jira/browse/JSPWIKI-1171<
> https://issues.apache.org/jira/browse/JSPWIKI-1171><
> > > https://issues.apache.org/jira/browse/JSPWIKI-1171<
> https://issues.apache.org/jira/browse/JSPWIKI-1171>>
> > > >
> > > >
> > > >
> > > > I’m recreating it here, in case this is the right place to report
> > things:
> > > >
> > > >
> > > >
> > > > Lots of pages aren't turning up in search.
> > > >
> > > >
> > > >
> > > > We cleared the lucene folder from the working directory, and it
> rebuilt
> > > it
> > > > - but quite quickly for 13,000 pages. And the total size of the
> Lucene
> > > > files was around 2.5MB, which seemed far too small.
> > > >
> > > >
> > > >
> > > > Doing a search failed to bring back pages that had the given word in
> > > their
> > > > name and content.
> > > >
> > > >
> > > >
> > > > Editing one of those pages (adding a space) caused it to then appear
> in
> > > > the searches.
> > > >
> > > >
> > > >
> > > > Checking the logs I found a lot of errors in the form:
> > > >
> > > >
> > > >
> > > > [ERROR] 2022-04-12 15:50:17.274 [main]
> o.a.w.r.DefaultReferenceManager
> > -
> > > > Error while trying to fetch a page name; trying to cope with the
> > > situation.
> > > >
> > > > org.apache.wiki.api.exceptions.ProviderException: Illegal page name
> > > >
> > > >
> > > >
> > > > (See attachment for whole exception trace)
> > > >
> > > >
> > > >
> > > > Sadly, it doesn't tell me what the illegal page names are.
> > > >
> > > >
> > > >
> > > > This wiki has been upgraded from version 2.8 - are there
> compatibility
> > > > issues with page names there?
> > > >
> > > >
> > > >
> > > > Any help you can give me gratefully received!
> > > >
> > > >
> > > >
> > > > Andy
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > <https://www.thephoenixgroup.com/>
> > > > *Andrew Ducker* | Senior Systems Developer
> > > > office: +441312458823 | mobile: 07980 86 86 39 | email:
> > > > andrew_ducker@thephoenixgroup.com
> > > > P Please help save the environment, print only if necessary.
> > > > [image: logobadgefinal]
> > > >
> > > >
> > > >
> > > > Confidentiality - This email is confidential.
> > > > Not meant for you? - If you don't think this email is meant for you,
> > > > please let us know. Do not copy or forward the information it
> contains,
> > > and
> > > > delete this email from your system.
> > > > Views expressed - Any personal views or opinions expressed in this
> > email
> > > > are the sender's, and do not necessarily reflect the views of the
> > Phoenix
> > > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > > Monitoring - We filter and monitor emails to protect our systems and
> to
> > > > keep them running smoothly.
> > > > Emailing us - Email isn't a secure form of communication. If you want
> > to
> > > > send us confidential information please send it by post. However, if
> > you
> > > do
> > > > communicate with us by email on any subject, you are giving us
> > permission
> > > > to email you back.
> > > > Phoning us - Calls may be monitored and/or recorded to protect both
> you
> > > > and us and help with our training. Call charges will vary.
> > > > Standard Life Assurance Limited is registered in Scotland (SC286833)
> at
> > > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and
> authorised
> > by
> > > > the Prudential Regulation Authority and regulated by the Financial
> > > Conduct
> > > > Authority and the Prudential Regulation Authority.
> > > www.standardlife.co.uk<http://www.standardlife.co.uk><
> http://www.standardlife.co.uk<http://www.standardlife.co.uk>>
> > > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > > Services Limited are companies in the Phoenix Group (Phoenix Group
> > > Holdings
> > > > plc and its subsidiaries). www.thephoenixgroup.com
> > > >
> > >
> > > Confidentiality - This email is confidential.
> > > Not meant for you? - If you don't think this email is meant for you,
> > > please let us know. Do not copy or forward the information it contains,
> > and
> > > delete this email from your system.
> > > Views expressed - Any personal views or opinions expressed in this
> email
> > > are the sender's, and do not necessarily reflect the views of the
> Phoenix
> > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > Monitoring - We filter and monitor emails to protect our systems and to
> > > keep them running smoothly.
> > > Emailing us - Email isn't a secure form of communication. If you want
> to
> > > send us confidential information please send it by post. However, if
> you
> > do
> > > communicate with us by email on any subject, you are giving us
> permission
> > > to email you back.
> > > Phoning us - Calls may be monitored and/or recorded to protect both you
> > > and us and help with our training. Call charges will vary.
> > > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised
> by
> > > the Prudential Regulation Authority and regulated by the Financial
> > Conduct
> > > Authority and the Prudential Regulation Authority.
> > www.standardlife.co.uk<http://www.standardlife.co.uk><
> > > http://www.standardlife.co.uk<http://www.standardlife.co.uk>>
> > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > Services Limited are companies in the Phoenix Group (Phoenix Group
> > Holdings
> > > plc and its subsidiaries). www.thephoenixgroup.com<
> > > http://www.thephoenixgroup.com>
> > >
> >
>
> Confidentiality - This email is confidential.
> Not meant for you? - If you don't think this email is meant for you,
> please let us know. Do not copy or forward the information it contains, and
> delete this email from your system.
> Views expressed - Any personal views or opinions expressed in this email
> are the sender's, and do not necessarily reflect the views of the Phoenix
> Group (Phoenix Group Holdings plc and its subsidiaries).
> Monitoring - We filter and monitor emails to protect our systems and to
> keep them running smoothly.
> Emailing us - Email isn't a secure form of communication. If you want to
> send us confidential information please send it by post. However, if you do
> communicate with us by email on any subject, you are giving us permission
> to email you back.
> Phoning us - Calls may be monitored and/or recorded to protect both you
> and us and help with our training. Call charges will vary.
> Standard Life Assurance Limited is registered in Scotland (SC286833) at
> Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> the Prudential Regulation Authority and regulated by the Financial Conduct
> Authority and the Prudential Regulation Authority. www.standardlife.co.uk<
> http://www.standardlife.co.uk>
> Standard Life Assurance Limited and Standard Life Assets and Employee
> Services Limited are companies in the Phoenix Group (Phoenix Group Holdings
> plc and its subsidiaries). www.thephoenixgroup.com<
> http://www.thephoenixgroup.com>
>

RE: [External] Re: Search not working on unmodified files

Posted by Andrew Ducker <an...@thephoenixgroup.com>.

Is there an example ehcache file somewhere?  It doesn’t get created anywhere automatically that I can see, and it’s not in the https://jspwiki-wiki.apache.org/Wiki.jsp?page=Configuration documentation.

Cheers,

Andy

From: Jim Willeke [mailto:jim@willeke.com]
Sent: 20 April 2022 21:35
To: user@jspwiki.apache.org
Subject: Re: [External] Re: Search not working on unmodified files

Alert! This is an external email sent from jim@willeke.com.

I have observed something similar.

Active Sessions 51
Uptime 0d, 18h 30m 39s
Number of pages 16390FYI: I too am seeing similar issues on 2.11.0-M8.

I have NOT observed the issues on 2.10.1.

The instance on version 2.10.1 (https://ldapwiki.com/wiki/SystemInfo<https://ldapwiki.com/wiki/SystemInfo>) is
busy
and the 2.11.0-M8 instance on has very low traffic.

On the version 2.11.0-M8, Cleared the Lucene and work directory and
restarted:

Active Sessions 1
Uptime 0d, 9h 11m 7s
Number of pages 16499

And one day later is shows:
Active Sessions 1
Uptime 1d, 10h 15m 43s
Number of pages 46

Then after using for a few minutes it now says:
Active Sessions 1
Uptime 1d, 10h 35m 52s
Number of pages 16500
(And yes one page was really added over the day)

Where Number of pages is the [{$totalpages}]

On the version 2.10.1 below is typical:

Active Sessions 51
Uptime 0d, 18h 30m 39s
Number of pages 16390

The ability to perform a search appears to be related to the
[{$totalpages}] variable. That is when it shows 46 almost nothing can be
found.

I do see in the log:
2022-04-20 10:12:22,050 [main] WARN
org.apache.wiki.providers.CachingProvider - seems
Ldapwiki.jspwiki.pageCache can't hold all pages from your page repository,
so we're delegating on the underlying provider instead. Please consider
increasing your cache sizes on ehcache.xml to avoid this behaviour

Which led to the finding the entry in
https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties<https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties>

"
*# By default, JSPWiki caches will hold up to 1.000 elements, except the
RSS cache, which will hold up to 250
elementsjspwiki.cache.custom-config-file = jspwiki-ehcache.xml*"

Both are behind nginx.

--
-jim
Jim Willeke


On Mon, Apr 18, 2022 at 3:40 PM Juan Pablo Santos Rodríguez <
juanpablo.santos@gmail.com> wrote:

> Hi!
>
> Looking at the source code, at jspwiki init, a full reindex is triggered.
> The full reindex is only performed if the Lucene dir is empty. As this is
> not the case, the initial full reindex isn't executed, and that's why you
> are not seeing anything on searches. Pages saves do trigger a partial
> reindex, though, so that explains the other side of the behaviour you're
> seeing.
>
> What files are inside the Lucene dir? If there are Lucene index files, can
> you open then with Luke to see where they came from, or the Lucene index
> version, or any other information? Is there any chance other jspwiki
> instance is using the same workDir?
>
> Last, emptying the lucene dir will enforce a full reindex next time the
> background thread responsible of performing Lucene reindexes runs, but it
> would be better to know why some files are getting created there outside
> your jspwiki instance.
>
>
> HTH,
> juan pablo
>
> El lun., 18 abr. 2022 12:32, Andrew Ducker <
> andrew_ducker@thephoenixgroup.com> escribió:
>
> > Hi,
> >
> > Thanks for the reply.
> >
> > We completely cleared the WorkDir, to be on the safe side, and that
> didn’t
> > help.
> >
> > It has access to the Work and wikipages folders, as it can create them
> > from scratch, and you can edit the Wiki files.
> >
> > No mentions of “unable to index” in the logs.
> >
> > In the jspwiki-custom.properties file there’s the following:
> > jspwiki.applicationName =IS Wiki
> > jspwiki.baseURL = <removed>
> > jspwiki.frontPage = Main
> > jspwiki.pageProvider =VersioningFileProvider
> > jspwiki.fileSystemProvider.pageDir =d:/JSPWiki/JSPWikiPages/Pages/wiki/
> > jspwiki.basicAttachmentProvider.storageDir
> > =d:/JSPWiki/JSPWikiPages/Attachments/wiki/
> > jspwiki.interWikiRef.Notes = Notes:%s
> > jspwiki.translatorReader.inlinePattern.1 = *.jpg
> > jspwiki.translatorReader.inlinePattern.2 = *.png
> > jspwiki.translatorReader.inlinePattern.3 = http://images.com/*<http://images.com/*>
> > jspwiki.rss.generate =true
> > jspwiki.rss.channelDescription = IS Wiki
> > mail.from =@mail.from@
> > mail.smtp.host<http://mail.smtp.host> =@mail.smtp.host@
> > jspwiki.workDir =d:/JSPWiki/WorkingDirectory/iswiki/
> > jspwiki.security = jaas
> >
> > Lucene-specific things in the logs:
> > [INFO ] 2022-04-13 13:44:48.094 [main] o.a.w.s.LuceneSearchProvider -
> > Lucene enabled, cache will be in:
> d:\JSPWiki\WorkingDirectory\iswiki\lucene
> > [WARN ] 2022-04-13 13:44:48.725 [JSPWiki Lucene Indexer]
> > o.a.w.WikiBackgroundThread - Starting up background thread: JSPWiki
> Lucene
> > Indexer.
> > [INFO ] 2022-04-13 13:45:48.732 [JSPWiki Lucene Indexer]
> > o.a.w.s.LuceneSearchProvider - Files found in Lucene directory, not
> > reindexing.
> >
> > So could it be starting to index, thinking it’s already indexed (despite
> > the whole WorkingDir being cleared out) and then stopping near-instantly?
> >
> > Thanks again,
> >
> > Andy
> >
> >
> > From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
> > Sent: 14 April 2022 16:25
> > To: user@jspwiki.apache.org
> > Subject: [External] Re: Search not working on unmodified files
> >
> > Alert! This is an external email sent from juanpablo.santos@gmail.com.
> >
> > Hi Andrew,
> >
> > haven't had the time to look in depth at JSPWIKI-1171, so just some
> > questions / random thoughts:
> >
> > JSPWiki has changed a lot since 2.8, but the wiki pages itself shouldn't
> > require any change, and should be readable by any version of JSPWiki. The
> > Lucene version has been upgraded several times, so it was definitely ok
> to
> > clear your Lucene folder. Also there is the refmgr.ser file and
> refmgr-attr
> > dir inside your workDir, which contain serialized information about page
> > references. I'm not 100% sure, but most probably the internal format has
> > changed since 2.8 (there was a global package rename on 2.9, when
> entering
> > the Apache incubator, and somewhat later a public API was pushed which
> may
> > have affected this format too) so you should delete those too and try to
> > restart your JSPWiki instance again and see if the situation persists;
> > perhaps deleting the entire workDir would be the safest approach. That's
> > what seems to be happening from the stacktrace attached at JSPWIKI-1171
> > (this list strips attachments, IIRC).
> >
> > Also, regarding the searches, would you mind confirming that the new
> > JSPWiki instance has permissions to read/write either the work and
> > wikipages directories/files? Also are there any errors logged when
> starting
> > JSPWiki? Specifically, the Lucene indexing is done on startup (if needed)
> > on the background, so eventually you should get all your pages indexed or
> > see log errors stating something like "Unable to index page $pageName,
> > continuing to next".
> >
> > Are you using a vanilla JSPWiki installation or do you have some
> > customizations? What parameters do you have on your
> > jspwiki-custom.properties file? I'm assuming you're deloying the war
> file?
> >
> >
> > best regards,
> > juan pablo
> >
> > On Thu, Apr 14, 2022 at 4:51 PM Andrew Ducker <
> > andrew_ducker@thephoenixgroup.com> wrote:
> >
> > > Hi!
> > >
> > >
> > >
> > > I am having a problem with an upgraded JSPWiki not indexing pages.
> > >
> > >
> > >
> > > I wasn’t sure where to report it, so added an issue on Jira:
> > >
> > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171><
> > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>>
> > >
> > >
> > >
> > > I’m recreating it here, in case this is the right place to report
> things:
> > >
> > >
> > >
> > > Lots of pages aren't turning up in search.
> > >
> > >
> > >
> > > We cleared the lucene folder from the working directory, and it rebuilt
> > it
> > > - but quite quickly for 13,000 pages. And the total size of the Lucene
> > > files was around 2.5MB, which seemed far too small.
> > >
> > >
> > >
> > > Doing a search failed to bring back pages that had the given word in
> > their
> > > name and content.
> > >
> > >
> > >
> > > Editing one of those pages (adding a space) caused it to then appear in
> > > the searches.
> > >
> > >
> > >
> > > Checking the logs I found a lot of errors in the form:
> > >
> > >
> > >
> > > [ERROR] 2022-04-12 15:50:17.274 [main] o.a.w.r.DefaultReferenceManager
> -
> > > Error while trying to fetch a page name; trying to cope with the
> > situation.
> > >
> > > org.apache.wiki.api.exceptions.ProviderException: Illegal page name
> > >
> > >
> > >
> > > (See attachment for whole exception trace)
> > >
> > >
> > >
> > > Sadly, it doesn't tell me what the illegal page names are.
> > >
> > >
> > >
> > > This wiki has been upgraded from version 2.8 - are there compatibility
> > > issues with page names there?
> > >
> > >
> > >
> > > Any help you can give me gratefully received!
> > >
> > >
> > >
> > > Andy
> > >
> > >
> > >
> > > --
> > >
> > >
> > >
> > >
> > >
> > > <https://www.thephoenixgroup.com/>
> > > *Andrew Ducker* | Senior Systems Developer
> > > office: +441312458823 | mobile: 07980 86 86 39 | email:
> > > andrew_ducker@thephoenixgroup.com
> > > P Please help save the environment, print only if necessary.
> > > [image: logobadgefinal]
> > >
> > >
> > >
> > > Confidentiality - This email is confidential.
> > > Not meant for you? - If you don't think this email is meant for you,
> > > please let us know. Do not copy or forward the information it contains,
> > and
> > > delete this email from your system.
> > > Views expressed - Any personal views or opinions expressed in this
> email
> > > are the sender's, and do not necessarily reflect the views of the
> Phoenix
> > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > Monitoring - We filter and monitor emails to protect our systems and to
> > > keep them running smoothly.
> > > Emailing us - Email isn't a secure form of communication. If you want
> to
> > > send us confidential information please send it by post. However, if
> you
> > do
> > > communicate with us by email on any subject, you are giving us
> permission
> > > to email you back.
> > > Phoning us - Calls may be monitored and/or recorded to protect both you
> > > and us and help with our training. Call charges will vary.
> > > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised
> by
> > > the Prudential Regulation Authority and regulated by the Financial
> > Conduct
> > > Authority and the Prudential Regulation Authority.
> > www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>>
> > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > Services Limited are companies in the Phoenix Group (Phoenix Group
> > Holdings
> > > plc and its subsidiaries). www.thephoenixgroup.com
> > >
> >
> > Confidentiality - This email is confidential.
> > Not meant for you? - If you don't think this email is meant for you,
> > please let us know. Do not copy or forward the information it contains,
> and
> > delete this email from your system.
> > Views expressed - Any personal views or opinions expressed in this email
> > are the sender's, and do not necessarily reflect the views of the Phoenix
> > Group (Phoenix Group Holdings plc and its subsidiaries).
> > Monitoring - We filter and monitor emails to protect our systems and to
> > keep them running smoothly.
> > Emailing us - Email isn't a secure form of communication. If you want to
> > send us confidential information please send it by post. However, if you
> do
> > communicate with us by email on any subject, you are giving us permission
> > to email you back.
> > Phoning us - Calls may be monitored and/or recorded to protect both you
> > and us and help with our training. Call charges will vary.
> > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> > the Prudential Regulation Authority and regulated by the Financial
> Conduct
> > Authority and the Prudential Regulation Authority.
> www.standardlife.co.uk<http://www.standardlife.co.uk><
> > http://www.standardlife.co.uk<http://www.standardlife.co.uk>>
> > Standard Life Assurance Limited and Standard Life Assets and Employee
> > Services Limited are companies in the Phoenix Group (Phoenix Group
> Holdings
> > plc and its subsidiaries). www.thephoenixgroup.com<
> > http://www.thephoenixgroup.com>
> >
>

Confidentiality - This email is confidential.
Not meant for you? - If you don't think this email is meant for you, please let us know. Do not copy or forward the information it contains, and delete this email from your system.
Views expressed - Any personal views or opinions expressed in this email are the sender's, and do not necessarily reflect the views of the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries).
Monitoring - We filter and monitor emails to protect our systems and to keep them running smoothly.
Emailing us - Email isn't a secure form of communication. If you want to send us confidential information please send it by post. However, if you do communicate with us by email on any subject, you are giving us permission to email you back.
Phoning us - Calls may be monitored and/or recorded to protect both you and us and help with our training. Call charges will vary.
Standard Life Assurance Limited is registered in Scotland (SC286833) at Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. www.standardlife.co.uk<http://www.standardlife.co.uk>
Standard Life Assurance Limited and Standard Life Assets and Employee Services Limited are companies in the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries). www.thephoenixgroup.com<http://www.thephoenixgroup.com>

Re: [External] Re: Search not working on unmodified files

Posted by Jim Willeke <ji...@willeke.com>.

I have observed something similar.

Active Sessions 51
Uptime 0d, 18h 30m 39s
Number of pages 16390FYI: I too am seeing similar issues on 2.11.0-M8.

I have NOT observed the issues on 2.10.1.

The instance on version 2.10.1 (https://ldapwiki.com/wiki/SystemInfo)  is
busy
and the 2.11.0-M8 instance on has very low traffic.

On the version 2.11.0-M8, Cleared the Lucene and work directory and
restarted:

Active Sessions 1
Uptime 0d, 9h 11m 7s
Number of pages 16499

And one day later is shows:
Active Sessions 1
Uptime 1d, 10h 15m 43s
Number of pages 46

Then after using for a few minutes it now says:
Active Sessions 1
Uptime 1d, 10h 35m 52s
Number of pages 16500
(And yes one page was really added over the day)

Where Number of pages is the [{$totalpages}]

On the version 2.10.1 below is typical:

Active Sessions 51
Uptime 0d, 18h 30m 39s
Number of pages 16390

The ability to perform a search appears to be related to the
[{$totalpages}] variable. That is when it shows 46 almost nothing can be
found.

I do see in the log:
2022-04-20 10:12:22,050 [main] WARN
org.apache.wiki.providers.CachingProvider  - seems
Ldapwiki.jspwiki.pageCache can't hold all pages from your page repository,
so we're delegating on the underlying provider instead. Please consider
increasing your cache sizes on ehcache.xml to avoid this behaviour

Which led to the finding the entry in
https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties

"
*# By default, JSPWiki caches will hold up to 1.000 elements, except the
RSS cache, which will hold up to 250
elementsjspwiki.cache.custom-config-file = jspwiki-ehcache.xml*"

Both are behind nginx.

--
-jim
Jim Willeke


On Mon, Apr 18, 2022 at 3:40 PM Juan Pablo Santos Rodríguez <
juanpablo.santos@gmail.com> wrote:

> Hi!
>
> Looking at the source code, at jspwiki init, a full reindex is triggered.
> The full reindex is only performed if the Lucene dir is empty. As this is
> not the case, the initial full reindex isn't executed, and that's why you
> are not seeing anything on searches. Pages saves do trigger a partial
> reindex, though, so that explains the other side of the behaviour you're
> seeing.
>
> What files are inside the Lucene dir? If there are Lucene index files, can
> you open then with Luke to see where they came from, or the Lucene index
> version, or any other information? Is there any chance other jspwiki
> instance is using the same workDir?
>
> Last, emptying the lucene dir will enforce a full reindex next time the
> background thread responsible of performing Lucene reindexes runs, but it
> would be better to know why some files are getting created there outside
> your jspwiki instance.
>
>
> HTH,
> juan pablo
>
> El lun., 18 abr. 2022 12:32, Andrew Ducker <
> andrew_ducker@thephoenixgroup.com> escribió:
>
> > Hi,
> >
> > Thanks for the reply.
> >
> > We completely cleared the WorkDir, to be on the safe side, and that
> didn’t
> > help.
> >
> > It has access to the Work and wikipages folders, as it can create them
> > from scratch, and you can edit the Wiki files.
> >
> > No mentions of “unable to index” in the logs.
> >
> > In the jspwiki-custom.properties file there’s the following:
> > jspwiki.applicationName =IS Wiki
> > jspwiki.baseURL = <removed>
> > jspwiki.frontPage = Main
> > jspwiki.pageProvider =VersioningFileProvider
> > jspwiki.fileSystemProvider.pageDir =d:/JSPWiki/JSPWikiPages/Pages/wiki/
> > jspwiki.basicAttachmentProvider.storageDir
> > =d:/JSPWiki/JSPWikiPages/Attachments/wiki/
> > jspwiki.interWikiRef.Notes = Notes:%s
> > jspwiki.translatorReader.inlinePattern.1 = *.jpg
> > jspwiki.translatorReader.inlinePattern.2 = *.png
> > jspwiki.translatorReader.inlinePattern.3 = http://images.com/*
> > jspwiki.rss.generate =true
> > jspwiki.rss.channelDescription = IS Wiki
> > mail.from =@mail.from@
> > mail.smtp.host =@mail.smtp.host@
> > jspwiki.workDir =d:/JSPWiki/WorkingDirectory/iswiki/
> > jspwiki.security = jaas
> >
> > Lucene-specific things in the logs:
> > [INFO ] 2022-04-13 13:44:48.094 [main] o.a.w.s.LuceneSearchProvider -
> > Lucene enabled, cache will be in:
> d:\JSPWiki\WorkingDirectory\iswiki\lucene
> > [WARN ] 2022-04-13 13:44:48.725 [JSPWiki Lucene Indexer]
> > o.a.w.WikiBackgroundThread - Starting up background thread: JSPWiki
> Lucene
> > Indexer.
> > [INFO ] 2022-04-13 13:45:48.732 [JSPWiki Lucene Indexer]
> > o.a.w.s.LuceneSearchProvider - Files found in Lucene directory, not
> > reindexing.
> >
> > So could it be starting to index, thinking it’s already indexed (despite
> > the whole WorkingDir being cleared out) and then stopping near-instantly?
> >
> > Thanks again,
> >
> > Andy
> >
> >
> > From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
> > Sent: 14 April 2022 16:25
> > To: user@jspwiki.apache.org
> > Subject: [External] Re: Search not working on unmodified files
> >
> > Alert! This is an external email sent from juanpablo.santos@gmail.com.
> >
> > Hi Andrew,
> >
> > haven't had the time to look in depth at JSPWIKI-1171, so just some
> > questions / random thoughts:
> >
> > JSPWiki has changed a lot since 2.8, but the wiki pages itself shouldn't
> > require any change, and should be readable by any version of JSPWiki. The
> > Lucene version has been upgraded several times, so it was definitely ok
> to
> > clear your Lucene folder. Also there is the refmgr.ser file and
> refmgr-attr
> > dir inside your workDir, which contain serialized information about page
> > references. I'm not 100% sure, but most probably the internal format has
> > changed since 2.8 (there was a global package rename on 2.9, when
> entering
> > the Apache incubator, and somewhat later a public API was pushed which
> may
> > have affected this format too) so you should delete those too and try to
> > restart your JSPWiki instance again and see if the situation persists;
> > perhaps deleting the entire workDir would be the safest approach. That's
> > what seems to be happening from the stacktrace attached at JSPWIKI-1171
> > (this list strips attachments, IIRC).
> >
> > Also, regarding the searches, would you mind confirming that the new
> > JSPWiki instance has permissions to read/write either the work and
> > wikipages directories/files? Also are there any errors logged when
> starting
> > JSPWiki? Specifically, the Lucene indexing is done on startup (if needed)
> > on the background, so eventually you should get all your pages indexed or
> > see log errors stating something like "Unable to index page $pageName,
> > continuing to next".
> >
> > Are you using a vanilla JSPWiki installation or do you have some
> > customizations? What parameters do you have on your
> > jspwiki-custom.properties file? I'm assuming you're deloying the war
> file?
> >
> >
> > best regards,
> > juan pablo
> >
> > On Thu, Apr 14, 2022 at 4:51 PM Andrew Ducker <
> > andrew_ducker@thephoenixgroup.com> wrote:
> >
> > > Hi!
> > >
> > >
> > >
> > > I am having a problem with an upgraded JSPWiki not indexing pages.
> > >
> > >
> > >
> > > I wasn’t sure where to report it, so added an issue on Jira:
> > >
> > > https://issues.apache.org/jira/browse/JSPWIKI-1171<
> > https://issues.apache.org/jira/browse/JSPWIKI-1171>
> > >
> > >
> > >
> > > I’m recreating it here, in case this is the right place to report
> things:
> > >
> > >
> > >
> > > Lots of pages aren't turning up in search.
> > >
> > >
> > >
> > > We cleared the lucene folder from the working directory, and it rebuilt
> > it
> > > - but quite quickly for 13,000 pages. And the total size of the Lucene
> > > files was around 2.5MB, which seemed far too small.
> > >
> > >
> > >
> > > Doing a search failed to bring back pages that had the given word in
> > their
> > > name and content.
> > >
> > >
> > >
> > > Editing one of those pages (adding a space) caused it to then appear in
> > > the searches.
> > >
> > >
> > >
> > > Checking the logs I found a lot of errors in the form:
> > >
> > >
> > >
> > > [ERROR] 2022-04-12 15:50:17.274 [main] o.a.w.r.DefaultReferenceManager
> -
> > > Error while trying to fetch a page name; trying to cope with the
> > situation.
> > >
> > > org.apache.wiki.api.exceptions.ProviderException: Illegal page name
> > >
> > >
> > >
> > > (See attachment for whole exception trace)
> > >
> > >
> > >
> > > Sadly, it doesn't tell me what the illegal page names are.
> > >
> > >
> > >
> > > This wiki has been upgraded from version 2.8 - are there compatibility
> > > issues with page names there?
> > >
> > >
> > >
> > > Any help you can give me gratefully received!
> > >
> > >
> > >
> > > Andy
> > >
> > >
> > >
> > > --
> > >
> > >
> > >
> > >
> > >
> > > <https://www.thephoenixgroup.com/>
> > > *Andrew Ducker* | Senior Systems Developer
> > > office: +441312458823 | mobile: 07980 86 86 39 | email:
> > > andrew_ducker@thephoenixgroup.com
> > > P Please help save the environment, print only if necessary.
> > > [image: logobadgefinal]
> > >
> > >
> > >
> > > Confidentiality - This email is confidential.
> > > Not meant for you? - If you don't think this email is meant for you,
> > > please let us know. Do not copy or forward the information it contains,
> > and
> > > delete this email from your system.
> > > Views expressed - Any personal views or opinions expressed in this
> email
> > > are the sender's, and do not necessarily reflect the views of the
> Phoenix
> > > Group (Phoenix Group Holdings plc and its subsidiaries).
> > > Monitoring - We filter and monitor emails to protect our systems and to
> > > keep them running smoothly.
> > > Emailing us - Email isn't a secure form of communication. If you want
> to
> > > send us confidential information please send it by post. However, if
> you
> > do
> > > communicate with us by email on any subject, you are giving us
> permission
> > > to email you back.
> > > Phoning us - Calls may be monitored and/or recorded to protect both you
> > > and us and help with our training. Call charges will vary.
> > > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised
> by
> > > the Prudential Regulation Authority and regulated by the Financial
> > Conduct
> > > Authority and the Prudential Regulation Authority.
> > www.standardlife.co.uk<http://www.standardlife.co.uk>
> > > Standard Life Assurance Limited and Standard Life Assets and Employee
> > > Services Limited are companies in the Phoenix Group (Phoenix Group
> > Holdings
> > > plc and its subsidiaries). www.thephoenixgroup.com
> > >
> >
> > Confidentiality - This email is confidential.
> > Not meant for you? - If you don't think this email is meant for you,
> > please let us know. Do not copy or forward the information it contains,
> and
> > delete this email from your system.
> > Views expressed - Any personal views or opinions expressed in this email
> > are the sender's, and do not necessarily reflect the views of the Phoenix
> > Group (Phoenix Group Holdings plc and its subsidiaries).
> > Monitoring - We filter and monitor emails to protect our systems and to
> > keep them running smoothly.
> > Emailing us - Email isn't a secure form of communication. If you want to
> > send us confidential information please send it by post. However, if you
> do
> > communicate with us by email on any subject, you are giving us permission
> > to email you back.
> > Phoning us - Calls may be monitored and/or recorded to protect both you
> > and us and help with our training. Call charges will vary.
> > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> > the Prudential Regulation Authority and regulated by the Financial
> Conduct
> > Authority and the Prudential Regulation Authority.
> www.standardlife.co.uk<
> > http://www.standardlife.co.uk>
> > Standard Life Assurance Limited and Standard Life Assets and Employee
> > Services Limited are companies in the Phoenix Group (Phoenix Group
> Holdings
> > plc and its subsidiaries). www.thephoenixgroup.com<
> > http://www.thephoenixgroup.com>
> >
>

RE: [External] Re: Search not working on unmodified files

Posted by Andrew Ducker <an...@thephoenixgroup.com>.

Hi,

Possibly got there.

We removed the OLD folder, which contains all of the history, and then it all scanned in.

We used Luke to check how many documents were indexed, and with the OLD subdirectory there it only indexed about 1,000 documents.  With that removed it got all of them (about 13,000 documents).

Does that make any sense why it would happen?

Andy

From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
Sent: 18 April 2022 20:39
To: user@jspwiki.apache.org
Subject: Re: [External] Re: Search not working on unmodified files

Alert! This is an external email sent from juanpablo.santos@gmail.com.

Hi!

Looking at the source code, at jspwiki init, a full reindex is triggered.
The full reindex is only performed if the Lucene dir is empty. As this is
not the case, the initial full reindex isn't executed, and that's why you
are not seeing anything on searches. Pages saves do trigger a partial
reindex, though, so that explains the other side of the behaviour you're
seeing.

What files are inside the Lucene dir? If there are Lucene index files, can
you open then with Luke to see where they came from, or the Lucene index
version, or any other information? Is there any chance other jspwiki
instance is using the same workDir?

Last, emptying the lucene dir will enforce a full reindex next time the
background thread responsible of performing Lucene reindexes runs, but it
would be better to know why some files are getting created there outside
your jspwiki instance.


HTH,
juan pablo

El lun., 18 abr. 2022 12:32, Andrew Ducker <
andrew_ducker@thephoenixgroup.com> escribió:

> Hi,
>
> Thanks for the reply.
>
> We completely cleared the WorkDir, to be on the safe side, and that didn’t
> help.
>
> It has access to the Work and wikipages folders, as it can create them
> from scratch, and you can edit the Wiki files.
>
> No mentions of “unable to index” in the logs.
>
> In the jspwiki-custom.properties file there’s the following:
> jspwiki.applicationName =IS Wiki
> jspwiki.baseURL = <removed>
> jspwiki.frontPage = Main
> jspwiki.pageProvider =VersioningFileProvider
> jspwiki.fileSystemProvider.pageDir =d:/JSPWiki/JSPWikiPages/Pages/wiki/
> jspwiki.basicAttachmentProvider.storageDir
> =d:/JSPWiki/JSPWikiPages/Attachments/wiki/
> jspwiki.interWikiRef.Notes = Notes:%s
> jspwiki.translatorReader.inlinePattern.1 = *.jpg
> jspwiki.translatorReader.inlinePattern.2 = *.png
> jspwiki.translatorReader.inlinePattern.3 = http://images.com/*<http://images.com/*>
> jspwiki.rss.generate =true
> jspwiki.rss.channelDescription = IS Wiki
> mail.from =@mail.from@
> mail.smtp.host<http://mail.smtp.host> =@mail.smtp.host@
> jspwiki.workDir =d:/JSPWiki/WorkingDirectory/iswiki/
> jspwiki.security = jaas
>
> Lucene-specific things in the logs:
> [INFO ] 2022-04-13 13:44:48.094 [main] o.a.w.s.LuceneSearchProvider -
> Lucene enabled, cache will be in: d:\JSPWiki\WorkingDirectory\iswiki\lucene
> [WARN ] 2022-04-13 13:44:48.725 [JSPWiki Lucene Indexer]
> o.a.w.WikiBackgroundThread - Starting up background thread: JSPWiki Lucene
> Indexer.
> [INFO ] 2022-04-13 13:45:48.732 [JSPWiki Lucene Indexer]
> o.a.w.s.LuceneSearchProvider - Files found in Lucene directory, not
> reindexing.
>
> So could it be starting to index, thinking it’s already indexed (despite
> the whole WorkingDir being cleared out) and then stopping near-instantly?
>
> Thanks again,
>
> Andy
>
>
> From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
> Sent: 14 April 2022 16:25
> To: user@jspwiki.apache.org
> Subject: [External] Re: Search not working on unmodified files
>
> Alert! This is an external email sent from juanpablo.santos@gmail.com.
>
> Hi Andrew,
>
> haven't had the time to look in depth at JSPWIKI-1171, so just some
> questions / random thoughts:
>
> JSPWiki has changed a lot since 2.8, but the wiki pages itself shouldn't
> require any change, and should be readable by any version of JSPWiki. The
> Lucene version has been upgraded several times, so it was definitely ok to
> clear your Lucene folder. Also there is the refmgr.ser file and refmgr-attr
> dir inside your workDir, which contain serialized information about page
> references. I'm not 100% sure, but most probably the internal format has
> changed since 2.8 (there was a global package rename on 2.9, when entering
> the Apache incubator, and somewhat later a public API was pushed which may
> have affected this format too) so you should delete those too and try to
> restart your JSPWiki instance again and see if the situation persists;
> perhaps deleting the entire workDir would be the safest approach. That's
> what seems to be happening from the stacktrace attached at JSPWIKI-1171
> (this list strips attachments, IIRC).
>
> Also, regarding the searches, would you mind confirming that the new
> JSPWiki instance has permissions to read/write either the work and
> wikipages directories/files? Also are there any errors logged when starting
> JSPWiki? Specifically, the Lucene indexing is done on startup (if needed)
> on the background, so eventually you should get all your pages indexed or
> see log errors stating something like "Unable to index page $pageName,
> continuing to next".
>
> Are you using a vanilla JSPWiki installation or do you have some
> customizations? What parameters do you have on your
> jspwiki-custom.properties file? I'm assuming you're deloying the war file?
>
>
> best regards,
> juan pablo
>
> On Thu, Apr 14, 2022 at 4:51 PM Andrew Ducker <
> andrew_ducker@thephoenixgroup.com> wrote:
>
> > Hi!
> >
> >
> >
> > I am having a problem with an upgraded JSPWiki not indexing pages.
> >
> >
> >
> > I wasn’t sure where to report it, so added an issue on Jira:
> >
> > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171><
> https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>>
> >
> >
> >
> > I’m recreating it here, in case this is the right place to report things:
> >
> >
> >
> > Lots of pages aren't turning up in search.
> >
> >
> >
> > We cleared the lucene folder from the working directory, and it rebuilt
> it
> > - but quite quickly for 13,000 pages. And the total size of the Lucene
> > files was around 2.5MB, which seemed far too small.
> >
> >
> >
> > Doing a search failed to bring back pages that had the given word in
> their
> > name and content.
> >
> >
> >
> > Editing one of those pages (adding a space) caused it to then appear in
> > the searches.
> >
> >
> >
> > Checking the logs I found a lot of errors in the form:
> >
> >
> >
> > [ERROR] 2022-04-12 15:50:17.274 [main] o.a.w.r.DefaultReferenceManager -
> > Error while trying to fetch a page name; trying to cope with the
> situation.
> >
> > org.apache.wiki.api.exceptions.ProviderException: Illegal page name
> >
> >
> >
> > (See attachment for whole exception trace)
> >
> >
> >
> > Sadly, it doesn't tell me what the illegal page names are.
> >
> >
> >
> > This wiki has been upgraded from version 2.8 - are there compatibility
> > issues with page names there?
> >
> >
> >
> > Any help you can give me gratefully received!
> >
> >
> >
> > Andy
> >
> >
> >
> > --
> >
> >
> >
> >
> >
> > <https://www.thephoenixgroup.com/>
> > *Andrew Ducker* | Senior Systems Developer
> > office: +441312458823 | mobile: 07980 86 86 39 | email:
> > andrew_ducker@thephoenixgroup.com
> > P Please help save the environment, print only if necessary.
> > [image: logobadgefinal]
> >
> >
> >
> > Confidentiality - This email is confidential.
> > Not meant for you? - If you don't think this email is meant for you,
> > please let us know. Do not copy or forward the information it contains,
> and
> > delete this email from your system.
> > Views expressed - Any personal views or opinions expressed in this email
> > are the sender's, and do not necessarily reflect the views of the Phoenix
> > Group (Phoenix Group Holdings plc and its subsidiaries).
> > Monitoring - We filter and monitor emails to protect our systems and to
> > keep them running smoothly.
> > Emailing us - Email isn't a secure form of communication. If you want to
> > send us confidential information please send it by post. However, if you
> do
> > communicate with us by email on any subject, you are giving us permission
> > to email you back.
> > Phoning us - Calls may be monitored and/or recorded to protect both you
> > and us and help with our training. Call charges will vary.
> > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> > the Prudential Regulation Authority and regulated by the Financial
> Conduct
> > Authority and the Prudential Regulation Authority.
> www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>>
> > Standard Life Assurance Limited and Standard Life Assets and Employee
> > Services Limited are companies in the Phoenix Group (Phoenix Group
> Holdings
> > plc and its subsidiaries). www.thephoenixgroup.com
> >
>
> Confidentiality - This email is confidential.
> Not meant for you? - If you don't think this email is meant for you,
> please let us know. Do not copy or forward the information it contains, and
> delete this email from your system.
> Views expressed - Any personal views or opinions expressed in this email
> are the sender's, and do not necessarily reflect the views of the Phoenix
> Group (Phoenix Group Holdings plc and its subsidiaries).
> Monitoring - We filter and monitor emails to protect our systems and to
> keep them running smoothly.
> Emailing us - Email isn't a secure form of communication. If you want to
> send us confidential information please send it by post. However, if you do
> communicate with us by email on any subject, you are giving us permission
> to email you back.
> Phoning us - Calls may be monitored and/or recorded to protect both you
> and us and help with our training. Call charges will vary.
> Standard Life Assurance Limited is registered in Scotland (SC286833) at
> Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> the Prudential Regulation Authority and regulated by the Financial Conduct
> Authority and the Prudential Regulation Authority. www.standardlife.co.uk<http://www.standardlife.co.uk><
> http://www.standardlife.co.uk<http://www.standardlife.co.uk>>
> Standard Life Assurance Limited and Standard Life Assets and Employee
> Services Limited are companies in the Phoenix Group (Phoenix Group Holdings
> plc and its subsidiaries). www.thephoenixgroup.com<
> http://www.thephoenixgroup.com>
>

Confidentiality - This email is confidential.
Not meant for you? - If you don't think this email is meant for you, please let us know. Do not copy or forward the information it contains, and delete this email from your system.
Views expressed - Any personal views or opinions expressed in this email are the sender's, and do not necessarily reflect the views of the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries).
Monitoring - We filter and monitor emails to protect our systems and to keep them running smoothly.
Emailing us - Email isn't a secure form of communication. If you want to send us confidential information please send it by post. However, if you do communicate with us by email on any subject, you are giving us permission to email you back.
Phoning us - Calls may be monitored and/or recorded to protect both you and us and help with our training. Call charges will vary.
Standard Life Assurance Limited is registered in Scotland (SC286833) at Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. www.standardlife.co.uk<http://www.standardlife.co.uk>
Standard Life Assurance Limited and Standard Life Assets and Employee Services Limited are companies in the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries). www.thephoenixgroup.com<http://www.thephoenixgroup.com>

Re: [External] Re: Search not working on unmodified files

Posted by Juan Pablo Santos Rodríguez <ju...@gmail.com>.

Hi!

Looking at the source code, at jspwiki init, a full reindex is triggered.
The full reindex is only performed if the Lucene dir is empty. As this is
not the case, the initial full reindex isn't executed, and that's why you
are not seeing anything on searches. Pages saves do trigger a partial
reindex, though, so that explains the other side of the behaviour you're
seeing.

What files are inside the Lucene dir? If there are Lucene index files, can
you open then with Luke to see where they came from, or the Lucene index
version, or any other information? Is there any chance other jspwiki
instance is using the same workDir?

Last, emptying the lucene dir will enforce a full reindex next time the
background thread responsible of performing Lucene reindexes runs, but it
would be better to know why some files are getting created there outside
your jspwiki instance.


HTH,
juan pablo

El lun., 18 abr. 2022 12:32, Andrew Ducker <
andrew_ducker@thephoenixgroup.com> escribió:

> Hi,
>
> Thanks for the reply.
>
> We completely cleared the WorkDir, to be on the safe side, and that didn’t
> help.
>
> It has access to the Work and wikipages folders, as it can create them
> from scratch, and you can edit the Wiki files.
>
> No mentions of “unable to index” in the logs.
>
> In the jspwiki-custom.properties file there’s the following:
> jspwiki.applicationName =IS Wiki
> jspwiki.baseURL = <removed>
> jspwiki.frontPage = Main
> jspwiki.pageProvider =VersioningFileProvider
> jspwiki.fileSystemProvider.pageDir =d:/JSPWiki/JSPWikiPages/Pages/wiki/
> jspwiki.basicAttachmentProvider.storageDir
> =d:/JSPWiki/JSPWikiPages/Attachments/wiki/
> jspwiki.interWikiRef.Notes = Notes:%s
> jspwiki.translatorReader.inlinePattern.1 = *.jpg
> jspwiki.translatorReader.inlinePattern.2 = *.png
> jspwiki.translatorReader.inlinePattern.3 = http://images.com/*
> jspwiki.rss.generate =true
> jspwiki.rss.channelDescription = IS Wiki
> mail.from =@mail.from@
> mail.smtp.host =@mail.smtp.host@
> jspwiki.workDir =d:/JSPWiki/WorkingDirectory/iswiki/
> jspwiki.security = jaas
>
> Lucene-specific things in the logs:
> [INFO ] 2022-04-13 13:44:48.094 [main] o.a.w.s.LuceneSearchProvider -
> Lucene enabled, cache will be in: d:\JSPWiki\WorkingDirectory\iswiki\lucene
> [WARN ] 2022-04-13 13:44:48.725 [JSPWiki Lucene Indexer]
> o.a.w.WikiBackgroundThread - Starting up background thread: JSPWiki Lucene
> Indexer.
> [INFO ] 2022-04-13 13:45:48.732 [JSPWiki Lucene Indexer]
> o.a.w.s.LuceneSearchProvider - Files found in Lucene directory, not
> reindexing.
>
> So could it be starting to index, thinking it’s already indexed (despite
> the whole WorkingDir being cleared out) and then stopping near-instantly?
>
> Thanks again,
>
> Andy
>
>
> From: Juan Pablo Santos Rodríguez [mailto:juanpablo.santos@gmail.com]
> Sent: 14 April 2022 16:25
> To: user@jspwiki.apache.org
> Subject: [External] Re: Search not working on unmodified files
>
> Alert! This is an external email sent from juanpablo.santos@gmail.com.
>
> Hi Andrew,
>
> haven't had the time to look in depth at JSPWIKI-1171, so just some
> questions / random thoughts:
>
> JSPWiki has changed a lot since 2.8, but the wiki pages itself shouldn't
> require any change, and should be readable by any version of JSPWiki. The
> Lucene version has been upgraded several times, so it was definitely ok to
> clear your Lucene folder. Also there is the refmgr.ser file and refmgr-attr
> dir inside your workDir, which contain serialized information about page
> references. I'm not 100% sure, but most probably the internal format has
> changed since 2.8 (there was a global package rename on 2.9, when entering
> the Apache incubator, and somewhat later a public API was pushed which may
> have affected this format too) so you should delete those too and try to
> restart your JSPWiki instance again and see if the situation persists;
> perhaps deleting the entire workDir would be the safest approach. That's
> what seems to be happening from the stacktrace attached at JSPWIKI-1171
> (this list strips attachments, IIRC).
>
> Also, regarding the searches, would you mind confirming that the new
> JSPWiki instance has permissions to read/write either the work and
> wikipages directories/files? Also are there any errors logged when starting
> JSPWiki? Specifically, the Lucene indexing is done on startup (if needed)
> on the background, so eventually you should get all your pages indexed or
> see log errors stating something like "Unable to index page $pageName,
> continuing to next".
>
> Are you using a vanilla JSPWiki installation or do you have some
> customizations? What parameters do you have on your
> jspwiki-custom.properties file? I'm assuming you're deloying the war file?
>
>
> best regards,
> juan pablo
>
> On Thu, Apr 14, 2022 at 4:51 PM Andrew Ducker <
> andrew_ducker@thephoenixgroup.com> wrote:
>
> > Hi!
> >
> >
> >
> > I am having a problem with an upgraded JSPWiki not indexing pages.
> >
> >
> >
> > I wasn’t sure where to report it, so added an issue on Jira:
> >
> > https://issues.apache.org/jira/browse/JSPWIKI-1171<
> https://issues.apache.org/jira/browse/JSPWIKI-1171>
> >
> >
> >
> > I’m recreating it here, in case this is the right place to report things:
> >
> >
> >
> > Lots of pages aren't turning up in search.
> >
> >
> >
> > We cleared the lucene folder from the working directory, and it rebuilt
> it
> > - but quite quickly for 13,000 pages. And the total size of the Lucene
> > files was around 2.5MB, which seemed far too small.
> >
> >
> >
> > Doing a search failed to bring back pages that had the given word in
> their
> > name and content.
> >
> >
> >
> > Editing one of those pages (adding a space) caused it to then appear in
> > the searches.
> >
> >
> >
> > Checking the logs I found a lot of errors in the form:
> >
> >
> >
> > [ERROR] 2022-04-12 15:50:17.274 [main] o.a.w.r.DefaultReferenceManager -
> > Error while trying to fetch a page name; trying to cope with the
> situation.
> >
> > org.apache.wiki.api.exceptions.ProviderException: Illegal page name
> >
> >
> >
> > (See attachment for whole exception trace)
> >
> >
> >
> > Sadly, it doesn't tell me what the illegal page names are.
> >
> >
> >
> > This wiki has been upgraded from version 2.8 - are there compatibility
> > issues with page names there?
> >
> >
> >
> > Any help you can give me gratefully received!
> >
> >
> >
> > Andy
> >
> >
> >
> > --
> >
> >
> >
> >
> >
> > <https://www.thephoenixgroup.com/>
> > *Andrew Ducker* | Senior Systems Developer
> > office: +441312458823 | mobile: 07980 86 86 39 | email:
> > andrew_ducker@thephoenixgroup.com
> > P Please help save the environment, print only if necessary.
> > [image: logobadgefinal]
> >
> >
> >
> > Confidentiality - This email is confidential.
> > Not meant for you? - If you don't think this email is meant for you,
> > please let us know. Do not copy or forward the information it contains,
> and
> > delete this email from your system.
> > Views expressed - Any personal views or opinions expressed in this email
> > are the sender's, and do not necessarily reflect the views of the Phoenix
> > Group (Phoenix Group Holdings plc and its subsidiaries).
> > Monitoring - We filter and monitor emails to protect our systems and to
> > keep them running smoothly.
> > Emailing us - Email isn't a secure form of communication. If you want to
> > send us confidential information please send it by post. However, if you
> do
> > communicate with us by email on any subject, you are giving us permission
> > to email you back.
> > Phoning us - Calls may be monitored and/or recorded to protect both you
> > and us and help with our training. Call charges will vary.
> > Standard Life Assurance Limited is registered in Scotland (SC286833) at
> > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> > the Prudential Regulation Authority and regulated by the Financial
> Conduct
> > Authority and the Prudential Regulation Authority.
> www.standardlife.co.uk<http://www.standardlife.co.uk>
> > Standard Life Assurance Limited and Standard Life Assets and Employee
> > Services Limited are companies in the Phoenix Group (Phoenix Group
> Holdings
> > plc and its subsidiaries). www.thephoenixgroup.com
> >
>
> Confidentiality - This email is confidential.
> Not meant for you? - If you don't think this email is meant for you,
> please let us know. Do not copy or forward the information it contains, and
> delete this email from your system.
> Views expressed - Any personal views or opinions expressed in this email
> are the sender's, and do not necessarily reflect the views of the Phoenix
> Group (Phoenix Group Holdings plc and its subsidiaries).
> Monitoring - We filter and monitor emails to protect our systems and to
> keep them running smoothly.
> Emailing us - Email isn't a secure form of communication. If you want to
> send us confidential information please send it by post. However, if you do
> communicate with us by email on any subject, you are giving us permission
> to email you back.
> Phoning us - Calls may be monitored and/or recorded to protect both you
> and us and help with our training. Call charges will vary.
> Standard Life Assurance Limited is registered in Scotland (SC286833) at
> Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by
> the Prudential Regulation Authority and regulated by the Financial Conduct
> Authority and the Prudential Regulation Authority. www.standardlife.co.uk<
> http://www.standardlife.co.uk>
> Standard Life Assurance Limited and Standard Life Assets and Employee
> Services Limited are companies in the Phoenix Group (Phoenix Group Holdings
> plc and its subsidiaries). www.thephoenixgroup.com<
> http://www.thephoenixgroup.com>
>