You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Daniel Naber <da...@t-online.de> on 2004/08/14 19:34:09 UTC
moving the analyzers into sandbox
Hi,
any objections against moving the German and Russian analyzers into the
sandbox? If not, I'd like to do that, but I'm not sure if we already
agreed on doing so. The current situation with analyzers both in lucene's
core and in the sandbox doesn't seem to make sense.
I suggest that we provide an lucene-analyzers.tar.gz with the upcoming
releases so that people don't have to check out the sandbox from CVS to
make use of the analyzers.
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Aug 15, 2004, at 8:00 AM, Daniel Naber wrote:
> On Saturday 14 August 2004 22:44, Erik Hatcher wrote:
>
>> +1 on moving those analyzers out.
>
> Technically this means to remove the files from lucene's core and
> re-add
> them in sandbox, doesn't it? Or is there some special case for moving
> (e.g. contact cvs admin so he can move files on the server)?
Right, we just add them to the sandbox and remove them from the core.
CVS doesn't support "move" operations. We could, though, (and this is
a separate discussion altogether) move to Subversion. Several Apache
projects already have and many more are going that direction.
> I agree. Some more ideas how we can improve the release process:
>
> -make sure the mirrors get the new version (don't know what's required
> for
> that)
We don't even have 1.4.1 mirrored yet. We need to be sure to make
that happen soon though. I've done it for the past couple of releases.
It's not that big of a deal, but does involve several manual steps
including signing each of the files locally and uploading them into the
correct spot at www.apache.org. There are instructions at the old
nagoya wiki somewhere and I have them archived in e-mail.
> I could create a small document that lists the things to do for each
> release. If you think that such a document is useful, should I put it
> into
> cvs or in the wiki?
I think the wiki is the best place. Most of the instructions are
actually out there, but they are not Lucene specific:
http://nagoya.apache.org/wiki/apachewiki.cgi?ReleaseManager
Having the steps on the Lucene wiki would be nice though.
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Daniel Naber <da...@t-online.de>.
On Saturday 14 August 2004 22:44, Erik Hatcher wrote:
> +1 on moving those analyzers out.
Technically this means to remove the files from lucene's core and re-add
them in sandbox, doesn't it? Or is there some special case for moving
(e.g. contact cvs admin so he can move files on the server)?
> > I suggest that we provide an lucene-analyzers.tar.gz with the upcoming
> > releases so that people don't have to check out the sandbox from CVS
> > to make use of the analyzers.
>
> We should go even further and release all sandbox components along with
> each release - this would give more exposure to the highlighter,
> snowball analyzer, and the others.
I agree. Some more ideas how we can improve the release process:
-Make sure the new version gets properly announced:
-on annoucements@jakarta.apache.org
-on freshmeat.net and maybe other services
-Make sure Bugzilla gets the new version number so people can select it
when they submit a bug report (1.4.1 is still missing)
-make sure the mirrors get the new version (don't know what's required for
that)
I could create a small document that lists the things to do for each
release. If you think that such a document is useful, should I put it into
cvs or in the wiki?
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Aug 16, 2004, at 5:15 PM, Daniel Naber wrote:
> On Sunday 15 August 2004 13:42, Otis Gospodnetic wrote:
>
>> +1 for releasing at least some Sandbox components. Analyzers,
>> Snowball
>> Analyzers and Highligher at least.
>>
>> Is this something you can/want to do, Daniel?
>
> The analyzers are now in the sandbox
Nice job moving things around. Be sure to update the CHANGES.txt file
with this info, especially with the info on changing the word list
loader package. That change may break some folks that are using that
class externally (although I'm personally fine with breaking BWC when
it is for the betterment of the codebase).
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Doug Cutting <cu...@apache.org>.
Erik Hatcher wrote:
> On Aug 16, 2004, at 5:15 PM, Daniel Naber wrote:
>
>> On Sunday 15 August 2004 13:42, Otis Gospodnetic wrote:
>>
>>> +1 for releasing at least some Sandbox components. Analyzers, Snowball
>>> Analyzers and Highligher at least.
>>>
>>> Is this something you can/want to do, Daniel?
>>
>> The analyzers are now in the sandbox, I'll leave it to our ant expert to
>> modify/create the appropriate build.xml for the other components...
>
> What is desired that currently isn't available with the top-level
> contributions directory build.xml?
+1 for releasing the sandbox components with the core.
But how should we do this? Should we bundle everything into one big
lucene-X.X.tar.gz file? Or should we have two files,
lucene-core-X.X.tar.gz and lucene-contrib-X.X.tar.gz? Or should we keep
the contributions each packaged separately, e.g. lucene-core-X.X.tar.gz,
lucene-snowball-X.X.tar.gz, etc. Or should we put all of the
contributions in a subdirectory, e.g., lucene-X.X.tar.gz,
contrib/lucene-snowball-X.X.tar.gz, etc. I prefer one of the latter,
but not strongly.
Whatever the case, I think we should have a single command line that
builds both the core and all of the sandbox contributions, runs unit
tests on everything, and places them all in single distribution
directory. If I'm not mistaken, I think that's the bit of ant wizardry
that Daniel was referring to.
In the end, I think the distinction between core and sandbox should be
less one of quality and more one of packaging.
Doug
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Aug 17, 2004, at 10:11 AM, Daniel Naber wrote:
> On Tuesday 17 August 2004 14:45, Erik Hatcher wrote:
>
>> src/java/org/apache/lucene/analysis/fr/FrenchStemmer.java:571:
>> duplicate case label
>> [javac] case '?':
>> [javac] ^
>
> "ant compile" works for me, probably because my system defaults to
> utf-8.
> But when I explicitly open the FrenchStemmer.java as utf-8, the special
> characters are okay, so it's in utf-8, just like GermanStemmer.java.
> So it
> should compile when the encoding is set to utf-8.
Running "ant compile -verbose", I get this:
[javac]
'/Users/erik/dev/jakarta/jakarta-lucene-sandbox/contributions/
analyzers/src/java'
[javac] '-encoding'
[javac] 'utf-8'
[javac] '-g'
[javac]
...
[javac]
/Users/erik/dev/jakarta/jakarta-lucene-sandbox/contributions/analyzers/
src/java/org/apache/lucene/analysis/fr/FrenchStemmer.java:571:
duplicate case label
[javac] case '?':
[javac] ^
And same type of errors with the DutchStemmer. I'm doing this on Mac
OS X.
I've done a fresh checkout of all "fr" files to no avail.
If someone can help me past compile errors, I'll gladly work on the
sandbox packaging :)
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Daniel Naber <da...@t-online.de>.
On Tuesday 17 August 2004 14:45, Erik Hatcher wrote:
> src/java/org/apache/lucene/analysis/fr/FrenchStemmer.java:571:
> duplicate case label
> [javac] case '?':
> [javac] ^
"ant compile" works for me, probably because my system defaults to utf-8.
But when I explicitly open the FrenchStemmer.java as utf-8, the special
characters are okay, so it's in utf-8, just like GermanStemmer.java. So it
should compile when the encoding is set to utf-8.
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Aug 16, 2004, at 5:32 PM, Daniel Naber wrote:
> However, "ant package" currently
> doesn't work in contributions/analyzers (first it cannot find an
> overview.html and if you add that it complains that
> contributions/analyzers/docs cannot be found).
I'm getting more serious errors with the analyzers project. Here is
what I see:
compile:
[javac] Compiling 26 source files to
/Users/erik/dev/jakarta/jakarta-lucene-sandbox/contributions/analyzers/
build/classes
[javac]
/Users/erik/dev/jakarta/jakarta-lucene-sandbox/contributions/analyzers/
src/java/org/apache/lucene/analysis/fr/FrenchStemmer.java:571:
duplicate case label
[javac] case '?':
[javac] ^
I've modified ../common.xml to have a build.encoding="UTF-8" and
modified the <javac>'s in there to use it (all locally) which got me
past the compilation issues of the GermanStemmer. I'm guessing the
FrenchStemmer is in a different encoding.
What needs to be done to alleviate this?
Thanks,
Erik
>
> Regards
> Daniel
>
> --
> http://www.danielnaber.de
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Daniel Naber <da...@t-online.de>.
On Monday 16 August 2004 23:16, Erik Hatcher wrote:
> What is desired that currently isn't available with the top-level
> contributions directory build.xml?
>
> The build packages each project into a .tar.gz including javadocs, and
> runs unit tests too.
I see. Then probably nothing is missing. However, "ant package" currently
doesn't work in contributions/analyzers (first it cannot find an
overview.html and if you add that it complains that
contributions/analyzers/docs cannot be found).
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Aug 16, 2004, at 5:15 PM, Daniel Naber wrote:
> On Sunday 15 August 2004 13:42, Otis Gospodnetic wrote:
>
>> +1 for releasing at least some Sandbox components. Analyzers,
>> Snowball
>> Analyzers and Highligher at least.
>>
>> Is this something you can/want to do, Daniel?
>
> The analyzers are now in the sandbox, I'll leave it to our ant expert
> to
> modify/create the appropriate build.xml for the other components...
What is desired that currently isn't available with the top-level
contributions directory build.xml?
The build packages each project into a .tar.gz including javadocs, and
runs unit tests too. I'd be happy to tinker with it if there is
something missing.
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Daniel Naber <da...@t-online.de>.
On Sunday 15 August 2004 13:42, Otis Gospodnetic wrote:
> +1 for releasing at least some Sandbox components. Analyzers, Snowball
> Analyzers and Highligher at least.
>
> Is this something you can/want to do, Daniel?
The analyzers are now in the sandbox, I'll leave it to our ant expert to
modify/create the appropriate build.xml for the other components...
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Otis Gospodnetic <ot...@yahoo.com>.
+1 for moving analyzers to the sandbox. We've talked about this
before, and I believe that is what we concluded with.
+1 for releasing at least some Sandbox components. Analyzers, Snowball
Analyzers and Highligher at least.
Is this something you can/want to do, Daniel?
Otis
--- Erik Hatcher <er...@ehatchersolutions.com> wrote:
>
> On Aug 14, 2004, at 1:34 PM, Daniel Naber wrote:
> > any objections against moving the German and Russian analyzers into
> the
> > sandbox? If not, I'd like to do that, but I'm not sure if we
> already
> > agreed on doing so. The current situation with analyzers both in
> > lucene's
> > core and in the sandbox doesn't seem to make sense.
>
> +1 on moving those analyzers out.
>
> > I suggest that we provide an lucene-analyzers.tar.gz with the
> upcoming
> > releases so that people don't have to check out the sandbox from
> CVS to
> > make use of the analyzers.
>
> We should go even further and release all sandbox components along
> with
> each release - this would give more exposure to the highlighter,
> snowball analyzer, and the others. Maybe we shouldn't package
> everything that way, as some things might tend to cause more support
> issues, such as WordNet which requires a few additional steps to make
>
> use of. Thoughts? At the very least though, analyzers, highlighter,
>
> and snowball make great sense to package and version with Lucene core
>
> releases, I think.
>
> Erik
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: moving the analyzers into sandbox
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Aug 14, 2004, at 1:34 PM, Daniel Naber wrote:
> any objections against moving the German and Russian analyzers into the
> sandbox? If not, I'd like to do that, but I'm not sure if we already
> agreed on doing so. The current situation with analyzers both in
> lucene's
> core and in the sandbox doesn't seem to make sense.
+1 on moving those analyzers out.
> I suggest that we provide an lucene-analyzers.tar.gz with the upcoming
> releases so that people don't have to check out the sandbox from CVS to
> make use of the analyzers.
We should go even further and release all sandbox components along with
each release - this would give more exposure to the highlighter,
snowball analyzer, and the others. Maybe we shouldn't package
everything that way, as some things might tend to cause more support
issues, such as WordNet which requires a few additional steps to make
use of. Thoughts? At the very least though, analyzers, highlighter,
and snowball make great sense to package and version with Lucene core
releases, I think.
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org