You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@lucene.apache.org by Peter Carlson <ca...@bookandhammer.com> on 2002/04/19 16:48:05 UTC

[VOTE] Lucene Sandbox committer nomination

I am nominating Eric Hatcher as a contributor to the Lucene-Sandbox Project.



On 4/4/02 7:36 AM, "Erik Hatcher" <li...@ehatchersolutions.com> wrote:

> How will committers to this new repository be selected?
> 
> I would like to have commit access to contribute my Lucene indexing Ant task
> as well as possibly a webapp that interfaces with an index.  And I would, of
> course, maintain as best I could.
> 
>   Erik
> 


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [VOTE] Lucene Sandbox committer nomination

Posted by Peter Carlson <ca...@bookandhammer.com>.

Great Erik

--Peter

On 4/19/02 5:36 PM, "Erik Hatcher" <li...@ehatchersolutions.com> wrote:

> Why thanks folks!  But spell my name right, will ya?!  :))
> 
> I won't have time to get into the sandbox and play for several weeks, but
> I'll be there, count on it!
> 
>   Erik
> 
> 
> ----- Original Message -----
> From: "Andrew C. Oliver" <ac...@apache.org>
> To: "Lucene Developers List" <lu...@jakarta.apache.org>
> Sent: Friday, April 19, 2002 1:46 PM
> Subject: Re: [VOTE] Lucene Sandbox committer nomination
> 
> 
>> +1
>> 
>> Peter Carlson wrote:
>> 
>>> I am nominating Eric Hatcher as a contributor to the Lucene-Sandbox
> Project.
>>> 
>>> 
>>> 
>>> On 4/4/02 7:36 AM, "Erik Hatcher" <li...@ehatchersolutions.com> wrote:
>>> 
>>>> How will committers to this new repository be selected?
>>>> 
>>>> I would like to have commit access to contribute my Lucene indexing Ant
> task
>>>> as well as possibly a webapp that interfaces with an index.  And I
> would, of
>>>> course, maintain as best I could.
>>>> 
>>>>  Erik
>>>> 
>>> 
>>> 
>>> --
>>> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
>>> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>>> 
>>> 
>> 
>> 
>> 
>> 
>> --
>> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
>> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>> 
>> 
> 
> 
> --
> To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
> 
> 


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [VOTE] Lucene Sandbox committer nomination

Posted by Erik Hatcher <li...@ehatchersolutions.com>.

Why thanks folks!  But spell my name right, will ya?!  :))

I won't have time to get into the sandbox and play for several weeks, but
I'll be there, count on it!

    Erik


----- Original Message -----
From: "Andrew C. Oliver" <ac...@apache.org>
To: "Lucene Developers List" <lu...@jakarta.apache.org>
Sent: Friday, April 19, 2002 1:46 PM
Subject: Re: [VOTE] Lucene Sandbox committer nomination


> +1
>
> Peter Carlson wrote:
>
> >I am nominating Eric Hatcher as a contributor to the Lucene-Sandbox
Project.
> >
> >
> >
> >On 4/4/02 7:36 AM, "Erik Hatcher" <li...@ehatchersolutions.com> wrote:
> >
> >>How will committers to this new repository be selected?
> >>
> >>I would like to have commit access to contribute my Lucene indexing Ant
task
> >>as well as possibly a webapp that interfaces with an index.  And I
would, of
> >>course, maintain as best I could.
> >>
> >>  Erik
> >>
> >
> >
> >--
> >To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> >For additional commands, e-mail:
<ma...@jakarta.apache.org>
> >
> >
>
>
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
>
>


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [VOTE] Lucene Sandbox committer nomination

Posted by "Andrew C. Oliver" <ac...@apache.org>.

+1

Peter Carlson wrote:

>I am nominating Eric Hatcher as a contributor to the Lucene-Sandbox Project.
>
>
>
>On 4/4/02 7:36 AM, "Erik Hatcher" <li...@ehatchersolutions.com> wrote:
>
>>How will committers to this new repository be selected?
>>
>>I would like to have commit access to contribute my Lucene indexing Ant task
>>as well as possibly a webapp that interfaces with an index.  And I would, of
>>course, maintain as best I could.
>>
>>  Erik
>>
>
>
>--
>To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
>For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>




--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [VOTE] Lucene Sandbox committer nomination

Posted by Kelvin Tan <ke...@relevanz.com>.

I'd definitely want to be able to maintain my contribs someplace other than
the mailing list. :)

Thinking about introducing a Lucene-Torque indexing package too, after some
clean-up.

----- Original Message -----
From: "Otis Gospodnetic" <ot...@yahoo.com>
To: "Lucene Developers List" <lu...@jakarta.apache.org>
Sent: Wednesday, April 24, 2002 11:56 PM
Subject: Re: [VOTE] Lucene Sandbox committer nomination


> I'm using his contribution, so +1, if he wants it, of course.
>
> Otis
>
> --- Peter Carlson <ca...@bookandhammer.com> wrote:
> > I am nominating Kelvin Tan as a contributor to the Lucene-Sandbox
> > Project.
> >
> > He had already contributed lots of code the Lucene Project including
> > a
> > chainable filter and multiple field searching.
> >
> > I think he will be a great addition to the team.
> >
> > --Peter
> >
> >
> > On 4/19/02 8:13 AM, "Otis Gospodnetic" <ot...@yahoo.com>
> > wrote:
> >
> > > Of course.
> > >
> > > Otis
> > >
> > > --- Peter Carlson <ca...@bookandhammer.com> wrote:
> > >> I am nominating Eric Hatcher as a contributor to the
> > Lucene-Sandbox
> > >> Project.
> > >>
> > >>
> > >>
> > >> On 4/4/02 7:36 AM, "Erik Hatcher" <li...@ehatchersolutions.com>
> > >> wrote:
> > >>
> > >>> How will committers to this new repository be selected?
> > >>>
> > >>> I would like to have commit access to contribute my Lucene
> > indexing
> > >> Ant task
> > >>> as well as possibly a webapp that interfaces with an index.  And
> > I
> > >> would, of
> > >>> course, maintain as best I could.
> > >>>
> > >>>   Erik
> > >>>
> > >>
> > >>
> > >> --
> > >> To unsubscribe, e-mail:
> > >> <ma...@jakarta.apache.org>
> > >> For additional commands, e-mail:
> > >> <ma...@jakarta.apache.org>
> > >>
> > >
> > >
> > > __________________________________________________
> > > Do You Yahoo!?
> > > Yahoo! Tax Center - online filing with TurboTax
> > > http://taxes.yahoo.com/
> > >
> > > --
> > > To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> > > For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> > >
> > >
> >
> >
> > --
> > To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> > For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> >
>
>
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Games - play chess, backgammon, pool and more
> http://games.yahoo.com/
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
>


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [VOTE] Lucene Sandbox committer nomination

Posted by Kelvin Tan <ke...@relevanz.com>.

Great! Thanks for the support.

Regards,
Kelvin

----- Original Message -----
From: "Peter Carlson" <ca...@bookandhammer.com>
To: "Lucene Developers List" <lu...@jakarta.apache.org>
Cc: "Kelvin Tan" <ke...@relevanz.com>
Sent: Wednesday, May 01, 2002 6:46 AM
Subject: Re: [VOTE] Lucene Sandbox committer nomination


> That's 3 + votes.
>
> I will have Kelvin added to the Lucene-Sandbox committer list.
>
> Glad to have you on board Kelvin.
>
> --Peter
>
>
>
> On 4/30/02 3:20 PM, "Eugene Gluzberg" <dr...@yahoo.com> wrote:
>
> > +1
> > Have fun
> > ----- Original Message -----
> > From: "Otis Gospodnetic" <ot...@yahoo.com>
> > To: "Lucene Developers List" <lu...@jakarta.apache.org>
> > Sent: Wednesday, April 24, 2002 11:56 AM
> > Subject: Re: [VOTE] Lucene Sandbox committer nomination
> >
> >
> >> I'm using his contribution, so +1, if he wants it, of course.
> >>
> >> Otis
> >>
> >> --- Peter Carlson <ca...@bookandhammer.com> wrote:
> >>> I am nominating Kelvin Tan as a contributor to the Lucene-Sandbox
> >>> Project.
> >>>
> >>> He had already contributed lots of code the Lucene Project including
> >>> a
> >>> chainable filter and multiple field searching.
> >>>
> >>> I think he will be a great addition to the team.
> >>>
> >>> --Peter
> >>>
> >>>
> >>> On 4/19/02 8:13 AM, "Otis Gospodnetic" <ot...@yahoo.com>
> >>> wrote:
> >>>
> >>>> Of course.
> >>>>
> >>>> Otis
> >>>>
> >>>> --- Peter Carlson <ca...@bookandhammer.com> wrote:
> >>>>> I am nominating Eric Hatcher as a contributor to the
> >>> Lucene-Sandbox
> >>>>> Project.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 4/4/02 7:36 AM, "Erik Hatcher" <li...@ehatchersolutions.com>
> >>>>> wrote:
> >>>>>
> >>>>>> How will committers to this new repository be selected?
> >>>>>>
> >>>>>> I would like to have commit access to contribute my Lucene
> >>> indexing
> >>>>> Ant task
> >>>>>> as well as possibly a webapp that interfaces with an index.  And
> >>> I
> >>>>> would, of
> >>>>>> course, maintain as best I could.
> >>>>>>
> >>>>>>   Erik
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> To unsubscribe, e-mail:
> >>>>> <ma...@jakarta.apache.org>
> >>>>> For additional commands, e-mail:
> >>>>> <ma...@jakarta.apache.org>
> >>>>>
> >>>>
> >>>>
> >>>> __________________________________________________
> >>>> Do You Yahoo!?
> >>>> Yahoo! Tax Center - online filing with TurboTax
> >>>> http://taxes.yahoo.com/
> >>>>
> >>>> --
> >>>> To unsubscribe, e-mail:
> >>> <ma...@jakarta.apache.org>
> >>>> For additional commands, e-mail:
> >>> <ma...@jakarta.apache.org>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> To unsubscribe, e-mail:
> >>> <ma...@jakarta.apache.org>
> >>> For additional commands, e-mail:
> >>> <ma...@jakarta.apache.org>
> >>>
> >>
> >>
> >> __________________________________________________
> >> Do You Yahoo!?
> >> Yahoo! Games - play chess, backgammon, pool and more
> >> http://games.yahoo.com/
> >>
> >> --
> >> To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> >> For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> >
> >
> > --
> > To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> > For additional commands, e-mail:
<ma...@jakarta.apache.org>
> >
> >
>


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [VOTE] Lucene Sandbox committer nomination

Posted by Peter Carlson <ca...@bookandhammer.com>.

That's 3 + votes.

I will have Kelvin added to the Lucene-Sandbox committer list.

Glad to have you on board Kelvin.

--Peter



On 4/30/02 3:20 PM, "Eugene Gluzberg" <dr...@yahoo.com> wrote:

> +1
> Have fun
> ----- Original Message -----
> From: "Otis Gospodnetic" <ot...@yahoo.com>
> To: "Lucene Developers List" <lu...@jakarta.apache.org>
> Sent: Wednesday, April 24, 2002 11:56 AM
> Subject: Re: [VOTE] Lucene Sandbox committer nomination
> 
> 
>> I'm using his contribution, so +1, if he wants it, of course.
>> 
>> Otis
>> 
>> --- Peter Carlson <ca...@bookandhammer.com> wrote:
>>> I am nominating Kelvin Tan as a contributor to the Lucene-Sandbox
>>> Project.
>>> 
>>> He had already contributed lots of code the Lucene Project including
>>> a
>>> chainable filter and multiple field searching.
>>> 
>>> I think he will be a great addition to the team.
>>> 
>>> --Peter
>>> 
>>> 
>>> On 4/19/02 8:13 AM, "Otis Gospodnetic" <ot...@yahoo.com>
>>> wrote:
>>> 
>>>> Of course.
>>>> 
>>>> Otis
>>>> 
>>>> --- Peter Carlson <ca...@bookandhammer.com> wrote:
>>>>> I am nominating Eric Hatcher as a contributor to the
>>> Lucene-Sandbox
>>>>> Project.
>>>>> 
>>>>> 
>>>>> 
>>>>> On 4/4/02 7:36 AM, "Erik Hatcher" <li...@ehatchersolutions.com>
>>>>> wrote:
>>>>> 
>>>>>> How will committers to this new repository be selected?
>>>>>> 
>>>>>> I would like to have commit access to contribute my Lucene
>>> indexing
>>>>> Ant task
>>>>>> as well as possibly a webapp that interfaces with an index.  And
>>> I
>>>>> would, of
>>>>>> course, maintain as best I could.
>>>>>> 
>>>>>>   Erik
>>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> To unsubscribe, e-mail:
>>>>> <ma...@jakarta.apache.org>
>>>>> For additional commands, e-mail:
>>>>> <ma...@jakarta.apache.org>
>>>>> 
>>>> 
>>>> 
>>>> __________________________________________________
>>>> Do You Yahoo!?
>>>> Yahoo! Tax Center - online filing with TurboTax
>>>> http://taxes.yahoo.com/
>>>> 
>>>> --
>>>> To unsubscribe, e-mail:
>>> <ma...@jakarta.apache.org>
>>>> For additional commands, e-mail:
>>> <ma...@jakarta.apache.org>
>>>> 
>>>> 
>>> 
>>> 
>>> --
>>> To unsubscribe, e-mail:
>>> <ma...@jakarta.apache.org>
>>> For additional commands, e-mail:
>>> <ma...@jakarta.apache.org>
>>> 
>> 
>> 
>> __________________________________________________
>> Do You Yahoo!?
>> Yahoo! Games - play chess, backgammon, pool and more
>> http://games.yahoo.com/
>> 
>> --
>> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
>> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
> 
> 
> --
> To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
> 
> 


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [VOTE] Lucene Sandbox committer nomination

Posted by Eugene Gluzberg <dr...@yahoo.com>.

+1
Have fun
----- Original Message -----
From: "Otis Gospodnetic" <ot...@yahoo.com>
To: "Lucene Developers List" <lu...@jakarta.apache.org>
Sent: Wednesday, April 24, 2002 11:56 AM
Subject: Re: [VOTE] Lucene Sandbox committer nomination


> I'm using his contribution, so +1, if he wants it, of course.
>
> Otis
>
> --- Peter Carlson <ca...@bookandhammer.com> wrote:
> > I am nominating Kelvin Tan as a contributor to the Lucene-Sandbox
> > Project.
> >
> > He had already contributed lots of code the Lucene Project including
> > a
> > chainable filter and multiple field searching.
> >
> > I think he will be a great addition to the team.
> >
> > --Peter
> >
> >
> > On 4/19/02 8:13 AM, "Otis Gospodnetic" <ot...@yahoo.com>
> > wrote:
> >
> > > Of course.
> > >
> > > Otis
> > >
> > > --- Peter Carlson <ca...@bookandhammer.com> wrote:
> > >> I am nominating Eric Hatcher as a contributor to the
> > Lucene-Sandbox
> > >> Project.
> > >>
> > >>
> > >>
> > >> On 4/4/02 7:36 AM, "Erik Hatcher" <li...@ehatchersolutions.com>
> > >> wrote:
> > >>
> > >>> How will committers to this new repository be selected?
> > >>>
> > >>> I would like to have commit access to contribute my Lucene
> > indexing
> > >> Ant task
> > >>> as well as possibly a webapp that interfaces with an index.  And
> > I
> > >> would, of
> > >>> course, maintain as best I could.
> > >>>
> > >>>   Erik
> > >>>
> > >>
> > >>
> > >> --
> > >> To unsubscribe, e-mail:
> > >> <ma...@jakarta.apache.org>
> > >> For additional commands, e-mail:
> > >> <ma...@jakarta.apache.org>
> > >>
> > >
> > >
> > > __________________________________________________
> > > Do You Yahoo!?
> > > Yahoo! Tax Center - online filing with TurboTax
> > > http://taxes.yahoo.com/
> > >
> > > --
> > > To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> > > For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> > >
> > >
> >
> >
> > --
> > To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> > For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> >
>
>
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Games - play chess, backgammon, pool and more
> http://games.yahoo.com/
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Web Crawler

Posted by Clemens Marschner <cm...@lanlab.de>.

Hi,

I have been writing a web crawler in Java for quite some time now. Since
Lucene doesn't contain one by itself, I wonder if you were interested in a
contribution within the Lucene project.

I would probably call it a 0.4. It has quite a modular design, it's
multithreaded and still pretty simple.

And it's optimized for speed. I spent some time with a profiler to get the
beast FAST and memory consumption low. It contains an optimized HTML parser
that just extracts the necessary information and doesn't waste time nor
objects.

I was able to get a maximum of 3.7 MB/sec on a 100MBit line and a MAN-style
network (a University campus with about 150 web servers).

Its only purpose is to crawl documents and links and store them somewhere.
Nothing is done with the documents (though it would be easy to incorporate
any computation steps, but this would probably shift the balance between IO
and CPU usage until one of them becomes a bottleneck). Any connection to the
Lucene engine has yet to be provided.

I have also made a lot of optimizations on RAM usage, but still some data
structures are kept in main memory (notably the hash of visited URLs),
limiting the number of files that can be crawled.

Since it's not a production release yet, it still has some limitations. Some
work still has to be done, I still have a lot of ideas, and pretty much of
the configuration is still made in the Java source code (well, at least,
most of it is concentrated in the main() method). Since I just used it for
myself, this was fine so far.

Cheers,

Clemens Marschner


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: [VOTE] Lucene Sandbox committer nomination

Posted by Otis Gospodnetic <ot...@yahoo.com>.

I'm using his contribution, so +1, if he wants it, of course.

Otis

--- Peter Carlson <ca...@bookandhammer.com> wrote:
> I am nominating Kelvin Tan as a contributor to the Lucene-Sandbox
> Project.
> 
> He had already contributed lots of code the Lucene Project including
> a
> chainable filter and multiple field searching.
> 
> I think he will be a great addition to the team.
> 
> --Peter
> 
> 
> On 4/19/02 8:13 AM, "Otis Gospodnetic" <ot...@yahoo.com>
> wrote:
> 
> > Of course.
> > 
> > Otis
> > 
> > --- Peter Carlson <ca...@bookandhammer.com> wrote:
> >> I am nominating Eric Hatcher as a contributor to the
> Lucene-Sandbox
> >> Project.
> >> 
> >> 
> >> 
> >> On 4/4/02 7:36 AM, "Erik Hatcher" <li...@ehatchersolutions.com>
> >> wrote:
> >> 
> >>> How will committers to this new repository be selected?
> >>> 
> >>> I would like to have commit access to contribute my Lucene
> indexing
> >> Ant task
> >>> as well as possibly a webapp that interfaces with an index.  And
> I
> >> would, of
> >>> course, maintain as best I could.
> >>> 
> >>>   Erik
> >>> 
> >> 
> >> 
> >> --
> >> To unsubscribe, e-mail:
> >> <ma...@jakarta.apache.org>
> >> For additional commands, e-mail:
> >> <ma...@jakarta.apache.org>
> >> 
> > 
> > 
> > __________________________________________________
> > Do You Yahoo!?
> > Yahoo! Tax Center - online filing with TurboTax
> > http://taxes.yahoo.com/
> > 
> > --
> > To unsubscribe, e-mail:  
> <ma...@jakarta.apache.org>
> > For additional commands, e-mail:
> <ma...@jakarta.apache.org>
> > 
> > 
> 
> 
> --
> To unsubscribe, e-mail:  
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
> 


__________________________________________________
Do You Yahoo!?
Yahoo! Games - play chess, backgammon, pool and more
http://games.yahoo.com/

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>