You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Mark Miller <ma...@gmail.com> on 2009/08/20 17:40:06 UTC

TokenStream JavaDoc

As a new user, reading the TokenStream javadoc is a bit intimidating I
think.

One thing that might help I think, is adding just a bit of example code
to the following section:

The workflow of the new TokenStream API is as follows:

1. Instantiation of TokenStream/TokenFilters which add/get attributes
to/from the AttributeSource.
2. The consumer calls reset().
3. the consumer retrieves attributes from the stream and stores local
references to all attributes it wants to access
4. The consumer calls incrementToken() until it returns false and
consumes the attributes after each call.

-- 
- Mark

http://www.lucidimagination.com




---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Re: TokenStream JavaDoc

Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Thu, Aug 20, 2009 at 11:44 AM, Tim Smith<ts...@attivio.com> wrote:
> Also, the javadoc should also mark that the consumer call end() as step
> 5 (don't think this was added when end() was put in)

And close()....
is there a way these could be consolidated?

-Yonik
http://www.lucidimagination.com


>  -- Tim
>
> Mark Miller wrote:
>> As a new user, reading the TokenStream javadoc is a bit intimidating I
>> think.
>>
>> One thing that might help I think, is adding just a bit of example code
>> to the following section:
>>
>> The workflow of the new TokenStream API is as follows:
>>
>> 1. Instantiation of TokenStream/TokenFilters which add/get attributes
>> to/from the AttributeSource.
>> 2. The consumer calls reset().
>> 3. the consumer retrieves attributes from the stream and stores local
>> references to all attributes it wants to access
>> 4. The consumer calls incrementToken() until it returns false and
>> consumes the attributes after each call.
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Re: TokenStream JavaDoc

Posted by Tim Smith <ts...@attivio.com>.
Also, the javadoc should also mark that the consumer call end() as step
5 (don't think this was added when end() was put in)

 -- Tim

Mark Miller wrote:
> As a new user, reading the TokenStream javadoc is a bit intimidating I
> think.
>
> One thing that might help I think, is adding just a bit of example code
> to the following section:
>
> The workflow of the new TokenStream API is as follows:
>
> 1. Instantiation of TokenStream/TokenFilters which add/get attributes
> to/from the AttributeSource.
> 2. The consumer calls reset().
> 3. the consumer retrieves attributes from the stream and stores local
> references to all attributes it wants to access
> 4. The consumer calls incrementToken() until it returns false and
> consumes the attributes after each call.
>
>   


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org