Custom TokenStream + custom Attributes

[Michal Krajňanský <mi...@gmail.com>]

Dear Lucene users,

I have implemented a custom tokenizer (derived from TokenStream).

I need to pass additional attributes to those standard in Lucene
(PositionIncrementAttribute, OffsetAttribute), that would represent the
word position in the tokenized sentence in the number of words and not
characters, as one usually passes through OffsetAttribute. (I need both.)

Is there a way of achieving this?

I tried to implement own Attribute class (derive a new interface and
implementing class). The code compiles ok but I am getting exception at
runtime about the class casting.

Thank you a lot in advance,


MK

Re: Custom TokenStream + custom Attributes

[Tomoko Uchida <to...@gmail.com>]

Hi,

I do not fully understand your requirements, but analyzers-kuromoji
(one of extended package for Japanese morphological analysis) has some
custom token attributes.
http://lucene.apache.org/core/6_0_1/analyzers-kuromoji/index.html?org/apache/lucene/analysis/ja/tokenattributes/package-summary.html

The implementation might be a good reference.

Hope that helps,
Tomoko

2016-06-08 20:44 GMT+09:00 Michal Krajňanský <mi...@gmail.com>:
> Dear Lucene users,
>
> I have implemented a custom tokenizer (derived from TokenStream).
>
> I need to pass additional attributes to those standard in Lucene
> (PositionIncrementAttribute, OffsetAttribute), that would represent the
> word position in the tokenized sentence in the number of words and not
> characters, as one usually passes through OffsetAttribute. (I need both.)
>
> Is there a way of achieving this?
>
> I tried to implement own Attribute class (derive a new interface and
> implementing class). The code compiles ok but I am getting exception at
> runtime about the class casting.
>
> Thank you a lot in advance,
>
>
> MK

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org