Write my Analyzer and Fillter, what would the dictionary-file-0 be like?

[Mengfei Ren <re...@gmail.com>]

Hi,

I'm try to write my own analyzer for Naive Bayes classification.
When I run the seq2sparse command and called -a MyANalyzer, The output
shows the algorithim used my filter and analyzer:

### Using length filter
### Using stopword fileter
### Using my filter
### Using my keyword filter


But if I check the dictionary-file-0 generated in this step, I  still find
all the tokens (not filtered!). Does that mean my analyzer didn't work, or
the dictionary-file-0 just record all tokens but the algorithm actually use
the filtered data?


Thanks

Mengfei Ren