You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Oliver Christ (JIRA)" <ji...@apache.org> on 2012/10/31 12:55:13 UTC
[jira] [Created] (LUCENE-4516) Suggesters: allow to associate a
user-specified key (int) with a string
Oliver Christ created LUCENE-4516:
-------------------------------------
Summary: Suggesters: allow to associate a user-specified key (int) with a string
Key: LUCENE-4516
URL: https://issues.apache.org/jira/browse/LUCENE-4516
Project: Lucene - Core
Issue Type: New Feature
Components: core/FSTs
Reporter: Oliver Christ
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-4516) Suggesters: allow to associate a
user-specified key (int) with a string
Posted by "Oliver Christ (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oliver Christ updated LUCENE-4516:
----------------------------------
Priority: Minor (was: Major)
> Suggesters: allow to associate a user-specified key (int) with a string
> -----------------------------------------------------------------------
>
> Key: LUCENE-4516
> URL: https://issues.apache.org/jira/browse/LUCENE-4516
> Project: Lucene - Core
> Issue Type: New Feature
> Components: core/FSTs
> Reporter: Oliver Christ
> Priority: Minor
>
> As a user, I'd like to associate a “foreign key” with a string (rather: final node) in the suggester index (in addition to the rank). For example, I’d like to add “Lucene in Action” with key 1933988177 (the ISBN) and some rank to a WFST or AnalyzingSuggester. A completion would return the completed string and the key associated with each entry (i.e. final nodes get a “key” field (int), which is returned in the LookupResult). That foreign key could also be used for fast de-duping (no more string/byte array comparisons).
> There may be workarounds for the “foreign key” use case –it seems that lots of data structures would be affected by storing a user-provided key with final nodes, which therefore may not be a viable path. It may be possible to encode the foreign key in the transducer’s output instead.
> *Discussion on java-user@lucene:*
> Mike McCandless:
> This is maybe the same idea as
> LUCENE-4491 ? Could you simply stuff your ISBN onto the end of the suggestion (ie enroll Lucene in
> Action|1933988177)?
> Dawid Weiss:
> Just remember that if your suffixes are unique then you'll be expanding the automaton quite a bit (unique suffix paths).
> D.
> Mike:
> That's a good point... encoding into the FST's output may be better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-4516) Suggesters: allow to associate a
user-specified key (int) with a string
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487703#comment-13487703 ]
Robert Muir commented on LUCENE-4516:
-------------------------------------
I don't think we should add an option to do this, instead you would just make your own suggester tailored to your use case?
For you it might be an int, for me maybe i have 6 long values, for another guy 2 bytes, and so on.
So I think someone should just fork one of our suggester impls and e.g. change the Outputs impl to suit their needs.
> Suggesters: allow to associate a user-specified key (int) with a string
> -----------------------------------------------------------------------
>
> Key: LUCENE-4516
> URL: https://issues.apache.org/jira/browse/LUCENE-4516
> Project: Lucene - Core
> Issue Type: New Feature
> Components: core/FSTs
> Reporter: Oliver Christ
> Priority: Minor
>
> As a user, I'd like to associate a “foreign key” with a string (rather: final node) in the suggester index (in addition to the rank). For example, I’d like to add “Lucene in Action” with key 1933988177 (the ISBN) and some rank to a WFST or AnalyzingSuggester. A completion would return the completed string and the key associated with each entry (i.e. final nodes get a “key” field (int), which is returned in the LookupResult). That foreign key could also be used for fast de-duping (no more string/byte array comparisons).
> There may be workarounds for the “foreign key” use case –it seems that lots of data structures would be affected by storing a user-provided key with final nodes, which therefore may not be a viable path. It may be possible to encode the foreign key in the transducer’s output instead.
> *Discussion on java-user@lucene:*
> Mike McCandless:
> This is maybe the same idea as
> LUCENE-4491 ? Could you simply stuff your ISBN onto the end of the suggestion (ie enroll Lucene in
> Action|1933988177)?
> Dawid Weiss:
> Just remember that if your suffixes are unique then you'll be expanding the automaton quite a bit (unique suffix paths).
> D.
> Mike:
> That's a good point... encoding into the FST's output may be better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-4516) Suggesters: allow to associate a
user-specified key (int) with a string
Posted by "Oliver Christ (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oliver Christ updated LUCENE-4516:
----------------------------------
Description:
As a user, I'd like to associate a “foreign key” with a string (rather: final node) in the suggester index (in addition to the rank). For example, I’d like to add “Lucene in Action” with key 1933988177 (the ISBN) and some rank to a WFST or AnalyzingSuggester. A completion would return the completed string and the key associated with each entry (i.e. final nodes get a “key” field (int), which is returned in the LookupResult). That foreign key could also be used for fast de-duping (no more string/byte array comparisons).
There may be workarounds for the “foreign key” use case –it seems that lots of data structures would be affected by storing a user-provided key with final nodes, which therefore may not be a viable path. It may be possible to encode the foreign key in the transducer’s output instead.
*Discussion on java-user@lucene:*
Mike McCandless:
This is maybe the same idea as
LUCENE-4491 ? Could you simply stuff your ISBN onto the end of the suggestion (ie enroll Lucene in
Action|1933988177)?
Dawid Weiss:
Just remember that if your suffixes are unique then you'll be expanding the automaton quite a bit (unique suffix paths).
D.
Mike:
That's a good point... encoding into the FST's output may be better.
> Suggesters: allow to associate a user-specified key (int) with a string
> -----------------------------------------------------------------------
>
> Key: LUCENE-4516
> URL: https://issues.apache.org/jira/browse/LUCENE-4516
> Project: Lucene - Core
> Issue Type: New Feature
> Components: core/FSTs
> Reporter: Oliver Christ
>
> As a user, I'd like to associate a “foreign key” with a string (rather: final node) in the suggester index (in addition to the rank). For example, I’d like to add “Lucene in Action” with key 1933988177 (the ISBN) and some rank to a WFST or AnalyzingSuggester. A completion would return the completed string and the key associated with each entry (i.e. final nodes get a “key” field (int), which is returned in the LookupResult). That foreign key could also be used for fast de-duping (no more string/byte array comparisons).
> There may be workarounds for the “foreign key” use case –it seems that lots of data structures would be affected by storing a user-provided key with final nodes, which therefore may not be a viable path. It may be possible to encode the foreign key in the transducer’s output instead.
> *Discussion on java-user@lucene:*
> Mike McCandless:
> This is maybe the same idea as
> LUCENE-4491 ? Could you simply stuff your ISBN onto the end of the suggestion (ie enroll Lucene in
> Action|1933988177)?
> Dawid Weiss:
> Just remember that if your suffixes are unique then you'll be expanding the automaton quite a bit (unique suffix paths).
> D.
> Mike:
> That's a good point... encoding into the FST's output may be better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-4516) Suggesters: allow to associate a
user-specified key (int) with a string
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487766#comment-13487766 ]
Simon Willnauer commented on LUCENE-4516:
-----------------------------------------
I agree with robert that this is maybe tailored. In general I think with the ideas in LUCENE-4491 we can make a good step forward to this and maybe enable it in a specialized suggester.
> Suggesters: allow to associate a user-specified key (int) with a string
> -----------------------------------------------------------------------
>
> Key: LUCENE-4516
> URL: https://issues.apache.org/jira/browse/LUCENE-4516
> Project: Lucene - Core
> Issue Type: New Feature
> Components: core/FSTs
> Reporter: Oliver Christ
> Priority: Minor
>
> As a user, I'd like to associate a “foreign key” with a string (rather: final node) in the suggester index (in addition to the rank). For example, I’d like to add “Lucene in Action” with key 1933988177 (the ISBN) and some rank to a WFST or AnalyzingSuggester. A completion would return the completed string and the key associated with each entry (i.e. final nodes get a “key” field (int), which is returned in the LookupResult). That foreign key could also be used for fast de-duping (no more string/byte array comparisons).
> There may be workarounds for the “foreign key” use case –it seems that lots of data structures would be affected by storing a user-provided key with final nodes, which therefore may not be a viable path. It may be possible to encode the foreign key in the transducer’s output instead.
> *Discussion on java-user@lucene:*
> Mike McCandless:
> This is maybe the same idea as
> LUCENE-4491 ? Could you simply stuff your ISBN onto the end of the suggestion (ie enroll Lucene in
> Action|1933988177)?
> Dawid Weiss:
> Just remember that if your suffixes are unique then you'll be expanding the automaton quite a bit (unique suffix paths).
> D.
> Mike:
> That's a good point... encoding into the FST's output may be better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org