You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "jackluo923 (via GitHub)" <gi...@apache.org> on 2023/06/07 16:55:28 UTC
[GitHub] [pinot] jackluo923 opened a new issue, #10863: Text-index does not support multi-token substring search where first and last tokens are partial
jackluo923 opened a new issue, #10863:
URL: https://github.com/apache/pinot/issues/10863
We've seen many cases where a user wants to search for a substring in a field with a text index. If all of the tokens in the query are complete words, we can directly use a phrase search:
`SELECT * FROM table WHERE text_match("col", '"substring match query"')`
However, if the first or last token is a partial word (e.g., `"string match que"`), the query will not return any results.
To work around the limitation, we can use this a query like this:
`SELECT * FROM table WHERE text_match("col", '".*string" AND match AND "que*"') AND "col" LIKE "%string match que%"`
But this is very slow and computationally expensive due to the LIKE which is necessary for validating the order of the tokens.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Text-index does not support multi-token substring search where first and last tokens are partial [pinot]
Posted by "chenboat (via GitHub)" <gi...@apache.org>.
chenboat commented on issue #10863:
URL: https://github.com/apache/pinot/issues/10863#issuecomment-2171881438
#12680 enabled the support for wildcard (including prefix and suffix) matching for the terms in a phrase search. @hpvd you can follow the https://docs.pinot.apache.org/basics/indexing/text-search-support#phrase-search-with-wildcard-term-matching to test the feature. We tested it with the default analyzer with Pinot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Text-index does not support multi-token substring search where first and last tokens are partial [pinot]
Posted by "hpvd (via GitHub)" <gi...@apache.org>.
hpvd commented on issue #10863:
URL: https://github.com/apache/pinot/issues/10863#issuecomment-2170730801
@chenboat does this merged feature solves this issue for you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Text-index does not support multi-token substring search where first and last tokens are partial [pinot]
Posted by "jackluo923 (via GitHub)" <gi...@apache.org>.
jackluo923 commented on issue #10863:
URL: https://github.com/apache/pinot/issues/10863#issuecomment-2171361952
@hpvd We'll likely back out the merged [feature](https://github.com/apache/pinot/pull/12680) and enable a similar feature via run-time configurable analyzer & query parser, as proposed in [this PR](https://github.com/apache/pinot/pull/13003). There are a couple of reasons for this:
1. There are some minor bugs and limitations with PR #12680 that cannot be fixed.
2. In production, there are many cases where the analyzer needs to be adjusted, and a custom query parser needs to accompany it.
We have been using PR#13003 in production over petabytes of data without issues and will work towards closing the PR, and this issue soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Text-index does not support multi-token substring search where first and last tokens are partial [pinot]
Posted by "hpvd (via GitHub)" <gi...@apache.org>.
hpvd commented on issue #10863:
URL: https://github.com/apache/pinot/issues/10863#issuecomment-2171363381
awsome! Many thanks for the detailed description of the current state and experience!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] chenboat commented on issue #10863: Text-index does not support multi-token substring search where first and last tokens are partial
Posted by "chenboat (via GitHub)" <gi...@apache.org>.
chenboat commented on issue #10863:
URL: https://github.com/apache/pinot/issues/10863#issuecomment-1581246120
This stackoverflow question suggests ElasticSearch (built on top of Lucene) supports similar substring search feature. https://stackoverflow.com/questions/44791075/in-elasticsearch-how-do-i-search-for-an-arbitrary-substring
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Text-index does not support multi-token substring search where first and last tokens are partial [pinot]
Posted by "github-actions[bot] (via GitHub) via commits" <co...@pinot.apache.org>.
github-actions[bot] commented on issue #10863:
URL: https://github.com/apache/pinot/issues/10863#issuecomment-6551076027
This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org