You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "xiangfu0 (via GitHub)" <gi...@apache.org> on 2026/04/22 16:31:19 UTC
[PR] Add SOUNDEX and DIFFERENCE scalar functions for phonetic string matching [pinot]
xiangfu0 opened a new pull request, #18292:
URL: https://github.com/apache/pinot/pull/18292
## Summary
- Adds `SOUNDEX(str)` scalar function: returns the 4-character Soundex code for a string (e.g. `SOUNDEX('Robert')` → `'R163'`). Null input returns null; empty string returns `'0000'` per SQL standard.
- Adds `DIFFERENCE(str1, str2)` scalar function: returns an integer 0–4 indicating phonetic similarity based on Soundex codes (4 = identical codes, 0 = no common positions). Null inputs are null-propagated by the framework.
- Both functions are backed by `org.apache.commons.codec.language.Soundex` (Apache Commons Codec, already a Pinot dependency — no new dep added).
- Follows existing `StringFunctions` conventions: `@ScalarFunction`, `nullableParameters = true` on `soundex`, primitive `int` return on `difference` consistent with `levenshteinDistance` / `hammingDistance`.
## Test plan
- [ ] `testSoundex()`: classic Soundex examples (Robert/Rupert → `R163`, Ashcraft → `A261`), empty string → `'0000'`, null → null
- [ ] `testDifference()`: identical codes (4), unrelated names (1), same string (4), empty-string pair (4), mixed empty (0)
- [ ] `./mvnw -pl pinot-common -Dtest=StringFunctionsTest test` passes
## Notes
- `SOUNDEX('')` returns `'0000'` rather than `''` to align with SQL Server / PostgreSQL / ANSI SQL standard behaviour.
- `difference()` declares a `catch (EncoderException)` block because the underlying `Soundex.difference(String, String)` carries the checked exception on its signature, but the exception is never thrown in practice when using a typed `Soundex` instance.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Add SOUNDEX and DIFFERENCE scalar functions for phonetic string matching [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18292:
URL: https://github.com/apache/pinot/pull/18292#issuecomment-4298585501
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/18292?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `72.97794%` with `294 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 35.27%. Comparing base ([`5d48eeb`](https://app.codecov.io/gh/apache/pinot/commit/5d48eeb3ea8354ba2a4295460cf107d66316130d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f806254`](https://app.codecov.io/gh/apache/pinot/commit/f806254da917de55f32d1205243c3c93f037619c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 9 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/pinot/pull/18292?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/pinot/sql/ddl/compile/PropertyMapping.java](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&filepath=pinot-sql-ddl%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fsql%2Fddl%2Fcompile%2FPropertyMapping.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3Qtc3FsLWRkbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc3FsL2RkbC9jb21waWxlL1Byb3BlcnR5TWFwcGluZy5qYXZh) | 65.35% | [71 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...troller/api/resources/PinotDdlRestletResource.java](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&filepath=pinot-controller%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcontroller%2Fapi%2Fresources%2FPinotDdlRestletResource.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1Bpbm90RGRsUmVzdGxldFJlc291cmNlLmphdmE=) | 70.05% | [37 Missing and 16 partials :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/pinot/sql/ddl/reverse/PropertyExtractor.java](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&filepath=pinot-sql-ddl%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fsql%2Fddl%2Freverse%2FPropertyExtractor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3Qtc3FsLWRkbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc3FsL2RkbC9yZXZlcnNlL1Byb3BlcnR5RXh0cmFjdG9yLmphdmE=) | 79.33% | [10 Missing and 15 partials :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../pinot/sql/parsers/parser/SqlPinotCreateTable.java](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&filepath=pinot-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fsql%2Fparsers%2Fparser%2FSqlPinotCreateTable.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9wYXJzZXIvU3FsUGlub3RDcmVhdGVUYWJsZS5qYXZh) | 35.13% | [24 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../org/apache/pinot/sql/ddl/compile/DdlCompiler.java](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&filepath=pinot-sql-ddl%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fsql%2Fddl%2Fcompile%2FDdlCompiler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3Qtc3FsLWRkbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc3FsL2RkbC9jb21waWxlL0RkbENvbXBpbGVyLmphdmE=) | 84.35% | [13 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../sql/parsers/parser/SqlPinotColumnDeclaration.java](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&filepath=pinot-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fsql%2Fparsers%2Fparser%2FSqlPinotColumnDeclaration.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9wYXJzZXIvU3FsUGlub3RDb2x1bW5EZWNsYXJhdGlvbi5qYXZh) | 48.57% | [18 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...he/pinot/sql/parsers/parser/SqlPinotDropTable.java](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&filepath=pinot-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fsql%2Fparsers%2Fparser%2FSqlPinotDropTable.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9wYXJzZXIvU3FsUGlub3REcm9wVGFibGUuamF2YQ==) | 47.36% | [10 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...rg/apache/pinot/sql/ddl/reverse/SchemaEmitter.java](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&filepath=pinot-sql-ddl%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fsql%2Fddl%2Freverse%2FSchemaEmitter.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3Qtc3FsLWRkbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc3FsL2RkbC9yZXZlcnNlL1NjaGVtYUVtaXR0ZXIuamF2YQ==) | 78.26% | [7 Missing and 3 partials :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../pinot/common/function/scalar/StringFunctions.java](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&filepath=pinot-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcommon%2Ffunction%2Fscalar%2FStringFunctions.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | 11.11% | [8 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...che/pinot/sql/parsers/parser/SqlPinotProperty.java](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&filepath=pinot-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fsql%2Fparsers%2Fparser%2FSqlPinotProperty.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9wYXJzZXIvU3FsUGlub3RQcm9wZXJ0eS5qYXZh) | 46.66% | [8 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| ... and [11 more](https://app.codecov.io/gh/apache/pinot/pull/18292?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
> :exclamation: There is a different number of reports uploaded between BASE (5d48eeb) and HEAD (f806254). Click for more details.
>
> <details><summary>HEAD has 36 uploads less than BASE</summary>
>
>| Flag | BASE (5d48eeb) | HEAD (f806254) |
>|------|------|------|
>|java-21|5|1|
>|unittests1|2|0|
>|unittests|4|1|
>|temurin|10|1|
>|java-11|5|0|
>|unittests2|2|1|
>|integration|6|0|
>|integration2|2|0|
>|integration1|2|0|
>|custom-integration1|2|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18292 +/- ##
=============================================
- Coverage 63.59% 35.27% -28.32%
+ Complexity 1658 1011 -647
=============================================
Files 3243 3270 +27
Lines 197343 198516 +1173
Branches 30547 30688 +141
=============================================
- Hits 125494 70024 -55470
- Misses 61814 122357 +60543
+ Partials 10035 6135 -3900
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `35.27% <72.97%> (-28.30%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `35.27% <72.97%> (-28.32%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `35.27% <72.97%> (-28.32%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/18292/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `35.27% <72.97%> (+0.21%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/18292?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Add SOUNDEX and DIFFERENCE scalar functions for phonetic string matching [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on PR #18292:
URL: https://github.com/apache/pinot/pull/18292#issuecomment-4298052156
Closing in favour of a cleaner branch that contains only the SOUNDEX/DIFFERENCE change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Add SOUNDEX and DIFFERENCE scalar functions for phonetic string matching [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 closed pull request #18292: Add SOUNDEX and DIFFERENCE scalar functions for phonetic string matching
URL: https://github.com/apache/pinot/pull/18292
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org