You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "jadami10 (via GitHub)" <gi...@apache.org> on 2025/06/05 20:32:37 UTC
[I] Combing a non-literal, arbitrarily sized list of strings [pinot]
jadami10 opened a new issue, #16022:
URL: https://github.com/apache/pinot/issues/16022
We have a query like below
```
SELECT
ARRAYTOSTRING(
ARRAY [
CONCAT('key_1:', json_extract_scalar(my_json, '$.key1', 'STRING', '')),
CONCAT('key_2:', json_extract_scalar(my_json, '$.key2', 'STRING', ''))
],
','
)
FROM table
```
under the hood, this fails because `ArrayLiteralTransformFunction only takes literals as arguments`. The issue is
- `ARRAYTOSTRING` only allows literal values
- `CONCAT` only allows 2 strings
So there is no way to actually support something like this without a bunch of nested `CONCAT` statements. I haven't worked too deeply in this part of the code before. Does it make more sense to
- support non-literal values for `ARRAY` (is this feasible?)
- add more iterations of `CONCAT` for more string options
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #16022: Combing a non-literal, arbitrarily sized list of strings
URL: https://github.com/apache/pinot/issues/16022
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "jadami10 (via GitHub)" <gi...@apache.org>.
jadami10 commented on issue #16022:
URL: https://github.com/apache/pinot/issues/16022#issuecomment-2952701687
Jackie, before we go solving a different problem, what would it take to undo the [existing](https://github.com/search?q=repo%3Aapache%2Fpinot+%22String+concat%22&type=code) concat implementation? Would we ever be comfortable turning `concat` strictly into the equivalent of `||` knowing it is a backwards incompat change? If we're going to ask if postgres supports array expressions, we should consider this as well.
But this also brings up, does it make sense to support `||` in Pinot?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "jadami10 (via GitHub)" <gi...@apache.org>.
jadami10 commented on issue #16022:
URL: https://github.com/apache/pinot/issues/16022#issuecomment-2971917237
I think the problem is https://github.com/apache/pinot/blob/ecba1cc2621bad58e59b3ff1b1c2f2fbc79c5b53/pinot-common/src/main/java/org/apache/pinot/common/function/scalar/string/StringFunctions.java#L36
it is not in line with any other database `concat` function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "jadami10 (via GitHub)" <gi...@apache.org>.
jadami10 commented on issue #16022:
URL: https://github.com/apache/pinot/issues/16022#issuecomment-2972006133
right, but I'm saying all other DBs do `concat(string1, ..., stringN) → varchar`.
so in postgres:
- concat("a", "b") -> "ab"
- concat("a", "b", "c") -> "abc"
- concat("a", "b", "c", "d") -> "abcd"
but in pinot
- concat("a", "b") -> "ab"
- concat("a", "b", "c") -> "acb"
- concat("a", "b", "c", "d") -> `Invalid number of arguments to function 'CONCAT'`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #16022:
URL: https://github.com/apache/pinot/issues/16022#issuecomment-2974752056
I see.. In this case we probably need to post a behavior change note and schedule it for a release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #16022:
URL: https://github.com/apache/pinot/issues/16022#issuecomment-2971946868
That is fine. Pinot will support both of them based on how many arguments are provided
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #16022:
URL: https://github.com/apache/pinot/issues/16022#issuecomment-2971874028
@jadami10 Here is the code from `org.apache.pinot.common.function.scalar.StringFunctions`:
```
/**
* Joins two input strings with no separator in between.
*/
@ScalarFunction
public static String concat(String input1, String input2) {
return input1 + input2;
}
```
So I believe Pinot supports both right now based on the argument count
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #16022:
URL: https://github.com/apache/pinot/issues/16022#issuecomment-4113208634
This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #16022:
URL: https://github.com/apache/pinot/issues/16022#issuecomment-2951318993
The problem for this is that `ArrayLiteralTransformFunction` only support literal (i.e. constant) as input, but not transform. @xiangfu0 Can you check if `ARRAY` can take expression in postgres? If so, we should modify `ArrayLiteralTransformFunction` to also support other `TransformFunction` as input
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "jadami10 (via GitHub)" <gi...@apache.org>.
jadami10 commented on issue #16022:
URL: https://github.com/apache/pinot/issues/16022#issuecomment-2946803873
maybe the problem is really [Pinot's concat function](https://docs.pinot.apache.org/functions-1/concat)? `CONCAT(col1, col2, seperator)` is different from other sql engine's i'm checking (postgres, sql server, trino/presto). `concat(string1, ..., stringN) → varchar` seems to be better?
though i don't know what a reasonable way to do this would be since you can't safely change pinot's behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Combing a non-literal, arbitrarily sized list of strings [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #16022:
URL: https://github.com/apache/pinot/issues/16022#issuecomment-4300860077
This issue has been closed because it has been inactive for 180 days. If you believe this issue is still relevant, please feel free to reopen it or create a new issue. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org