You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "abhioncbr (via GitHub)" <gi...@apache.org> on 2023/05/14 02:53:15 UTC
[GitHub] [pinot] abhioncbr opened a new pull request, #10765: [multistage]: Added support for Json Data Type.
abhioncbr opened a new pull request, #10765:
URL: https://github.com/apache/pinot/pull/10765
- Added initial support for the `json` data type in multistage.
cc @walterddr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1195857950
##########
pinot-query-runtime/src/test/resources/queries/JsonType.json:
##########
@@ -0,0 +1,41 @@
+{
+ "json_data": {
+ "tables": {
+ "jsonTbl": {
+ "schema": [
+ {"name": "jsonCol", "type": "JSON"},
+ {"name": "stringCol", "type": "STRING"}
+ ],
+ "inputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ }
+ },
+ "queries": [
+ {
+ "sql": "SELECT jsonCol FROM {jsonTbl}",
+ "outputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}"]
+ ]
+ },
+ {
+ "sql": "SELECT jsonCol, stringCol FROM {jsonTbl}",
+ "outputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ },
+ {
+ "sql": "SELECT jsonCol, stringCol FROM {jsonTbl} where stringCol='str33'",
+ "outputs": [
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
Review Comment:
I added more tests.
I skipped the `select * ` because I am not sure how to skip the `ts` column. The other one selecting both `json` and `string` columns is already there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on PR #10765:
URL: https://github.com/apache/pinot/pull/10765#issuecomment-1548862176
@walterddr Updated as per the comments. Please have a look. thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #10765:
URL: https://github.com/apache/pinot/pull/10765#issuecomment-1546797795
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/10765?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#10765](https://app.codecov.io/gh/apache/pinot/pull/10765?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (3018263) into [master](https://app.codecov.io/gh/apache/pinot/commit/0912544cce9307462c5931abdc61131ec2d50bfa?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (0912544) will **decrease** coverage by `56.68%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #10765 +/- ##
=============================================
- Coverage 70.38% 13.71% -56.68%
+ Complexity 6467 439 -6028
=============================================
Files 2152 2098 -54
Lines 115689 113207 -2482
Branches 17471 17174 -297
=============================================
- Hits 81433 15523 -65910
- Misses 28603 96407 +67804
+ Partials 5653 1277 -4376
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `13.71% <0.00%> (-0.06%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://app.codecov.io/gh/apache/pinot/pull/10765?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../java/org/apache/pinot/query/type/TypeFactory.java](https://app.codecov.io/gh/apache/pinot/pull/10765?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcXVlcnktcGxhbm5lci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcXVlcnkvdHlwZS9UeXBlRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (-70.38%)` | :arrow_down: |
... and [1701 files with indirect coverage changes](https://app.codecov.io/gh/apache/pinot/pull/10765/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1194104653
##########
pinot-query-runtime/src/test/resources/queries/JsonType.json:
##########
@@ -0,0 +1,22 @@
+{
+ "json_data": {
+ "tables": {
+ "jsonTbl": {
+ "schema": [
+ {"name": "jsonCol", "type": "JSON"},
+ {"name": "stringCol", "type": "STRING"}
+ ],
+ "inputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ }
+ },
+ "queries": [
+ {"sql": "SELECT jsonCol FROM {jsonTbl}"},
+ {"sql": "SELECT jsonCol, stringCol FROM {jsonTbl}"},
+ {"sql": "SELECT jsonCol, stringCol FROM {jsonTbl} where stringCol='str33'"}
Review Comment:
yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1194103588
##########
pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/QueryRunnerTestBase.java:
##########
@@ -156,7 +157,11 @@ protected List<Object[]> queryH2(String sql)
while (h2ResultSet.next()) {
Object[] row = new Object[columnCount];
for (int i = 0; i < columnCount; i++) {
- row[i] = h2ResultSet.getObject(i + 1);
+ if (h2ResultSet.getMetaData().getColumnType(i + 1) == Types.OTHER) {
+ row[i] = h2ResultSet.getString(i + 1);
Review Comment:
H2 has the support [support](https://www.h2database.com/html/datatypes.html#json_type), but we can not test transformation functions with H2. I'll use your suggested one. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1195307453
##########
pinot-query-runtime/src/test/resources/queries/JsonType.json:
##########
@@ -0,0 +1,41 @@
+{
+ "json_data": {
+ "tables": {
+ "jsonTbl": {
+ "schema": [
+ {"name": "jsonCol", "type": "JSON"},
+ {"name": "stringCol", "type": "STRING"}
+ ],
+ "inputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ }
+ },
+ "queries": [
+ {
+ "sql": "SELECT jsonCol FROM {jsonTbl}",
+ "outputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}"]
+ ]
+ },
+ {
+ "sql": "SELECT jsonCol, stringCol FROM {jsonTbl}",
+ "outputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ },
+ {
+ "sql": "SELECT jsonCol, stringCol FROM {jsonTbl} where stringCol='str33'",
+ "outputs": [
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
Review Comment:
can you also add
- `SELECT stringCol FROM {jsonTbl} where jsonCol = '<escaped json str>'` to check the equal comparison?
- `SELECT ... where jsonCol LIKE '%val222%'` for string function apply to json columns?
- `SELECT * `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1194102710
##########
pinot-query-runtime/src/test/resources/queries/JsonType.json:
##########
@@ -0,0 +1,22 @@
+{
+ "json_data": {
+ "tables": {
+ "jsonTbl": {
+ "schema": [
+ {"name": "jsonCol", "type": "JSON"},
+ {"name": "stringCol", "type": "STRING"}
+ ],
+ "inputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ }
+ },
+ "queries": [
+ {"sql": "SELECT jsonCol FROM {jsonTbl}"},
+ {"sql": "SELECT jsonCol, stringCol FROM {jsonTbl}"},
+ {"sql": "SELECT jsonCol, stringCol FROM {jsonTbl} where stringCol='str33'"}
Review Comment:
ah. i see so this is only to support JSON as string return? got it. that's good in its own.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1195857950
##########
pinot-query-runtime/src/test/resources/queries/JsonType.json:
##########
@@ -0,0 +1,41 @@
+{
+ "json_data": {
+ "tables": {
+ "jsonTbl": {
+ "schema": [
+ {"name": "jsonCol", "type": "JSON"},
+ {"name": "stringCol", "type": "STRING"}
+ ],
+ "inputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ }
+ },
+ "queries": [
+ {
+ "sql": "SELECT jsonCol FROM {jsonTbl}",
+ "outputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}"]
+ ]
+ },
+ {
+ "sql": "SELECT jsonCol, stringCol FROM {jsonTbl}",
+ "outputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ },
+ {
+ "sql": "SELECT jsonCol, stringCol FROM {jsonTbl} where stringCol='str33'",
+ "outputs": [
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
Review Comment:
I added more tests.
I skipped the `select * ` because I am not sure how to skip the `ts` column from the output. The other one selecting both `json` and `string` columns is already there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1193065738
##########
pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/QueryRunnerTestBase.java:
##########
@@ -386,14 +391,22 @@ protected void addTableToH2(String tableName, Schema schema)
h2FieldNamesAndTypes.toArray(new String[h2FieldNamesAndTypes.size()])) + ")").execute();
}
+ private String getH2InsertQueryParam(String h2FieldNameAndType) {
+ String type = h2FieldNameAndType.split(" ")[1];
+ if ("json".equalsIgnoreCase(type)) {
+ return "? FORMAT JSON";
Review Comment:
This is needed for the comparison with Pinot rows. As per the [documentation](https://www.h2database.com/html/datatypes.html#json_type), this will save the data in `java.lang.String` format in H2 db.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1193066014
##########
pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/QueryRunnerTestBase.java:
##########
@@ -156,7 +157,11 @@ protected List<Object[]> queryH2(String sql)
while (h2ResultSet.next()) {
Object[] row = new Object[columnCount];
for (int i = 0; i < columnCount; i++) {
- row[i] = h2ResultSet.getObject(i + 1);
+ if (h2ResultSet.getMetaData().getColumnType(i + 1) == Types.OTHER) {
+ row[i] = h2ResultSet.getString(i + 1);
Review Comment:
This invokes the [method](http://javadox.com/com.h2database/h2/1.4.200/org/h2/value/ValueJson.html#getString()) to get the correct String data for `json` column data type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1193066014
##########
pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/QueryRunnerTestBase.java:
##########
@@ -156,7 +157,11 @@ protected List<Object[]> queryH2(String sql)
while (h2ResultSet.next()) {
Object[] row = new Object[columnCount];
for (int i = 0; i < columnCount; i++) {
- row[i] = h2ResultSet.getObject(i + 1);
+ if (h2ResultSet.getMetaData().getColumnType(i + 1) == Types.OTHER) {
+ row[i] = h2ResultSet.getString(i + 1);
Review Comment:
This invokes the [method](http://javadox.com/com.h2database/h2/1.4.200/org/h2/value/ValueJson.html#getString()) to get the correct String data for `json` column data type from H2 db.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1195307453
##########
pinot-query-runtime/src/test/resources/queries/JsonType.json:
##########
@@ -0,0 +1,41 @@
+{
+ "json_data": {
+ "tables": {
+ "jsonTbl": {
+ "schema": [
+ {"name": "jsonCol", "type": "JSON"},
+ {"name": "stringCol", "type": "STRING"}
+ ],
+ "inputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ }
+ },
+ "queries": [
+ {
+ "sql": "SELECT jsonCol FROM {jsonTbl}",
+ "outputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}"]
+ ]
+ },
+ {
+ "sql": "SELECT jsonCol, stringCol FROM {jsonTbl}",
+ "outputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ },
+ {
+ "sql": "SELECT jsonCol, stringCol FROM {jsonTbl} where stringCol='str33'",
+ "outputs": [
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
Review Comment:
can you also add
- `SELECT stringCol FROM {jsonTbl} where jsonCol = '<escaped json str>'` to check the equal comparison?
- `SELECT * `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1194101025
##########
pinot-query-runtime/src/test/resources/queries/JsonType.json:
##########
@@ -0,0 +1,22 @@
+{
+ "json_data": {
+ "tables": {
+ "jsonTbl": {
+ "schema": [
+ {"name": "jsonCol", "type": "JSON"},
+ {"name": "stringCol", "type": "STRING"}
+ ],
+ "inputs": [
+ ["{\"key1\":\"val1\",\"key2\":\"val2\"}", "str1"],
+ ["{\"key11\":\"val11\",\"key22\":\"val22\"}", "str22"],
+ ["{\"key111\":\"val111\",\"key222\":{\"key222_a\":\"val222_a\"}}", "str33"]
+ ]
+ }
+ },
+ "queries": [
+ {"sql": "SELECT jsonCol FROM {jsonTbl}"},
+ {"sql": "SELECT jsonCol, stringCol FROM {jsonTbl}"},
+ {"sql": "SELECT jsonCol, stringCol FROM {jsonTbl} where stringCol='str33'"}
Review Comment:
I have a separate [issue](https://github.com/apache/pinot/issues/10767) for that. Would you like me to add support for those in this PR or a separate one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] abhioncbr commented on a diff in pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "abhioncbr (via GitHub)" <gi...@apache.org>.
abhioncbr commented on code in PR #10765:
URL: https://github.com/apache/pinot/pull/10765#discussion_r1194103588
##########
pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/QueryRunnerTestBase.java:
##########
@@ -156,7 +157,11 @@ protected List<Object[]> queryH2(String sql)
while (h2ResultSet.next()) {
Object[] row = new Object[columnCount];
for (int i = 0; i < columnCount; i++) {
- row[i] = h2ResultSet.getObject(i + 1);
+ if (h2ResultSet.getMetaData().getColumnType(i + 1) == Types.OTHER) {
+ row[i] = h2ResultSet.getString(i + 1);
Review Comment:
H2 has the [json support](https://www.h2database.com/html/datatypes.html#json_type), but we can not test transformation functions with H2. I'll use your suggested one. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr merged pull request #10765: [multistage]: Added support for Json Data Type.
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr merged PR #10765:
URL: https://github.com/apache/pinot/pull/10765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org