You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/03/05 21:26:21 UTC

[GitHub] [incubator-pinot] amrishlal opened a new pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

amrishlal opened a new pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651


   ## Description
   JSON data is stored in columns of type STRING, so there is nothing preventing the column from storing bad json string. Bad JSON string in columns will cause json_extract_scalar to throw an exception which would terminate query processing. However, when json_extract_scalar is used within the WHERE clause, we should return the default value instead of throwing exception. This will allow the predicate to be evaluated to either true or false and hence allow the query to complete successfully. Returning default value from json_extract_scalar is an undocumented feature. Ideally, json_extract_scalar should return NULL in when it encounters bad JSON. However, NULL support is currently pending, so this is the best we can do.
   
   A good description should include pointers to an issue or design document, etc.
   ## Upgrade Notes
   Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)
   * [ ] Yes (Please label as **<code>backward-incompat</code>**, and complete the section below on Release Notes)
   
   Does this PR fix a zero-downtime upgrade introduced earlier?
   * [ ] Yes (Please label this as **<code>backward-incompat</code>**, and complete the section below on Release Notes)
   
   Does this PR otherwise need attention when creating release notes? Things to consider:
   - New configuration options
   - Deprecation of configurations
   - Signature changes to public methods/interfaces
   - New plugins added or old plugins removed
   * [ ] Yes (Please label this PR as **<code>release-notes</code>** and complete the section on Release Notes)
   ## Release Notes
   If you have tagged this as either backward-incompat or release-notes,
   you MUST add text here that you would like to see appear in release notes of the
   next release.
   
   If you have a series of commits adding or enabling a feature, then
   add this section only in final commit that marks the feature completed.
   Refer to earlier release notes to see examples of text
   
   ## Documentation
   If you have introduced a new feature or configuration, please add it to the documentation as well.
   See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] siddharthteotia merged pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

Posted by GitBox <gi...@apache.org>.
siddharthteotia merged pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] amrishlal commented on a change in pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651#discussion_r589764720



##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java
##########
@@ -282,6 +282,36 @@ public void testJsonMatchArrayWithIndex() {
     Assert.assertEquals(iterator.next()[0], "goofy");
   }
 
+  /** Evaluate json_extract_scalar over string column that does not contain valid json data. */
+  @Test
+  public void testJsonExtractScalarAgainstInvalidJson() {
+    // json_extract_scalar throws exception since we are trying to parse a non-JSON string.
+    Operator operator1 = getOperatorForSqlQuery(
+        "select count(*) FROM testTable WHERE json_extract_scalar(stringColumn, '$.name.first', 'INT') = 0");
+    try {
+      IntermediateResultsBlock block1 = (IntermediateResultsBlock) operator1.nextBlock();
+      Assert.assertTrue(false);
+    } catch (RuntimeException re) {
+      Assert.assertEquals(re.toString(),
+          "java.lang.RuntimeException: Illegal Json Path: [$.name.first], when reading [daffy duck]");
+    }
+
+    // JSON data is stored in columns of type STRING, so there is nothing preventing the column from storing bad json
+    // string. Bad JSON string in columns will cause json_extract_scalar to throw an exception which would terminate
+    // query processing. However, when json_extract_scalar is used within the WHERE clause, we should return the
+    // default value instead of throwing exception. This will allow the predicate to be evaluated to either true or
+    // false and hence allow the query to complete successfully. Returning default value from json_extract_scalar is
+    // an undocumented feature. Ideally, json_extract_scalar should return NULL in when it encounters bad JSON. However,
+    // NULL support is currently pending, so this is the best we can do.
+    Operator operator2 = getOperatorForSqlQuery(
+        "select count(*) FROM testTable WHERE json_extract_scalar(stringColumn, '$.name.first', 'INT', 0) = 0");
+
+    IntermediateResultsBlock block2 = (IntermediateResultsBlock) operator2.nextBlock();
+    Collection<Object[]> rows = block2.getSelectionResult();
+
+    Assert.assertEquals(block2.getAggregationResult().get(0), 9l);

Review comment:
       Done.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] amrishlal commented on a change in pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651#discussion_r589768294



##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java
##########
@@ -282,6 +282,36 @@ public void testJsonMatchArrayWithIndex() {
     Assert.assertEquals(iterator.next()[0], "goofy");
   }
 
+  /** Evaluate json_extract_scalar over string column that does not contain valid json data. */
+  @Test
+  public void testJsonExtractScalarAgainstInvalidJson() {
+    // json_extract_scalar throws exception since we are trying to parse a non-JSON string.
+    Operator operator1 = getOperatorForSqlQuery(
+        "select count(*) FROM testTable WHERE json_extract_scalar(stringColumn, '$.name.first', 'INT') = 0");
+    try {
+      IntermediateResultsBlock block1 = (IntermediateResultsBlock) operator1.nextBlock();
+      Assert.assertTrue(false);
+    } catch (RuntimeException re) {
+      Assert.assertEquals(re.toString(),

Review comment:
       Removed validation.

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/JsonExtractScalarTransformFunction.java
##########
@@ -131,7 +131,7 @@ public TransformResultMetadata getResultMetadata() {
     final String[] stringValuesSV = _jsonFieldTransformFunction.transformToStringValuesSV(projectionBlock);
     final int[] results = new int[projectionBlock.getNumDocs()];
     for (int i = 0; i < results.length; i++) {
-      Object read = JsonPath.read(stringValuesSV[i], _jsonPath);
+      Object read = JsonPath.using(JSON_PARSER_CONFIG).parse(stringValuesSV[i]).read(_jsonPath);

Review comment:
       I am a bit hesitant to do this since since it will allocate static memory for Parser, but if this function is never used in query, the parser will still take up memory space. Although, it seems like there is scope for both performance and memory improvement here. For example, currently `JsonExtractScalarFunction` object will be constructed fresh for each query that uses `json_extract_scalar` and if the query has multiple instances of `json_extract_scalar` then `JsonExtractScalarFunction` will be constructed multiple times. So one option may be to add a lazy function object cache to `TransformationFunctionFactory` to reuse function objects. Although, this is probably better handled in its own PR.
   
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] amrishlal commented on a change in pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651#discussion_r589768151



##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java
##########
@@ -282,6 +282,36 @@ public void testJsonMatchArrayWithIndex() {
     Assert.assertEquals(iterator.next()[0], "goofy");
   }
 
+  /** Evaluate json_extract_scalar over string column that does not contain valid json data. */
+  @Test
+  public void testJsonExtractScalarAgainstInvalidJson() {
+    // json_extract_scalar throws exception since we are trying to parse a non-JSON string.
+    Operator operator1 = getOperatorForSqlQuery(
+        "select count(*) FROM testTable WHERE json_extract_scalar(stringColumn, '$.name.first', 'INT') = 0");
+    try {
+      IntermediateResultsBlock block1 = (IntermediateResultsBlock) operator1.nextBlock();
+      Assert.assertTrue(false);

Review comment:
       Done




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

Posted by GitBox <gi...@apache.org>.
jackjlli commented on a change in pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651#discussion_r589672613



##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java
##########
@@ -282,6 +282,36 @@ public void testJsonMatchArrayWithIndex() {
     Assert.assertEquals(iterator.next()[0], "goofy");
   }
 
+  /** Evaluate json_extract_scalar over string column that does not contain valid json data. */
+  @Test
+  public void testJsonExtractScalarAgainstInvalidJson() {
+    // json_extract_scalar throws exception since we are trying to parse a non-JSON string.
+    Operator operator1 = getOperatorForSqlQuery(
+        "select count(*) FROM testTable WHERE json_extract_scalar(stringColumn, '$.name.first', 'INT') = 0");
+    try {
+      IntermediateResultsBlock block1 = (IntermediateResultsBlock) operator1.nextBlock();
+      Assert.assertTrue(false);

Review comment:
       `Assert.assertFail("Your expected error message.")`
   

##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java
##########
@@ -282,6 +282,36 @@ public void testJsonMatchArrayWithIndex() {
     Assert.assertEquals(iterator.next()[0], "goofy");
   }
 
+  /** Evaluate json_extract_scalar over string column that does not contain valid json data. */
+  @Test
+  public void testJsonExtractScalarAgainstInvalidJson() {
+    // json_extract_scalar throws exception since we are trying to parse a non-JSON string.
+    Operator operator1 = getOperatorForSqlQuery(
+        "select count(*) FROM testTable WHERE json_extract_scalar(stringColumn, '$.name.first', 'INT') = 0");
+    try {
+      IntermediateResultsBlock block1 = (IntermediateResultsBlock) operator1.nextBlock();
+      Assert.assertTrue(false);
+    } catch (RuntimeException re) {
+      Assert.assertEquals(re.toString(),
+          "java.lang.RuntimeException: Illegal Json Path: [$.name.first], when reading [daffy duck]");
+    }
+
+    // JSON data is stored in columns of type STRING, so there is nothing preventing the column from storing bad json
+    // string. Bad JSON string in columns will cause json_extract_scalar to throw an exception which would terminate
+    // query processing. However, when json_extract_scalar is used within the WHERE clause, we should return the
+    // default value instead of throwing exception. This will allow the predicate to be evaluated to either true or
+    // false and hence allow the query to complete successfully. Returning default value from json_extract_scalar is
+    // an undocumented feature. Ideally, json_extract_scalar should return NULL in when it encounters bad JSON. However,
+    // NULL support is currently pending, so this is the best we can do.
+    Operator operator2 = getOperatorForSqlQuery(
+        "select count(*) FROM testTable WHERE json_extract_scalar(stringColumn, '$.name.first', 'INT', 0) = 0");
+
+    IntermediateResultsBlock block2 = (IntermediateResultsBlock) operator2.nextBlock();
+    Collection<Object[]> rows = block2.getSelectionResult();
+
+    Assert.assertEquals(block2.getAggregationResult().get(0), 9l);

Review comment:
       Nit: `9L`

##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java
##########
@@ -282,6 +282,36 @@ public void testJsonMatchArrayWithIndex() {
     Assert.assertEquals(iterator.next()[0], "goofy");
   }
 
+  /** Evaluate json_extract_scalar over string column that does not contain valid json data. */
+  @Test
+  public void testJsonExtractScalarAgainstInvalidJson() {
+    // json_extract_scalar throws exception since we are trying to parse a non-JSON string.
+    Operator operator1 = getOperatorForSqlQuery(
+        "select count(*) FROM testTable WHERE json_extract_scalar(stringColumn, '$.name.first', 'INT') = 0");
+    try {
+      IntermediateResultsBlock block1 = (IntermediateResultsBlock) operator1.nextBlock();
+      Assert.assertTrue(false);
+    } catch (RuntimeException re) {
+      Assert.assertEquals(re.toString(),

Review comment:
       It'd be good not to validate the actual message, as the message may evolve. 

##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java
##########
@@ -282,6 +282,36 @@ public void testJsonMatchArrayWithIndex() {
     Assert.assertEquals(iterator.next()[0], "goofy");
   }
 
+  /** Evaluate json_extract_scalar over string column that does not contain valid json data. */
+  @Test
+  public void testJsonExtractScalarAgainstInvalidJson() {
+    // json_extract_scalar throws exception since we are trying to parse a non-JSON string.
+    Operator operator1 = getOperatorForSqlQuery(
+        "select count(*) FROM testTable WHERE json_extract_scalar(stringColumn, '$.name.first', 'INT') = 0");
+    try {
+      IntermediateResultsBlock block1 = (IntermediateResultsBlock) operator1.nextBlock();
+      Assert.assertTrue(false);
+    } catch (RuntimeException re) {
+      Assert.assertEquals(re.toString(),
+          "java.lang.RuntimeException: Illegal Json Path: [$.name.first], when reading [daffy duck]");
+    }
+
+    // JSON data is stored in columns of type STRING, so there is nothing preventing the column from storing bad json
+    // string. Bad JSON string in columns will cause json_extract_scalar to throw an exception which would terminate
+    // query processing. However, when json_extract_scalar is used within the WHERE clause, we should return the
+    // default value instead of throwing exception. This will allow the predicate to be evaluated to either true or
+    // false and hence allow the query to complete successfully. Returning default value from json_extract_scalar is
+    // an undocumented feature. Ideally, json_extract_scalar should return NULL in when it encounters bad JSON. However,
+    // NULL support is currently pending, so this is the best we can do.
+    Operator operator2 = getOperatorForSqlQuery(
+        "select count(*) FROM testTable WHERE json_extract_scalar(stringColumn, '$.name.first', 'INT', 0) = 0");
+
+    IntermediateResultsBlock block2 = (IntermediateResultsBlock) operator2.nextBlock();
+    Collection<Object[]> rows = block2.getSelectionResult();
+
+    Assert.assertEquals(block2.getAggregationResult().get(0), 9l);

Review comment:
       And it might be good to add a comment to explain what 9L denotes, like what count you're trying to fetch here?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] amrishlal commented on a change in pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651#discussion_r590596002



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/JsonExtractScalarTransformFunction.java
##########
@@ -131,7 +131,7 @@ public TransformResultMetadata getResultMetadata() {
     final String[] stringValuesSV = _jsonFieldTransformFunction.transformToStringValuesSV(projectionBlock);
     final int[] results = new int[projectionBlock.getNumDocs()];
     for (int i = 0; i < results.length; i++) {
-      Object read = JsonPath.read(stringValuesSV[i], _jsonPath);
+      Object read = JsonPath.using(JSON_PARSER_CONFIG).parse(stringValuesSV[i]).read(_jsonPath);

Review comment:
       I have added the constant for now, although at some point in future it would be good to move towards having a function object cache in `TransformationFunctionFactory`.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651#discussion_r589712457



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/JsonExtractScalarTransformFunction.java
##########
@@ -131,7 +131,7 @@ public TransformResultMetadata getResultMetadata() {
     final String[] stringValuesSV = _jsonFieldTransformFunction.transformToStringValuesSV(projectionBlock);
     final int[] results = new int[projectionBlock.getNumDocs()];
     for (int i = 0; i < results.length; i++) {
-      Object read = JsonPath.read(stringValuesSV[i], _jsonPath);
+      Object read = JsonPath.using(JSON_PARSER_CONFIG).parse(stringValuesSV[i]).read(_jsonPath);

Review comment:
       Shall we cache `JsonPath.using(JSON_PARSER_CONFIG)` as a constant? I think it is thread-safe




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651#issuecomment-791742340


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=h1) Report
   > Merging [#6651](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=desc) (c2b6d17) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `0.60%`.
   > The diff coverage is `62.61%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6651/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6651      +/-   ##
   ==========================================
   - Coverage   66.44%   65.84%   -0.61%     
   ==========================================
     Files        1075     1361     +286     
     Lines       54773    66517   +11744     
     Branches     8168     9702    +1534     
   ==========================================
   + Hits        36396    43795    +7399     
   - Misses      15700    19599    +3899     
   - Partials     2677     3123     +446     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.84% <62.61%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...pinot/broker/api/resources/PinotClientRequest.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdENsaWVudFJlcXVlc3QuamF2YQ==) | `0.00% <0.00%> (-27.28%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <ø> (+9.52%)` | :arrow_up: |
   | [...n/java/org/apache/pinot/client/BrokerResponse.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Jyb2tlclJlc3BvbnNlLmphdmE=) | `100.00% <ø> (ø)` | |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <ø> (-13.29%)` | :arrow_down: |
   | [...org/apache/pinot/client/DynamicBrokerSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0R5bmFtaWNCcm9rZXJTZWxlY3Rvci5qYXZh) | `82.85% <ø> (+10.12%)` | :arrow_up: |
   | [...n/java/org/apache/pinot/client/ExecutionStats.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0V4ZWN1dGlvblN0YXRzLmphdmE=) | `68.88% <ø> (ø)` | |
   | ... and [1246 more](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=footer). Last update [1dbdd67...c2b6d17](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651#discussion_r589842243



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/JsonExtractScalarTransformFunction.java
##########
@@ -131,7 +131,7 @@ public TransformResultMetadata getResultMetadata() {
     final String[] stringValuesSV = _jsonFieldTransformFunction.transformToStringValuesSV(projectionBlock);
     final int[] results = new int[projectionBlock.getNumDocs()];
     for (int i = 0; i < results.length; i++) {
-      Object read = JsonPath.read(stringValuesSV[i], _jsonPath);
+      Object read = JsonPath.using(JSON_PARSER_CONFIG).parse(stringValuesSV[i]).read(_jsonPath);

Review comment:
       It's just one single object on top of the Configuration which can be reused. I would not worry about the memory overhead of one single object. Also, it should never be initialized if this function is never used thus this class is not loaded.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io commented on pull request #6651: Don't throw exception in json_extract_scalar when default value is available.

Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #6651:
URL: https://github.com/apache/incubator-pinot/pull/6651#issuecomment-791742340


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=h1) Report
   > Merging [#6651](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=desc) (50f4465) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `0.60%`.
   > The diff coverage is `62.64%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6651/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6651      +/-   ##
   ==========================================
   - Coverage   66.44%   65.84%   -0.61%     
   ==========================================
     Files        1075     1360     +285     
     Lines       54773    66537   +11764     
     Branches     8168     9701    +1533     
   ==========================================
   + Hits        36396    43808    +7412     
   - Misses      15700    19608    +3908     
   - Partials     2677     3121     +444     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.84% <62.64%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...pinot/broker/api/resources/PinotClientRequest.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdENsaWVudFJlcXVlc3QuamF2YQ==) | `0.00% <0.00%> (-27.28%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <ø> (+9.52%)` | :arrow_up: |
   | [...n/java/org/apache/pinot/client/BrokerResponse.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Jyb2tlclJlc3BvbnNlLmphdmE=) | `100.00% <ø> (ø)` | |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <ø> (-13.29%)` | :arrow_down: |
   | [...org/apache/pinot/client/DynamicBrokerSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0R5bmFtaWNCcm9rZXJTZWxlY3Rvci5qYXZh) | `82.85% <ø> (+10.12%)` | :arrow_up: |
   | [...n/java/org/apache/pinot/client/ExecutionStats.java](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0V4ZWN1dGlvblN0YXRzLmphdmE=) | `68.88% <ø> (ø)` | |
   | ... and [1243 more](https://codecov.io/gh/apache/incubator-pinot/pull/6651/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=footer). Last update [df767c0...50f4465](https://codecov.io/gh/apache/incubator-pinot/pull/6651?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org