You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/03/03 18:29:04 UTC
[GitHub] [incubator-pinot] johighley opened a new issue #6634: JSON Ingestion: Int parse of String with leading spaces is failing, and failing silently
johighley opened a new issue #6634:
URL: https://github.com/apache/incubator-pinot/issues/6634
I'm ingesting JSON data to a realtime table. The value is a JSON String but is a numeric value with leading spaces:
{ "account":" 123", ....}
If the table column type is Double, then this record will load successfully -- it ignores the leading spaces. If the column type is Int, then the record will not load and there is no error in the server logs (or any log I can find). It fails silently. If I remove the leading spaces, then the String will load successfully as an Int.
1) When converting a JSON String value to an Int column type, whitespace needs to be trimmed prior to parsing.
2) A record ingestion should not fail silently. I'm guessing Integer.parseInt is being used and throwing a NumberFormatException that is being eaten.
Code is version 0.7.0-SNAPSHOT-e62addb3b381e89d3afe24847a1bff06e7682246
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] johighley edited a comment on issue #6634: JSON Ingestion: Int parse of String with leading spaces is failing, and failing silently
Posted by GitBox <gi...@apache.org>.
johighley edited a comment on issue #6634:
URL: https://github.com/apache/incubator-pinot/issues/6634#issuecomment-790117575
we're running the docker images. controller, server, and broker are each their own container. I grep'd for 'ERROR' in pinotServer.log and there's nothing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6634: JSON Ingestion: Int parse of String with leading spaces is failing, and failing silently
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6634:
URL: https://github.com/apache/incubator-pinot/issues/6634#issuecomment-789982465
@fx19880617 Any idea why the error message is not properly logged?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] johighley commented on issue #6634: JSON Ingestion: Int parse of String with leading spaces is failing, and failing silently
Posted by GitBox <gi...@apache.org>.
johighley commented on issue #6634:
URL: https://github.com/apache/incubator-pinot/issues/6634#issuecomment-789977456
Running the docker images. There's no error message in pinotServer.log. Would it be somewhere else?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] johighley commented on issue #6634: JSON Ingestion: Int parse of String with leading spaces is failing, and failing silently
Posted by GitBox <gi...@apache.org>.
johighley commented on issue #6634:
URL: https://github.com/apache/incubator-pinot/issues/6634#issuecomment-790117575
we're running the docker images. controller, server, and broker are each their own container
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #6634: JSON Ingestion: Int parse of String with leading spaces is failing, and failing silently
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #6634:
URL: https://github.com/apache/incubator-pinot/issues/6634#issuecomment-790050261
it should be the log file inside the container. Are you running pinot server separately or pinot quickstart all in one?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6634: JSON Ingestion: Int parse of String with leading spaces is failing, and failing silently
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6634:
URL: https://github.com/apache/incubator-pinot/issues/6634#issuecomment-789973936
You should see the following `ERROR` log in the server log: `Caught exception while transforming the record: ...` with the exception trace.
The reason why double works is because while converting from `" 123"` to numeric value, `Integer.parseInt()` throws exception, but `Double.parseDouble()` works fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6634: JSON Ingestion: Int parse of String with leading spaces is failing, and failing silently
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6634:
URL: https://github.com/apache/incubator-pinot/issues/6634#issuecomment-789987656
#6635 to trim the string before parsing
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org