You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/08/06 10:26:12 UTC
[GitHub] [druid] clintropolis opened a new pull request #11556: fix parse exception handling for stream parsers
clintropolis opened a new pull request #11556:
URL: https://github.com/apache/druid/pull/11556
Fixes #10784.
### Description
This PR does the ugly/easy fix to `FilteringCloseableInputRowIterator` to better handle `ParseException` thrown during iteration, due to the fact that some transforming iterators will call `next` on their delegated iterator whenever their own `hasNext` method is called. The trigger in the case of #10784 is the `FluentIterable` transform used by `StreamChunkParser.parseWithParser`, but I would imagine that others are possible due to all the layers of iterators.
Switching the iterator to be `Pair<InputRow, ParseException>` would be a fairly disruptive change as far as I can tell, because its sort of iterators all the way down. This would be changing the individual parsers/format readers to mean not throwing ParseException, unless I'm missing something, making it similar to the `sample` interface, which uses `InputRowListPlusRawValues` to provide an iterator of lists of rows, or a parse exception. This isn't necessarily unreasonable to do, but significantly more effort than making this change.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [x] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s merged pull request #11556: fix parse exception handling for stream parsers
Posted by GitBox <gi...@apache.org>.
suneet-s merged pull request #11556:
URL: https://github.com/apache/druid/pull/11556
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org