You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/10/21 19:24:44 UTC
[GitHub] [pinot] dongxiaoman opened a new issue #7612: Pinot query ArrayIndexOutOfBounds for some multi-value column queries
dongxiaoman opened a new issue #7612:
URL: https://github.com/apache/pinot/issues/7612
When we run the below query on our cluster via Pinot UI, we get `"errorCode": 200,
"message": "QueryExecutionError:\njava.lang.ArrayIndexOutOfBoundsException"`
The query is simple as
`select account_key from point_entry where account_key = '1fedf01cdb9aba9395172b5832445d1e' and account_fields_values not in ('intercompany_clearing') limit 1 option(timeoutMs=10000)`
Both of the columns in the query are multi-value columns.
The Server error log is below:
```
Caught exception while executing operator of index: 4 (query: QueryContext{_tableName='point_entry_OFFLINE', _selectExpressions=[account_key], _aliasList=[null], _filter=(account_key = '1fedf01cdb9aba9395172b5832445d1e' AND account_fields_values NOT IN ('intercompany_clearing','cash','merchant_balance','fx_clearing') AND created_at <= '1634813999'), _groupByExpressions=null, _havingFilter=null, _orderByExpressions=null, _limit=100, _offset=0, _queryOptions={responseFormat=sql, groupByMode=sql, timeoutMs=9863}, _debugOptions=null, _brokerRequest=BrokerRequest(querySource:QuerySource(tableName:point_entry_OFFLINE), pinotQuery:PinotQuery(dataSource:DataSource(tableName:point_entry_OFFLINE), selectList:[Expression(type:IDENTIFIER, identifier:Identifier(name:account_key))], filterExpression:Expression(type:FUNCTION, functionCall:Function(operator:AND, operands:[Expression(type:FUNCTION, functionCall:Function(operator:EQUALS, operands:[Expression(type:IDENTIFIER, identifier:Identifier(n
ame:account_key)), Expression(type:LITERAL, literal:<Literal stringValue:1fedf01cdb9aba9395172b5832445d1e>)])), Expression(type:FUNCTION, functionCall:Function(operator:NOT_IN, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:account_fields_values)), Expression(type:LITERAL, literal:<Literal stringValue:intercompany_clearing>), Expression(type:LITERAL, literal:<Literal stringValue:cash>), Expression(type:LITERAL, literal:<Literal stringValue:merchant_balance>), Expression(type:LITERAL, literal:<Literal stringValue:fx_clearing>)])), Expression(type:FUNCTION, functionCall:Function(operator:LESS_THAN_OR_EQUAL, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:created_at)), Expression(type:LITERAL, literal:<Literal stringValue:1634813999>)]))])), orderByList:[], limit:100, queryOptions:{responseFormat=sql, groupByMode=sql, timeoutMs=9863}))})
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on issue #7612: Pinot query ArrayIndexOutOfBounds for some multi-value column queries
Posted by GitBox <gi...@apache.org>.
walterddr commented on issue #7612:
URL: https://github.com/apache/pinot/issues/7612#issuecomment-996438555
I think this ticket can be closed.
The root cause was that one of the very old segment wasn't properly conformed with the new indexed segments. thus the query always fails when hitting that specific segment. (@dongxiaoman plz correct me if I remembered wrong).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on issue #7612: Pinot query ArrayIndexOutOfBounds for some multi-value column queries
Posted by GitBox <gi...@apache.org>.
walterddr commented on issue #7612:
URL: https://github.com/apache/pinot/issues/7612#issuecomment-951047921
looks like the sever log is different from the query pasted, server log has 4 string literals in the NOT_IN filter expression but the query above only has 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] dongxiaoman commented on issue #7612: Pinot query ArrayIndexOutOfBounds for some multi-value column queries
Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on issue #7612:
URL: https://github.com/apache/pinot/issues/7612#issuecomment-951606249
For now we have pin pointed to a segment that was generated long ago. Could be the data that was wrongly generated; for now we can work around this by adding a time filter to skip those segments. Will keep digging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #7612: Pinot query ArrayIndexOutOfBounds for some multi-value column queries
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #7612:
URL: https://github.com/apache/pinot/issues/7612#issuecomment-952114362
Does this happen to one single segment or all segments generated during that period of time?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] dongxiaoman closed issue #7612: Pinot query ArrayIndexOutOfBounds for some multi-value column queries
Posted by GitBox <gi...@apache.org>.
dongxiaoman closed issue #7612:
URL: https://github.com/apache/pinot/issues/7612
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] icefury71 commented on issue #7612: Pinot query ArrayIndexOutOfBounds for some multi-value column queries
Posted by GitBox <gi...@apache.org>.
icefury71 commented on issue #7612:
URL: https://github.com/apache/pinot/issues/7612#issuecomment-996320557
@dongxiaoman can this issue be closed ? Can you update the ticket with the resolution steps as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] dongxiaoman commented on issue #7612: Pinot query ArrayIndexOutOfBounds for some multi-value column queries
Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on issue #7612:
URL: https://github.com/apache/pinot/issues/7612#issuecomment-999774558
Forgot to close it. Happens only to one segment that is probably generated by some version of library that may have issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] dongxiaoman commented on issue #7612: Pinot query ArrayIndexOutOfBounds for some multi-value column queries
Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on issue #7612:
URL: https://github.com/apache/pinot/issues/7612#issuecomment-951356252
```
2021/10/25 21:32:07.808 ERROR [BaseCombineOperator] [pqw-7] Caught exception while executing operator of index: 4 (query: QueryContext{_tableName='point_entry_OFFLINE', _selectExpressions=[account_key], _aliasList=[null], _filter=(account_key = 'xxxx' AND account_fields_values NOT IN ('intercompany_clearing') AND created_at <= '1635181199'), _groupByExpressions=null, _havingFilter=null, _orderByExpressions=null, _limit=1, _offset=0, _queryOptions={responseFormat=sql, groupByMode=sql, timeoutMs=9926}, _debugOptions=null, _brokerRequest=BrokerRequest(querySource:QuerySource(tableName:point_entry_OFFLINE), pinotQuery:PinotQuery(dataSource:DataSource(tableName:point_entry_OFFLINE), selectList:[Expression(type:IDENTIFIER, identifier:Identifier(name:account_key))], filterExpression:Expression(type:FUNCTION, functionCall:Function(operator:AND, operands:[Expression(type:FUNCTION, functionCall:Function(operator:EQUALS, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:accoun
t_key)), Expression(type:LITERAL, literal:<Literal stringValue:1fedf01cdb9aba9395172b5832445d1e>)])), Expression(type:FUNCTION, functionCall:Function(operator:NOT_IN, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:account_fields_values)), Expression(type:LITERAL, literal:<Literal stringValue:intercompany_clearing>)])), Expression(type:FUNCTION, functionCall:Function(operator:LESS_THAN_OR_EQUAL, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:created_at)), Expression(type:LITERAL, literal:<Literal stringValue:1635181199>)]))])), orderByList:[], limit:1, queryOptions:{responseFormat=sql, groupByMode=sql, timeoutMs=9926}))})
java.lang.ArrayIndexOutOfBoundsException: 8
at org.apache.pinot.segment.local.io.util.PinotDataBitSet.readInt(PinotDataBitSet.java:127) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.segment.local.io.util.FixedBitIntReaderWriter.readInt(FixedBitIntReaderWriter.java:42) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.segment.local.segment.index.readers.forward.FixedBitMVForwardIndexReader.getDictIdMV(FixedBitMVForwardIndexReader.java:126) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.segment.local.segment.index.readers.forward.FixedBitMVForwardIndexReader.getDictIdMV(FixedBitMVForwardIndexReader.java:51) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.dociditerators.MVScanDocIdIterator.applyAnd(MVScanDocIdIterator.java:80) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.docidsets.AndDocIdSet.iterator(AndDocIdSet.java:128) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.DocIdSetOperator.getNextBlock(DocIdSetOperator.java:63) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.DocIdSetOperator.getNextBlock(DocIdSetOperator.java:35) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.ProjectionOperator.getNextBlock(ProjectionOperator.java:57) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.ProjectionOperator.getNextBlock(ProjectionOperator.java:30) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.transform.PassThroughTransformOperator.getNextBlock(PassThroughTransformOperator.java:47) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.transform.PassThroughTransformOperator.getNextBlock(PassThroughTransformOperator.java:31) ~[pinot-all-0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-jar-with-dependencies.jar:0.9.0-2021-10-01-9e37c7f68-SNAPSHOT-9e37c7f680ef3c5256eb30a7b9b43e8f338b2ccd]
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #7612: Pinot query ArrayIndexOutOfBounds for some multi-value column queries
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #7612:
URL: https://github.com/apache/pinot/issues/7612#issuecomment-950590051
Thanks for reporting this. Could you please try more queries on these 2 columns? A full stack trace from the server log would also help debugging the issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org