You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "geekboood (via GitHub)" <gi...@apache.org> on 2023/06/05 13:59:04 UTC
[GitHub] [pinot] geekboood opened a new issue, #10844: Long respond time with simple filter query
geekboood opened a new issue, #10844:
URL: https://github.com/apache/pinot/issues/10844
I have one pinot server instance useing version 0.12.1, with 8000+ segments and each segments contains 1048576 rows. I create inverted index and bloom filter (and set on heap) for this filter field, but the query time still too slow. Here is the tracing detail.
`{\"ColumnValueSegmentPruner Time\":18},{\"SegmentPrunerService Time\":35},{\"CombinePlanNode Time\":7192},{\"SelectionOrderByCombineOperator Time\":813},{\"InstanceResponseOperator Time\":813}`
```
"numServersQueried": 1,
"numServersResponded": 1,
"numSegmentsQueried": 8507,
"numSegmentsProcessed": 427,
"numSegmentsMatched": 19,
"numConsumingSegmentsQueried": 0,
"numConsumingSegmentsProcessed": 0,
"numConsumingSegmentsMatched": 0,
"numDocsScanned": 628,
"numEntriesScannedInFilter": 0,
"numEntriesScannedPostFilter": 1256,
"numGroupsLimitReached": false,
"totalDocs": 8920236032,
"timeUsedMs": 8098,
```
What could be the problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] geekboood commented on issue #10844: Long respond time with simple filter query
Posted by "geekboood (via GitHub)" <gi...@apache.org>.
geekboood commented on issue #10844:
URL: https://github.com/apache/pinot/issues/10844#issuecomment-1580687414
My Pinot server run on HDD so this seems normal. Does Pinot has mechanism to put index on SSD, such as a seperate index directory?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] geekboood commented on issue #10844: Long respond time with simple filter query
Posted by "geekboood (via GitHub)" <gi...@apache.org>.
geekboood commented on issue #10844:
URL: https://github.com/apache/pinot/issues/10844#issuecomment-1632650695
Adding SSD improved overall performance. I add a bcache layer using SSD to Pinot data drive and the performance drastically improved, both initial starting and querying. For Pinot, it has static data segment files so this might be a good idea to balance between performance and cost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10844: Long respond time with simple filter query
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10844:
URL: https://github.com/apache/pinot/issues/10844#issuecomment-1589832541
Do you have SSD attached to your servers? If so, you may change the data directory in the server config to point to the SSD. You may reference this page for the server config: https://docs.pinot.apache.org/configuration-reference/server
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] geekboood closed issue #10844: Long respond time with simple filter query
Posted by "geekboood (via GitHub)" <gi...@apache.org>.
geekboood closed issue #10844: Long respond time with simple filter query
URL: https://github.com/apache/pinot/issues/10844
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10844: Long respond time with simple filter query
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10844:
URL: https://github.com/apache/pinot/issues/10844#issuecomment-1579322732
Is this during a cold start? Do you have SSD or HDD on your server? `CombinePlanNode` takes majority of the time, which means binary search on the dictionary to find the matching dictionary id might be the root cause.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org