You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/02/24 02:34:04 UTC
[GitHub] [spark] maropu commented on pull request #29542: [SPARK-32703][SQL] Replace deprecated API calls from SpecificParquetRecordReaderBase
maropu commented on pull request #29542:
URL: https://github.com/apache/spark/pull/29542#issuecomment-784704067
Hi, all. Sorry for my late comment. It seems some TPCDS queries (e.g., q9, q54, q31, .... See [this](https://docs.google.com/spreadsheets/d/1V8xoKR9ElU-rOXMH84gb5BbLEw0XAPTJY8c8aZeIqus/edit#gid=55812358) for more details) got slower after merging this commit. Are they expected performance changes? For example, I've checked the peformance of TPCDS `q9` with/without the commit;
```
// The current master
OpenJDK 64-Bit Server VM 1.8.0_265-b01 on Linux 4.14.203-156.332.amzn2.x86_64
Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
TPCDS Snappy: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
q9 59071 59510 622 0.0 1640854387.2 1.0X
// Revert this commit
OpenJDK 64-Bit Server VM 1.8.0_265-b01 on Linux 4.14.203-156.332.amzn2.x86_64
Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
TPCDS Snappy: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
q9 28963 29057 132 0.0 804530503.1 1.0X
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org