You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "ajantha-bhat (via GitHub)" <gi...@apache.org> on 2023/01/30 05:00:24 UTC
[GitHub] [iceberg] ajantha-bhat opened a new pull request, #6696: Build: Bump Arrow from 10.0.1 to 11.0.0
ajantha-bhat opened a new pull request, #6696:
URL: https://github.com/apache/iceberg/pull/6696
release notes
https://arrow.apache.org/release/11.0.0.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on pull request #6696: Build: Bump Arrow from 10.0.1 to 11.0.0
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on PR #6696:
URL: https://github.com/apache/iceberg/pull/6696#issuecomment-1418451836
[With Arrow 10.0.1]
run-benchmark (SparkParquetWritersNestedDataBenchmark) 5m31s
run-benchmark (SparkParquetWritersFlatDataBenchmark) 5m13s
run-benchmark (SparkParquetReadersNestedDataBenchmark) 4m28s
run-benchmark (SparkParquetReadersFlatDataBenchmark) 5m11s
[With Arrow-11.0.0]
run-benchmark (SparkParquetWritersNestedDataBenchmark) 4m32s
run-benchmark (SparkParquetWritersFlatDataBenchmark) 5m9s
run-benchmark (SparkParquetReadersNestedDataBenchmark) 4m29s
run-benchmark (SparkParquetReadersFlatDataBenchmark) 5m13s
It dumps the JMH outputs into a local file. I am sure whether I can access those files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on pull request #6696: Build: Bump Arrow from 10.0.1 to 11.0.0
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on PR #6696:
URL: https://github.com/apache/iceberg/pull/6696#issuecomment-1418643318
```
Benchmark [Arrow-11.0.0] Mode Cnt Score Error Units
SparkParquetWritersNestedDataBenchmark.writeUsingIcebergWriter ss 5 4.152 ± 0.123 s/op
SparkParquetWritersNestedDataBenchmark.writeUsingSparkWriter ss 5 4.325 ± 0.089 s/op
SparkParquetWritersFlatDataBenchmark.writeUsingIcebergWriter ss 5 6.411 ± 0.048 s/op
SparkParquetWritersFlatDataBenchmark.writeUsingSparkWriter ss 5 6.725 ± 0.207 s/op
SparkParquetReadersNestedDataBenchmark.readUsingIcebergReader ss 5 0.591 ± 0.079 s/op
SparkParquetReadersNestedDataBenchmark.readUsingIcebergReaderUnsafe ss 5 0.679 ± 0.011 s/op
SparkParquetReadersNestedDataBenchmark.readUsingSparkReader ss 5 0.626 ± 0.059 s/op
SparkParquetReadersNestedDataBenchmark.readWithProjectionUsingIcebergReader ss 5 0.286 ± 0.029 s/op
SparkParquetReadersNestedDataBenchmark.readWithProjectionUsingIcebergReaderUnsafe ss 5 0.365 ± 0.036 s/op
SparkParquetReadersNestedDataBenchmark.readWithProjectionUsingSparkReader ss 5 0.333 ± 0.031 s/op
SparkParquetReadersFlatDataBenchmark.readUsingIcebergReader ss 5 1.012 ± 0.051 s/op
SparkParquetReadersFlatDataBenchmark.readUsingIcebergReaderUnsafe ss 5 1.223 ± 0.046 s/op
SparkParquetReadersFlatDataBenchmark.readUsingSparkReader ss 5 1.128 ± 0.018 s/op
SparkParquetReadersFlatDataBenchmark.readWithProjectionUsingIcebergReader ss 5 0.582 ± 0.023 s/op
SparkParquetReadersFlatDataBenchmark.readWithProjectionUsingIcebergReaderUnsafe ss 5 0.733 ± 0.013 s/op
SparkParquetReadersFlatDataBenchmark.readWithProjectionUsingSparkReader ss 5 0.648 ± 0.021 s/op
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on pull request #6696: Build: Bump Arrow from 10.0.1 to 11.0.0
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #6696:
URL: https://github.com/apache/iceberg/pull/6696#issuecomment-1432901683
Thanks @ajantha-bhat for bumping Arrow and running the benchmarks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra commented on pull request #6696: Build: Bump Arrow from 10.0.1 to 11.0.0
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on PR #6696:
URL: https://github.com/apache/iceberg/pull/6696#issuecomment-1418623938
@ajantha-bhat the results are stored and linked under [benchmark-results](https://github.com/ajantha-bhat/iceberg/suites/10704631127/artifacts/541607117) for each benchmark run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra commented on pull request #6696: Build: Bump Arrow from 10.0.1 to 11.0.0
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on PR #6696:
URL: https://github.com/apache/iceberg/pull/6696#issuecomment-1417512864
@ajantha-bhat could you please copy/paste the actual benchmark numbers from the linked benchmark runs to have an easier comparison of the numbers here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on pull request #6696: Build: Bump Arrow from 10.0.1 to 11.0.0
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on PR #6696:
URL: https://github.com/apache/iceberg/pull/6696#issuecomment-1418644100
<img width="1199" alt="image" src="https://user-images.githubusercontent.com/5889404/216913511-533f8206-c974-4e78-b2a7-4416c04a1c06.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on pull request #6696: Build: Bump Arrow from 10.0.1 to 11.0.0
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on PR #6696:
URL: https://github.com/apache/iceberg/pull/6696#issuecomment-1411534472
> @ajantha-bhat, can you please run benchmarks to ensure there are no regressions?
Benchmarks:
[With Arrow 10.0.1] https://github.com/ajantha-bhat/iceberg/actions/runs/4061513944
[With Arrow-11.0.0] https://github.com/ajantha-bhat/iceberg/actions/runs/4061518292
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #6696: Build: Bump Arrow from 10.0.1 to 11.0.0
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #6696:
URL: https://github.com/apache/iceberg/pull/6696#issuecomment-1410962799
@ajantha-bhat, can you please run benchmarks to ensure there are no regressions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko merged pull request #6696: Build: Bump Arrow from 10.0.1 to 11.0.0
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko merged PR #6696:
URL: https://github.com/apache/iceberg/pull/6696
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org