You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "arifazmidd (via GitHub)" <gi...@apache.org> on 2023/12/07 22:23:15 UTC
[I] DeltaLakeToIcebergMigration [iceberg]
arifazmidd opened a new issue, #9246:
URL: https://github.com/apache/iceberg/issues/9246
### Query engine
_No response_
### Question
I'm using [DeltaLakeToIcebergMigrationActionsProvider](https://iceberg.apache.org/docs/1.3.0/delta-lake-migration/) and am seeing it takes about 2 mins to migrate a table with 200 parquet files. Is this the expected performance? This would be quite slow for large tables.
I notice that it needs to read every parquet files footer. Is this required when it's simply creating a snapshot of the table/is there any workaround?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] DeltaLakeToIcebergMigration Performance [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #9246: DeltaLakeToIcebergMigration Performance
URL: https://github.com/apache/iceberg/issues/9246
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] DeltaLakeToIcebergMigration Performance [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #9246:
URL: https://github.com/apache/iceberg/issues/9246#issuecomment-2597166268
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] DeltaLakeToIcebergMigration Performance [iceberg]
Posted by "nk1506 (via GitHub)" <gi...@apache.org>.
nk1506 commented on issue #9246:
URL: https://github.com/apache/iceberg/issues/9246#issuecomment-1867357185
I think performance should not be proportional to no of parquet files. Do you have more infos in terms of metadata files? like how many chechpoint parquet files , etc?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org