You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "coolderli (via GitHub)" <gi...@apache.org> on 2023/02/21 03:03:09 UTC
[GitHub] [iceberg] coolderli opened a new issue, #6890: Renaming a table may conflict with the new table name
coolderli opened a new issue, #6890:
URL: https://github.com/apache/iceberg/issues/6890
### Apache Iceberg version
1.0.0
### Query engine
Spark
### Please describe the bug 🐞
I searched the issues but didn't find the answer, there is my problem.
I query the table with the name `app_hr_talent_empl_df_1d_v2_drop` but I got `File does not exist: /xxxx/xxdb/app_hr_talent_empl_df_1d_v2/metadata/00243-109f2aee-78a2-42b1-bfba-6a69e18b919b.metadata.json`
I rename the table `app_hr_talent_empl_df_1d_v2` to `app_hr_talent_empl_df_1d_v2_drop` and then I create a new table with the name `app_hr_talent_empl_df_1d_v2`.
So I think the orphans clean produces of table `app_hr_talent_empl_df_1d_v2`delete the file `/xxxx/xxdb/app_hr_talent_empl_df_1d_v2/metadata/00243-109f2aee-78a2-42b1-bfba-6a69e18b919b.metadata.json`.
How should we deal with this situation, Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Renaming a table may conflict with the new table with old table name [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #6890: Renaming a table may conflict with the new table with old table name
URL: https://github.com/apache/iceberg/issues/6890
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] szehon-ho commented on issue #6890: Renaming a table may conflict with the new table with old table name
Posted by "szehon-ho (via GitHub)" <gi...@apache.org>.
szehon-ho commented on issue #6890:
URL: https://github.com/apache/iceberg/issues/6890#issuecomment-1438905193
I remember there's some related thoughts on the matter: https://github.com/apache/iceberg/issues/4159
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra commented on issue #6890: Renaming a table may conflict with the new table with old table name
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on issue #6890:
URL: https://github.com/apache/iceberg/issues/6890#issuecomment-1438335009
@coolderli could you please describe your catalog configuration and which operations you are executing? That may help in pinpointing down the issue. Also logs would be helpful in case you have them.
This might be a shot in the dark, but could you try with [`cache-enabled = false`](https://iceberg.apache.org/docs/latest/configuration/#catalog-properties) or run `REFRESH`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] szehon-ho commented on issue #6890: Renaming a table may conflict with the new table with old table name
Posted by "szehon-ho (via GitHub)" <gi...@apache.org>.
szehon-ho commented on issue #6890:
URL: https://github.com/apache/iceberg/issues/6890#issuecomment-1438893112
Yea this is a tough problem Ive thought about before as well, the only fix is to create table with new location as Ajantha mentioned. I feel the only fix would have to maintain your own index of table to location, and then check if a location is taken before allowing to create the table at that location. I'm not sure if that's a feature we can have catalog support at some point. (Though it wont help cross-catalog cases).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] coolderli commented on issue #6890: Renaming a table may conflict with the new table with old table name
Posted by "coolderli (via GitHub)" <gi...@apache.org>.
coolderli commented on issue #6890:
URL: https://github.com/apache/iceberg/issues/6890#issuecomment-1438351030
@nastra Thank you for your attention.
I used the hive catalog. We periodically perform orphan file cleanup for each table.
As described above, I renamed table_a to table_b and recreated table_a. Then table_a and table_b have the same data directory.
The table_a's orphan file cleanup program will clean up table_b's normal files, so when we query table_b encountered FileNotFoundException.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #6890: Renaming a table may conflict with the new table with old table name
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6890:
URL: https://github.com/apache/iceberg/issues/6890#issuecomment-1685435930
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Renaming a table may conflict with the new table with old table name [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6890:
URL: https://github.com/apache/iceberg/issues/6890#issuecomment-1874709577
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on issue #6890: Renaming a table may conflict with the new table with old table name
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on issue #6890:
URL: https://github.com/apache/iceberg/issues/6890#issuecomment-1438420603
True. As both the tables share the same path, running remove orphan files on one table can clean up the live files of another table.
I don't think we have a fix for this based on the current remove orphan files design.
For Nessie Catalog, I am using UUID for the table path to avoid this problem. Maybe we need to enforce this to all the catalogs (I can work on this once people agree to this change)
As a workaround, when you create a new table (you can specify the new table location in the create table statement or table properties)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org