[GitHub] [iceberg] rdblue commented on issue #1306: Spark 3: Consider providing better support for path-based tables

[GitBox <gi...@apache.org>]

rdblue commented on issue #1306:
URL: https://github.com/apache/iceberg/issues/1306#issuecomment-670971549


   I was also thinking about adding support for identifiers with a path. We could use that to delegate from `IcebergSource` to a catalog, which would the bring support for all of the `SaveMode` options in the v1 `DataFrameWriter`. Since that write path is very confusing and often broken, I think adding that is a very minor benefit.
   
   I think there may already be support for tables that are identified using ```iceberg.`fs:/path/to/table````. That works for v1 sources, like Parquet. Russell's option could work as well.
   
   Isn't the main blocker that we need an atomic operation? HadoopTableOperations works for HDFS tables, but any other file system, including `file:`, needs something for coordination. I'm not sure what that would be for path-based tables.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org