You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/01/18 04:45:17 UTC
[GitHub] [iceberg] HeartSaVioR edited a comment on issue #2041: Out of order fields are corrupting data with Insert Into SQL on Spark 3.0
HeartSaVioR edited a comment on issue #2041:
URL: https://github.com/apache/iceberg/issues/2041#issuecomment-761976271
Yeah, I think @RussellSpitzer is correct on this. And my quick google says matching by column name during INSERT INTO ... SELECT isn't something other RDBMS would also support.
The safest way to do this is providing explicit column list for both source and target, but Spark doesn't support providing explicit column list in target side, so your only workaround is knowing the order of columns in target table in prior, and ensure your select statement is matching the column list via "explicitly" providing column list instead of simply using "*".
```
spark.sql("insert into target select f1, f2, f3 from input").show(false)
```
This would work. I'd rather say it's safer if Spark supports explicit column list on INSERT INTO, but that's not a matter of Iceberg and you're welcome to ask this to Spark user mailing list. At least as of now I don't see any issue for this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org