You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/09/03 21:44:36 UTC
[GitHub] [iceberg] wypoon opened a new issue #3070: Precedence if both snapshot-id and as-of-timestamp are specified as options in Spark DataFrameReader
wypoon opened a new issue #3070:
URL: https://github.com/apache/iceberg/issues/3070
From the code in (spark2) `Reader#tasks()` and (spark3) `SparkBatchQueryScan#tasks()`, it would appear that if both "snapshot-id" and "as-of-timestamp" are specified as options in the `DataFrameReader`, then the timestamp takes precedence:
```
if (snapshotId != null) {
scan = scan.useSnapshot(snapshotId);
}
if (asOfTimestamp != null) {
scan = scan.asOfTime(asOfTimestamp);
}
```
The call to `asOfTime` on the `TableScan` would replace the snapshot set by the `useSnapshot`.
We don't disallow setting both options. However, I think that if the "snapshot-id" is set, it should take precedence.
I am interested to know what the community thinks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] wypoon closed issue #3070: Precedence if both snapshot-id and as-of-timestamp are specified as options in Spark DataFrameReader
Posted by GitBox <gi...@apache.org>.
wypoon closed issue #3070:
URL: https://github.com/apache/iceberg/issues/3070
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] wypoon commented on issue #3070: Precedence if both snapshot-id and as-of-timestamp are specified as options in Spark DataFrameReader
Posted by GitBox <gi...@apache.org>.
wypoon commented on issue #3070:
URL: https://github.com/apache/iceberg/issues/3070#issuecomment-924214108
I was mistaken. We don't allow setting both options. I don't know how I missed that!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org