You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "jackye1995 (via GitHub)" <gi...@apache.org> on 2024/05/21 03:48:41 UTC
[PR] View: add property to describe advisory read mode [iceberg]
jackye1995 opened a new pull request, #10362:
URL: https://github.com/apache/iceberg/pull/10362
This is basically the concept of secure view, but separated into different levels based on our experience of implementing it in EMR Spark. I also considered making this a spec change, but using a property seems sufficient here.
@danielcweeks @nastra @amogh-jahagirdar @rdblue @wmoustafa @jzhuge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] View: add property to describe advisory read mode [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10362:
URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2453235736
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] View: add property to describe advisory read mode [iceberg]
Posted by "jackye1995 (via GitHub)" <gi...@apache.org>.
jackye1995 commented on PR #10362:
URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2125460867
> It feels like this is less an attribute of the view definition and more of a decision based on policy
yes that is correct. This is a type of access decision, and it is mentioned in that proposal. The same ask is coming from multiple products at the same time.
Are you proposing this as a change to the REST spec directly? The reason I am proposing a change to the definition first is because user can define a default behavior for our engine to follow independent of its owning catalog or policy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] View: add property to describe advisory read mode [iceberg]
Posted by "jackye1995 (via GitHub)" <gi...@apache.org>.
jackye1995 commented on PR #10362:
URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2124943243
Here is another doc in BigQuery explaining possible attacks: https://cloud.google.com/bigquery/docs/best-practices-row-level-security#limit-side-channel-attacks
And to enforce this, features like row-level security "do not participate in query pruning on partitioned and clustered tables", as described in https://cloud.google.com/bigquery/docs/best-practices-row-level-security#filtering_on_partitioned_columns_impacts_performancesecurity#filtering_on_partitioned_columns_impacts_performance
That is for row-level security, but it's a similar concept for view that we are talking about here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] View: add property to describe advisory read mode [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #10362: View: add property to describe advisory read mode
URL: https://github.com/apache/iceberg/pull/10362
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] View: add property to describe advisory read mode [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #10362:
URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2123228246
@jackye1995 can you help me understand the case that this is protecting against? I'm not sure I understand the loophole that this is attempting to close.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] View: add property to describe advisory read mode [iceberg]
Posted by "jackye1995 (via GitHub)" <gi...@apache.org>.
jackye1995 commented on PR #10362:
URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2123261221
This allows an engine to distinguish if a view is a secure view, and act accordingly. As described in similar systems like https://docs.snowflake.com/en/user-guide/views-secure:
> Some of the internal optimizations for views require access to the underlying data in the base tables for the view. This access might allow data that is hidden from users of the view to be exposed through user code, such as user-defined functions, or other programmatic methods. Secure views do not utilize these optimizations, ensuring that users have no access to the underlying data.
And what we are saying here is, in between utilize the optimizations (open) and not utilize the optimizations (isolated), there is an intermediate state which is trying to only utilize optimizations that the engine considers as safe, which is the protected mode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] View: add property to describe advisory read mode [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10362:
URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2467006408
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] View: add property to describe advisory read mode [iceberg]
Posted by "jackye1995 (via GitHub)" <gi...@apache.org>.
jackye1995 commented on PR #10362:
URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2124810239
Pushdown might be not a good word to use here, if there is a better wording let me know.
The Snowflake doc provides a good case: https://docs.snowflake.com/en/user-guide/views-secure#how-might-data-be-exposed-by-a-non-secure-view. A filter to get additional information out of a view like `1/iff(color = 'Purple', 0, 1) = 1` is what we are trying to have different ways to handle. In the latest EMR Spark security features we distinguish those concepts when evaluating a view in an execution plan. So to have a good integration with Iceberg view, we would like to introduce this concept.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] View: add property to describe advisory read mode [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on PR #10362:
URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2125327982
@jackye1995 it seems like the mode would be based on the policy applied to the subject performing the query. For example, if someone has full access to the data, all optimizations should be applied. If someone has restricted access, then you would want to signal that.
It feels like this is less an attribute of the view definition and more of a decision based on policy. Shouldn't this just be handled in the "access decision" proposal?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] View: add property to describe advisory read mode [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #10362:
URL: https://github.com/apache/iceberg/pull/10362#issuecomment-2123427489
Can you give an example? I can't think of any pushdown operation that exposes data to user code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org