You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Thomas Wang <w...@datability.io> on 2021/09/23 20:59:10 UTC
Exact S3 Permissions to allow a flink job to use s3 for checkpointing
Hi,
I'm trying to figure out what exact s3 permissions does a flink job need to
work appropriately when using s3 for checkpointing. Currently, I have the
following IAM Policy, but it seems insufficient. Can anyone help me figure
this out? Thanks.
{
Action = [
"s3:PutObject",
"s3:GetObject",
]
Effect = "Allow"
Resource = "arn:aws:s3:::<BUCKET_NAME>/<SOME_PATH>/*"
},
Thomas
Re: Exact S3 Permissions to allow a flink job to use s3 for
checkpointing
Posted by "Meissner, Dylan" <dy...@nordstrom.com>.
Thomas, I use only these actions for checkpoints and savepoints: s3:GetObject, s3:PutObject, s3:DeleteObject, and s3:ListBucket
Cheers,
Dylan
________________________________
From: Guowei Ma <gu...@gmail.com>
Sent: Friday, September 24, 2021 12:47 AM
To: Thomas Wang <w...@datability.io>
Cc: user <us...@flink.apache.org>
Subject: Re: Exact S3 Permissions to allow a flink job to use s3 for checkpointing
Hi, Thomas
I am not an expert of s3 but I think Flinkneed write/read/delete(maybe list) permission of the path(bucket).
BTW, What error did you encounter?
Best,
Guowei
On Fri, Sep 24, 2021 at 5:00 AM Thomas Wang <w@...@datability.io>> wrote:
Hi,
I'm trying to figure out what exact s3 permissions does a flink job need to work appropriately when using s3 for checkpointing. Currently, I have the following IAM Policy, but it seems insufficient. Can anyone help me figure this out? Thanks.
{
Action = [
"s3:PutObject",
"s3:GetObject",
]
Effect = "Allow"
Resource = "arn:aws:s3:::<BUCKET_NAME>/<SOME_PATH>/*"
},
Thomas
Re: Exact S3 Permissions to allow a flink job to use s3 for checkpointing
Posted by Guowei Ma <gu...@gmail.com>.
Hi, Thomas
I am not an expert of s3 but I think Flinkneed write/read/delete(maybe
list) permission of the path(bucket).
BTW, What error did you encounter?
Best,
Guowei
On Fri, Sep 24, 2021 at 5:00 AM Thomas Wang <w...@datability.io> wrote:
> Hi,
>
> I'm trying to figure out what exact s3 permissions does a flink job need
> to work appropriately when using s3 for checkpointing. Currently, I have
> the following IAM Policy, but it seems insufficient. Can anyone help me
> figure this out? Thanks.
>
> {
> Action = [
> "s3:PutObject",
> "s3:GetObject",
> ]
> Effect = "Allow"
> Resource = "arn:aws:s3:::<BUCKET_NAME>/<SOME_PATH>/*"
> },
>
> Thomas
>