[jira] [Commented] (HADOOP-14576) DynamoDB tables may leave ACTIVE state after initial connection

["Aaron Fabbri (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-14576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16075123#comment-16075123 ] 

Aaron Fabbri commented on HADOOP-14576:
---------------------------------------

Thanks for filing the JIRA, [~mackrorysd].

{quote}
My concern with failing over to S3 for non-auth read is what happens when you're listing stuff that isn't consistent on S3 yet. IMO non-auth mode is really just to enable lazily loading data that already existed or that is added outside of S3Guard. I don't think it should weaken guarantees in the presence of partitioning events in DynamoDB.
{quote}

Yeah I think both arguments have merit.

We could argue that failing back to basic S3 without consistency is better than failing a job.  We could also have a configuration flag that lets users choose either behavior.    Since the chance of inconsistency is pretty low,  there is a good probability that running in degraded mode (no S3Guard) until the table comes back would be successful.

I'm not sure authoritative mode matters?  The client being in S3guard authoritative mode just means the the FS client *may* skip round trips to S3 *if* the metadatastore reports it has a full listing.   Since MetadatStore throws an error, the "if metadatastore reports full listing" condition is not met.

> DynamoDB tables may leave ACTIVE state after initial connection
> ---------------------------------------------------------------
>
>                 Key: HADOOP-14576
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14576
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: HADOOP-13345
>            Reporter: Sean Mackrory
>
> We currently only anticipate tables not being in the ACTIVE state when first connecting. It is possible for a table to be in the ACTIVE state and move to an UPDATING state during partitioning events. Attempts to read or write during that time will result in an AmazonServerException getting thrown. We should try to handle that better...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org