[jira] [Commented] (HADOOP-14215) DynamoDB client should waitForActive on existing tables

["Mingliang Liu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15957642#comment-15957642 ] 

Mingliang Liu commented on HADOOP-14215:
----------------------------------------

During test, I found a corner case beyond v0 patch: when the table is being created by another process, it waits it for active w/ v0 patch, and it will try to verify the table version compatibility immediately. This can happen *before* the version marker is put by the other process that is creating the table. This is a race condition. The exception is like:
{code}
java.io.IOException: S3Guard table lacks version marker. Table: mliu-s3guard
	at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.verifyVersionCompatibility(DynamoDBMetadataStore.java:764)
	at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initTable(DynamoDBMetadataStore.java:727)
	at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:239)
	at org.apache.hadoop.fs.s3a.s3guard.S3Guard.getMetadataStore(S3Guard.java:94)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:282)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3257)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:473)
{code}

So I think v0 patch is not enough to address this race condition.

p.s. I also remember that [~rajesh.balamohan] met the similar problem while he was testing S3Guard with Hive. That is to say, w/o v0 patch, the problem is existing (though hidden mostly by not active).

> DynamoDB client should waitForActive on existing tables
> -------------------------------------------------------
>
>                 Key: HADOOP-14215
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14215
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Sean Mackrory
>            Assignee: Sean Mackrory
>         Attachments: HADOOP-14215-HADOOP-13345.000.patch
>
>
> I saw a case where 2 separate applications tried to use the same non-pre-existing table with table.create = true at about the same time. One failed with a ResourceInUse exception. If a table does not exist, we attempt to create it and then wait for it to enter the active state. If another jumps in in the middle of that, the table may exist, thus bypassing our call to waitForActive(), and then try to use the table immediately.
> While we're at it, let's also make sure that the race condition where a table might get created between checking if it exists and attempting to create it is handled gracefully.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org