[jira] [Updated] (HADOOP-14323) ITestS3GuardListConsistency failure w/ Local, authoritative metadata store

["Aaron Fabbri (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-14323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aaron Fabbri updated HADOOP-14323:
----------------------------------
    Attachment: HADOOP-14323-HADOOP-13345.001.patch

Attaching patch.

The problem here is that the test is using {{S3Guard.getMetadataStore()}} which is creating a new {{MetadataStore}} instance.  This works with dynamo because multiple instances share a back end.  For LocalMetadataStore, however, it will create a new, empty, store, thus the null result.

This is the main fix:

{noformat}
@@ -194,7 +201,7 @@ public void testListStatusWriteBack() throws Exception {

     // Metadata store without write-back should still only contain /OnS3AndMS,
     // because newly discovered /OnS3 is not written back to metadata store
-    mdResults = S3Guard.getMetadataStore(noWriteBack).listChildren(directory);
+    mdResults = noWriteBack.getMetadataStore().listChildren(directory);
     assertEquals("Metadata store without write back should still only know "
             + "about /OnS3AndMS, but it has: " + mdResults,
         1, mdResults.numEntries());
{noformat}

To [~stevel@apache.org]'s question: Null returns here are allowed:  A MetadataStore may decide not to retain entries due to their own eviction policy.  MetadataStoreTestBase has an {{allowMissing()}} predicate that avoids test failures for the longer-running MetadataStore contract tests.  Null returns in this case will not happen with the default sizing of the LocalMetadataStore, though, so I think this patch is sufficient.

> ITestS3GuardListConsistency failure w/ Local, authoritative metadata store
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-14323
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14323
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: s3
>    Affects Versions: HADOOP-13345
>            Reporter: Aaron Fabbri
>            Assignee: Aaron Fabbri
>            Priority: Minor
>         Attachments: HADOOP-14323-HADOOP-13345.001.patch
>
>
> When doing some testing for HADOOP-14266 I noticed this test failure:
> {noformat}
> java.lang.NullPointerException: null
>         at org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency.testListStatusWriteBack(ITestS3GuardListConsistency.java:317)
> {noformat}
> I was running with LocalMetadataStore and {{fs.s3a.metadatastore.authoritative}} set to true.  I haven't been testing this mode recently so not sure if this case ever worked.  Lower priority but we should fix it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org