[jira] [Comment Edited] (HBASE-20984) Add/Modify test case to check custom hbase.wal.dir outside hdfs filesystem

["Sean Busbey (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-20984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16723208#comment-16723208 ] 

Sean Busbey edited comment on HBASE-20984 at 12/17/18 5:46 PM:
---------------------------------------------------------------

{quote}
{code}

--- a/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
+++ b/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
@@ -551,16 +551,16 @@ public class HBaseTestingUtility extends HBaseZKTestingUtility {
   /**
    * Sets up a new path in test filesystem to be used by tests.
    */
-  private Path getNewDataTestDirOnTestFS() throws IOException {
+  private Path getNewDataTestDirOnTestFS(FileSystem fs) throws IOException {
     //The file system can be either local, mini dfs, or if the configuration
     //is supplied externally, it can be an external cluster FS. If it is a local
     //file system, the tests should use getBaseTestDir, otherwise, we can use
     //the working directory, and create a unique sub dir there
-    FileSystem fs = getTestFileSystem();
+
     Path newDataTestDir;
     String randomStr = getRandomUUID().toString();
     if (fs.getUri().getScheme().equals(FileSystem.getLocal(conf).getUri().getScheme())) {
-      newDataTestDir = new Path(getDataTestDir(), randomStr);
+      newDataTestDir = new Path("file:///"+getDataTestDir(), randomStr);
       File dataTestDir = new File(newDataTestDir.toString());
       if (deleteOnExit()) dataTestDir.deleteOnExit();
     } else {
{code}
{quote}

this is a little brittle, since it assumes the scheme for the Filesystem returned by {{FileSystem.getLocal(conf)}} will be {{file://}}, but that's unlikely to change anytime soon. (unless the Hadoop encrypted filesystem supports directly wrapping a local fs instance?)

Could you change this to use {{makeQualified}} from the {{fs}} instead of inserting a scheme in the string? Or does that break things?


was (Author: busbey):
{quote}
{code}

562	    if (fs.getUri().getScheme().equals(FileSystem.getLocal(conf).getUri().getScheme())) {	562	    if (fs.getUri().getScheme().equals(FileSystem.getLocal(conf).getUri().getScheme())) {
563	      newDataTestDir = new Path(getDataTestDir(), randomStr);	563	      newDataTestDir = new Path("file:///"+getDataTestDir(), randomStr);
564	      File dataTestDir = new File(newDataTestDir.toString());	564	      File dataTestDir = new File(newDataTestDir.toString());
565	      if (deleteOnExit()) dataTestDir.deleteOnExit();	565	      if (deleteOnExit()) dataTestDir.deleteOnExit();
566	    } else {	566	    } else {
{code}
{quote}

this is a little brittle, since it assumes the scheme for the Filesystem returned by {{FileSystem.getLocal(conf)}} will be {{file://}}, but that's unlikely to change anytime soon. (unless the Hadoop encrypted filesystem supports directly wrapping a local fs instance?)

Could you change this to use {{makeQualified}} from the {{fs}} instead of inserting a scheme in the string? Or does that break things?

> Add/Modify test case to check custom hbase.wal.dir outside hdfs filesystem
> --------------------------------------------------------------------------
>
>                 Key: HBASE-20984
>                 URL: https://issues.apache.org/jira/browse/HBASE-20984
>             Project: HBase
>          Issue Type: Bug
>          Components: test, wal
>            Reporter: Sakthi
>            Assignee: Sakthi
>            Priority: Minor
>         Attachments: hbase-20984.master.001.patch, hbase-20984.master.002.patch
>
>
> The current setup in TestWALFactory tries to create custom WAL directory outside hdfs but ends up creating a custom WAL directory inside hdfs. In TestWALFactory.java:
> {code:java}
> public static void setUpBeforeClass() throws Exception {
>     CommonFSUtils.setWALRootDir(TEST_UTIL.getConfiguration(), new Path("file:///tmp/wal")); // A local filesystem WAL is attempted
> ...
>     hbaseDir = TEST_UTIL.createRootDir();
>     hbaseWALDir = TEST_UTIL.createWALRootDir(); // But a directory inside hdfs is created here using HBaseTestingUtility#getNewDataTestDirOnTestFS
> }
> {code}
> The change was made in HBASE-20723



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)