You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "ramkrishna.s.vasudevan (Created) (JIRA)" <ji...@apache.org> on 2011/12/12 13:02:30 UTC
[jira] [Created] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Failure of creating split dir if it already exists prevents splits from happening further
-----------------------------------------------------------------------------------------
Key: HBASE-5009
URL: https://issues.apache.org/jira/browse/HBASE-5009
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
The scenario is
-> The split of a region takes a long time
-> The deletion of the splitDir fails due to HDFS problems.
-> Subsequent splits also fail after that.
{code}
private static void createSplitDir(final FileSystem fs, final Path splitdir)
throws IOException {
if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
}
{code}
Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173777#comment-13173777 ]
Zhihong Yu commented on HBASE-5009:
-----------------------------------
How long would the loop below take ?
{code}
+ while (!threadPool.isTerminated()) {
+ Thread.sleep(50);
+ }
throw new IOException("Took too long to split the" +
" files and create the references, aborting split");
{code}
I think with the above loop, at least the message in the IOE should be changed because the splitting may take much longer than this.fileSplitTimeout ms.
Should we introduce another timeout for the new loop :-)
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171080#comment-13171080 ]
Zhihong Yu commented on HBASE-5009:
-----------------------------------
In HBASE-5009.patch, I don't see the call to awaitTermination().
I think we should utilize the timeout for awaitTermination() and throw exception if its return value is false.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173236#comment-13173236 ]
Zhihong Yu commented on HBASE-5009:
-----------------------------------
I think we should use threadPool.awaitTermination() where a timeout can be specified so that we don't wait indefinitely.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170933#comment-13170933 ]
ramkrishna.s.vasudevan commented on HBASE-5009:
-----------------------------------------------
The threadPool.awaitTermination() will ensure that the background thread pool is completed.
Also FSUtils.create() was doing an exists check and then createNewFile(). It was followed by create(). Changed it to fs.create(p, false). If the path had existed it will throw exception.
Pls review
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176626#comment-13176626 ]
Hudson commented on HBASE-5009:
-------------------------------
Integrated in HBase-0.92 #211 (See [https://builds.apache.org/job/HBase-0.92/211/])
HBASE-5009 Failure of creating split dir if it already exists prevents splits from happening further
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/Reference.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174135#comment-13174135 ]
Zhihong Yu commented on HBASE-5009:
-----------------------------------
+1 on patch if tests pass.
Thanks for your work Ramkrishna.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5009:
------------------------------------------
Fix Version/s: 0.90.6
0.92.1
Affects Version/s: (was: 0.90.6)
Status: Patch Available (was: Open)
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5009:
------------------------------------------
Attachment: HBASE-5009_Branch90.patch
Patch for 0.90
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5009:
------------------------------
Status: Patch Available (was: Open)
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5009:
------------------------------------------
Attachment: HBASE-5009.patch
Patch for trunk
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176478#comment-13176478 ]
Hadoop QA commented on HBASE-5009:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12508723/5009.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated -151 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 76 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/606//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/606//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/606//console
This message is automatically generated.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5009:
------------------------------
Status: Open (was: Patch Available)
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176467#comment-13176467 ]
Zhihong Yu commented on HBASE-5009:
-----------------------------------
Patch for 0.90 passes tests:
{code}
Tests run: 702, Failures: 0, Errors: 0, Skipped: 9
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:18:08.131s
{code}
Integrated to 0.90 branch.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "gaojinchao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169266#comment-13169266 ]
gaojinchao commented on HBASE-5009:
-----------------------------------
Yes , that is th root reason, I think we should guarantee all children threads is stoped.
At same time, splitdir is not useful, we alse delete it. It seems no harm
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170212#comment-13170212 ]
ramkrishna.s.vasudevan commented on HBASE-5009:
-----------------------------------------------
I am working on this and some more testing and analysis is going on.. will submit a patch after we are ok with it. :)
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167638#comment-13167638 ]
stack commented on HBASE-5009:
------------------------------
@Ram What you think? We should just accept it if splitDir exists already rather than fail? I think I did the above. Was thinking 'fail fast' if come across the unexpected. I'd be fine w/ changing it.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168848#comment-13168848 ]
stack commented on HBASE-5009:
------------------------------
That seems reasonable Ram.
NN won't recreate the dir in 0.92 I believe (now its on hadoop-1.0.0RC2).
How you think we should fix? Should we be more forgiving of case where the splitdir is in place?
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Anoop Sam John (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169264#comment-13169264 ]
Anoop Sam John commented on HBASE-5009:
---------------------------------------
Due to the max wait time out elapse, when we drop the split attempt, we just shutdown the thread pool which do the StoreFileSplitter tasks. This will not guarantee the stop of the threads . Dont you think this could be an issue?
After the time out the split log thread will start the rollback but some threads which it had started still might be there alive and can do some work afterwards.
Ram - The split rollback need to ensure the closure of these threads also?
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168150#comment-13168150 ]
ramkrishna.s.vasudevan commented on HBASE-5009:
-----------------------------------------------
@Stack
After analysis, it was found that spitStoreFiles() is done by a different thread pool.
These threads take more time to complete > fileSplitTimeout(30secs). In that case we shut down the thread pool. But by the time the request to create the split files has reached NN but not yet processed.
By the time we go with roll back and delete the splitDir. Just after this deletion is done the creation of splitfiles is successful in the NN. So the deleted dir is created again.
Hence subsequent splits are failing. Correct me if am wrong.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173730#comment-13173730 ]
ramkrishna.s.vasudevan commented on HBASE-5009:
-----------------------------------------------
@Ted
{code}
boolean stillRunning = !threadPool.awaitTermination(
this.fileSplitTimeout, TimeUnit.MILLISECONDS);
{code}
We already do that. We wait for 30 secs. But the problem is the threads that were spawned are still alive and later they issue the create command to NN. Thus after we rollback and delete the splitdir it is again created by these threads.
Correct me if am wrong Ted.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176660#comment-13176660 ]
Hudson commented on HBASE-5009:
-------------------------------
Integrated in HBase-TRUNK #2587 (See [https://builds.apache.org/job/HBase-TRUNK/2587/])
HBASE-5009 Failure of creating split dir if it already exists prevents splits from happening further
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/Reference.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5009:
------------------------------------------
Status: Open (was: Patch Available)
Cancelling and resubmitting as the tests hanged
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174164#comment-13174164 ]
Hadoop QA commented on HBASE-5009:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12507675/HBASE-5009.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated -152 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/567//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/567//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/567//console
This message is automatically generated.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5009:
------------------------------
Attachment: 5009.txt
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176678#comment-13176678 ]
Hudson commented on HBASE-5009:
-------------------------------
Integrated in HBase-TRUNK-security #52 (See [https://builds.apache.org/job/HBase-TRUNK-security/52/])
HBASE-5009 Failure of creating split dir if it already exists prevents splits from happening further
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/Reference.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176574#comment-13176574 ]
Zhihong Yu commented on HBASE-5009:
-----------------------------------
Integrated to 0.92 and TRUNK.
Thanks for the patch Ramkrishna.
Thanks for the comments, Stack, Jinchao and Anoop.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171413#comment-13171413 ]
ramkrishna.s.vasudevan commented on HBASE-5009:
-----------------------------------------------
@Ted
sorry i meant the isTerminated() api to check whether the threads have been terminated.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5009:
------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174077#comment-13174077 ]
ramkrishna.s.vasudevan commented on HBASE-5009:
-----------------------------------------------
@Ted
Not sure how long it may take.. but it should not be too long.. But this will ensure that the background threads are completed. If not we may get more such errors
{code}
org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /hbase/Htable_UFDR_015/914f3eff1aa7649b7a99b8ce86f3f39c/splits/42423e22baa0c02897a27f92a265f2ad/value/1480356236187113033.914f3eff1aa7649b7a99b8ce86f3f39c File does not exist. [Lease. Holder: DFSClient_hb_rs_193-195-18-165,20020,1324435953313_1324435953584_377812698_27, pendingcreates: 1]
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1786)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1777)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:1834)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:1820)
at org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:738)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:547)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1116)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1112)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1110)
at org.apache.hadoop.ipc.Client.call(Client.java:954)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:245)
at $Proxy6.complete(Unknown Source)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.huawei.isap.ump.ha.client.RPCRetryAndSwitchInvoker.invokeMethod(RPCRetryAndSwitchInvoker.java:183)
at com.huawei.isap.ump.ha.client.RPCRetryAndSwitchInvoker.invokeMethod(RPCRetryAndSwitchInvoker.java:171)
at com.huawei.isap.ump.ha.client.RPCRetryAndSwitchInvoker.invoke(RPCRetryAndSwitchInvoker.java:76)
at $Proxy6.complete(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:4307)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:4211)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:63)
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:88)
at org.apache.hadoop.hbase.io.Reference.write(Reference.java:133)
at org.apache.hadoop.hbase.regionserver.StoreFile.split(StoreFile.java:651)
at org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFile(SplitTransaction.java:498)
{code}
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.6
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177046#comment-13177046 ]
Hudson commented on HBASE-5009:
-------------------------------
Integrated in HBase-0.92-security #51 (See [https://builds.apache.org/job/HBase-0.92-security/51/])
HBASE-5009 Failure of creating split dir if it already exists prevents splits from happening further
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/Reference.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176433#comment-13176433 ]
Zhihong Yu commented on HBASE-5009:
-----------------------------------
Pressing 'Submit' again wouldn't run the tests.
This is because Hadoop QA remembers the attachment Id of the patch for which test suite was executed.
Please attach the same copy of patch for TRUNK again.
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5009) Failure of creating split dir if it
already exists prevents splits from happening further
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5009:
------------------------------
Status: Patch Available (was: Open)
> Failure of creating split dir if it already exists prevents splits from happening further
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-5009
> URL: https://issues.apache.org/jira/browse/HBASE-5009
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.1, 0.90.6
>
> Attachments: 5009.txt, HBASE-5009.patch, HBASE-5009_Branch90.patch
>
>
> The scenario is
> -> The split of a region takes a long time
> -> The deletion of the splitDir fails due to HDFS problems.
> -> Subsequent splits also fail after that.
> {code}
> private static void createSplitDir(final FileSystem fs, final Path splitdir)
> throws IOException {
> if (fs.exists(splitdir)) throw new IOException("Splitdir already exits? " + splitdir);
> if (!fs.mkdirs(splitdir)) throw new IOException("Failed create of " + splitdir);
> }
> {code}
> Correct me if am wrong? If it is an issue can we change the behaviour of throwing exception?
> Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira