You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (Created) (JIRA)" <ji...@apache.org> on 2011/10/07 18:01:30 UTC
[jira] [Created] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
The update of .tableinfo is not atomic; we remove then rename
-------------------------------------------------------------
Key: HBASE-4553
URL: https://issues.apache.org/jira/browse/HBASE-4553
Project: HBase
Issue Type: Task
Reporter: stack
This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141382#comment-13141382 ]
stack commented on HBASE-4553:
------------------------------
The above TestLogRolling was because could not get lock on dir.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140957#comment-13140957 ]
jiraposter@reviews.apache.org commented on HBASE-4553:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2656/
-----------------------------------------------------------
(Updated 2011-11-01 05:47:41.332884)
Review request for hbase.
Summary
-------
Retry.
This addresses bug hbase-4553.
https://issues.apache.org/jira/browse/hbase-4553
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/HConstants.java 943b774
src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 46ca765
src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java b9d4b90
src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java 24570c9
src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 36d90c3
src/main/java/org/apache/hadoop/hbase/util/HMerge.java a6f6b69
src/main/java/org/apache/hadoop/hbase/util/Merge.java 3aa980f
src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 1cfeea8
src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java 8404d1b
src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 49bcf02
src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java b2cb233
src/test/java/org/apache/hadoop/hbase/util/TestFSTableDescriptors.java 8fce6ec
src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java 1ad30e6
src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java ffc8d9d
src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java 88dc9de
Diff: https://reviews.apache.org/r/2656/diff
Testing
-------
Thanks,
Michael
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Attachment: 4553-v10.txt
Here is another version w/ more javadoc explaining whats going on and some more tests.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Patch Available (was: Open)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Attachment: 4553-v13.txt
Draining servers test case needed updating to match the changes this patch brings on.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Patch Available (was: Open)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Patch Available (was: Open)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Open (was: Patch Available)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Attachment: 3446-v8.txt
Change .tableinfo naming so its now .tableinfo.SEQUENCEID. The sequenceid always rolls forward (next seqid is based off the old one). This removes hole where we could have no .tableinfo in case where editors crash at inopportune time (TODO: Same for .regioninfo -- should be a single .regioninfo rather than one per region).
Bulk of patch is moving TableDescriptor utility out of FSUtils into the FSTableDescriptors class.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Attachment: 4553-v9.txt
Add back creating of files in .tmp dir so we don't have partials if crash.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Open (was: Patch Available)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-4553) The update of .tableinfo
is not atomic; we remove then rename
Posted by "Ted Yu (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128081#comment-13128081 ]
Ted Yu edited comment on HBASE-4553 at 10/15/11 5:45 AM:
---------------------------------------------------------
There was an issue in testTableAdminAndMetadata.
The wait condition was flawed. Here it goes.
was (Author: thehellmaker):
There was an issue in testTableAdminAndMetadata.
The wait condition was flawed. Ted helped me out with a patch. Here it goes.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Work started) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HBASE-4553 started by stack.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Open (was: Patch Available)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Assignee: stack
Status: Patch Available (was: Open)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140958#comment-13140958 ]
jiraposter@reviews.apache.org commented on HBASE-4553:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2656/
-----------------------------------------------------------
Review request for hbase.
Summary
-------
Retry.
This addresses bug hbase-4553.
https://issues.apache.org/jira/browse/hbase-4553
Diffs
-----
Diff: https://reviews.apache.org/r/2656/diff
Testing
-------
Thanks,
Michael
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144321#comment-13144321 ]
Hadoop QA commented on HBASE-4553:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502501/4553-v13.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 27 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -164 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
org.apache.hadoop.hbase.master.TestMasterFailover
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/177//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/177//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/177//console
This message is automatically generated.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143615#comment-13143615 ]
Hadoop QA commented on HBASE-4553:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502233/4553-v12.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -164 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/166//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/166//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/166//console
This message is automatically generated.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142829#comment-13142829 ]
Hadoop QA commented on HBASE-4553:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501986/4553-v11.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/151//console
This message is automatically generated.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143572#comment-13143572 ]
Hadoop QA commented on HBASE-4553:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502227/4553-v12.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -164 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/164//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/164//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/164//console
This message is automatically generated.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Attachment: 4553-v5.txt
Not done yet. What I have so far.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 4553-v5.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123616#comment-13123616 ]
stack commented on HBASE-4553:
------------------------------
@Todd Thanks boss.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Attachment: 4553-v12.txt
Rebase against trunk.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141501#comment-13141501 ]
Hadoop QA commented on HBASE-4553:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501803/4553-v10.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -166 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/125//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/125//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/125//console
This message is automatically generated.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Open (was: Patch Available)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123567#comment-13123567 ]
stack commented on HBASE-4553:
------------------------------
So, I see this in this failure up on jenkins in https://builds.apache.org/job/HBase-0.92/53/artifact/trunk/target/surefire-reports/org.apache.hadoop.hbase.avro.TestAvroServer-output.txt
I see that we updated .tableinfo and a check on the files modtime at about the same time fails with filenotfound.
Todd, should we spin here till file shows up?
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141387#comment-13141387 ]
jiraposter@reviews.apache.org commented on HBASE-4553:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2656/
-----------------------------------------------------------
(Updated 2011-11-01 17:52:00.184660)
Review request for hbase.
Changes
-------
Updated diff; more javadoc and tests.
Summary
-------
Retry.
This addresses bug hbase-4553.
https://issues.apache.org/jira/browse/hbase-4553
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 46ca765
src/main/java/org/apache/hadoop/hbase/HConstants.java 76278a2
src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java b9d4b90
src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java 24570c9
src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 36d90c3
src/main/java/org/apache/hadoop/hbase/util/HMerge.java a6f6b69
src/main/java/org/apache/hadoop/hbase/util/Merge.java 3aa980f
src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 0a6f703
src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java 8404d1b
src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 49bcf02
src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java b2cb233
src/test/java/org/apache/hadoop/hbase/util/TestFSTableDescriptors.java 8fce6ec
src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java 1ad30e6
src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 21a57ca
src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java e4b3d75
Diff: https://reviews.apache.org/r/2656/diff
Testing
-------
Thanks,
Michael
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143466#comment-13143466 ]
Hadoop QA commented on HBASE-4553:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502178/4553-v12.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -164 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint
org.apache.hadoop.hbase.TestGlobalMemStoreSize
org.apache.hadoop.hbase.master.TestMasterFailover
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/158//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/158//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/158//console
This message is automatically generated.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123603#comment-13123603 ]
Todd Lipcon commented on HBASE-4553:
------------------------------------
atomic operations on filesystems are tricky... to do this "correctly" in the face of crashes, we need to have some process either do a rollback or roll-forward to recover from failures. Something like:
writer:
- create tableinfo.tmp
- delete tableinfo
- rename tableinfo.tmp to tableinfo
reader:
- try to read tableinfo
- on IOE (block missing, etc), that means that the file was deleted underneath. So spin until the file open succeeds.
if the writer crashes between the delete and rename, we need someone else to come in and "finish" the operation.
IMO we need some general purpose way of allowing the master to keep an "intent log" in ZK for this kind of thing - and then if the master fails over, it can complete the operation.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Open (was: Patch Available)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144435#comment-13144435 ]
stack commented on HBASE-4553:
------------------------------
I ran test suite on local ubuntu and all passed except roll log; it had this:
{code}
Tests in error:
testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling): Cannot lock storage /export1/stack/hbase/target/test-data/74208e88-deaa-4d3c-a6ed-d316c5aa5ef2/dfscluster_0515d0e6-8fcd-47c0-8b34-9126722fc1cb/dfs/data/data3. The directory is already locked.
{code}
which is unrelated. Committing v13.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Attachment: 4553-v11.txt
Same as v10.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144497#comment-13144497 ]
Hudson commented on HBASE-4553:
-------------------------------
Integrated in HBase-0.92 #109 (See [https://builds.apache.org/job/HBase-0.92/109/])
HBASE-4553 The update of .tableinfo is not atomic; we remove then rename
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HMerge.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/Merge.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestFSTableDescriptors.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Attachment: 4553-v12.txt
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141013#comment-13141013 ]
Hadoop QA commented on HBASE-4553:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501740/4553-v9.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -166 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.master.TestMasterFailover
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/119//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/119//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/119//console
This message is automatically generated.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Patch Available (was: Open)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128074#comment-13128074 ]
Ted Yu commented on HBASE-4553:
-------------------------------
I don't see FNFE in TestAvroServer-output although the test failure is consistent.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Priority: Critical
> Fix For: 0.92.0
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Attachment: 4553-v12.txt
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Priority: Critical (was: Major)
Fix Version/s: 0.92.0
Marking critical for 0.92. I see this in unit tests from time to time -- TestAvroServer does it pretty reliably. If no .tableinfo, we're kinda hosed so needs to work. I'm on this one.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Priority: Critical
> Fix For: 0.92.0
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129188#comment-13129188 ]
Ted Yu commented on HBASE-4553:
-------------------------------
We should impose an upper limit on the amount of time we wait in the loop.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140959#comment-13140959 ]
stack commented on HBASE-4553:
------------------------------
Retried upload to rb (Sorry for the no diff LarsH).
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Attachment: 4553-v12.txt
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Open (was: Patch Available)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Patch Available (was: Open)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143312#comment-13143312 ]
Hadoop QA commented on HBASE-4553:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12502166/4553-v12.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -164 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.master.TestMasterFailover
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/155//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/155//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/155//console
This message is automatically generated.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Akash Ashok (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130956#comment-13130956 ]
Akash Ashok commented on HBASE-4553:
------------------------------------
https://issues.apache.org/jira/browse/HBASE-4621 opened for the TestCaseFix . Shall modify and submit the patch out there. Thanks
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 4553-v5.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140679#comment-13140679 ]
stack commented on HBASE-4553:
------------------------------
I need to put back the bit where .tableinfo is made in a tmp dir.... Will make a new patch later.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129660#comment-13129660 ]
stack commented on HBASE-4553:
------------------------------
Patch looks fine Akash (long as you ensure an upper bound on test as per Ted suggestion). I'd suggest you do it in another issue though rather than here. This one is about .tableinfo updating. Thanks boss.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Open (was: Patch Available)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Akash Ashok (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Akash Ashok updated HBASE-4553:
-------------------------------
Attachment: HBase-4553-TestAvroServer.patch
There was an issue in testTableAdminAndMetadata.
The wait condition was flawed. Ted helped me out with a patch. Here it goes.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140666#comment-13140666 ]
Hadoop QA commented on HBASE-4553:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501695/3446-v8.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/115//console
This message is automatically generated.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v5.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Patch Available (was: In Progress)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed branch and trunk.
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4553:
-------------------------
Status: Patch Available (was: Open)
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not
atomic; we remove then rename
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144579#comment-13144579 ]
Hudson commented on HBASE-4553:
-------------------------------
Integrated in HBase-TRUNK #2411 (See [https://builds.apache.org/job/HBase-TRUNK/2411/])
HBASE-4553 The update of .tableinfo is not atomic; we remove then rename
stack :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HMerge.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/Merge.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestFSTableDescriptors.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java
> The update of .tableinfo is not atomic; we remove then rename
> -------------------------------------------------------------
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists already. In 0.20+ its better but still 'some' issues if existing reader when file is renamed. This issue is about fixing this (though we depend on fix first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira