You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2011/03/29 23:33:05 UTC
[jira] [Created] (HDFS-1795) Port 0.20-append changes onto
0.20-security-203
Port 0.20-append changes onto 0.20-security-203
-----------------------------------------------
Key: HDFS-1795
URL: https://issues.apache.org/jira/browse/HDFS-1795
Project: Hadoop HDFS
Issue Type: Task
Reporter: Andrew Purtell
Attachments: security-append-patches.zip
Port 0.20-append changes onto 0.20-security-203.
I started with a Git repository cloned from git://git.apache.org/hadoop-common.git . Branch 'branch-0.20-security-203' was used as the starting point for the work. I then enumerated over the 0.20-append specific patches in 'branch-0.20-append'. Each was applied if not already via cherry pick except for as noted below. This process in effect replayed the evolution of 0.20-append branch on top of 0.20-security-203.
The specific functional changes that HBase absolutely relies upon are specially mentioned.
Generally I ran the full test suite after each change. There were a couple of exceptions where pairs of adjacent change sets were strongly related, in which case I applied them in sequence, then ran the test suite. During this process I encountered no test failures except for one test in TestFileAppend4, a test brought in from the append branch, and I still need to dig in to see if this is a real problem or if the test needs to be changed to work on top of security-203.
{noformat}
commit b9ad012eaf3915c2169a02a7130b54cbcc1d8a89
Author: Dhruba Borthakur <dh...@apache.org>
Date: Fri Jun 4 07:20:10 2010 +0000
HDFS-200. Support append and sync for hadoop 0.20 branch.
Required for HBase
commit c968e11b5a60fc6f28e4e43fbbc8a99e7e49a659
Author: Dhruba Borthakur <dh...@apache.org>
Date: Wed Jun 9 23:09:07 2010 +0000
HDFS-101. DFSClient correctly detects second datanode failure in write
pipeline. (Nicolas Spiegelberg via dhruba)
Excluded
Already in 0.20-security-203 according to search of Git change log
commit 9f7e5ed2ff47444a1dcd12ed34796929d5b9f7d5
Author: Dhruba Borthakur <dh...@apache.org>
Date: Wed Jun 9 23:12:21 2010 +0000
HDFS-988. Fix bug where savenameSpace can corrupt edits log.
(Nicolas Spiegelberg via dhruba)
commit dfbbd6fbadaa95c54a1040b4fe8854b1b858d7a5
Author: Dhruba Borthakur <dh...@apache.org>
Date: Thu Jun 10 18:46:03 2010 +0000
HDFS-826. Allow a mechanism for an application to detect that
datanode(s) have died in the write pipeline. (dhruba)
Required for HBase
Commit be8d32503d30208a2d7772b3b4b2a270938a4004
Author: Dhruba Borthakur <dh...@apache.org>
Date: Thu Jun 10 22:25:39 2010 +0000
HDFS-142. Blocks that are being written by a client are stored in the
blocksBeingWritten directory.
(Dhruba Borthakur, Nicolas Spiegelberg, Todd Lipcon via dhruba)
commit 856efc2e95aaacc597d669c1b053634ff752dbec
Author: Dhruba Borthakur <dh...@apache.org>
Date: Fri Jun 11 00:48:41 2010 +0000
HDFS-630. Client can exclude specific nodes in the write pipeline.
(Nicolas Spiegelberg via dhruba)
Required for HBase
commit 2da1a05fc0cc0429229e87694977bae2ba370625
Author: Dhruba Borthakur <dh...@apache.org>
Date: Fri Jun 11 01:02:13 2010 +0000
HDFS-457. Better handling of volume failure in DataNode Storage.
(Nicolas Spiegelberg via dhruba)
Excluded
Already in 0.20-security-203 according to search of Git change log
commit bd42393cd3a3a731ea98b25ddb528ad03a1ab4af
Author: Dhruba Borthakur <dh...@apache.org>
Date: Fri Jun 11 23:37:38 2010 +0000
HDFS-1054. remove sleep before retry for allocating a block.
(Todd Lipcon via dhruba)
commit 120441b9e571a5703ac39b47608e87182f0f4972
Author: Dhruba Borthakur <dh...@apache.org>
Date: Wed Jun 16 20:53:12 2010 +0000
HDFS-445. pread should refetch block locations when necessary.
(Todd Lipcon via dhruba)
Excluded
Already in 0.20-security-203 according to search of Git change log
commit 2004aa453ba6b7ee2045093ba313ef8551a7f8da
Author: Dhruba Borthakur <dh...@apache.org>
Date: Wed Jun 16 20:59:10 2010 +0000
HDFS-561. Fix write pipeline
commit 2a8227b0e6be8937fc4a654899be2a22c1f6efbe
Author: Dhruba Borthakur <dh...@apache.org>
Date: Wed Jun 16 21:13:24 2010 +0000
HDFS-927. DFSInputStream retries too many times for new block
locations. (Todd Lipcon via dhruba)
Excluded
Already in 0.20-security-203 according to search of Git change log
commit b1e49dbf50a429cf01b636caa2666ff81ed2a016
Author: Dhruba Borthakur <dh...@apache.org>
Date: Wed Jun 16 21:21:45 2010 +0000
HDFS-1215. Fix unti test TestNodeCount.
(Todd Lipcon via dhruba)
Excluded
TestNodeCount not failing on current working branch
commit b5078c57478deda9974ed40897045d38b100b9fc
Author: Dhruba Borthakur <dh...@apache.org>
Date: Wed Jun 16 21:25:38 2010 +0000
HDFS-606. Fix ConcurrentModificationException in
invalidateCorruptReplicas. (Todd Lipcon via dhruba)
commit c3d35f8cdd761cbcd0a4ef851e37ccdb2d25d7e0
Author: Dhruba Borthakur <dh...@apache.org>
Date: Wed Jun 16 21:42:53 2010 +0000
HDFS-1216. Upgrade to junit 4.5
(Todd Lipcon via dhruba)
Excluded
Already using Junit 4.5 on current working branch
commit c21b6834e982b0bf2edaf61dae8a0028076ccb60
Author: Dhruba Borthakur <dh...@apache.org>
Date: Thu Jun 17 01:34:56 2010 +0000
HDFS-1207. FSNamesystem.stallReplicationWork should be volatile.
(Todd Lipcon via dhruba)
commit e8a1a09fec716bb013f94f1c14f4ae8f8409ee68
Author: Dhruba Borthakur <dh...@apache.org>
Date: Wed Jun 16 21:57:39 2010 +0000
HDFS-1141. completeFile does not check lease ownership.
(Todd Lipcon via dhruba)
commit ca458a07a6d8cb5cda8c589b1a23dabffacef7fe
Author: Dhruba Borthakur <dh...@apache.org>
Date: Thu Jun 17 01:44:32 2010 +0000
HDFS-1204. Lease expiration should recover single files,
not entire lease holder (Sam Rash via dhruba)
commit 62441fbd516ec9132619d448a1051554d29d2dba
Author: Dhruba Borthakur <dh...@apache.org>
Date: Thu Jun 17 01:52:50 2010 +0000
HDFS-1210. DFSClient should log exception when block recovery fails.
(Todd Lipcon via dhruba)
Not a required change but a minor useful one
commit 08b67cdf9ff7dada8aa86a2443a0ffc1a0a83c20
Author: Dhruba Borthakur <dh...@apache.org>
Date: Thu Jun 17 01:59:30 2010 +0000
HDFS-1211. Block receiver should not log "rewind" packets at INFO
level. (Todd Lipcon via dhruba)
Not a required change but a minor useful one
commit 0c9911a623891e82c40d4c1c3a523f3a85a85216
Author: Dhruba Borthakur <dh...@apache.org>
Date: Sun Jun 20 08:31:54 2010 +0000
HDFS-1118. Fix socketleak on DFSClient.
(Zheng Shao via dhruba)
commit db9989c091b37446e4f4d468b24c4bd08b1ea337
Author: Tsz-wo Sze <sz...@apache.org>
Date: Mon Jun 21 21:26:45 2010 +0000
MAPREDUCE-1880. Fix BigDecimal.divide(..) in the pi example.
Excluded
Not an HDFS change
commit 11fa325540526da57a9e178463017853058ba921
Author: Dhruba Borthakur <dh...@apache.org>
Date: Tue Jun 22 05:55:48 2010 +0000
HDFS-1254. Support append/sync via the default configuration.
(dhruba)
commit 9e405ca177aba719b04cfd7d33d400d311bf8db7
Author: Dhruba Borthakur <dh...@apache.org>
Date: Thu Jul 1 08:37:30 2010 +0000
HDFS-1057. Concurrent readers hit ChecksumExceptions if following
a writer to very end of file (Sam Rash via dhruba)
commit 6512b527cda8efa0c622a8657af96ac785625773
Author: Dhruba Borthakur <dh...@apache.org>
Date: Tue Jul 6 06:52:20 2010 +0000
HDFS-1202. DataBlockScanner throws NPE when updated before
initialized. (Todd Lipcon via dhruba)
commit 116e71fea80c31042520f364625ba7b1647cdb09
Author: Tsz-wo Sze <sz...@apache.org>
Date: Tue Jul 6 17:39:25 2010 +0000
HDFS-1258. Clearing namespace quota on "/" corrupts fs image.
commit ca88978b319debe146e9988d882393033b1d2d62
Author: Tsz-wo Sze <sz...@apache.org>
Date: Wed Sep 22 20:52:25 2010 +0000
MAPREDUCE-1280. Update Eclipse plugin to the new eclipse.jdt API.
Excluded
Not an HDFS change
commit a68b163fc31c3d82c63b69ae99e9954fef4df4af
Author: Hairong Kuang <ha...@apache.org>
Date: Mon Sep 27 18:14:20 2010 +0000
HDFS-1346. DFSClient receives out of order packet ack.
commit 7083b46208f56776be9ada2d0e60d95cb4fdb67e
Author: Hairong Kuang <ha...@apache.org>
Date: Mon Nov 1 06:17:40 2010 +0000
HDFS-724. Use a bidirectional heartbeat to detect stuck pipeline.
commit eb2346ea7726c33715f3e651f4babcb6e6c3f0c2
Author: Hairong Kuang <ha...@apache.org>
Date: Tue Nov 16 17:53:17 2010 +0000
Fix incorrect heartbeat ack serialization in HDFS-724 for append 0.20.
Combined with 7083b462
commit 0e996bdba420fd0031490457e27dda67f58de18f
Author: Eli Collins <el...@apache.org>
Date: Tue Nov 9 01:59:27 2010 +0000
HDFS-1240. svn merge -c 1032810 from branch-0.20.
Excluded
TestDFSShell not failing on working branch
Commit eb2346ea7726c33715f3e651f4babcb6e6c3f0c2
Author: Hairong Kuang <ha...@apache.org>
Date: Tue Nov 16 17:53:17 2010 +0000
HDFS-895. Allow hflush/sync to occur in parallel with new writes to
the file
Required to mitigate HLog flush penalty
Need to raise default heap for tests for HDFS-895 unit tests to pass
commit 9a3094f3a085817d377ce69fb8eb002f966f992b
Author: Dhruba Borthakur <dh...@apache.org>
Date: Fri Dec 10 05:38:06 2010 +0000
HDFS-1520. Lightweight NameNode operation recoverLease to trigger
lease recovery. (Hairong Kuang via dhruba)
Required by HBase for master to seize ownership of and quickly split
HLogs of failed regionservers
commit f9436bf6bd2f5bc4f8cebe2781c5ce628db31239
Author: Hairong Kuang <ha...@apache.org>
Date: Fri Jan 7 20:11:38 2011 +0000
HDFS-1555. Disallow pipeline recovery if a file is already being lease
Excluded
Bad commit
commit 966ef38cb13d75ada40de2b9951496ddcb6918a4
Author: Hairong Kuang <ha...@apache.org>
Date: Fri Jan 7 20:19:40 2011 +0000
Revert the change made to HDFS-1555:
Excluded
commit e499be8c74a871ef06aa8a74c621e68d50167d3d
Author: Hairong Kuang <ha...@apache.org>
Date: Fri Jan 7 20:25:23 2011 +0000
HDFS-1555. Disallow pipelien recovery if a file is already being
lease recovered
commit df0d79cc2b09438c079fdf10b913936492117917
Author: Hairong Kuang <ha...@apache.org>
Date: Mon Jan 10 19:01:36 2011 +0000
HDFS-1554. New semantics for recoverLease.
Required by HBase for master to seize ownership of and quickly split
HLogs of failed regionservers
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1795) Port 0.20-append changes onto
0.20-security-203
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HDFS-1795:
---------------------------------
Attachment: security-append-patches.zip
> Port 0.20-append changes onto 0.20-security-203
> -----------------------------------------------
>
> Key: HDFS-1795
> URL: https://issues.apache.org/jira/browse/HDFS-1795
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: Andrew Purtell
> Attachments: security-append-patches.zip
>
>
> Port 0.20-append changes onto 0.20-security-203.
> I started with a Git repository cloned from git://git.apache.org/hadoop-common.git . Branch 'branch-0.20-security-203' was used as the starting point for the work. I then enumerated over the 0.20-append specific patches in 'branch-0.20-append'. Each was applied if not already via cherry pick except for as noted below. This process in effect replayed the evolution of 0.20-append branch on top of 0.20-security-203.
> The specific functional changes that HBase absolutely relies upon are specially mentioned.
> Generally I ran the full test suite after each change. There were a couple of exceptions where pairs of adjacent change sets were strongly related, in which case I applied them in sequence, then ran the test suite. During this process I encountered no test failures except for one test in TestFileAppend4, a test brought in from the append branch, and I still need to dig in to see if this is a real problem or if the test needs to be changed to work on top of security-203.
> {noformat}
> commit b9ad012eaf3915c2169a02a7130b54cbcc1d8a89
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Fri Jun 4 07:20:10 2010 +0000
> HDFS-200. Support append and sync for hadoop 0.20 branch.
>
> Required for HBase
> commit c968e11b5a60fc6f28e4e43fbbc8a99e7e49a659
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 9 23:09:07 2010 +0000
> HDFS-101. DFSClient correctly detects second datanode failure in write
> pipeline. (Nicolas Spiegelberg via dhruba)
> Excluded
> Already in 0.20-security-203 according to search of Git change log
> commit 9f7e5ed2ff47444a1dcd12ed34796929d5b9f7d5
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 9 23:12:21 2010 +0000
> HDFS-988. Fix bug where savenameSpace can corrupt edits log.
> (Nicolas Spiegelberg via dhruba)
> commit dfbbd6fbadaa95c54a1040b4fe8854b1b858d7a5
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 10 18:46:03 2010 +0000
> HDFS-826. Allow a mechanism for an application to detect that
> datanode(s) have died in the write pipeline. (dhruba)
> Required for HBase
> Commit be8d32503d30208a2d7772b3b4b2a270938a4004
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 10 22:25:39 2010 +0000
> HDFS-142. Blocks that are being written by a client are stored in the
> blocksBeingWritten directory.
> (Dhruba Borthakur, Nicolas Spiegelberg, Todd Lipcon via dhruba)
> commit 856efc2e95aaacc597d669c1b053634ff752dbec
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Fri Jun 11 00:48:41 2010 +0000
> HDFS-630. Client can exclude specific nodes in the write pipeline.
> (Nicolas Spiegelberg via dhruba)
> Required for HBase
> commit 2da1a05fc0cc0429229e87694977bae2ba370625
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Fri Jun 11 01:02:13 2010 +0000
> HDFS-457. Better handling of volume failure in DataNode Storage.
> (Nicolas Spiegelberg via dhruba)
> Excluded
> Already in 0.20-security-203 according to search of Git change log
> commit bd42393cd3a3a731ea98b25ddb528ad03a1ab4af
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Fri Jun 11 23:37:38 2010 +0000
> HDFS-1054. remove sleep before retry for allocating a block.
> (Todd Lipcon via dhruba)
> commit 120441b9e571a5703ac39b47608e87182f0f4972
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 20:53:12 2010 +0000
> HDFS-445. pread should refetch block locations when necessary.
> (Todd Lipcon via dhruba)
> Excluded
> Already in 0.20-security-203 according to search of Git change log
> commit 2004aa453ba6b7ee2045093ba313ef8551a7f8da
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 20:59:10 2010 +0000
> HDFS-561. Fix write pipeline
> commit 2a8227b0e6be8937fc4a654899be2a22c1f6efbe
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 21:13:24 2010 +0000
> HDFS-927. DFSInputStream retries too many times for new block
> locations. (Todd Lipcon via dhruba)
> Excluded
> Already in 0.20-security-203 according to search of Git change log
> commit b1e49dbf50a429cf01b636caa2666ff81ed2a016
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 21:21:45 2010 +0000
> HDFS-1215. Fix unti test TestNodeCount.
> (Todd Lipcon via dhruba)
> Excluded
> TestNodeCount not failing on current working branch
> commit b5078c57478deda9974ed40897045d38b100b9fc
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 21:25:38 2010 +0000
> HDFS-606. Fix ConcurrentModificationException in
> invalidateCorruptReplicas. (Todd Lipcon via dhruba)
> commit c3d35f8cdd761cbcd0a4ef851e37ccdb2d25d7e0
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 21:42:53 2010 +0000
> HDFS-1216. Upgrade to junit 4.5
> (Todd Lipcon via dhruba)
> Excluded
> Already using Junit 4.5 on current working branch
> commit c21b6834e982b0bf2edaf61dae8a0028076ccb60
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 17 01:34:56 2010 +0000
> HDFS-1207. FSNamesystem.stallReplicationWork should be volatile.
> (Todd Lipcon via dhruba)
> commit e8a1a09fec716bb013f94f1c14f4ae8f8409ee68
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 21:57:39 2010 +0000
> HDFS-1141. completeFile does not check lease ownership.
> (Todd Lipcon via dhruba)
> commit ca458a07a6d8cb5cda8c589b1a23dabffacef7fe
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 17 01:44:32 2010 +0000
> HDFS-1204. Lease expiration should recover single files,
> not entire lease holder (Sam Rash via dhruba)
> commit 62441fbd516ec9132619d448a1051554d29d2dba
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 17 01:52:50 2010 +0000
> HDFS-1210. DFSClient should log exception when block recovery fails.
> (Todd Lipcon via dhruba)
> Not a required change but a minor useful one
> commit 08b67cdf9ff7dada8aa86a2443a0ffc1a0a83c20
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 17 01:59:30 2010 +0000
> HDFS-1211. Block receiver should not log "rewind" packets at INFO
> level. (Todd Lipcon via dhruba)
> Not a required change but a minor useful one
> commit 0c9911a623891e82c40d4c1c3a523f3a85a85216
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Sun Jun 20 08:31:54 2010 +0000
> HDFS-1118. Fix socketleak on DFSClient.
> (Zheng Shao via dhruba)
> commit db9989c091b37446e4f4d468b24c4bd08b1ea337
> Author: Tsz-wo Sze <sz...@apache.org>
> Date: Mon Jun 21 21:26:45 2010 +0000
> MAPREDUCE-1880. Fix BigDecimal.divide(..) in the pi example.
> Excluded
> Not an HDFS change
> commit 11fa325540526da57a9e178463017853058ba921
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Tue Jun 22 05:55:48 2010 +0000
> HDFS-1254. Support append/sync via the default configuration.
> (dhruba)
> commit 9e405ca177aba719b04cfd7d33d400d311bf8db7
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jul 1 08:37:30 2010 +0000
> HDFS-1057. Concurrent readers hit ChecksumExceptions if following
> a writer to very end of file (Sam Rash via dhruba)
> commit 6512b527cda8efa0c622a8657af96ac785625773
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Tue Jul 6 06:52:20 2010 +0000
> HDFS-1202. DataBlockScanner throws NPE when updated before
> initialized. (Todd Lipcon via dhruba)
> commit 116e71fea80c31042520f364625ba7b1647cdb09
> Author: Tsz-wo Sze <sz...@apache.org>
> Date: Tue Jul 6 17:39:25 2010 +0000
> HDFS-1258. Clearing namespace quota on "/" corrupts fs image.
> commit ca88978b319debe146e9988d882393033b1d2d62
> Author: Tsz-wo Sze <sz...@apache.org>
> Date: Wed Sep 22 20:52:25 2010 +0000
> MAPREDUCE-1280. Update Eclipse plugin to the new eclipse.jdt API.
> Excluded
> Not an HDFS change
> commit a68b163fc31c3d82c63b69ae99e9954fef4df4af
> Author: Hairong Kuang <ha...@apache.org>
> Date: Mon Sep 27 18:14:20 2010 +0000
> HDFS-1346. DFSClient receives out of order packet ack.
> commit 7083b46208f56776be9ada2d0e60d95cb4fdb67e
> Author: Hairong Kuang <ha...@apache.org>
> Date: Mon Nov 1 06:17:40 2010 +0000
> HDFS-724. Use a bidirectional heartbeat to detect stuck pipeline.
> commit eb2346ea7726c33715f3e651f4babcb6e6c3f0c2
> Author: Hairong Kuang <ha...@apache.org>
> Date: Tue Nov 16 17:53:17 2010 +0000
> Fix incorrect heartbeat ack serialization in HDFS-724 for append 0.20.
> Combined with 7083b462
> commit 0e996bdba420fd0031490457e27dda67f58de18f
> Author: Eli Collins <el...@apache.org>
> Date: Tue Nov 9 01:59:27 2010 +0000
> HDFS-1240. svn merge -c 1032810 from branch-0.20.
> Excluded
> TestDFSShell not failing on working branch
> Commit eb2346ea7726c33715f3e651f4babcb6e6c3f0c2
> Author: Hairong Kuang <ha...@apache.org>
> Date: Tue Nov 16 17:53:17 2010 +0000
> HDFS-895. Allow hflush/sync to occur in parallel with new writes to
> the file
> Required to mitigate HLog flush penalty
> Need to raise default heap for tests for HDFS-895 unit tests to pass
> commit 9a3094f3a085817d377ce69fb8eb002f966f992b
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Fri Dec 10 05:38:06 2010 +0000
> HDFS-1520. Lightweight NameNode operation recoverLease to trigger
> lease recovery. (Hairong Kuang via dhruba)
> Required by HBase for master to seize ownership of and quickly split
> HLogs of failed regionservers
> commit f9436bf6bd2f5bc4f8cebe2781c5ce628db31239
> Author: Hairong Kuang <ha...@apache.org>
> Date: Fri Jan 7 20:11:38 2011 +0000
> HDFS-1555. Disallow pipeline recovery if a file is already being lease
> Excluded
> Bad commit
> commit 966ef38cb13d75ada40de2b9951496ddcb6918a4
> Author: Hairong Kuang <ha...@apache.org>
> Date: Fri Jan 7 20:19:40 2011 +0000
> Revert the change made to HDFS-1555:
> Excluded
> commit e499be8c74a871ef06aa8a74c621e68d50167d3d
> Author: Hairong Kuang <ha...@apache.org>
> Date: Fri Jan 7 20:25:23 2011 +0000
> HDFS-1555. Disallow pipelien recovery if a file is already being
> lease recovered
> commit df0d79cc2b09438c079fdf10b913936492117917
> Author: Hairong Kuang <ha...@apache.org>
> Date: Mon Jan 10 19:01:36 2011 +0000
> HDFS-1554. New semantics for recoverLease.
> Required by HBase for master to seize ownership of and quickly split
> HLogs of failed regionservers
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-1795) Port 0.20-append changes onto
0.20-security-203
Posted by "Eli Collins (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins resolved HDFS-1795.
-------------------------------
Resolution: Fixed
Fix Version/s: 0.20.205.0
> Port 0.20-append changes onto 0.20-security-203
> -----------------------------------------------
>
> Key: HDFS-1795
> URL: https://issues.apache.org/jira/browse/HDFS-1795
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: Andrew Purtell
> Fix For: 0.20.205.0
>
> Attachments: security-append-patches.zip
>
>
> Port 0.20-append changes onto 0.20-security-203.
> I started with a Git repository cloned from git://git.apache.org/hadoop-common.git . Branch 'branch-0.20-security-203' was used as the starting point for the work. I then enumerated over the 0.20-append specific patches in 'branch-0.20-append'. Each was applied if not already via cherry pick except for as noted below. This process in effect replayed the evolution of 0.20-append branch on top of 0.20-security-203.
> The specific functional changes that HBase absolutely relies upon are specially mentioned.
> Generally I ran the full test suite after each change. There were a couple of exceptions where pairs of adjacent change sets were strongly related, in which case I applied them in sequence, then ran the test suite. During this process I encountered no test failures except for one test in TestFileAppend4, a test brought in from the append branch, and I still need to dig in to see if this is a real problem or if the test needs to be changed to work on top of security-203.
> {noformat}
> commit b9ad012eaf3915c2169a02a7130b54cbcc1d8a89
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Fri Jun 4 07:20:10 2010 +0000
> HDFS-200. Support append and sync for hadoop 0.20 branch.
>
> Required for HBase
> commit c968e11b5a60fc6f28e4e43fbbc8a99e7e49a659
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 9 23:09:07 2010 +0000
> HDFS-101. DFSClient correctly detects second datanode failure in write
> pipeline. (Nicolas Spiegelberg via dhruba)
> Excluded
> Already in 0.20-security-203 according to search of Git change log
> commit 9f7e5ed2ff47444a1dcd12ed34796929d5b9f7d5
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 9 23:12:21 2010 +0000
> HDFS-988. Fix bug where savenameSpace can corrupt edits log.
> (Nicolas Spiegelberg via dhruba)
> commit dfbbd6fbadaa95c54a1040b4fe8854b1b858d7a5
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 10 18:46:03 2010 +0000
> HDFS-826. Allow a mechanism for an application to detect that
> datanode(s) have died in the write pipeline. (dhruba)
> Required for HBase
> Commit be8d32503d30208a2d7772b3b4b2a270938a4004
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 10 22:25:39 2010 +0000
> HDFS-142. Blocks that are being written by a client are stored in the
> blocksBeingWritten directory.
> (Dhruba Borthakur, Nicolas Spiegelberg, Todd Lipcon via dhruba)
> commit 856efc2e95aaacc597d669c1b053634ff752dbec
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Fri Jun 11 00:48:41 2010 +0000
> HDFS-630. Client can exclude specific nodes in the write pipeline.
> (Nicolas Spiegelberg via dhruba)
> Required for HBase
> commit 2da1a05fc0cc0429229e87694977bae2ba370625
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Fri Jun 11 01:02:13 2010 +0000
> HDFS-457. Better handling of volume failure in DataNode Storage.
> (Nicolas Spiegelberg via dhruba)
> Excluded
> Already in 0.20-security-203 according to search of Git change log
> commit bd42393cd3a3a731ea98b25ddb528ad03a1ab4af
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Fri Jun 11 23:37:38 2010 +0000
> HDFS-1054. remove sleep before retry for allocating a block.
> (Todd Lipcon via dhruba)
> commit 120441b9e571a5703ac39b47608e87182f0f4972
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 20:53:12 2010 +0000
> HDFS-445. pread should refetch block locations when necessary.
> (Todd Lipcon via dhruba)
> Excluded
> Already in 0.20-security-203 according to search of Git change log
> commit 2004aa453ba6b7ee2045093ba313ef8551a7f8da
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 20:59:10 2010 +0000
> HDFS-561. Fix write pipeline
> commit 2a8227b0e6be8937fc4a654899be2a22c1f6efbe
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 21:13:24 2010 +0000
> HDFS-927. DFSInputStream retries too many times for new block
> locations. (Todd Lipcon via dhruba)
> Excluded
> Already in 0.20-security-203 according to search of Git change log
> commit b1e49dbf50a429cf01b636caa2666ff81ed2a016
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 21:21:45 2010 +0000
> HDFS-1215. Fix unti test TestNodeCount.
> (Todd Lipcon via dhruba)
> Excluded
> TestNodeCount not failing on current working branch
> commit b5078c57478deda9974ed40897045d38b100b9fc
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 21:25:38 2010 +0000
> HDFS-606. Fix ConcurrentModificationException in
> invalidateCorruptReplicas. (Todd Lipcon via dhruba)
> commit c3d35f8cdd761cbcd0a4ef851e37ccdb2d25d7e0
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 21:42:53 2010 +0000
> HDFS-1216. Upgrade to junit 4.5
> (Todd Lipcon via dhruba)
> Excluded
> Already using Junit 4.5 on current working branch
> commit c21b6834e982b0bf2edaf61dae8a0028076ccb60
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 17 01:34:56 2010 +0000
> HDFS-1207. FSNamesystem.stallReplicationWork should be volatile.
> (Todd Lipcon via dhruba)
> commit e8a1a09fec716bb013f94f1c14f4ae8f8409ee68
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Wed Jun 16 21:57:39 2010 +0000
> HDFS-1141. completeFile does not check lease ownership.
> (Todd Lipcon via dhruba)
> commit ca458a07a6d8cb5cda8c589b1a23dabffacef7fe
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 17 01:44:32 2010 +0000
> HDFS-1204. Lease expiration should recover single files,
> not entire lease holder (Sam Rash via dhruba)
> commit 62441fbd516ec9132619d448a1051554d29d2dba
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 17 01:52:50 2010 +0000
> HDFS-1210. DFSClient should log exception when block recovery fails.
> (Todd Lipcon via dhruba)
> Not a required change but a minor useful one
> commit 08b67cdf9ff7dada8aa86a2443a0ffc1a0a83c20
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jun 17 01:59:30 2010 +0000
> HDFS-1211. Block receiver should not log "rewind" packets at INFO
> level. (Todd Lipcon via dhruba)
> Not a required change but a minor useful one
> commit 0c9911a623891e82c40d4c1c3a523f3a85a85216
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Sun Jun 20 08:31:54 2010 +0000
> HDFS-1118. Fix socketleak on DFSClient.
> (Zheng Shao via dhruba)
> commit db9989c091b37446e4f4d468b24c4bd08b1ea337
> Author: Tsz-wo Sze <sz...@apache.org>
> Date: Mon Jun 21 21:26:45 2010 +0000
> MAPREDUCE-1880. Fix BigDecimal.divide(..) in the pi example.
> Excluded
> Not an HDFS change
> commit 11fa325540526da57a9e178463017853058ba921
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Tue Jun 22 05:55:48 2010 +0000
> HDFS-1254. Support append/sync via the default configuration.
> (dhruba)
> commit 9e405ca177aba719b04cfd7d33d400d311bf8db7
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Thu Jul 1 08:37:30 2010 +0000
> HDFS-1057. Concurrent readers hit ChecksumExceptions if following
> a writer to very end of file (Sam Rash via dhruba)
> commit 6512b527cda8efa0c622a8657af96ac785625773
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Tue Jul 6 06:52:20 2010 +0000
> HDFS-1202. DataBlockScanner throws NPE when updated before
> initialized. (Todd Lipcon via dhruba)
> commit 116e71fea80c31042520f364625ba7b1647cdb09
> Author: Tsz-wo Sze <sz...@apache.org>
> Date: Tue Jul 6 17:39:25 2010 +0000
> HDFS-1258. Clearing namespace quota on "/" corrupts fs image.
> commit ca88978b319debe146e9988d882393033b1d2d62
> Author: Tsz-wo Sze <sz...@apache.org>
> Date: Wed Sep 22 20:52:25 2010 +0000
> MAPREDUCE-1280. Update Eclipse plugin to the new eclipse.jdt API.
> Excluded
> Not an HDFS change
> commit a68b163fc31c3d82c63b69ae99e9954fef4df4af
> Author: Hairong Kuang <ha...@apache.org>
> Date: Mon Sep 27 18:14:20 2010 +0000
> HDFS-1346. DFSClient receives out of order packet ack.
> commit 7083b46208f56776be9ada2d0e60d95cb4fdb67e
> Author: Hairong Kuang <ha...@apache.org>
> Date: Mon Nov 1 06:17:40 2010 +0000
> HDFS-724. Use a bidirectional heartbeat to detect stuck pipeline.
> commit eb2346ea7726c33715f3e651f4babcb6e6c3f0c2
> Author: Hairong Kuang <ha...@apache.org>
> Date: Tue Nov 16 17:53:17 2010 +0000
> Fix incorrect heartbeat ack serialization in HDFS-724 for append 0.20.
> Combined with 7083b462
> commit 0e996bdba420fd0031490457e27dda67f58de18f
> Author: Eli Collins <el...@apache.org>
> Date: Tue Nov 9 01:59:27 2010 +0000
> HDFS-1240. svn merge -c 1032810 from branch-0.20.
> Excluded
> TestDFSShell not failing on working branch
> Commit eb2346ea7726c33715f3e651f4babcb6e6c3f0c2
> Author: Hairong Kuang <ha...@apache.org>
> Date: Tue Nov 16 17:53:17 2010 +0000
> HDFS-895. Allow hflush/sync to occur in parallel with new writes to
> the file
> Required to mitigate HLog flush penalty
> Need to raise default heap for tests for HDFS-895 unit tests to pass
> commit 9a3094f3a085817d377ce69fb8eb002f966f992b
> Author: Dhruba Borthakur <dh...@apache.org>
> Date: Fri Dec 10 05:38:06 2010 +0000
> HDFS-1520. Lightweight NameNode operation recoverLease to trigger
> lease recovery. (Hairong Kuang via dhruba)
> Required by HBase for master to seize ownership of and quickly split
> HLogs of failed regionservers
> commit f9436bf6bd2f5bc4f8cebe2781c5ce628db31239
> Author: Hairong Kuang <ha...@apache.org>
> Date: Fri Jan 7 20:11:38 2011 +0000
> HDFS-1555. Disallow pipeline recovery if a file is already being lease
> Excluded
> Bad commit
> commit 966ef38cb13d75ada40de2b9951496ddcb6918a4
> Author: Hairong Kuang <ha...@apache.org>
> Date: Fri Jan 7 20:19:40 2011 +0000
> Revert the change made to HDFS-1555:
> Excluded
> commit e499be8c74a871ef06aa8a74c621e68d50167d3d
> Author: Hairong Kuang <ha...@apache.org>
> Date: Fri Jan 7 20:25:23 2011 +0000
> HDFS-1555. Disallow pipelien recovery if a file is already being
> lease recovered
> commit df0d79cc2b09438c079fdf10b913936492117917
> Author: Hairong Kuang <ha...@apache.org>
> Date: Mon Jan 10 19:01:36 2011 +0000
> HDFS-1554. New semantics for recoverLease.
> Required by HBase for master to seize ownership of and quickly split
> HLogs of failed regionservers
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira