You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by na...@apache.org on 2019/09/17 11:13:11 UTC
[hadoop] branch ozone-0.4.1 updated (dafb1c0 -> 0fb42e5)
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a change to branch ozone-0.4.1
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from dafb1c0 HDDS-2124. Random next links
new 6c0f861 HDDS-2111. XSS fragments can be injected to the S3g landing page
new 7f823d5 HDDS-2114. Rename does not preserve non-explicitly created interim directories. Contributed by Lokesh Jain & Istvan Fajth.
new 0fb42e5 HDDS-2132. TestKeyValueContainer is failing (#1457).
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../keyvalue/helpers/KeyValueContainerUtil.java | 5 +++--
.../hadoop/fs/ozone/BasicOzoneFileSystem.java | 24 ++++++++++++++++-----
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 25 ++++++++++++++++++++++
.../src/main/resources/webapps/static/index.html | 8 +++++--
.../src/main/resources/webapps/static/s3g.js} | 11 ++++------
5 files changed, 57 insertions(+), 16 deletions(-)
copy hadoop-ozone/{ozone-recon/src/main/resources/webapps/recon/ozone-recon-web/src/constants/breadcrumbs.constants.tsx => s3gateway/src/main/resources/webapps/static/s3g.js} (77%)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 03/03: HDDS-2132. TestKeyValueContainer is failing (#1457).
Posted by na...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a commit to branch ozone-0.4.1
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 0fb42e514be56a43d732f1f7efc67171dd0224e1
Author: Doroszlai, Attila <64...@users.noreply.github.com>
AuthorDate: Tue Sep 17 11:01:33 2019 +0200
HDDS-2132. TestKeyValueContainer is failing (#1457).
(cherry picked from commit e54977f888e1a855e9f88b9fa41e0c8794bd0881)
---
.../ozone/container/keyvalue/helpers/KeyValueContainerUtil.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/KeyValueContainerUtil.java b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/KeyValueContainerUtil.java
index c0e7d78..2c02ad5 100644
--- a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/KeyValueContainerUtil.java
+++ b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/KeyValueContainerUtil.java
@@ -198,8 +198,9 @@ public final class KeyValueContainerUtil {
kvContainerData.setKeyCount(liveKeys.size());
byte[] bcsId = metadata.getStore().get(DFSUtil.string2Bytes(
OzoneConsts.BLOCK_COMMIT_SEQUENCE_ID_PREFIX));
- Preconditions.checkNotNull(bcsId);
- kvContainerData.updateBlockCommitSequenceId(Longs.fromByteArray(bcsId));
+ if (bcsId != null) {
+ kvContainerData.updateBlockCommitSequenceId(Longs.fromByteArray(bcsId));
+ }
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 02/03: HDDS-2114. Rename does not preserve non-explicitly
created interim directories. Contributed by Lokesh Jain & Istvan Fajth.
Posted by na...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a commit to branch ozone-0.4.1
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 7f823d5e3c32817f06b3dd4e1b5ab8b8275dc030
Author: Mukul Kumar Singh <ms...@apache.org>
AuthorDate: Tue Sep 17 10:47:00 2019 +0530
HDDS-2114. Rename does not preserve non-explicitly created interim directories. Contributed by Lokesh Jain & Istvan Fajth.
(cherry picked from commit 292bce7908bf4830c793a3f4e80376819c038379)
---
.../hadoop/fs/ozone/BasicOzoneFileSystem.java | 24 ++++++++++++++++-----
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 25 ++++++++++++++++++++++
2 files changed, 44 insertions(+), 5 deletions(-)
diff --git a/hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java b/hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java
index 27bc925..5299e6a 100644
--- a/hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java
+++ b/hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java
@@ -374,7 +374,11 @@ public class BasicOzoneFileSystem extends FileSystem {
}
}
RenameIterator iterator = new RenameIterator(src, dst);
- return iterator.iterate();
+ boolean result = iterator.iterate();
+ if (result) {
+ createFakeParentDirectory(src);
+ }
+ return result;
}
private class DeleteIterator extends OzoneListingIterator {
@@ -459,10 +463,7 @@ public class BasicOzoneFileSystem extends FileSystem {
if (result) {
// If this delete operation removes all files/directories from the
// parent direcotry, then an empty parent directory must be created.
- Path parent = f.getParent();
- if (parent != null && !parent.isRoot()) {
- createFakeDirectoryIfNecessary(parent);
- }
+ createFakeParentDirectory(f);
}
return result;
@@ -475,6 +476,19 @@ public class BasicOzoneFileSystem extends FileSystem {
* @param f path to the fake parent directory
* @throws IOException
*/
+ private void createFakeParentDirectory(Path f) throws IOException {
+ Path parent = f.getParent();
+ if (parent != null && !parent.isRoot()) {
+ createFakeDirectoryIfNecessary(parent);
+ }
+ }
+
+ /**
+ * Create a fake directory key if it does not already exist.
+ *
+ * @param f path to the fake directory
+ * @throws IOException
+ */
private void createFakeDirectoryIfNecessary(Path f) throws IOException {
String key = pathToKey(f);
if (!key.isEmpty() && !o3Exists(f)) {
diff --git a/hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java b/hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java
index ac8f11f..434e8f0 100644
--- a/hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java
+++ b/hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java
@@ -51,6 +51,7 @@ import org.junit.rules.Timeout;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
/**
@@ -285,6 +286,30 @@ public class TestOzoneFileSystem {
fileStatus2.equals(dir12.toString()));
}
+ @Test
+ public void testNonExplicitlyCreatedPathExistsAfterItsLeafsWereRemoved()
+ throws Exception {
+ Path source = new Path("/source");
+ Path interimPath = new Path(source, "interimPath");
+ Path leafInsideInterimPath = new Path(interimPath, "leaf");
+ Path target = new Path("/target");
+ Path leafInTarget = new Path(target, "leaf");
+
+ fs.mkdirs(source);
+ fs.mkdirs(target);
+ fs.mkdirs(leafInsideInterimPath);
+ assertTrue(fs.rename(leafInsideInterimPath, leafInTarget));
+
+ // after rename listStatus for interimPath should succeed and
+ // interimPath should have no children
+ FileStatus[] statuses = fs.listStatus(interimPath);
+ assertNotNull("liststatus returns a null array", statuses);
+ assertEquals("Statuses array is not empty", 0, statuses.length);
+ FileStatus fileStatus = fs.getFileStatus(interimPath);
+ assertEquals("FileStatus does not point to interimPath",
+ interimPath.getName(), fileStatus.getPath().getName());
+ }
+
private KeyInfo getKey(Path keyPath, boolean isDirectory)
throws IOException, OzoneException {
String key = o3fs.pathToKey(keyPath);
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 01/03: HDDS-2111. XSS fragments can be injected to the S3g
landing page
Posted by na...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a commit to branch ozone-0.4.1
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 6c0f86149c9765a581c09bbe9345d224a597eee6
Author: Márton Elek <el...@apache.org>
AuthorDate: Sat Sep 14 05:33:05 2019 +0200
HDDS-2111. XSS fragments can be injected to the S3g landing page
Signed-off-by: Anu Engineer <ae...@apache.org>
(cherry picked from commit 2358e53e9c9f8489b24648b1017eb856d4bd42b0)
---
.../src/main/resources/webapps/static/index.html | 8 ++++++--
.../src/main/resources/webapps/static/s3g.js | 23 ++++++++++++++++++++++
2 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/hadoop-ozone/s3gateway/src/main/resources/webapps/static/index.html b/hadoop-ozone/s3gateway/src/main/resources/webapps/static/index.html
index 68939ef..b20bf35 100644
--- a/hadoop-ozone/s3gateway/src/main/resources/webapps/static/index.html
+++ b/hadoop-ozone/s3gateway/src/main/resources/webapps/static/index.html
@@ -21,6 +21,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta http-equiv="Content-Security-Policy" content="script-src 'self';">
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
<meta name="description" content="Apache Hadoop Ozone S3 gateway">
@@ -68,12 +69,15 @@
<p>For example with aws-cli:</p>
- <pre>aws s3api --endpoint <script>document.write(window.location.href.replace("static/", ""))</script> create-bucket --bucket=wordcount</pre>
+ <pre>aws s3api --endpoint <span id="s3gurl"></span> create-bucket --bucket=wordcount</pre>
<p>For more information, please check the <a href="docs">documentation.</a>
</p>
</div><!-- /.container -->
-<script src="static/bootstrap-3.4.1/js/bootstrap.min.js"></script>
+<script src="jquery-3.4.1.min.js"></script>
+<script src="bootstrap-3.4.1/js/bootstrap.min.js"></script>
+<script src="s3g.js"></script>
+
</body>
</html>
diff --git a/hadoop-ozone/s3gateway/src/main/resources/webapps/static/s3g.js b/hadoop-ozone/s3gateway/src/main/resources/webapps/static/s3g.js
new file mode 100644
index 0000000..8b1e977
--- /dev/null
+++ b/hadoop-ozone/s3gateway/src/main/resources/webapps/static/s3g.js
@@ -0,0 +1,23 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+window.onload = function () {
+ var safeurl = window.location.protocol + "//" + window.location.host + window.location.pathname;
+ safeurl = safeurl.replace("static/", "");
+ document.getElementById('s3gurl').innerHTML = safeurl;
+};
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org