You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by na...@apache.org on 2019/01/25 06:56:31 UTC
[hadoop] branch trunk updated (3c60303 -> a448b05)
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from 3c60303 HADOOP-16065. -Ddynamodb should be -Ddynamo in AWS SDK testing document.
new c6d901a HDDS-1006. AuditParser assumes incorrect log format. Contributed by Dinesh Chitlangia.
new 8ff9578 HDDS-1007. Add robot test for AuditParser. Contributed by Dinesh Chitlangia.
new 45c4cfe HDDS-906. Display the ozone version on SCM/OM web ui instead of Hadoop version. Contributed by Doroszlai, Attila.
new 2181b18 HDDS-990. Typos in Ozone doc. Contributed by Doroszlai, Attila.
new 5dae1a0 HDDS-973. HDDS/Ozone fail to build on Windows. Contributed by Xiaoyu Yao.
new a448b05 HDDS-1009. TestAbortMultipartUpload is missing the apache license text. Contributed by Dinesh Chitlangia.
The 6 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../org/apache/hadoop/utils/HddsVersionInfo.java | 70 ++------------------
.../{HddsVersionInfo.java => VersionInfo.java} | 65 +++++++------------
hadoop-hdds/docs/content/BucketCommands.md | 10 +--
hadoop-hdds/docs/content/Dozone.md | 17 +++--
hadoop-hdds/docs/content/JavaApi.md | 36 +++++------
hadoop-hdds/docs/content/KeyCommands.md | 6 +-
hadoop-hdds/docs/content/Rest.md | 2 +-
hadoop-hdds/docs/content/S3.md | 6 +-
hadoop-hdds/docs/content/VolumeCommands.md | 2 +-
hadoop-hdds/docs/pom.xml | 8 ++-
.../hadoop/hdds/server/ServiceRuntimeInfoImpl.java | 15 +++--
.../hdds/scm/server/StorageContainerManager.java | 24 ++++---
.../apache/hadoop/ozone/util/OzoneVersionInfo.java | 75 ++--------------------
.../src/main/smoketest/auditparser/parser.robot | 40 ++++++++++++
hadoop-ozone/dist/src/main/smoketest/test.sh | 2 +
.../hadoop/ozone/TestStorageContainerManager.java | 6 ++
.../apache/hadoop/ozone/om/TestOzoneManager.java | 8 +++
.../org/apache/hadoop/ozone/om/OzoneManager.java | 2 +
.../s3/endpoint/TestAbortMultipartUpload.java | 19 ++++++
.../ozone/audit/parser/common/DatabaseHelper.java | 12 ++--
.../tools/src/test/resources/testaudit.log | 18 +++---
21 files changed, 191 insertions(+), 252 deletions(-)
copy hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/{HddsVersionInfo.java => VersionInfo.java} (59%)
create mode 100644 hadoop-ozone/dist/src/main/smoketest/auditparser/parser.robot
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 03/06: HDDS-906. Display the ozone version on SCM/OM web
ui instead of Hadoop version. Contributed by Doroszlai, Attila.
Posted by na...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 45c4cfe790bd6d7962698555c634a42b38c6cad1
Author: Nanda kumar <na...@gmail.com>
AuthorDate: Fri Jan 25 12:16:29 2019 +0530
HDDS-906. Display the ozone version on SCM/OM web ui instead of Hadoop version.
Contributed by Doroszlai, Attila.
---
.../org/apache/hadoop/utils/HddsVersionInfo.java | 70 ++------------------
.../{HddsVersionInfo.java => VersionInfo.java} | 65 +++++++------------
.../hadoop/hdds/server/ServiceRuntimeInfoImpl.java | 15 +++--
.../hdds/scm/server/StorageContainerManager.java | 24 ++++---
.../apache/hadoop/ozone/util/OzoneVersionInfo.java | 75 ++--------------------
.../hadoop/ozone/TestStorageContainerManager.java | 6 ++
.../apache/hadoop/ozone/om/TestOzoneManager.java | 8 +++
.../org/apache/hadoop/ozone/om/OzoneManager.java | 2 +
8 files changed, 68 insertions(+), 197 deletions(-)
diff --git a/hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/HddsVersionInfo.java b/hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/HddsVersionInfo.java
index 88b61f6..13c62b7 100644
--- a/hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/HddsVersionInfo.java
+++ b/hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/HddsVersionInfo.java
@@ -20,86 +20,24 @@ package org.apache.hadoop.utils;
import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;
-import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.util.ClassUtil;
-import org.apache.hadoop.util.ThreadUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import java.io.IOException;
-import java.io.InputStream;
-import java.util.Properties;
-
/**
* This class returns build information about Hadoop components.
*/
@InterfaceAudience.Public
@InterfaceStability.Stable
-public class HddsVersionInfo {
+public final class HddsVersionInfo {
private static final Logger LOG = LoggerFactory.getLogger(
HddsVersionInfo.class);
- public static final HddsVersionInfo HDDS_VERSION_INFO =
- new HddsVersionInfo("hdds");
-
- private Properties info;
-
- protected HddsVersionInfo(String component) {
- info = new Properties();
- String versionInfoFile = component + "-version-info.properties";
- InputStream is = null;
- try {
- is = ThreadUtil.getResourceAsStream(
- HddsVersionInfo.class.getClassLoader(),
- versionInfoFile);
- info.load(is);
- } catch (IOException ex) {
- LoggerFactory.getLogger(getClass()).warn("Could not read '" +
- versionInfoFile + "', " + ex.toString(), ex);
- } finally {
- IOUtils.closeStream(is);
- }
- }
-
- protected String getVersion() {
- return info.getProperty("version", "Unknown");
- }
+ public static final VersionInfo HDDS_VERSION_INFO =
+ new VersionInfo("hdds");
- protected String getRevision() {
- return info.getProperty("revision", "Unknown");
- }
-
- protected String getBranch() {
- return info.getProperty("branch", "Unknown");
- }
-
- protected String getDate() {
- return info.getProperty("date", "Unknown");
- }
-
- protected String getUser() {
- return info.getProperty("user", "Unknown");
- }
-
- protected String getUrl() {
- return info.getProperty("url", "Unknown");
- }
-
- protected String getSrcChecksum() {
- return info.getProperty("srcChecksum", "Unknown");
- }
-
- public String getBuildVersion() {
- return HDDS_VERSION_INFO.getVersion() +
- " from " + HDDS_VERSION_INFO.getRevision() +
- " by " + getUser() +
- " source checksum " + getSrcChecksum();
- }
-
- protected String getProtocVersion() {
- return info.getProperty("protocVersion", "Unknown");
- }
+ private HddsVersionInfo() {}
public static void main(String[] args) {
System.out.println("Using HDDS " + HDDS_VERSION_INFO.getVersion());
diff --git a/hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/HddsVersionInfo.java b/hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/VersionInfo.java
similarity index 59%
copy from hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/HddsVersionInfo.java
copy to hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/VersionInfo.java
index 88b61f6..9b86934 100644
--- a/hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/HddsVersionInfo.java
+++ b/hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/VersionInfo.java
@@ -15,15 +15,12 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-
package org.apache.hadoop.utils;
import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;
import org.apache.hadoop.io.IOUtils;
-import org.apache.hadoop.util.ClassUtil;
import org.apache.hadoop.util.ThreadUtil;
-import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
@@ -35,24 +32,17 @@ import java.util.Properties;
*/
@InterfaceAudience.Public
@InterfaceStability.Stable
-public class HddsVersionInfo {
-
- private static final Logger LOG = LoggerFactory.getLogger(
- HddsVersionInfo.class);
+public class VersionInfo {
- public static final HddsVersionInfo HDDS_VERSION_INFO =
- new HddsVersionInfo("hdds");
+ private final Properties info = new Properties();
- private Properties info;
-
- protected HddsVersionInfo(String component) {
- info = new Properties();
+ public VersionInfo(String component) {
String versionInfoFile = component + "-version-info.properties";
InputStream is = null;
try {
is = ThreadUtil.getResourceAsStream(
- HddsVersionInfo.class.getClassLoader(),
- versionInfoFile);
+ getClass().getClassLoader(),
+ versionInfoFile);
info.load(is);
} catch (IOException ex) {
LoggerFactory.getLogger(getClass()).warn("Could not read '" +
@@ -62,57 +52,46 @@ public class HddsVersionInfo {
}
}
- protected String getVersion() {
+ public String getRelease() {
+ return info.getProperty("release", "Unknown");
+ }
+
+ public String getVersion() {
return info.getProperty("version", "Unknown");
}
- protected String getRevision() {
+ public String getRevision() {
return info.getProperty("revision", "Unknown");
}
- protected String getBranch() {
+ public String getBranch() {
return info.getProperty("branch", "Unknown");
}
- protected String getDate() {
+ public String getDate() {
return info.getProperty("date", "Unknown");
}
- protected String getUser() {
+ public String getUser() {
return info.getProperty("user", "Unknown");
}
- protected String getUrl() {
+ public String getUrl() {
return info.getProperty("url", "Unknown");
}
- protected String getSrcChecksum() {
+ public String getSrcChecksum() {
return info.getProperty("srcChecksum", "Unknown");
}
- public String getBuildVersion() {
- return HDDS_VERSION_INFO.getVersion() +
- " from " + HDDS_VERSION_INFO.getRevision() +
- " by " + getUser() +
- " source checksum " + getSrcChecksum();
- }
-
- protected String getProtocVersion() {
+ public String getProtocVersion() {
return info.getProperty("protocVersion", "Unknown");
}
- public static void main(String[] args) {
- System.out.println("Using HDDS " + HDDS_VERSION_INFO.getVersion());
- System.out.println(
- "Source code repository " + HDDS_VERSION_INFO.getUrl() + " -r " +
- HDDS_VERSION_INFO.getRevision());
- System.out.println("Compiled by " + HDDS_VERSION_INFO.getUser() + " on "
- + HDDS_VERSION_INFO.getDate());
- System.out.println(
- "Compiled with protoc " + HDDS_VERSION_INFO.getProtocVersion());
- System.out.println(
- "From source with checksum " + HDDS_VERSION_INFO.getSrcChecksum());
- LOG.debug("This command was run using " +
- ClassUtil.findContainingJar(HddsVersionInfo.class));
+ public String getBuildVersion() {
+ return getVersion() +
+ " from " + getRevision() +
+ " by " + getUser() +
+ " source checksum " + getSrcChecksum();
}
}
diff --git a/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/ServiceRuntimeInfoImpl.java b/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/ServiceRuntimeInfoImpl.java
index 36d6b64..850d96e 100644
--- a/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/ServiceRuntimeInfoImpl.java
+++ b/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/ServiceRuntimeInfoImpl.java
@@ -17,7 +17,7 @@
package org.apache.hadoop.hdds.server;
-import org.apache.hadoop.util.VersionInfo;
+import org.apache.hadoop.utils.VersionInfo;
/**
* Helper base class to report the standard version and runtime information.
@@ -26,21 +26,26 @@ import org.apache.hadoop.util.VersionInfo;
public class ServiceRuntimeInfoImpl implements ServiceRuntimeInfo {
private long startedTimeInMillis;
+ private final VersionInfo versionInfo;
+
+ protected ServiceRuntimeInfoImpl(VersionInfo versionInfo) {
+ this.versionInfo = versionInfo;
+ }
@Override
public String getVersion() {
- return VersionInfo.getVersion() + ", r" + VersionInfo.getRevision();
+ return versionInfo.getVersion() + ", r" + versionInfo.getRevision();
}
@Override
public String getSoftwareVersion() {
- return VersionInfo.getVersion();
+ return versionInfo.getVersion();
}
@Override
public String getCompileInfo() {
- return VersionInfo.getDate() + " by " + VersionInfo.getUser() + " from "
- + VersionInfo.getBranch();
+ return versionInfo.getDate() + " by " + versionInfo.getUser() + " from "
+ + versionInfo.getBranch();
}
@Override
diff --git a/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java b/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java
index a7949db..ca98033 100644
--- a/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java
+++ b/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java
@@ -44,20 +44,17 @@ import org.apache.hadoop.hdds.scm.command.CommandStatusReportHandler;
import org.apache.hadoop.hdds.scm.container.CloseContainerEventHandler;
import org.apache.hadoop.hdds.scm.container.ContainerActionsHandler;
import org.apache.hadoop.hdds.scm.container.ContainerID;
+import org.apache.hadoop.hdds.scm.container.ContainerInfo;
import org.apache.hadoop.hdds.scm.container.ContainerManager;
+import org.apache.hadoop.hdds.scm.container.ContainerReportHandler;
import org.apache.hadoop.hdds.scm.container.IncrementalContainerReportHandler;
import org.apache.hadoop.hdds.scm.container.SCMContainerManager;
-import org.apache.hadoop.hdds.scm.container.ContainerReportHandler;
-import org.apache.hadoop.hdds.scm.container.replication
- .ReplicationActivityStatus;
-import org.apache.hadoop.hdds.scm.container.replication.ReplicationManager;
-import org.apache.hadoop.hdds.scm.container.ContainerInfo;
-import org.apache.hadoop.hdds.scm.container.placement.algorithms
- .ContainerPlacementPolicy;
-import org.apache.hadoop.hdds.scm.container.placement.algorithms
- .SCMContainerPlacementCapacity;
+import org.apache.hadoop.hdds.scm.container.placement.algorithms.ContainerPlacementPolicy;
+import org.apache.hadoop.hdds.scm.container.placement.algorithms.SCMContainerPlacementCapacity;
import org.apache.hadoop.hdds.scm.container.placement.metrics.ContainerStat;
import org.apache.hadoop.hdds.scm.container.placement.metrics.SCMMetrics;
+import org.apache.hadoop.hdds.scm.container.replication.ReplicationActivityStatus;
+import org.apache.hadoop.hdds.scm.container.replication.ReplicationManager;
import org.apache.hadoop.hdds.scm.events.SCMEvents;
import org.apache.hadoop.hdds.scm.exceptions.SCMException;
import org.apache.hadoop.hdds.scm.exceptions.SCMException.ResultCodes;
@@ -95,8 +92,7 @@ import org.apache.hadoop.security.authentication.client.AuthenticationException;
import org.apache.hadoop.util.GenericOptionsParser;
import org.apache.hadoop.util.JvmPauseMonitor;
import org.apache.hadoop.util.StringUtils;
-
-
+import org.apache.hadoop.utils.HddsVersionInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -110,10 +106,11 @@ import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.TimeUnit;
+
+import static org.apache.hadoop.hdds.scm.ScmConfigKeys.HDDS_SCM_KERBEROS_KEYTAB_FILE_KEY;
+import static org.apache.hadoop.hdds.scm.ScmConfigKeys.HDDS_SCM_KERBEROS_PRINCIPAL_KEY;
import static org.apache.hadoop.hdds.scm.ScmConfigKeys.HDDS_SCM_WATCHER_TIMEOUT_DEFAULT;
import static org.apache.hadoop.ozone.OzoneConfigKeys.OZONE_ENABLED;
-import static org.apache.hadoop.hdds.scm.ScmConfigKeys.HDDS_SCM_KERBEROS_PRINCIPAL_KEY;
-import static org.apache.hadoop.hdds.scm.ScmConfigKeys.HDDS_SCM_KERBEROS_KEYTAB_FILE_KEY;
import static org.apache.hadoop.util.ExitUtil.terminate;
/**
@@ -209,6 +206,7 @@ public final class StorageContainerManager extends ServiceRuntimeInfoImpl
*/
private StorageContainerManager(OzoneConfiguration conf)
throws IOException, AuthenticationException {
+ super(HddsVersionInfo.HDDS_VERSION_INFO);
configuration = conf;
StorageContainerManager.initMetrics();
diff --git a/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/OzoneVersionInfo.java b/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/OzoneVersionInfo.java
index c408b03..8378c5f 100644
--- a/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/OzoneVersionInfo.java
+++ b/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/OzoneVersionInfo.java
@@ -20,90 +20,25 @@ package org.apache.hadoop.ozone.util;
import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;
-import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.util.ClassUtil;
-import org.apache.hadoop.util.ThreadUtil;
import org.apache.hadoop.utils.HddsVersionInfo;
+import org.apache.hadoop.utils.VersionInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import java.io.IOException;
-import java.io.InputStream;
-import java.util.Properties;
-
/**
* This class returns build information about Hadoop components.
*/
@InterfaceAudience.Public
@InterfaceStability.Stable
-public class OzoneVersionInfo {
+public final class OzoneVersionInfo {
private static final Logger LOG =
LoggerFactory.getLogger(OzoneVersionInfo.class);
- private Properties info;
-
- protected OzoneVersionInfo(String component) {
- info = new Properties();
- String versionInfoFile = component + "-version-info.properties";
- InputStream is = null;
- try {
- is = ThreadUtil
- .getResourceAsStream(OzoneVersionInfo.class.getClassLoader(),
- versionInfoFile);
- info.load(is);
- } catch (IOException ex) {
- LoggerFactory.getLogger(getClass()).warn("Could not read '" +
- versionInfoFile + "', " + ex.toString(), ex);
- } finally {
- IOUtils.closeStream(is);
- }
- }
-
- protected String getVersion() {
- return info.getProperty("version", "Unknown");
- }
-
- protected String getRelease() {
- return info.getProperty("release", "Unknown");
- }
-
- protected String getRevision() {
- return info.getProperty("revision", "Unknown");
- }
-
- protected String getBranch() {
- return info.getProperty("branch", "Unknown");
- }
-
- protected String getDate() {
- return info.getProperty("date", "Unknown");
- }
-
- protected String getUser() {
- return info.getProperty("user", "Unknown");
- }
-
- protected String getUrl() {
- return info.getProperty("url", "Unknown");
- }
-
- protected String getSrcChecksum() {
- return info.getProperty("srcChecksum", "Unknown");
- }
-
- protected String getBuildVersion() {
- return getVersion() +
- " from " + getRevision() +
- " by " + getUser() +
- " source checksum " + getSrcChecksum();
- }
-
- protected String getProtocVersion() {
- return info.getProperty("protocVersion", "Unknown");
- }
+ public static final VersionInfo OZONE_VERSION_INFO =
+ new VersionInfo("ozone");
- private static final OzoneVersionInfo OZONE_VERSION_INFO =
- new OzoneVersionInfo("ozone");
+ private OzoneVersionInfo() {}
public static void main(String[] args) {
System.out.println(
diff --git a/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestStorageContainerManager.java b/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestStorageContainerManager.java
index c79da17..1e2524d 100644
--- a/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestStorageContainerManager.java
+++ b/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestStorageContainerManager.java
@@ -62,6 +62,7 @@ import org.apache.hadoop.ozone.protocol.commands.SCMCommand;
import org.apache.hadoop.security.authentication.client.AuthenticationException;
import org.apache.hadoop.test.GenericTestUtils;
import org.apache.hadoop.util.ExitUtil;
+import org.apache.hadoop.utils.HddsVersionInfo;
import org.junit.Assert;
import org.junit.Rule;
@@ -474,5 +475,10 @@ public class TestStorageContainerManager {
ScmInfo scmInfo = scm.getClientProtocolServer().getScmInfo();
Assert.assertEquals(clusterId, scmInfo.getClusterId());
Assert.assertEquals(scmId, scmInfo.getScmId());
+
+ String expectedVersion = HddsVersionInfo.HDDS_VERSION_INFO.getVersion();
+ String actualVersion = scm.getSoftwareVersion();
+ Assert.assertEquals(expectedVersion, actualVersion);
}
+
}
diff --git a/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManager.java b/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManager.java
index 26c2a58..45231f1 100644
--- a/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManager.java
+++ b/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManager.java
@@ -38,6 +38,7 @@ import org.apache.hadoop.ozone.protocol.proto
.OzoneManagerProtocolProtos.ServicePort;
import org.apache.hadoop.hdds.protocol.proto.HddsProtos;
import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.VolumeList;
+import org.apache.hadoop.ozone.util.OzoneVersionInfo;
import org.apache.hadoop.ozone.web.handlers.BucketArgs;
import org.apache.hadoop.ozone.web.handlers.KeyArgs;
import org.apache.hadoop.ozone.web.handlers.UserArgs;
@@ -1395,4 +1396,11 @@ public class TestOzoneManager {
LifeCycle.State.RUNNING,
cluster.getOzoneManager().getOmRatisServerState());
}
+
+ @Test
+ public void testVersion() {
+ String expectedVersion = OzoneVersionInfo.OZONE_VERSION_INFO.getVersion();
+ String actualVersion = cluster.getOzoneManager().getSoftwareVersion();
+ Assert.assertEquals(expectedVersion, actualVersion);
+ }
}
diff --git a/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java b/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
index a921a03..9c15e31 100644
--- a/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
+++ b/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
@@ -99,6 +99,7 @@ import org.apache.hadoop.ozone.security.acl.OzoneObjInfo;
import org.apache.hadoop.ozone.security.acl.RequestContext;
import org.apache.hadoop.ozone.security.OzoneBlockTokenSecretManager;
import org.apache.hadoop.ozone.security.OzoneDelegationTokenSecretManager;
+import org.apache.hadoop.ozone.util.OzoneVersionInfo;
import org.apache.hadoop.security.AccessControlException;
import org.apache.hadoop.security.SecurityUtil;
import org.apache.hadoop.security.UserGroupInformation;
@@ -223,6 +224,7 @@ public final class OzoneManager extends ServiceRuntimeInfoImpl
private volatile boolean isOmRpcServerRunning = false;
private OzoneManager(OzoneConfiguration conf) throws IOException {
+ super(OzoneVersionInfo.OZONE_VERSION_INFO);
Preconditions.checkNotNull(conf);
configuration = conf;
omStorage = new OMStorage(conf);
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 01/06: HDDS-1006. AuditParser assumes incorrect log
format. Contributed by Dinesh Chitlangia.
Posted by na...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit c6d901af77efd7d0bdea7a0258932eac627a4b09
Author: Nanda kumar <na...@gmail.com>
AuthorDate: Fri Jan 25 12:11:58 2019 +0530
HDDS-1006. AuditParser assumes incorrect log format.
Contributed by Dinesh Chitlangia.
---
.../ozone/audit/parser/common/DatabaseHelper.java | 12 +++++++-----
hadoop-ozone/tools/src/test/resources/testaudit.log | 18 +++++++++---------
2 files changed, 16 insertions(+), 14 deletions(-)
diff --git a/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/common/DatabaseHelper.java b/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/common/DatabaseHelper.java
index 4b3154b..a7282b2 100644
--- a/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/common/DatabaseHelper.java
+++ b/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/common/DatabaseHelper.java
@@ -166,18 +166,20 @@ public final class DatabaseHelper {
tempEntry.appendException(currentLine);
} else {
entry = StringUtils.stripAll(currentLine.split("\\|"));
+ String[] ops =
+ entry[5].substring(entry[5].indexOf('=') + 1).split(" ", 2);
tempEntry = new AuditEntry.Builder()
.setTimestamp(entry[0])
.setLevel(entry[1])
.setLogger(entry[2])
.setUser(entry[3].substring(entry[3].indexOf('=') + 1))
.setIp(entry[4].substring(entry[4].indexOf('=') + 1))
- .setOp(entry[5].substring(entry[5].indexOf('=') + 1))
- .setParams(entry[6])
- .setResult(entry[7].substring(entry[7].indexOf('=') + 1))
+ .setOp(ops[0])
+ .setParams(ops[1])
+ .setResult(entry[6].substring(entry[6].indexOf('=') + 1))
.build();
- if(entry.length == 9){
- tempEntry.setException(entry[8]);
+ if(entry.length == 8){
+ tempEntry.setException(entry[7]);
}
}
if(nextLine == null || nextLine.matches(ParserConsts.DATE_REGEX)){
diff --git a/hadoop-ozone/tools/src/test/resources/testaudit.log b/hadoop-ozone/tools/src/test/resources/testaudit.log
index 6d472b5..4c56f35 100644
--- a/hadoop-ozone/tools/src/test/resources/testaudit.log
+++ b/hadoop-ozone/tools/src/test/resources/testaudit.log
@@ -1,15 +1,15 @@
-2018-09-06 01:57:22,996 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=ALLOCATE_KEY | {volume=vol-7-67105, bucket=bucket-0-68911, key=key-246-29031, dataSize=10240, replicationType=STAND_ALONE, replicationFactor=ONE, keyLocationInfo=null} | ret=SUCCESS |
-2018-09-06 01:57:22,997 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=COMMIT_KEY | {volume=vol-4-88912, bucket=bucket-0-27678, key=key-241-42688, dataSize=10240, replicationType=null, replicationFactor=null, keyLocationInfo=[org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo@25bd7387], clientID=61987500296} | ret=SUCCESS |
-2018-09-06 01:57:22,997 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=COMMIT_KEY | {volume=vol-1-59303, bucket=bucket-0-47510, key=key-248-17213, dataSize=10240, replicationType=null, replicationFactor=null, keyLocationInfo=[org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo@788f5bea], clientID=61990833797} | ret=SUCCESS |
-2018-09-06 01:58:08,035 | ERROR | OMAudit | user=hadoop | ip=172.18.0.4 | op=CREATE_VOLUME | {admin=hadoop, owner=tom, volume=dcv, creationTime=0, quotaInBytes=1152921504606846976} | ret=FAILURE | org.apache.hadoop.ozone.om.exceptions.OMException
+2018-09-06 01:57:22,996 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=ALLOCATE_KEY {volume=vol-7-67105, bucket=bucket-0-68911, key=key-246-29031, dataSize=10240, replicationType=STAND_ALONE, replicationFactor=ONE, keyLocationInfo=null} | ret=SUCCESS |
+2018-09-06 01:57:22,997 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=COMMIT_KEY {volume=vol-4-88912, bucket=bucket-0-27678, key=key-241-42688, dataSize=10240, replicationType=null, replicationFactor=null, keyLocationInfo=[org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo@25bd7387], clientID=61987500296} | ret=SUCCESS |
+2018-09-06 01:57:22,997 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=COMMIT_KEY {volume=vol-1-59303, bucket=bucket-0-47510, key=key-248-17213, dataSize=10240, replicationType=null, replicationFactor=null, keyLocationInfo=[org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo@788f5bea], clientID=61990833797} | ret=SUCCESS |
+2018-09-06 01:58:08,035 | ERROR | OMAudit | user=hadoop | ip=172.18.0.4 | op=CREATE_VOLUME {admin=hadoop, owner=tom, volume=dcv, creationTime=0, quotaInBytes=1152921504606846976} | ret=FAILURE | org.apache.hadoop.ozone.om.exceptions.OMException
at org.apache.hadoop.ozone.om.VolumeManagerImpl.createVolume(VolumeManagerImpl.java:137)
at org.apache.hadoop.ozone.om.OzoneManager.createVolume(OzoneManager.java:469)
-2018-09-06 01:58:18,447 | ERROR | OMAudit | user=hadoop | ip=172.18.0.4 | op=CREATE_BUCKET | {volume=dcv, bucket=dcb, acls=[USER:hadoop:rw, GROUP:users:rw], isVersionEnabled=false, storageType=DISK, creationTime=0} | ret=FAILURE | org.apache.hadoop.ozone.om.exceptions.OMException: Bucket already exist
+2018-09-06 01:58:18,447 | ERROR | OMAudit | user=hadoop | ip=172.18.0.4 | op=CREATE_BUCKET {volume=dcv, bucket=dcb, acls=[USER:hadoop:rw, GROUP:users:rw], isVersionEnabled=false, storageType=DISK, creationTime=0} | ret=FAILURE | org.apache.hadoop.ozone.om.exceptions.OMException: Bucket already exist
at org.apache.hadoop.ozone.om.BucketManagerImpl.createBucket(BucketManagerImpl.java:98)
at org.apache.hadoop.ozone.om.OzoneManager.createBucket(OzoneManager.java:694)
-2018-09-06 01:59:36,686 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=DELETE_KEY | {volume=dcv, bucket=dcb, key=dck1, dataSize=0, replicationType=null, replicationFactor=null, keyLocationInfo=null} | ret=SUCCESS |
-2018-09-06 01:59:41,027 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=DELETE_KEY | {volume=dcv, bucket=dcb, key=dck2, dataSize=0, replicationType=null, replicationFactor=null, keyLocationInfo=null} | ret=SUCCESS |
-2018-09-06 01:59:47,169 | ERROR | OMAudit | user=hadoop | ip=172.18.0.4 | op=DELETE_KEY | {volume=dcv, bucket=dcb, key=dck2, dataSize=0, replicationType=null, replicationFactor=null, keyLocationInfo=null} | ret=FAILURE | org.apache.hadoop.ozone.om.exceptions.OMException: Key not found
+2018-09-06 01:59:36,686 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=DELETE_KEY {volume=dcv, bucket=dcb, key=dck1, dataSize=0, replicationType=null, replicationFactor=null, keyLocationInfo=null} | ret=SUCCESS |
+2018-09-06 01:59:41,027 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=DELETE_KEY {volume=dcv, bucket=dcb, key=dck2, dataSize=0, replicationType=null, replicationFactor=null, keyLocationInfo=null} | ret=SUCCESS |
+2018-09-06 01:59:47,169 | ERROR | OMAudit | user=hadoop | ip=172.18.0.4 | op=DELETE_KEY {volume=dcv, bucket=dcb, key=dck2, dataSize=0, replicationType=null, replicationFactor=null, keyLocationInfo=null} | ret=FAILURE | org.apache.hadoop.ozone.om.exceptions.OMException: Key not found
at org.apache.hadoop.ozone.om.KeyManagerImpl.deleteKey(KeyManagerImpl.java:448)
at org.apache.hadoop.ozone.om.OzoneManager.deleteKey(OzoneManager.java:892)
-2018-09-06 01:60:22,900 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=ALLOCATE_KEY | {volume=vol-8-67105, bucket=bucket-0-68911, key=key-246-29031, dataSize=10240, replicationType=STAND_ALONE, replicationFactor=ONE, keyLocationInfo=null} | ret=SUCCESS |
+2018-09-06 01:60:22,900 | INFO | OMAudit | user=hadoop | ip=172.18.0.4 | op=ALLOCATE_KEY {volume=vol-8-67105, bucket=bucket-0-68911, key=key-246-29031, dataSize=10240, replicationType=STAND_ALONE, replicationFactor=ONE, keyLocationInfo=null} | ret=SUCCESS |
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 05/06: HDDS-973. HDDS/Ozone fail to build on Windows.
Contributed by Xiaoyu Yao.
Posted by na...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 5dae1a0c663cf9ab0e1e0463e5121afb0fa4a83e
Author: Nanda kumar <na...@gmail.com>
AuthorDate: Fri Jan 25 12:19:11 2019 +0530
HDDS-973. HDDS/Ozone fail to build on Windows.
Contributed by Xiaoyu Yao.
---
hadoop-hdds/docs/pom.xml | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/hadoop-hdds/docs/pom.xml b/hadoop-hdds/docs/pom.xml
index 43e6b19..2e22302 100644
--- a/hadoop-hdds/docs/pom.xml
+++ b/hadoop-hdds/docs/pom.xml
@@ -46,7 +46,13 @@ http://maven.apache.org/xsd/maven-4.0.0.xsd">
</execution>
</executions>
<configuration>
- <executable>dev-support/bin/generate-site.sh</executable>
+ <executable>${shell-executable}</executable>
+ <workingDirectory>${project.build.directory}</workingDirectory>
+ <arguments>
+ <argument>${basedir}/dev-support/bin/generate-site.sh</argument>
+ <argument>${hdds.version}</argument>
+ <argument>${project.build.directory}</argument>
+ </arguments>
</configuration>
</plugin>
<plugin>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 02/06: HDDS-1007. Add robot test for AuditParser.
Contributed by Dinesh Chitlangia.
Posted by na...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 8ff9578126cb97bb4958fde78a7f08c6e2b30f4b
Author: Nanda kumar <na...@gmail.com>
AuthorDate: Fri Jan 25 12:14:10 2019 +0530
HDDS-1007. Add robot test for AuditParser.
Contributed by Dinesh Chitlangia.
---
.../src/main/smoketest/auditparser/parser.robot | 40 ++++++++++++++++++++++
hadoop-ozone/dist/src/main/smoketest/test.sh | 2 ++
2 files changed, 42 insertions(+)
diff --git a/hadoop-ozone/dist/src/main/smoketest/auditparser/parser.robot b/hadoop-ozone/dist/src/main/smoketest/auditparser/parser.robot
new file mode 100644
index 0000000..2afdb5a
--- /dev/null
+++ b/hadoop-ozone/dist/src/main/smoketest/auditparser/parser.robot
@@ -0,0 +1,40 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+Documentation Smoketest ozone cluster startup
+Library OperatingSystem
+Resource ../commonlib.robot
+
+*** Test Cases ***
+
+Initiating freon to generate data
+ ${result} = Execute ozone freon randomkeys --numOfVolumes 5 --numOfBuckets 5 --numOfKeys 5 --numOfThreads 1
+ Wait Until Keyword Succeeds 3min 10sec Should contain ${result} Number of Keys added: 125
+ Should Not Contain ${result} ERROR
+
+Testing audit parser
+ ${logfile} = Execute ls -t /opt/hadoop/logs | grep om-audit | head -1
+ Execute ozone auditparser /opt/hadoop/audit.db load "/opt/hadoop/logs/${logfile}"
+ ${result} = Execute ozone auditparser /opt/hadoop/audit.db template top5cmds
+ Should Contain ${result} ALLOCATE_KEY
+ ${result} = Execute ozone auditparser /opt/hadoop/audit.db template top5users
+ Should Contain ${result} hadoop
+ ${result} = Execute ozone auditparser /opt/hadoop/audit.db query "select count(*) from audit where op='CREATE_VOLUME' and RESULT='SUCCESS'"
+ Should Contain ${result} 5
+ ${result} = Execute ozone auditparser /opt/hadoop/audit.db query "select count(*) from audit where op='CREATE_BUCKET' and RESULT='SUCCESS'"
+ Should Contain ${result} 5
+ ${result} = Execute ozone auditparser /opt/hadoop/audit.db query "select count(*) from audit where RESULT='FAILURE'"
+ Should Contain ${result} 0
diff --git a/hadoop-ozone/dist/src/main/smoketest/test.sh b/hadoop-ozone/dist/src/main/smoketest/test.sh
index b447481..04f5abd 100755
--- a/hadoop-ozone/dist/src/main/smoketest/test.sh
+++ b/hadoop-ozone/dist/src/main/smoketest/test.sh
@@ -140,6 +140,8 @@ if [ "$RUN_ALL" = true ]; then
#
# We select the test suites and execute them on multiple type of clusters
#
+ DEFAULT_TESTS=("auditparser")
+ execute_tests auditparser "${DEFAULT_TESTS[@]}"
DEFAULT_TESTS=("security")
execute_tests ozonesecure "${DEFAULT_TESTS[@]}"
DEFAULT_TESTS=("basic")
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 04/06: HDDS-990. Typos in Ozone doc. Contributed by
Doroszlai, Attila.
Posted by na...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 2181b188500eb01b7851cadcbe3c8fbf24245035
Author: Nanda kumar <na...@gmail.com>
AuthorDate: Fri Jan 25 12:17:36 2019 +0530
HDDS-990. Typos in Ozone doc.
Contributed by Doroszlai, Attila.
---
hadoop-hdds/docs/content/BucketCommands.md | 10 ++++-----
hadoop-hdds/docs/content/Dozone.md | 17 +++++++-------
hadoop-hdds/docs/content/JavaApi.md | 36 +++++++++++++++---------------
hadoop-hdds/docs/content/KeyCommands.md | 6 ++---
hadoop-hdds/docs/content/Rest.md | 2 +-
hadoop-hdds/docs/content/S3.md | 6 ++---
hadoop-hdds/docs/content/VolumeCommands.md | 2 +-
7 files changed, 39 insertions(+), 40 deletions(-)
diff --git a/hadoop-hdds/docs/content/BucketCommands.md b/hadoop-hdds/docs/content/BucketCommands.md
index 619cd19..5afd999 100644
--- a/hadoop-hdds/docs/content/BucketCommands.md
+++ b/hadoop-hdds/docs/content/BucketCommands.md
@@ -32,7 +32,7 @@ Ozone shell supports the following bucket commands.
### Create
-The bucket create command allows a user to create a bucket.
+The bucket create command allows users to create a bucket.
***Params:***
@@ -49,7 +49,7 @@ Since no scheme was specified this command defaults to O3 (RPC) protocol.
### Delete
-The bucket delete commands allows an user to delete a volume. If the
+The bucket delete command allows users to delete a bucket. If the
bucket is not empty then this command will fail.
***Params:***
@@ -59,7 +59,7 @@ bucket is not empty then this command will fail.
| Uri | The name of the bucket
{{< highlight bash >}}
-ozone sh volume delete /hive/jan
+ozone sh bucket delete /hive/jan
{{< /highlight >}}
The above command will delete _jan_ bucket if it is empty.
@@ -81,7 +81,7 @@ The above command will print out the information about _jan_ bucket.
### List
-The bucket list commands allows uset to list the buckets in a volume.
+The bucket list command allows users to list the buckets in a volume.
***Params:***
@@ -96,7 +96,7 @@ The bucket list commands allows uset to list the buckets in a volume.
ozone sh bucket list /hive
{{< /highlight >}}
-This command will list all buckets on the volume _hive_.
+This command will list all buckets on the volume _hive_.
diff --git a/hadoop-hdds/docs/content/Dozone.md b/hadoop-hdds/docs/content/Dozone.md
index f6efb0f..559134b 100644
--- a/hadoop-hdds/docs/content/Dozone.md
+++ b/hadoop-hdds/docs/content/Dozone.md
@@ -23,7 +23,7 @@ menu:
limitations under the License.
-->
-Dozone stands for docker for ozone. Ozone supports docker to make it easy to develop and test ozone. Starting a docker based ozone container is simple.
+Dozone stands for docker for ozone. Ozone supports docker to make it easy to develop and test ozone. Starting a docker-based ozone container is simple.
In the `compose/ozone` directory there are two files that define the docker and ozone settings.
@@ -41,13 +41,13 @@ docker-compose up -d
to run a ozone cluster on docker.
-This command will launch a Namenode, OM, SCM and a data node.
+This command will launch OM, SCM and a data node.
-To access the OM UI, one can run 'http://localhost:9874'.
+To access the OM UI, one can view http://localhost:9874.
_Please note_: dozone does not map the data node ports to the 9864. Instead, it maps to the ephemeral port range. So many examples in the command shell will not work if you run those commands from the host machine. To find out where the data node port is listening, you can run the `docker ps` command or always ssh into a container before running ozone commands.
-To shutdown a running docker based ozone cluster, please run
+To shutdown a running docker-based ozone cluster, please run
{{< highlight bash >}}
docker-compose down
@@ -63,7 +63,7 @@ Useful Docker & Ozone Commands
If you make any modifications to ozone, the simplest way to test it is to run freon and unit tests.
-Here are the instructions to run freon in a docker based cluster.
+Here are the instructions to run freon in a docker-based cluster.
{{< highlight bash >}}
docker-compose exec datanode bash
@@ -76,7 +76,7 @@ Now we can execute freon for load generation.
ozone freon randomkeys --numOfVolumes=10 --numOfBuckets 10 --numOfKeys 10
{{< /highlight >}}
-Here is a set helpful commands while working with docker for ozone.
+Here is a set of helpful commands for working with docker for ozone.
To check the status of the components:
{{< highlight bash >}}
@@ -93,9 +93,8 @@ docker-compose logs scm
As the WebUI ports are forwarded to the external machine, you can check the web UI:
* For the Storage Container Manager: http://localhost:9876
-* For the Ozone Managerr: http://localhost:9874
-* For the Datanode: check the port with docker ps (as there could be multiple data node ports are mapped to the ephemeral port range)
-* For the Namenode: http://localhost:9870
+* For the Ozone Manager: http://localhost:9874
+* For the Datanode: check the port with `docker ps` (as there could be multiple data nodes, ports are mapped to the ephemeral port range)
You can start multiple data nodes with:
diff --git a/hadoop-hdds/docs/content/JavaApi.md b/hadoop-hdds/docs/content/JavaApi.md
index e538f4b..de7a9fd 100644
--- a/hadoop-hdds/docs/content/JavaApi.md
+++ b/hadoop-hdds/docs/content/JavaApi.md
@@ -25,8 +25,8 @@ menu:
Introduction
-------------
-Ozone ships with it own client library, that supports both RPC(Remote
-Procedure call) and REST(Representational State Transfer). This library is
+Ozone ships with its own client library that supports both RPC (Remote
+Procedure Call) and REST (Representational State Transfer). This library is
the primary user interface to ozone.
It is trivial to switch from RPC to REST or vice versa, by setting the
@@ -77,10 +77,10 @@ An object store represents an active cluster against which the client is working
{{< highlight java >}}
// Let us create a volume to store our game assets.
// This uses default arguments for creating that volume.
-objectStore.createVolume(“assets”);
+objectStore.createVolume("assets");
// Let us verify that the volume got created.
-OzoneVolume assets = objectStore.getVolume(“assets”);
+OzoneVolume assets = objectStore.getVolume("assets");
{{< /highlight >}}
@@ -92,8 +92,8 @@ Once you have a volume, you can create buckets inside the volume.
{{< highlight bash >}}
// Let us create a bucket called videos.
-assets.createBucket(“videos”);
-Ozonebucket video = assets.getBucket(“videos”);
+assets.createBucket("videos");
+OzoneBucket video = assets.getBucket("videos");
{{< /highlight >}}
At this point we have a usable volume and a bucket. Our volume is called assets and bucket is called videos.
@@ -106,20 +106,20 @@ With a bucket object the users can now read and write keys. The following code r
{{< highlight bash >}}
// read data from the file, this is a user provided function.
-byte [] vidoeData = readFile(“into.mp4”);
+byte [] videoData = readFile("intro.mp4");
// Create an output stream and write data.
-OzoneOutputStream videoStream = video.createKey(“intro.mp4”, 1048576);
+OzoneOutputStream videoStream = video.createKey("intro.mp4", 1048576);
videoStream.write(videoData);
// Close the stream when it is done.
- videoStream.close();
+videoStream.close();
// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
// Let us allocate a byte array to hold the video first.
byte[] data = new byte[(int)1048576];
-OzoneInputStream introStream = video.readKey(“intro.mp4”);
+OzoneInputStream introStream = video.readKey("intro.mp4");
// read intro.mp4 into the data buffer
introStream.read(data);
introStream.close();
@@ -137,31 +137,31 @@ ObjectStore objectStore = ozClient.getObjectStore();
// Let us create a volume to store our game assets.
// This default arguments for creating that volume.
-objectStore.createVolume(“assets”);
+objectStore.createVolume("assets");
// Let us verify that the volume got created.
-OzoneVolume assets = objectStore.getVolume(“assets”);
+OzoneVolume assets = objectStore.getVolume("assets");
// Let us create a bucket called videos.
-assets.createBucket(“videos”);
-Ozonebucket video = assets.getBucket(“videos”);
+assets.createBucket("videos");
+OzoneBucket video = assets.getBucket("videos");
// read data from the file, this is assumed to be a user provided function.
-byte [] vidoeData = readFile(“into.mp4”);
+byte [] videoData = readFile("intro.mp4");
// Create an output stream and write data.
-OzoneOutputStream videoStream = video.createKey(“intro.mp4”, 1048576);
+OzoneOutputStream videoStream = video.createKey("intro.mp4", 1048576);
videoStream.write(videoData);
// Close the stream when it is done.
- videoStream.close();
+videoStream.close();
// We can use the same bucket to read the file that we just wrote, by creating an input Stream.
// Let us allocate a byte array to hold the video first.
byte[] data = new byte[(int)1048576];
-OzoneInputStream introStream = video.readKey(“into.mp4”);
+OzoneInputStream introStream = video.readKey("intro.mp4");
introStream.read(data);
// Close the stream when it is done.
diff --git a/hadoop-hdds/docs/content/KeyCommands.md b/hadoop-hdds/docs/content/KeyCommands.md
index 0139a28..1a77762 100644
--- a/hadoop-hdds/docs/content/KeyCommands.md
+++ b/hadoop-hdds/docs/content/KeyCommands.md
@@ -69,7 +69,7 @@ The above command will put the sales.orc as a new key into _/hive/jan/corrected-
### Delete
-The delete key command removes the key from the bucket.
+The key delete command removes the key from the bucket.
***Params:***
@@ -102,7 +102,7 @@ key.
### List
-The key list commands allows user to list all keys in a bucket.
+The key list command allows user to list all keys in a bucket.
***Params:***
@@ -117,7 +117,7 @@ The key list commands allows user to list all keys in a bucket.
ozone sh key list /hive/jan
{{< /highlight >}}
-This command will list all key in the bucket _/hive/jan_.
+This command will list all keys in the bucket _/hive/jan_.
diff --git a/hadoop-hdds/docs/content/Rest.md b/hadoop-hdds/docs/content/Rest.md
index a25d3ab..b3faac0 100644
--- a/hadoop-hdds/docs/content/Rest.md
+++ b/hadoop-hdds/docs/content/Rest.md
@@ -219,7 +219,7 @@ this request gets all volumes owned by *bilbo* and each volume's name contains p
### Create Bucket
-This API allows an user to create a bucket in a volume.
+This API allows users to create a bucket in a volume.
Schema:
diff --git a/hadoop-hdds/docs/content/S3.md b/hadoop-hdds/docs/content/S3.md
index e8897b1..a451310 100644
--- a/hadoop-hdds/docs/content/S3.md
+++ b/hadoop-hdds/docs/content/S3.md
@@ -54,7 +54,7 @@ By default it uses the path-style addressing. To use virtual host style URLs set
## Bucket browser
-Bucket's could be browsed from the browser with adding `?browser=true` to the bucket URL.
+Buckets could be browsed from the browser by adding `?browser=true` to the bucket URL.
For example the content of the 'testbucket' could be checked from the browser using the URL http://localhost:9878/testbucket?browser=true
@@ -92,7 +92,7 @@ HEAD Object | implemented |
Security is not yet implemented, you can *use* any AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
-Note: Ozone has a notion for 'volumes' which is missing from the S3 Rest endpoint. Under the hood S3 bucket names are mapped to Ozone 'volume/bucket' locations (depending from the given authentication information).
+Note: Ozone has a notion for 'volumes' which is missing from the S3 Rest endpoint. Under the hood S3 bucket names are mapped to Ozone 'volume/bucket' locations (depending on the given authentication information).
To show the storage location of a S3 bucket, use the `ozone sh bucket path <bucketname>` command.
@@ -122,7 +122,7 @@ aws s3 ls --endpoint http://localhost:9878 s3://buckettest
### S3 Fuse driver (goofys)
-Goofys is a S3 FUSE driver. It could be used to any mount any Ozone bucket as posix file system:
+Goofys is a S3 FUSE driver. It could be used to mount any Ozone bucket as posix file system:
```
diff --git a/hadoop-hdds/docs/content/VolumeCommands.md b/hadoop-hdds/docs/content/VolumeCommands.md
index 6f024ef..fecda40 100644
--- a/hadoop-hdds/docs/content/VolumeCommands.md
+++ b/hadoop-hdds/docs/content/VolumeCommands.md
@@ -52,7 +52,7 @@ volume has a quota of 1TB, and the owner is _bilbo_.
### Delete
-The volume delete commands allows an administrator to delete a volume. If the
+The volume delete command allows an administrator to delete a volume. If the
volume is not empty then this command will fail.
***Params:***
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 06/06: HDDS-1009. TestAbortMultipartUpload is missing the
apache license text. Contributed by Dinesh Chitlangia.
Posted by na...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nanda pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit a448b05287452d5063610fea3c10634040e205ad
Author: Nanda kumar <na...@gmail.com>
AuthorDate: Fri Jan 25 12:20:12 2019 +0530
HDDS-1009. TestAbortMultipartUpload is missing the apache license text.
Contributed by Dinesh Chitlangia.
---
.../ozone/s3/endpoint/TestAbortMultipartUpload.java | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestAbortMultipartUpload.java b/hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestAbortMultipartUpload.java
index 61cf48a..76d4a12 100644
--- a/hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestAbortMultipartUpload.java
+++ b/hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestAbortMultipartUpload.java
@@ -1,3 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
package org.apache.hadoop.ozone.s3.endpoint;
import org.apache.hadoop.ozone.client.OzoneClientStub;
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org