You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by va...@apache.org on 2017/02/21 06:59:52 UTC
[1/2] hadoop git commit: YARN-6170. TimelineReaderServer should wait
to join with HttpServer2 (Sangjin Lee via Varun Saxena)
Repository: hadoop
Updated Branches:
refs/heads/YARN-5355-branch-2 42b69405f -> fc5438f06
YARN-6170. TimelineReaderServer should wait to join with HttpServer2 (Sangjin Lee via Varun Saxena)
(cherry picked from commit 649deb72fbb62568b4ea0d67444df6faaaed169d)
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/26593053
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/26593053
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/26593053
Branch: refs/heads/YARN-5355-branch-2
Commit: 2659305349ab2622c9fbbacea3979062d66d24b9
Parents: 42b6940
Author: Varun Saxena <va...@apache.org>
Authored: Sat Feb 11 19:21:45 2017 +0530
Committer: Varun Saxena <va...@apache.org>
Committed: Tue Feb 21 12:28:33 2017 +0530
----------------------------------------------------------------------
.../timelineservice/reader/TimelineReaderServer.java | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hadoop/blob/26593053/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineReaderServer.java
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineReaderServer.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineReaderServer.java
index 116cc2a..8c5e72d 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineReaderServer.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineReaderServer.java
@@ -110,6 +110,16 @@ public class TimelineReaderServer extends CompositeService {
startTimelineReaderWebApp();
}
+ private void join() {
+ // keep the main thread that started the server up until it receives a stop
+ // signal
+ if (readerWebServer != null) {
+ try {
+ readerWebServer.join();
+ } catch (InterruptedException ignore) {}
+ }
+ }
+
@Override
protected void serviceStop() throws Exception {
if (readerWebServer != null) {
@@ -185,6 +195,7 @@ public class TimelineReaderServer extends CompositeService {
Configuration conf = new YarnConfiguration();
conf.setBoolean(YarnConfiguration.TIMELINE_SERVICE_ENABLED, true);
conf.setFloat(YarnConfiguration.TIMELINE_SERVICE_VERSION, 2.0f);
- startTimelineReaderServer(args, conf);
+ TimelineReaderServer server = startTimelineReaderServer(args, conf);
+ server.join();
}
}
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[2/2] hadoop git commit: YARN-6159. Documentation changes for
TimelineV2Client (Naganarasimha G R via Varun Saxena)
Posted by va...@apache.org.
YARN-6159. Documentation changes for TimelineV2Client (Naganarasimha G R via Varun Saxena)
(cherry picked from commit 6ba61d20d3f65e40ea8e3a49d5beebe34f04aab4)
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/fc5438f0
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/fc5438f0
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/fc5438f0
Branch: refs/heads/YARN-5355-branch-2
Commit: fc5438f067621b4c6b5639e102fe8839a5006cba
Parents: 2659305
Author: Varun Saxena <va...@apache.org>
Authored: Tue Feb 21 12:25:37 2017 +0530
Committer: Varun Saxena <va...@apache.org>
Committed: Tue Feb 21 12:29:27 2017 +0530
----------------------------------------------------------------------
.../src/site/markdown/TimelineServiceV2.md | 44 ++++++++------------
1 file changed, 18 insertions(+), 26 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hadoop/blob/fc5438f0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServiceV2.md
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServiceV2.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServiceV2.md
index 02b0562..7d36a4a 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServiceV2.md
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServiceV2.md
@@ -340,56 +340,48 @@ To write MapReduce framework data to Timeline Service v.2, enable the following
This section is for YARN application developers that want to integrate with Timeline Service v.2.
-Developers can continue to use the `TimelineClient` API to publish per-framework data to the
-Timeline Service v.2. You only need to instantiate the right type of the client to write to v.2.
-On the other hand, the entity/object API for v.2 is different than v.1 as the object model is
-significantly changed. The v.2 timeline entity class is
-`org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity` whereas the v.1 class is
-`org.apache.hadoop.yarn.api.records.timeline.TimelineEntity`. The methods on `TimelineClient`
-suitable for writing to Timeline Service v.2 are clearly delineated, and they use the v.2
-types as arguments.
+Developers need to use the `TimelineV2Client` API to publish per-framework data to the
+Timeline Service v.2. The entity/object API for v.2 is different than v.1 as
+the object model is significantly changed. The v.2 timeline entity class is
+`org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity`.
Timeline Service v.2 `putEntities` methods come in 2 varieties: `putEntities` and
`putEntitiesAsync`. The former is a blocking operation which must be used for writing more
critical data (e.g. lifecycle events). The latter is a non-blocking operation. Note that neither
has a return value.
-Creating a `TimelineClient` for v.2 involves passing in the application id to the factory method.
+Creating a `TimelineV2Client` involves passing in the application id to the static method
+`TimelineV2Client.createTimelineClient`.
For example:
// Create and start the Timeline client v.2
- TimelineClient client = TimelineClient.createTimelineClient(appId);
- client.init(conf);
- client.start();
+ TimelineV2Client timelineClient =
+ TimelineV2Client.createTimelineClient(appId);
+ timelineClient.init(conf);
+ timelineClient.start();
try {
TimelineEntity myEntity = new TimelineEntity();
- myEntity.setEntityType("MY_APPLICATION");
- myEntity.setEntityId("MyApp1")
+ myEntity.setType("MY_APPLICATION");
+ myEntity.setId("MyApp1");
// Compose other entity info
// Blocking write
- client.putEntities(entity);
+ timelineClient.putEntities(myEntity);
TimelineEntity myEntity2 = new TimelineEntity();
// Compose other info
// Non-blocking write
- timelineClient.putEntitiesAsync(entity);
+ timelineClient.putEntitiesAsync(myEntity2);
- } catch (IOException e) {
- // Handle the exception
- } catch (RuntimeException e) {
- // In Hadoop 2.6, if attempts submit information to the Timeline Server fail more than the retry limit,
- // a RuntimeException will be raised. This may change in future releases, being
- // replaced with a IOException that is (or wraps) that which triggered retry failures.
- } catch (YarnException e) {
+ } catch (IOException | YarnException e) {
// Handle the exception
} finally {
// Stop the Timeline client
- client.stop();
+ timelineClient.stop();
}
As evidenced above, you need to specify the YARN application id to be able to write to the Timeline
@@ -397,9 +389,9 @@ Service v.2. Note that currently you need to be on the cluster to be able to wri
Service. For example, an application master or code in the container can write to the Timeline
Service, while an off-cluster MapReduce job submitter cannot.
-After creating the timeline client, user also needs to set the timeline collector address for the application. If `AMRMClient` is used then by registering the timeline client by calling `AMRMClient#registerTimelineClient` is sufficient.
+After creating the timeline v2 client, user also needs to set the timeline collector address for the application. If `AMRMClient` is used then by registering the timeline client by calling `AMRMClient#registerTimelineV2Client` is sufficient.
- amRMClient.registerTimelineClient(timelineClient)\u037e
+ amRMClient.registerTimelineV2Client(timelineClient)\u037e
Else address needs to be retrieved from the AM allocate response and need to be set in timeline client explicitly.
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org