You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by gi...@apache.org on 2018/06/14 13:58:20 UTC
mesos-site git commit: Updated the website built from mesos SHA:
e25385a.
Repository: mesos-site
Updated Branches:
refs/heads/asf-site ac68e3297 -> 166f7d4d1
Updated the website built from mesos SHA: e25385a.
Project: http://git-wip-us.apache.org/repos/asf/mesos-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos-site/commit/166f7d4d
Tree: http://git-wip-us.apache.org/repos/asf/mesos-site/tree/166f7d4d
Diff: http://git-wip-us.apache.org/repos/asf/mesos-site/diff/166f7d4d
Branch: refs/heads/asf-site
Commit: 166f7d4d18780306617e65891eea240e7ae066c9
Parents: ac68e32
Author: jenkins <bu...@apache.org>
Authored: Thu Jun 14 13:58:15 2018 +0000
Committer: jenkins <bu...@apache.org>
Committed: Thu Jun 14 13:58:15 2018 +0000
----------------------------------------------------------------------
content/documentation/index.html | 1 +
content/documentation/latest/index.html | 1 +
.../latest/operator-http-api/index.html | 4 +-
.../documentation/latest/recordio/index.html | 249 +++++++++++++++++++
.../latest/scheduler-http-api/index.html | 37 +--
.../documentation/operator-http-api/index.html | 4 +-
content/documentation/recordio/index.html | 249 +++++++++++++++++++
.../documentation/scheduler-http-api/index.html | 37 +--
content/sitemap.xml | 8 +
9 files changed, 514 insertions(+), 76 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/166f7d4d/content/documentation/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/index.html b/content/documentation/index.html
index eef417a..ae66025 100644
--- a/content/documentation/index.html
+++ b/content/documentation/index.html
@@ -229,6 +229,7 @@
<li><a href="endpoints/">HTTP Endpoints</a> for available HTTP endpoints.</li>
<li><a href="/documentation/latest/./api-client-libraries/">API Client Libraries</a> lists client libraries for the HTTP APIs.</li>
<li><a href="/documentation/latest/./versioning/">API Versioning</a> describes HTTP API and release versioning.</li>
+<li><a href="/documentation/latest/./recordio/">RecordIO</a> describes the RecordIO format used by the streaming endpoints of the HTTP API.</li>
<li>Writing Frameworks
<ul>
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/166f7d4d/content/documentation/latest/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/latest/index.html b/content/documentation/latest/index.html
index 1714cf4..dda56f1 100644
--- a/content/documentation/latest/index.html
+++ b/content/documentation/latest/index.html
@@ -229,6 +229,7 @@
<li><a href="endpoints/">HTTP Endpoints</a> for available HTTP endpoints.</li>
<li><a href="/documentation/latest/./api-client-libraries/">API Client Libraries</a> lists client libraries for the HTTP APIs.</li>
<li><a href="/documentation/latest/./versioning/">API Versioning</a> describes HTTP API and release versioning.</li>
+<li><a href="/documentation/latest/./recordio/">RecordIO</a> describes the RecordIO format used by the streaming endpoints of the HTTP API.</li>
<li>Writing Frameworks
<ul>
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/166f7d4d/content/documentation/latest/operator-http-api/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/latest/operator-http-api/index.html b/content/documentation/latest/operator-http-api/index.html
index cc9fce2..c71206a 100644
--- a/content/documentation/latest/operator-http-api/index.html
+++ b/content/documentation/latest/operator-http-api/index.html
@@ -117,7 +117,7 @@
<p>For requests that Mesos can answer synchronously and immediately, an HTTP response will be sent with status <strong>200 OK</strong>, possibly including a response body encoded in JSON or Protobuf. The encoding depends on the <strong>Accept</strong> header present in the request (the default encoding is JSON). Responses will be gzip compressed if the <strong>Accept-Encoding</strong> header is set to “gzip”.</p>
-<p>For requests that require asynchronous processing (e.g., <code>RESERVE_RESOURCES</code>), an HTTP response will be sent with status <strong>202 Accepted</strong>. For requests that result in a stream of events (<code>SUBSCRIBE</code>), a streaming HTTP response with <a href="/documentation/latest/./scheduler-http-api/#recordio-response-format">RecordIO</a> encoding is sent. Currently, gzip compression is not supported for streaming responses.</p>
+<p>For requests that require asynchronous processing (e.g., <code>RESERVE_RESOURCES</code>), an HTTP response will be sent with status <strong>202 Accepted</strong>. For requests that result in a stream of events (<code>SUBSCRIBE</code>), a streaming HTTP response with <a href="/documentation/latest/./recordio/">RecordIO</a> encoding is sent. Currently, gzip compression is not supported for streaming responses.</p>
<h2>Master API</h2>
@@ -2543,7 +2543,7 @@ Transfer-Encoding: chunked
<more events>
</code></pre>
-<p>The client is expected to keep a <strong>persistent</strong> connection open to the endpoint even after getting a <code>SUBSCRIBED</code> HTTP Response event. This is indicated by “Connection: keep-alive” and “Transfer-Encoding: chunked” headers with <em>no</em> “Content-Length” header set. All subsequent events generated by Mesos are streamed on this connection. The master encodes each Event in <a href="/documentation/latest/./scheduler-http-api/#recordio-response-format">RecordIO</a> format, i.e., string representation of length of the event in bytes followed by JSON or binary Protobuf encoded event.</p>
+<p>The client is expected to keep a <strong>persistent</strong> connection open to the endpoint even after getting a <code>SUBSCRIBED</code> HTTP Response event. This is indicated by “Connection: keep-alive” and “Transfer-Encoding: chunked” headers with <em>no</em> “Content-Length” header set. All subsequent events generated by Mesos are streamed on this connection. The master encodes each Event in <a href="/documentation/latest/./recordio/">RecordIO</a> format, i.e., string representation of length of the event in bytes followed by JSON or binary Protobuf encoded event.</p>
<p>The following events are currently sent by the master. The canonical source of this information is at <a href="https://github.com/apache/mesos/blob/master/include/mesos/v1/master/master.proto">master.proto</a>. Note that when sending JSON encoded events, master encodes raw bytes in Base64 and strings in UTF-8.</p>
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/166f7d4d/content/documentation/latest/recordio/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/latest/recordio/index.html b/content/documentation/latest/recordio/index.html
new file mode 100644
index 0000000..359eef9
--- /dev/null
+++ b/content/documentation/latest/recordio/index.html
@@ -0,0 +1,249 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title>Apache Mesos - RecordIO Data Format</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <meta property="og:locale" content="en_US"/>
+ <meta property="og:type" content="website"/>
+ <meta property="og:title" content="Apache Mesos"/>
+ <meta property="og:site_name" content="Apache Mesos"/>
+ <meta property="og:url" content="http://mesos.apache.org/"/>
+ <meta property="og:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/>
+ <meta property="og:description"
+ content="Apache Mesos abstracts resources away from machines,
+ enabling fault-tolerant and elastic distributed systems
+ to easily be built and run effectively."/>
+
+ <meta name="twitter:card" content="summary"/>
+ <meta name="twitter:site" content="@ApacheMesos"/>
+ <meta name="twitter:title" content="Apache Mesos"/>
+ <meta name="twitter:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/>
+ <meta name="twitter:description"
+ content="Apache Mesos abstracts resources away from machines,
+ enabling fault-tolerant and elastic distributed systems
+ to easily be built and run effectively."/>
+
+ <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
+ <link href="../../../assets/css/main.css" rel="stylesheet" />
+
+
+ <!-- Google Analytics Magic -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-20226872-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <!-- magical breadcrumbs -->
+ <div class="topnav">
+ <div class="container">
+ <ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+
+ <li><a href="http://mesos.apache.org">Apache Mesos</a></li>
+ <li><a href="/documentation
+/">Documentation
+</a></li>
+ </ul><!-- /.breadcrumb -->
+ </div><!-- /.container -->
+ </div><!-- /.topnav -->
+
+ <!-- navbar excitement -->
+<div class="navbar navbar-default navbar-static-top" role="navigation">
+ <div class="container">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#mesos-menu" aria-expanded="false">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo"/></a>
+ </div><!-- /.navbar-header -->
+
+ <div class="navbar-collapse collapse" id="mesos-menu">
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/getting-started/">Getting Started</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Community</a></li>
+ </ul>
+ </div><!-- /#mesos-menu -->
+ </div><!-- /.container -->
+</div><!-- /.navbar -->
+
+<div class="content">
+ <div class="container">
+ <div class="row-fluid">
+ <div class="col-md-4">
+ <h4>If you're new to Mesos</h4>
+ <p>See the <a href="/getting-started/">getting started</a> page for more
+ information about downloading, building, and deploying Mesos.</p>
+
+ <h4>If you'd like to get involved or you're looking for support</h4>
+ <p>See our <a href="/community/">community</a> page for more details.</p>
+ </div>
+ <div class="col-md-8">
+ <h1>RecordIO Data Format</h1>
+
+<h2>Overview</h2>
+
+<p>RecordIO is the name for a set of binary data exchange formats.
+The basic idea is to divide the data into individual chunks, called ‘records’,
+and then to prepend to every record its length in bytes, followed by the data.</p>
+
+<p>Since there is no formal specification of the RecordIO format, there tend
+to be slight incompatibilities between RecordIO implementations. Common
+differences include, for example, a magic value at the beginning of the
+stream to indicate the stream format, encoding the record length using
+fixed-size fields with native integers or extra fields to suppord compressed
+record content.</p>
+
+<p>Therefore, when using a third-party library, one needs to verify that
+the implementation matches the RecordIO format used by Mesos, as described
+below.</p>
+
+<h2>Mesos RecordIO Format</h2>
+
+<p>The <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html#sec2.1">BNF grammar</a>
+for a RecordIO-encoded streaming response is:</p>
+
+<pre><code> records = *record
+
+ record = record-size LF record-data
+
+ record-size = 1*DIGIT
+ record-data = record-size(OCTET)
+</code></pre>
+
+<p><code>record-size</code> should be interpreted as an unsigned 64-bit integer (<code>uint64</code>).</p>
+
+<p>For example, a stream may look like:</p>
+
+<pre><code>128\n
+{"type": "SUBSCRIBED","subscribed": {"framework_id": {"value":"12220-3440-12532-2345"},"heartbeat_interval_seconds":15.0}20\n
+{"type":"HEARTBEAT"}675\n
+...
+</code></pre>
+
+<p>In pseudo-code, this could be parsed with something like the following:</p>
+
+<pre><code> while (true) {
+ do {
+ lengthBytes = readline()
+ } while (lengthBytes.length < 1)
+
+ messageLength = parseInt(lengthBytes);
+ messageBytes = read(messageLength);
+ process(messageBytes);
+ }
+</code></pre>
+
+<p>Network intermediaries (e.g., proxies) are free to change the chunk
+boundaries; this should not have any effect on the recipient application.</p>
+
+<h2>Streaming HTTP requests</h2>
+
+<p>Within Mesos, RecordIO is used as data format for API endpoints
+that support streaming responses. The implementation expects the
+client to be aware of the following non-standard HTTP header fields
+when using such endpoints:</p>
+
+<p>The <code>Message-Accept</code> header must be set by clients that request a
+streaming response from the server by setting the <code>Accept</code> header to
+<code>application/recordio</code>. It specifies which media types the client
+accepts encoded in the RecordIO stream, and its content must have the
+same format as the standard HTTP <code>Accept</code> header.</p>
+
+<p>The <code>Message-Content-Type</code> is set by the server when streaming the
+response to an API call. It specifies the content type of the data
+encoded in the RecordIO stream and has the same format as the
+standard HTTP <code>Content-Type</code> header.</p>
+
+<p>Currently, the only supported MIME types for both header fields are
+<code>application/json</code> and <code>application/x-protobuf</code>.</p>
+
+ </div>
+</div>
+
+ </div><!-- /.container -->
+</div><!-- /.content -->
+
+<hr>
+
+
+
+ <!-- footer -->
+ <div class="footer">
+ <div class="container">
+
+ <div class="col-md-3">
+ <a href="https://www.apache.org/events/current-event.html">
+ <img src="https://www.apache.org/events/current-event-234x60.png"/>
+ </a>
+ </div>
+
+ <div class="col-md-3 social-blk">
+ <span class="social">
+ <a href="https://twitter.com/ApacheMesos"
+ class="twitter-follow-button"
+ data-show-count="false" data-size="large">Follow @ApacheMesos</a>
+ <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
+ <a href="https://twitter.com/intent/tweet?button_hashtag=mesos"
+ class="twitter-hashtag-button"
+ data-size="large"
+ data-related="ApacheMesos">Tweet #mesos</a>
+ <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
+ </span>
+ </div>
+
+ <div class="col-md-6 trademark">
+ <p>© 2012-2018 <a href="http://apache.org">The Apache Software Foundation</a>.
+ Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.
+ <p>
+ </div>
+
+ </div><!-- /.container -->
+ </div><!-- /.footer -->
+
+ <!-- JS -->
+ <script src="//code.jquery.com/jquery-1.11.0.min.js"></script>
+ <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js"></script>
+ <script src="//cdnjs.cloudflare.com/ajax/libs/anchor-js/4.1.0/anchor.min.js"></script>
+
+ <!-- Inject anchors for all headings on the page, see https://www.bryanbraun.com/anchorjs. -->
+ <script type="text/javascript">
+ anchors.options = {
+ placement: 'right',
+ ariaLabel: 'Permalink',
+ };
+
+ // The default is to not add anchors to h1, but we have pages with multiple h1 headers,
+ // and we do want to put anchors on those.
+ anchors.add('h1, h2, h3, h4, h5, h6');
+ </script>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/166f7d4d/content/documentation/latest/scheduler-http-api/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/latest/scheduler-http-api/index.html b/content/documentation/latest/scheduler-http-api/index.html
index 85a3a94..a70db96 100644
--- a/content/documentation/latest/scheduler-http-api/index.html
+++ b/content/documentation/latest/scheduler-http-api/index.html
@@ -153,42 +153,7 @@ considered stable and is the recommended way to develop new Mesos schedulers.</p
<h3>RecordIO response format</h3>
-<p>The response returned from the <code>SUBSCRIBE</code> call (see <a href="#subscribe">below</a>) is encoded in RecordIO format, which essentially prepends to a single record (either JSON or serialized Protobuf) its length in bytes, followed by a newline and then the data:</p>
-
-<p>The <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html#sec2.1">BNF grammar</a> for a RecordIO-encoded streaming response is:</p>
-
-<pre><code> records = *record
-
- record = record-size LF record-data
-
- record-size = 1*DIGIT
- record-data = record-size(OCTET)
-</code></pre>
-
-<p><code>record-size</code> should be interpreted as an unsigned 64-bit integer (<code>uint64</code>).</p>
-
-<p>For example, a stream may look like:</p>
-
-<pre><code>128\n
-{"type": "SUBSCRIBED","subscribed": {"framework_id": {"value":"12220-3440-12532-2345"},"heartbeat_interval_seconds":15.0}20\n
-{"type":"HEARTBEAT"}675\n
-...
-</code></pre>
-
-<p>In pseudo-code, this could be parsed with something like the following:</p>
-
-<pre><code> while (true) {
- do {
- lengthBytes = readline()
- } while (lengthBytes.length < 1)
-
- messageLength = parseInt(lengthBytes);
- messageBytes = read(messageLength);
- process(messageBytes);
- }
-</code></pre>
-
-<p>Network intermediaries (e.g., proxies) are free to change the chunk boundaries; this should not have any effect on the recipient application (scheduler). We wanted a way to delimit/encode two events for JSON/Protobuf responses consistently and RecordIO format allowed us to do that.</p>
+<p>The response returned from the <code>SUBSCRIBE</code> call (see <a href="#subscribe">below</a>) is encoded in RecordIO format, which essentially prepends to a single record (either JSON or serialized Protobuf) its length in bytes, followed by a newline and then the data. See <a href="/documentation/latest/./recordio/">RecordIO Format</a> for details.</p>
<p><a id="subscribe"></a></p>
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/166f7d4d/content/documentation/operator-http-api/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/operator-http-api/index.html b/content/documentation/operator-http-api/index.html
index cfaaa0e..195fb5a 100644
--- a/content/documentation/operator-http-api/index.html
+++ b/content/documentation/operator-http-api/index.html
@@ -117,7 +117,7 @@
<p>For requests that Mesos can answer synchronously and immediately, an HTTP response will be sent with status <strong>200 OK</strong>, possibly including a response body encoded in JSON or Protobuf. The encoding depends on the <strong>Accept</strong> header present in the request (the default encoding is JSON). Responses will be gzip compressed if the <strong>Accept-Encoding</strong> header is set to “gzip”.</p>
-<p>For requests that require asynchronous processing (e.g., <code>RESERVE_RESOURCES</code>), an HTTP response will be sent with status <strong>202 Accepted</strong>. For requests that result in a stream of events (<code>SUBSCRIBE</code>), a streaming HTTP response with <a href="/documentation/latest/./scheduler-http-api/#recordio-response-format">RecordIO</a> encoding is sent. Currently, gzip compression is not supported for streaming responses.</p>
+<p>For requests that require asynchronous processing (e.g., <code>RESERVE_RESOURCES</code>), an HTTP response will be sent with status <strong>202 Accepted</strong>. For requests that result in a stream of events (<code>SUBSCRIBE</code>), a streaming HTTP response with <a href="/documentation/latest/./recordio/">RecordIO</a> encoding is sent. Currently, gzip compression is not supported for streaming responses.</p>
<h2>Master API</h2>
@@ -2543,7 +2543,7 @@ Transfer-Encoding: chunked
<more events>
</code></pre>
-<p>The client is expected to keep a <strong>persistent</strong> connection open to the endpoint even after getting a <code>SUBSCRIBED</code> HTTP Response event. This is indicated by “Connection: keep-alive” and “Transfer-Encoding: chunked” headers with <em>no</em> “Content-Length” header set. All subsequent events generated by Mesos are streamed on this connection. The master encodes each Event in <a href="/documentation/latest/./scheduler-http-api/#recordio-response-format">RecordIO</a> format, i.e., string representation of length of the event in bytes followed by JSON or binary Protobuf encoded event.</p>
+<p>The client is expected to keep a <strong>persistent</strong> connection open to the endpoint even after getting a <code>SUBSCRIBED</code> HTTP Response event. This is indicated by “Connection: keep-alive” and “Transfer-Encoding: chunked” headers with <em>no</em> “Content-Length” header set. All subsequent events generated by Mesos are streamed on this connection. The master encodes each Event in <a href="/documentation/latest/./recordio/">RecordIO</a> format, i.e., string representation of length of the event in bytes followed by JSON or binary Protobuf encoded event.</p>
<p>The following events are currently sent by the master. The canonical source of this information is at <a href="https://github.com/apache/mesos/blob/master/include/mesos/v1/master/master.proto">master.proto</a>. Note that when sending JSON encoded events, master encodes raw bytes in Base64 and strings in UTF-8.</p>
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/166f7d4d/content/documentation/recordio/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/recordio/index.html b/content/documentation/recordio/index.html
new file mode 100644
index 0000000..11c7706
--- /dev/null
+++ b/content/documentation/recordio/index.html
@@ -0,0 +1,249 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title>Apache Mesos - RecordIO Data Format</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <meta property="og:locale" content="en_US"/>
+ <meta property="og:type" content="website"/>
+ <meta property="og:title" content="Apache Mesos"/>
+ <meta property="og:site_name" content="Apache Mesos"/>
+ <meta property="og:url" content="http://mesos.apache.org/"/>
+ <meta property="og:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/>
+ <meta property="og:description"
+ content="Apache Mesos abstracts resources away from machines,
+ enabling fault-tolerant and elastic distributed systems
+ to easily be built and run effectively."/>
+
+ <meta name="twitter:card" content="summary"/>
+ <meta name="twitter:site" content="@ApacheMesos"/>
+ <meta name="twitter:title" content="Apache Mesos"/>
+ <meta name="twitter:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/>
+ <meta name="twitter:description"
+ content="Apache Mesos abstracts resources away from machines,
+ enabling fault-tolerant and elastic distributed systems
+ to easily be built and run effectively."/>
+
+ <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
+ <link href="../../assets/css/main.css" rel="stylesheet" />
+
+
+ <!-- Google Analytics Magic -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-20226872-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <!-- magical breadcrumbs -->
+ <div class="topnav">
+ <div class="container">
+ <ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+
+ <li><a href="http://mesos.apache.org">Apache Mesos</a></li>
+ <li><a href="/documentation
+/">Documentation
+</a></li>
+ </ul><!-- /.breadcrumb -->
+ </div><!-- /.container -->
+ </div><!-- /.topnav -->
+
+ <!-- navbar excitement -->
+<div class="navbar navbar-default navbar-static-top" role="navigation">
+ <div class="container">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#mesos-menu" aria-expanded="false">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo"/></a>
+ </div><!-- /.navbar-header -->
+
+ <div class="navbar-collapse collapse" id="mesos-menu">
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/getting-started/">Getting Started</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Community</a></li>
+ </ul>
+ </div><!-- /#mesos-menu -->
+ </div><!-- /.container -->
+</div><!-- /.navbar -->
+
+<div class="content">
+ <div class="container">
+ <div class="row-fluid">
+ <div class="col-md-4">
+ <h4>If you're new to Mesos</h4>
+ <p>See the <a href="/getting-started/">getting started</a> page for more
+ information about downloading, building, and deploying Mesos.</p>
+
+ <h4>If you'd like to get involved or you're looking for support</h4>
+ <p>See our <a href="/community/">community</a> page for more details.</p>
+ </div>
+ <div class="col-md-8">
+ <h1>RecordIO Data Format</h1>
+
+<h2>Overview</h2>
+
+<p>RecordIO is the name for a set of binary data exchange formats.
+The basic idea is to divide the data into individual chunks, called ‘records’,
+and then to prepend to every record its length in bytes, followed by the data.</p>
+
+<p>Since there is no formal specification of the RecordIO format, there tend
+to be slight incompatibilities between RecordIO implementations. Common
+differences include, for example, a magic value at the beginning of the
+stream to indicate the stream format, encoding the record length using
+fixed-size fields with native integers or extra fields to suppord compressed
+record content.</p>
+
+<p>Therefore, when using a third-party library, one needs to verify that
+the implementation matches the RecordIO format used by Mesos, as described
+below.</p>
+
+<h2>Mesos RecordIO Format</h2>
+
+<p>The <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html#sec2.1">BNF grammar</a>
+for a RecordIO-encoded streaming response is:</p>
+
+<pre><code> records = *record
+
+ record = record-size LF record-data
+
+ record-size = 1*DIGIT
+ record-data = record-size(OCTET)
+</code></pre>
+
+<p><code>record-size</code> should be interpreted as an unsigned 64-bit integer (<code>uint64</code>).</p>
+
+<p>For example, a stream may look like:</p>
+
+<pre><code>128\n
+{"type": "SUBSCRIBED","subscribed": {"framework_id": {"value":"12220-3440-12532-2345"},"heartbeat_interval_seconds":15.0}20\n
+{"type":"HEARTBEAT"}675\n
+...
+</code></pre>
+
+<p>In pseudo-code, this could be parsed with something like the following:</p>
+
+<pre><code> while (true) {
+ do {
+ lengthBytes = readline()
+ } while (lengthBytes.length < 1)
+
+ messageLength = parseInt(lengthBytes);
+ messageBytes = read(messageLength);
+ process(messageBytes);
+ }
+</code></pre>
+
+<p>Network intermediaries (e.g., proxies) are free to change the chunk
+boundaries; this should not have any effect on the recipient application.</p>
+
+<h2>Streaming HTTP requests</h2>
+
+<p>Within Mesos, RecordIO is used as data format for API endpoints
+that support streaming responses. The implementation expects the
+client to be aware of the following non-standard HTTP header fields
+when using such endpoints:</p>
+
+<p>The <code>Message-Accept</code> header must be set by clients that request a
+streaming response from the server by setting the <code>Accept</code> header to
+<code>application/recordio</code>. It specifies which media types the client
+accepts encoded in the RecordIO stream, and its content must have the
+same format as the standard HTTP <code>Accept</code> header.</p>
+
+<p>The <code>Message-Content-Type</code> is set by the server when streaming the
+response to an API call. It specifies the content type of the data
+encoded in the RecordIO stream and has the same format as the
+standard HTTP <code>Content-Type</code> header.</p>
+
+<p>Currently, the only supported MIME types for both header fields are
+<code>application/json</code> and <code>application/x-protobuf</code>.</p>
+
+ </div>
+</div>
+
+ </div><!-- /.container -->
+</div><!-- /.content -->
+
+<hr>
+
+
+
+ <!-- footer -->
+ <div class="footer">
+ <div class="container">
+
+ <div class="col-md-3">
+ <a href="https://www.apache.org/events/current-event.html">
+ <img src="https://www.apache.org/events/current-event-234x60.png"/>
+ </a>
+ </div>
+
+ <div class="col-md-3 social-blk">
+ <span class="social">
+ <a href="https://twitter.com/ApacheMesos"
+ class="twitter-follow-button"
+ data-show-count="false" data-size="large">Follow @ApacheMesos</a>
+ <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
+ <a href="https://twitter.com/intent/tweet?button_hashtag=mesos"
+ class="twitter-hashtag-button"
+ data-size="large"
+ data-related="ApacheMesos">Tweet #mesos</a>
+ <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
+ </span>
+ </div>
+
+ <div class="col-md-6 trademark">
+ <p>© 2012-2018 <a href="http://apache.org">The Apache Software Foundation</a>.
+ Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.
+ <p>
+ </div>
+
+ </div><!-- /.container -->
+ </div><!-- /.footer -->
+
+ <!-- JS -->
+ <script src="//code.jquery.com/jquery-1.11.0.min.js"></script>
+ <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js"></script>
+ <script src="//cdnjs.cloudflare.com/ajax/libs/anchor-js/4.1.0/anchor.min.js"></script>
+
+ <!-- Inject anchors for all headings on the page, see https://www.bryanbraun.com/anchorjs. -->
+ <script type="text/javascript">
+ anchors.options = {
+ placement: 'right',
+ ariaLabel: 'Permalink',
+ };
+
+ // The default is to not add anchors to h1, but we have pages with multiple h1 headers,
+ // and we do want to put anchors on those.
+ anchors.add('h1, h2, h3, h4, h5, h6');
+ </script>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/166f7d4d/content/documentation/scheduler-http-api/index.html
----------------------------------------------------------------------
diff --git a/content/documentation/scheduler-http-api/index.html b/content/documentation/scheduler-http-api/index.html
index 9ab3136..fee8c39 100644
--- a/content/documentation/scheduler-http-api/index.html
+++ b/content/documentation/scheduler-http-api/index.html
@@ -153,42 +153,7 @@ considered stable and is the recommended way to develop new Mesos schedulers.</p
<h3>RecordIO response format</h3>
-<p>The response returned from the <code>SUBSCRIBE</code> call (see <a href="#subscribe">below</a>) is encoded in RecordIO format, which essentially prepends to a single record (either JSON or serialized Protobuf) its length in bytes, followed by a newline and then the data:</p>
-
-<p>The <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html#sec2.1">BNF grammar</a> for a RecordIO-encoded streaming response is:</p>
-
-<pre><code> records = *record
-
- record = record-size LF record-data
-
- record-size = 1*DIGIT
- record-data = record-size(OCTET)
-</code></pre>
-
-<p><code>record-size</code> should be interpreted as an unsigned 64-bit integer (<code>uint64</code>).</p>
-
-<p>For example, a stream may look like:</p>
-
-<pre><code>128\n
-{"type": "SUBSCRIBED","subscribed": {"framework_id": {"value":"12220-3440-12532-2345"},"heartbeat_interval_seconds":15.0}20\n
-{"type":"HEARTBEAT"}675\n
-...
-</code></pre>
-
-<p>In pseudo-code, this could be parsed with something like the following:</p>
-
-<pre><code> while (true) {
- do {
- lengthBytes = readline()
- } while (lengthBytes.length < 1)
-
- messageLength = parseInt(lengthBytes);
- messageBytes = read(messageLength);
- process(messageBytes);
- }
-</code></pre>
-
-<p>Network intermediaries (e.g., proxies) are free to change the chunk boundaries; this should not have any effect on the recipient application (scheduler). We wanted a way to delimit/encode two events for JSON/Protobuf responses consistently and RecordIO format allowed us to do that.</p>
+<p>The response returned from the <code>SUBSCRIBE</code> call (see <a href="#subscribe">below</a>) is encoded in RecordIO format, which essentially prepends to a single record (either JSON or serialized Protobuf) its length in bytes, followed by a newline and then the data. See <a href="/documentation/latest/./recordio/">RecordIO Format</a> for details.</p>
<p><a id="subscribe"></a></p>
http://git-wip-us.apache.org/repos/asf/mesos-site/blob/166f7d4d/content/sitemap.xml
----------------------------------------------------------------------
diff --git a/content/sitemap.xml b/content/sitemap.xml
index fdb0149..cff1860 100644
--- a/content/sitemap.xml
+++ b/content/sitemap.xml
@@ -573,6 +573,10 @@
<lastmod>2018-06-14T00:00:00+00:00</lastmod>
</url>
<url>
+ <loc>http://mesos.apache.org/documentation/latest/recordio/</loc>
+ <lastmod>2018-06-14T00:00:00+00:00</lastmod>
+ </url>
+ <url>
<loc>http://mesos.apache.org/documentation/latest/docker-containerizer/</loc>
<lastmod>2018-06-14T00:00:00+00:00</lastmod>
</url>
@@ -18525,6 +18529,10 @@
<lastmod>2018-06-14T00:00:00+00:00</lastmod>
</url>
<url>
+ <loc>http://mesos.apache.org/documentation/recordio/</loc>
+ <lastmod>2018-06-14T00:00:00+00:00</lastmod>
+ </url>
+ <url>
<loc>http://mesos.apache.org/documentation/docker-containerizer/</loc>
<lastmod>2018-06-14T00:00:00+00:00</lastmod>
</url>