You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by gi...@apache.org on 2020/08/06 07:55:43 UTC
[airflow-site] branch asf-site updated: Update asf-site to output
generated at fd501a8
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 7d4aa67 Update asf-site to output generated at fd501a8
7d4aa67 is described below
commit 7d4aa67a4428781918120fdd1bc0aa1844dc4268
Author: potiuk <ja...@potiuk.com>
AuthorDate: Thu Aug 6 07:55:32 2020 +0000
Update asf-site to output generated at fd501a8
---
_gen/docs-index.json | 32 ++--
_gen/indexes/en/blog-index.json | 2 +-
_gen/indexes/en/blog-posts.json | 2 +-
blog/airflow-1.10.10/index.html | 8 +-
blog/airflow-1.10.8-1.10.9/index.html | 4 +-
blog/airflow-survey/index.html | 4 +-
blog/announcing-new-website/index.html | 4 +-
.../index.html | 4 +-
.../index.html | 4 +-
.../index.html | 4 +-
.../index.html | 192 ++++++++++++---------
blog/index.html | 33 ++++
blog/index.xml | 101 +++++++++++
.../index.html | 4 +-
blog/tags/community/index.html | 11 +-
blog/tags/community/index.xml | 4 +-
blog/tags/development/index.html | 5 +
blog/tags/documentation/index.html | 5 +
blog/tags/release/index.html | 5 +
blog/tags/{documentation => rest-api}/index.html | 33 ++--
blog/tags/rest-api/index.xml | 119 +++++++++++++
blog/tags/survey/index.html | 5 +
blog/tags/users/index.html | 5 +
community/index.html | 174 ++++++++++++++++---
index.html | 30 ++--
index.xml | 101 +++++++++++
search/index.html | 4 +-
sitemap.xml | 110 ++++++------
tags/index.html | 2 +-
tags/index.xml | 2 +-
use-cases/adobe/index.html | 4 +-
use-cases/big-fish-games/index.html | 4 +-
use-cases/dish/index.html | 4 +-
use-cases/experity/index.html | 4 +-
use-cases/onefootball/index.html | 4 +-
35 files changed, 787 insertions(+), 246 deletions(-)
diff --git a/_gen/docs-index.json b/_gen/docs-index.json
index c4d4e20..bc8758e 100644
--- a/_gen/docs-index.json
+++ b/_gen/docs-index.json
@@ -1,16 +1,16 @@
-{
- "stable": "1.10.11",
- "versions": [
- "1.10.1",
- "1.10.10",
- "1.10.11",
- "1.10.2",
- "1.10.3",
- "1.10.4",
- "1.10.5",
- "1.10.6",
- "1.10.7",
- "1.10.8",
- "1.10.9"
- ]
-}
+{
+ "stable": "1.10.11",
+ "versions": [
+ "1.10.1",
+ "1.10.10",
+ "1.10.11",
+ "1.10.2",
+ "1.10.3",
+ "1.10.4",
+ "1.10.5",
+ "1.10.6",
+ "1.10.7",
+ "1.10.8",
+ "1.10.9"
+ ]
+}
diff --git a/_gen/indexes/en/blog-index.json b/_gen/indexes/en/blog-index.json
index fccb8ef..3bed3c5 100644
--- a/_gen/indexes/en/blog-index.json
+++ b/_gen/indexes/en/blog-index.json
@@ -1 +1 @@
-{"version":"2.3.8","fields":["title","description","author","content","tags","url"],"fieldVectors":[["title/It's a \"Breeze\" to develop Apache Airflow",[0,1.805,1,0.76,2,0.219,3,0.339,4,0.469]],["description/It's a \"Breeze\" to develop Apache Airflow",[2,0.236,4,0.329,5,1.268,6,0.534,7,1.268,8,0.702,9,0.534,10,0.702,11,0.534,12,0.4,13,1.268,14,0.4,15,1.268,16,0.034]],["author/It's a \"Breeze\" to develop Apache Airflow",[11,0.76,17,0.999]],["content/It's a \"Breeze\" to develop Apache [...]
\ No newline at end of file
+{"version":"2.3.8","fields":["title","description","author","content","tags","url"],"fieldVectors":[["title/It's a \"Breeze\" to develop Apache Airflow",[0,1.906,1,0.855,2,0.163,3,0.38,4,0.479]],["description/It's a \"Breeze\" to develop Apache Airflow",[2,0.177,4,0.342,5,1.358,6,0.609,7,1.358,8,0.78,9,0.609,10,0.609,11,0.609,12,0.472,13,1.358,14,0.472,15,1.358,16,0.1]],["author/It's a \"Breeze\" to develop Apache Airflow",[11,0.855,17,1.095]],["content/It's a \"Breeze\" to develop Apach [...]
\ No newline at end of file
diff --git a/_gen/indexes/en/blog-posts.json b/_gen/indexes/en/blog-posts.json
index a484d48..41d118a 100644
--- a/_gen/indexes/en/blog-posts.json
+++ b/_gen/indexes/en/blog-posts.json
@@ -1 +1 @@
-[{"content":"## The story behind the Airflow Breeze tool\nInitially, we started contributing to this fantastic open-source project [Apache Airflow] with a team of three which then grew to five. When we kicked it off a year ago, I realized pretty soon where the biggest bottlenecks and areas for improvement in terms of productivity were. Even with the help of our client, who provided us with a “homegrown” development environment it took us literally days to set it up and learn some basics. [...]
\ No newline at end of file
+[{"content":"## The story behind the Airflow Breeze tool\nInitially, we started contributing to this fantastic open-source project [Apache Airflow] with a team of three which then grew to five. When we kicked it off a year ago, I realized pretty soon where the biggest bottlenecks and areas for improvement in terms of productivity were. Even with the help of our client, who provided us with a “homegrown” development environment it took us literally days to set it up and learn some basics. [...]
\ No newline at end of file
diff --git a/blog/airflow-1.10.10/index.html b/blog/airflow-1.10.10/index.html
index 9c29bea..7da9f9a 100644
--- a/blog/airflow-1.10.10/index.html
+++ b/blog/airflow-1.10.10/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-04-09T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-04-10T18:44:33+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Apache Airflow 1.10.10">
<meta itemprop="description" content="We are happy to present Apache Airflow 1.10.10">
<meta itemprop="datePublished" content="2020-04-09T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-04-10T18:44:33+01:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="1143">
@@ -687,10 +687,10 @@ objc[1873]: +[__NSPlaceholderDate initialize] may have been in progress in anoth
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Previous</button>
</a>
- <a >
+ <a href="/blog/implementing-stable-api-for-apache-airflow/">
-<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Next</button>
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Next</button>
</a>
</div>
diff --git a/blog/airflow-1.10.8-1.10.9/index.html b/blog/airflow-1.10.8-1.10.9/index.html
index 7bc0e70..4f3c072 100644
--- a/blog/airflow-1.10.8-1.10.9/index.html
+++ b/blog/airflow-1.10.8-1.10.9/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-02-23T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-02-23T22:34:43+00:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Apache Airflow 1.10.8 & 1.10.9">
<meta itemprop="description" content="We are happy to present the new 1.10.8 and 1.10.9 releases of Apache Airflow.">
<meta itemprop="datePublished" content="2020-02-23T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-02-23T22:34:43+00:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="437">
diff --git a/blog/airflow-survey/index.html b/blog/airflow-survey/index.html
index eaa6851..5c09368 100644
--- a/blog/airflow-survey/index.html
+++ b/blog/airflow-survey/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-12-11T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-02-19T10:34:11+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Airflow Survey 2019">
<meta itemprop="description" content="Receiving and adjusting to our users’ feedback is a must. Let’s see who Airflow users are, how they play with it, and what they miss.">
<meta itemprop="datePublished" content="2019-12-11T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-02-19T10:34:11+01:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="1775">
diff --git a/blog/announcing-new-website/index.html b/blog/announcing-new-website/index.html
index 60b4160..a4d2fa4 100644
--- a/blog/announcing-new-website/index.html
+++ b/blog/announcing-new-website/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-12-11T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-02-19T10:34:11+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="New Airflow website">
<meta itemprop="description" content="We are thrilled about our new website!">
<meta itemprop="datePublished" content="2019-12-11T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-02-19T10:34:11+01:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="282">
diff --git a/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html b/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html
index edbb63b..b8a9a37 100644
--- a/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html
+++ b/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-11-22T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-02-11T08:20:10+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="ApacheCon Europe 2019 — Thoughts and Insights by Airflow Committers">
<meta itemprop="description" content="Here come some thoughts by Airflow committers and contributors from the ApacheCon Europe 2019. Get to know the ASF community!">
<meta itemprop="datePublished" content="2019-11-22T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-02-11T08:20:10+01:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="138">
diff --git a/blog/documenting-using-local-development-environments/index.html b/blog/documenting-using-local-development-environments/index.html
index 5820cc2..f078cbe 100644
--- a/blog/documenting-using-local-development-environments/index.html
+++ b/blog/documenting-using-local-development-environments/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-11-22T00:00:00+00:00" />
-<meta property="article:modified_time" content="2019-12-30T16:35:47+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Documenting using local development environment">
<meta itemprop="description" content="The story behind documenting local development environment of Apache Airflow">
<meta itemprop="datePublished" content="2019-11-22T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2019-12-30T16:35:47+01:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="256">
diff --git a/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html b/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html
index 52897bb..dc43459 100644
--- a/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html
+++ b/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html
@@ -37,14 +37,14 @@ About Me I have been writing tech articles on medium as well as my blog for the
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-12-20T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-02-08T05:17:09+05:30" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Experience in Google Season of Docs 2019 with Apache Airflow">
<meta itemprop="description" content="I came across Google Season of Docs (GSoD) almost by accident, thanks to my extensive HackerNews and Twitter addiction. I was familiar with the Google Summer of Code but not with this program. It turns out it was the inaugural phase. I read the details, and the process felt a lot like GSoC except that this was about documentation.
About Me I have been writing tech articles on medium as well as my blog for the past 1.">
<meta itemprop="datePublished" content="2019-12-20T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-02-08T05:17:09+05:30" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="1521">
diff --git a/blog/its-a-breeze-to-develop-apache-airflow/index.html b/blog/implementing-stable-api-for-apache-airflow/index.html
similarity index 86%
copy from blog/its-a-breeze-to-develop-apache-airflow/index.html
copy to blog/implementing-stable-api-for-apache-airflow/index.html
index 574c47e..62f8e97 100644
--- a/blog/its-a-breeze-to-develop-apache-airflow/index.html
+++ b/blog/implementing-stable-api-for-apache-airflow/index.html
@@ -29,30 +29,30 @@
<meta name="msapplication-TileImage" content="/favicons/ms-icon-144x144.png">
<meta name="theme-color" content="#ffffff">
-<title>It's a "Breeze" to develop Apache Airflow | Apache Airflow</title><meta property="og:title" content="It's a "Breeze" to develop Apache Airflow" />
-<meta property="og:description" content="A Principal Software Engineer's journey to developer productivity. Learn how Jarek and his team speeded up and simplified Airflow development for the community." />
+<title>Implementing Stable API for Apache Airflow | Apache Airflow</title><meta property="og:title" content="Implementing Stable API for Apache Airflow" />
+<meta property="og:description" content="An Outreachy intern's progress report on contributing to Apache Airflow REST API." />
<meta property="og:type" content="article" />
-<meta property="og:url" content="/blog/its-a-breeze-to-develop-apache-airflow/" />
+<meta property="og:url" content="/blog/implementing-stable-api-for-apache-airflow/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:published_time" content="2019-11-22T00:00:00+00:00" />
-<meta property="article:modified_time" content="2019-11-26T12:05:52+01:00" /><meta property="og:site_name" content="Apache Airflow" />
-<meta itemprop="name" content="It's a "Breeze" to develop Apache Airflow">
-<meta itemprop="description" content="A Principal Software Engineer's journey to developer productivity. Learn how Jarek and his team speeded up and simplified Airflow development for the community.">
+<meta property="article:published_time" content="2020-07-19T00:00:00+00:00" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta itemprop="name" content="Implementing Stable API for Apache Airflow">
+<meta itemprop="description" content="An Outreachy intern's progress report on contributing to Apache Airflow REST API.">
-<meta itemprop="datePublished" content="2019-11-22T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2019-11-26T12:05:52+01:00" />
-<meta itemprop="wordCount" content="127">
+<meta itemprop="datePublished" content="2020-07-19T00:00:00+00:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
+<meta itemprop="wordCount" content="703">
-<meta itemprop="keywords" content="Development," />
+<meta itemprop="keywords" content="REST API," />
<meta name="twitter:card" content="summary_large_image"/>
<meta name="twitter:image" content="/images/feature-image.png"/>
-<meta name="twitter:title" content="It's a "Breeze" to develop Apache Airflow"/>
-<meta name="twitter:description" content="A Principal Software Engineer's journey to developer productivity. Learn how Jarek and his team speeded up and simplified Airflow development for the community."/>
+<meta name="twitter:title" content="Implementing Stable API for Apache Airflow"/>
+<meta name="twitter:description" content="An Outreachy intern's progress report on contributing to Apache Airflow REST API."/>
<script type="application/javascript">
@@ -80,7 +80,7 @@ if (!doNotTrack) {
crossorigin="anonymous"></script>
-<meta name="description" content="A Principal Software Engineer's journey to developer productivity. Learn how Jarek and his team speeded up and simplified Airflow development for the community." />
+<meta name="description" content="An Outreachy intern's progress report on contributing to Apache Airflow REST API." />
@@ -379,19 +379,17 @@ if (!doNotTrack) {
<div class="tags-container">
- <a class="tag" href="/blog/tags/development/">Development</a>
-
</div>
- <span class="bodytext__medium--brownish-grey">Fri, Nov 22, 2019</span>
+ <span class="bodytext__medium--brownish-grey">Sun, Jul 19, 2020</span>
</div>
- <p class="blogpost-content__metadata--title">It's a "Breeze" to develop Apache Airflow</p>
+ <p class="blogpost-content__metadata--title">Implementing Stable API for Apache Airflow</p>
<div class="blogpost-content__metadata--author">
<span class="blogpost-content__metadata--author">
- Jarek Potiuk
+ Ephraim Anierobi
</span>
- <a href="https://twitter.com/higrys/" class="blogpost-content__metadata--social-media-icon">
+ <a href="https://twitter.com/ephraimbuddy/" class="blogpost-content__metadata--social-media-icon">
<svg xmlns="http://www.w3.org/2000/svg" width="22" height="21" viewBox="0 0 22 21">
<g id="Group_1746" data-name="Group 1746" transform="translate(.076 -.055)">
<ellipse id="Ellipse_19" cx="11" cy="10.5" fill="#51504f" data-name="Ellipse 19" rx="11" ry="10.5"
@@ -407,7 +405,7 @@ if (!doNotTrack) {
</a>
- <a href="https://github.com/potiuk/" class="blogpost-content__metadata--social-media-icon">
+ <a href="https://github.com/ephraimbuddy/" class="blogpost-content__metadata--social-media-icon">
<svg xmlns="http://www.w3.org/2000/svg" width="21.737" height="21.2" viewBox="0 0 21.737 21.2">
<path id="Path_1378" d="M33.971 1181.31a10.87 10.87 0 0 0-3.435 21.182c.543.1.742-.236.742-.524 0-.258-.009-.941-.015-1.848-3.023.657-3.661-1.457-3.661-1.457a2.876 2.876 0 0 0-1.207-1.59c-.987-.674.075-.661.075-.661a2.283 2.283 0 0 1 1.665 1.12 2.314 2.314 0 0 0 3.163.9 2.322 2.322 0 0 1 .69-1.453c-2.413-.274-4.951-1.207-4.951-5.371a4.2 4.2 0 0 1 1.119-2.917 3.908 3.908 0 0 1 .107-2.876s.913-.292 2.989 1.114a10.3 10.3 0 0 1 5.442 0c2.075-1.406 2.986-1.114 2.986-1.114a3.9 3.9 0 0 1 .1 [...]
</svg>
@@ -415,32 +413,98 @@ if (!doNotTrack) {
</a>
- <a href="https://linkedin.com/in/jarekpotiuk/" class="blogpost-content__metadata--social-media-icon">
- <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 20 21">
- <g id="Group_1745" data-name="Group 1745" transform="translate(.155 -.055)">
- <ellipse id="Ellipse_20" cx="10" cy="10.5" fill="#51504f" data-name="Ellipse 20" rx="10" ry="10.5" transform="translate(-.155 .055)"/>
- <g id="Group_698" data-name="Group 698" transform="translate(5.843 5.004)">
- <path id="Path_644" d="M-1311.072 1423.962a.9.9 0 0 1-.972.9.888.888 0 0 1-.937-.9.9.9 0 0 1 .961-.9.892.892 0 0 1 .948.9zm-1.862 7.413v-5.8h1.8v5.8z" fill="#fff" data-name="Path 644" transform="translate(1312.981 -1423.061)"/>
- <path id="Path_645" d="M-1284.253 1448.767c0-.723-.024-1.328-.047-1.85h1.565l.083.806h.035a2.084 2.084 0 0 1 1.791-.937c1.186 0 2.076.795 2.076 2.5v3.428h-1.8v-3.214c0-.747-.261-1.257-.914-1.257a.989.989 0 0 0-.925.676 1.29 1.29 0 0 0-.06.451v3.345h-1.8z" fill="#fff" data-name="Path 645" transform="translate(1287.182 -1444.402)"/>
- </g>
- </g>
-</svg>
-
- </a>
-
</div>
- <p class="blogpost-content__metadata--description">A Principal Software Engineer's journey to developer productivity. Learn how Jarek and his team speeded up and simplified Airflow development for the community.</p>
+ <p class="blogpost-content__metadata--description">An Outreachy intern's progress report on contributing to Apache Airflow REST API.</p>
</div>
<div class="markdown-content">
-<h2 id="the-story-behind-the-airflow-breeze-tool">The story behind the Airflow Breeze tool</h2>
+<p>My <a href="https://outreachy.org" target="_blank">Outreachy internship</a> is coming to its ends which is also the best time to look back and
+reflect on the progress so far.</p>
+
+<p>The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post,
+I will be sharing my progress so far.</p>
+
+<p>We started a bit late implementing the REST API because it took time for the OpenAPI 3.0
+specification we were to use for the project to be merged. Thanks to <a href="https://github.com/mik-laj" target="_blank">Kamil</a>,
+who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I
+implemented and the challenges I encountered, including how I overcame them.</p>
+
+<h3 id="implementing-the-read-only-connection-endpoints">Implementing The Read-Only Connection Endpoints</h3>
+
+<p>The <a href="https://github.com/apache/airflow/pull/9095" target="_blank">read-only connection endpoints</a> were the first endpoint I implemented. Looking back,
+I can see how much I have improved.</p>
+
+<p>I started by implementing the database schema for the Connection table using <a href="https://marshmallow.readthedocs.io/en/2.x-line/" target="_blank">Marshmallow 2</a>.
+We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder
+is deeply integrated to Apache Airflow. This meant I had to unlearn Marshmallow 3 that I had
+ been studying before this realization, but thankfully, <a href="https://marshmallow.readthedocs.io/en/stable/index.html" target="_blank">Marshmallow 3</a> isn’t too
+ different, so I was able to start using Marshmallow 2 in no time.</p>
+
+<p>This first PR would have been more difficult than it was unless there had been any reference
+endpoint to look at. <a href="https://github.com/mik-laj" target="_blank">Kamil</a> implemented a <a href="https://github.com/apache/airflow/pull/9045" target="_blank">draft PR</a> in which I took inspiration from.
+Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that
+ I learned using <a href="https://github.com/wolever/parameterized" target="_blank">parameterized</a> in unit tests :D.</p>
+
+<h3 id="implementing-the-read-only-dagruns-endpoints">Implementing The Read-Only DagRuns Endpoints</h3>
+
+<p>This <a href="https://github.com/apache/airflow/pull/9153" target="_blank">endpoint</a> came with its many challenges, especially on filtering with <code>datetimes</code>.
+This was because the <code>connexion</code> library we were using to build the REST API was not validating
+date-time format in OpenAPI 3.0 specification, what I eventually found out, was intentional.
+Connexion dropped <code>strict-rfc3339</code> because of the later license which is not compatible with
+Apache 2.0 license.</p>
+
+<p>I implemented a workaround on this, by defining a function called <code>conn_parse_datetime</code> in the
+API utils module. This was later refactored and thankfully, <a href="https://github.com/mik-laj" target="_blank">Kamil</a>
+ implemented a decorator that allowed us to have cleaner code on the views while using this function.</p>
+
+<p>Then we tried using <code>rfc3339-validator</code> whose license is compatible with Apache 2.0 licence but
+ later discarded this because with our custom date parser we were able to use duration and
+ not just date times.</p>
+
+<h3 id="other-endpoints">Other Endpoints</h3>
+
+<p>I implemented some different other endpoints. One peculiar issue I faced was because of Marshmallow 2
+not giving error when extra fields are in the request body. I implemented a <code>validate_unknown</code>
+method on the schema to handle this. Thankfully, Flask-AppBuilder updated to using Marshmallow 3,
+we quickly updated Flask-AppBuilder in Apache Airflow and started using Marshmallow 3 too.</p>
+
+<p>Here are some PRs I contributed that are related to the REST API:</p>
+
+<ol>
+<li><p><a href="https://github.com/apache/airflow/pull/9227" target="_blank">Add event log endpoints</a>
+The event log would help users get information on operations performed at the UI</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9266" target="_blank">Add CRUD endpoints for connection</a>
+This PR performs DELETE, PATCH and POST operations on <code>Connection</code></p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9331" target="_blank">Add log endpoint</a>
+This PR enables users to get Task Instances log entries</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9431" target="_blank">Move limit & offset to kwargs in views plus work on a configurable maximum limit</a>
+This helped us in having a neat code on the views and added configurable maximum limit on query results.</p></li>
-<p>Initially, we started contributing to this fantastic open-source project [Apache Airflow] with a team of three which then grew to five. When we kicked it off a year ago, I realized pretty soon where the biggest bottlenecks and areas for improvement in terms of productivity were. Even with the help of our client, who provided us with a “homegrown” development environment it took us literally days to set it up and learn some basics.</p>
+<li><p><a href="https://github.com/apache/airflow/pull/9648" target="_blank">Update FlaskAppBuilder to v3</a>
+This enabled Airflow to start using v3 of Flask App Builder and also made it possible for the API to use
+ a modern database serializer/deserializer</p></li>
-<p>That is how the journey to increased productivity in Apache Airflow began. The result? The Airflow Breeze open-source tool. Jarek Potiuk, an Airflow Committer, will tell you all about it.</p>
+<li><p><a href="https://github.com/apache/airflow/pull/9771" target="_blank">Add migration guide from the experimental REST API to the stable REST API</a>
+This would enable users to start using the stable REST API in less time.</p></li>
+</ol>
-<p>You can learn <a href="https://www.polidea.com/blog/its-a-breeze-to-develop-apache-airflow/?utm_source=ApacheAirflowBlog&utm_medium=Npaid&utm_campaign=Blog&utm_term=Article&utm_content=AAB_NOP_BLG_ART_AB_001" target="_blank">how and why it’s a “Breeze” to Develop Apache Airflow on Polidea blog</a>.</p>
+<h3 id="follow-ups">Follow-Ups</h3>
+
+<p>There is still lots of works to be done on the REST API including writing helpful documentation.
+I still follow up on these and hopefully, we will complete the REST API before the internship ends.</p>
+
+<p>I am very grateful to my mentors, <a href="https://github.com/potiuk" target="_blank">Jarek</a> and <a href="https://github.com/kaxil" target="_blank">Kaxil</a> for their
+patience with me and for surviving my never-ending questions. <a href="https://github.com/mik-laj" target="_blank">Kamil</a> and <a href="https://github.com/turbaszek" target="_blank">Tomek</a>
+have been very supportive and I appreciate them for their support and amazing code reviews.</p>
+
+<p>Thanks to <a href="https://github.com/leahecole" target="_blank">Leah E. Cole</a> and <a href="https://github.com/mschickensoup" target="_blank">Karolina Rosół</a>, for their
+wonderful reviews. I’m grateful.</p>
+
+<p>Thanks for reading!</p>
</div>
@@ -498,16 +562,16 @@ if (!doNotTrack) {
<div class="pager">
- <a >
+ <a href="/blog/airflow-1.10.10/">
-<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Previous</button>
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Previous</button>
</a>
- <a href="/blog/documenting-using-local-development-environments/">
+ <a >
-<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Next</button>
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Next</button>
</a>
</div>
@@ -520,46 +584,6 @@ if (!doNotTrack) {
- <div class="blog-pager">
- <h5 class="header__xsmall--greyish-brown">Read also</h5>
- <div class="pager">
- <div class="list-items">
-
- <div class="list-item list-item--wide">
-
-
-<div class="card">
- <div class="box-event__blogpost">
- <div class="box-event__blogpost--metadata">
- <div class="tags-container">
-
-
- <a class="tag" href="/blog/tags/development/">Development</a>
-
-
- </div>
- <span class="bodytext__medium--brownish-grey box-event__blogpost--date">Fri, Nov 22, 2019</span>
- </div>
- <p class="box-event__blogpost--header">Documenting using local development environment</p>
- <p class="box-event__blogpost--author">Elena Fedotova</p>
- <p class="box-event__blogpost--description">The story behind documenting local development environment of Apache Airflow</p>
- <div class="mt-auto">
- <a href="/blog/documenting-using-local-development-environments/">
-
-
-<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read more</button>
-
- </a>
- </div>
- </div>
-</div>
-
- </div>
-
- </div>
- </div>
- </div>
-
</div>
@@ -568,7 +592,7 @@ if (!doNotTrack) {
<div class="base-layout--button">
- <a href=https://github.com/apache/airflow-site/edit/master/landing-pages/site/content/en/blog/Its-a-breeze-to-develop-apache-airflow.md>
+ <a href=https://github.com/apache/airflow-site/edit/master/landing-pages/site/content/en/blog/implementing-stable-API-for-Apache-Airflow.md>
diff --git a/blog/index.html b/blog/index.html
index 3c4e8ec..e91c208 100644
--- a/blog/index.html
+++ b/blog/index.html
@@ -400,6 +400,10 @@ if (!doNotTrack) {
+ <a class="tag" href="/blog/tags/rest-api/">Rest api</a>
+
+
+
<a class="tag" href="/blog/tags/survey/">Survey</a>
@@ -424,6 +428,35 @@ if (!doNotTrack) {
<div class="tags-container">
+
+ </div>
+ <span class="bodytext__medium--brownish-grey box-event__blogpost--date">Sun, Jul 19, 2020</span>
+ </div>
+ <p class="box-event__blogpost--header">Implementing Stable API for Apache Airflow</p>
+ <p class="box-event__blogpost--author">Ephraim Anierobi</p>
+ <p class="box-event__blogpost--description">An Outreachy intern's progress report on contributing to Apache Airflow REST API.</p>
+ <div class="mt-auto">
+ <a href="/blog/implementing-stable-api-for-apache-airflow/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
<a class="tag" href="/blog/tags/release/">Release</a>
diff --git a/blog/index.xml b/blog/index.xml
index 3416a84..34046f9 100644
--- a/blog/index.xml
+++ b/blog/index.xml
@@ -14,6 +14,107 @@
<item>
+ <title>Blog: Implementing Stable API for Apache Airflow</title>
+ <link>/blog/implementing-stable-api-for-apache-airflow/</link>
+ <pubDate>Sun, 19 Jul 2020 00:00:00 +0000</pubDate>
+
+ <guid>/blog/implementing-stable-api-for-apache-airflow/</guid>
+ <description>
+
+
+
+
+<p>My <a href="https://outreachy.org" target="_blank">Outreachy internship</a> is coming to its ends which is also the best time to look back and
+reflect on the progress so far.</p>
+
+<p>The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post,
+I will be sharing my progress so far.</p>
+
+<p>We started a bit late implementing the REST API because it took time for the OpenAPI 3.0
+specification we were to use for the project to be merged. Thanks to <a href="https://github.com/mik-laj" target="_blank">Kamil</a>,
+who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I
+implemented and the challenges I encountered, including how I overcame them.</p>
+
+<h3 id="implementing-the-read-only-connection-endpoints">Implementing The Read-Only Connection Endpoints</h3>
+
+<p>The <a href="https://github.com/apache/airflow/pull/9095" target="_blank">read-only connection endpoints</a> were the first endpoint I implemented. Looking back,
+I can see how much I have improved.</p>
+
+<p>I started by implementing the database schema for the Connection table using <a href="https://marshmallow.readthedocs.io/en/2.x-line/" target="_blank">Marshmallow 2</a>.
+We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder
+is deeply integrated to Apache Airflow. This meant I had to unlearn Marshmallow 3 that I had
+ been studying before this realization, but thankfully, <a href="https://marshmallow.readthedocs.io/en/stable/index.html" target="_blank">Marshmallow 3</a> isn&rsquo;t too
+ different, so I was able to start using Marshmallow 2 in no time.</p>
+
+<p>This first PR would have been more difficult than it was unless there had been any reference
+endpoint to look at. <a href="https://github.com/mik-laj" target="_blank">Kamil</a> implemented a <a href="https://github.com/apache/airflow/pull/9045" target="_blank">draft PR</a> in which I took inspiration from.
+Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that
+ I learned using <a href="https://github.com/wolever/parameterized" target="_blank">parameterized</a> in unit tests :D.</p>
+
+<h3 id="implementing-the-read-only-dagruns-endpoints">Implementing The Read-Only DagRuns Endpoints</h3>
+
+<p>This <a href="https://github.com/apache/airflow/pull/9153" target="_blank">endpoint</a> came with its many challenges, especially on filtering with <code>datetimes</code>.
+This was because the <code>connexion</code> library we were using to build the REST API was not validating
+date-time format in OpenAPI 3.0 specification, what I eventually found out, was intentional.
+Connexion dropped <code>strict-rfc3339</code> because of the later license which is not compatible with
+Apache 2.0 license.</p>
+
+<p>I implemented a workaround on this, by defining a function called <code>conn_parse_datetime</code> in the
+API utils module. This was later refactored and thankfully, <a href="https://github.com/mik-laj" target="_blank">Kamil</a>
+ implemented a decorator that allowed us to have cleaner code on the views while using this function.</p>
+
+<p>Then we tried using <code>rfc3339-validator</code> whose license is compatible with Apache 2.0 licence but
+ later discarded this because with our custom date parser we were able to use duration and
+ not just date times.</p>
+
+<h3 id="other-endpoints">Other Endpoints</h3>
+
+<p>I implemented some different other endpoints. One peculiar issue I faced was because of Marshmallow 2
+not giving error when extra fields are in the request body. I implemented a <code>validate_unknown</code>
+method on the schema to handle this. Thankfully, Flask-AppBuilder updated to using Marshmallow 3,
+we quickly updated Flask-AppBuilder in Apache Airflow and started using Marshmallow 3 too.</p>
+
+<p>Here are some PRs I contributed that are related to the REST API:</p>
+
+<ol>
+<li><p><a href="https://github.com/apache/airflow/pull/9227" target="_blank">Add event log endpoints</a>
+The event log would help users get information on operations performed at the UI</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9266" target="_blank">Add CRUD endpoints for connection</a>
+This PR performs DELETE, PATCH and POST operations on <code>Connection</code></p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9331" target="_blank">Add log endpoint</a>
+This PR enables users to get Task Instances log entries</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9431" target="_blank">Move limit &amp; offset to kwargs in views plus work on a configurable maximum limit</a>
+This helped us in having a neat code on the views and added configurable maximum limit on query results.</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9648" target="_blank">Update FlaskAppBuilder to v3</a>
+This enabled Airflow to start using v3 of Flask App Builder and also made it possible for the API to use
+ a modern database serializer/deserializer</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9771" target="_blank">Add migration guide from the experimental REST API to the stable REST API</a>
+This would enable users to start using the stable REST API in less time.</p></li>
+</ol>
+
+<h3 id="follow-ups">Follow-Ups</h3>
+
+<p>There is still lots of works to be done on the REST API including writing helpful documentation.
+I still follow up on these and hopefully, we will complete the REST API before the internship ends.</p>
+
+<p>I am very grateful to my mentors, <a href="https://github.com/potiuk" target="_blank">Jarek</a> and <a href="https://github.com/kaxil" target="_blank">Kaxil</a> for their
+patience with me and for surviving my never-ending questions. <a href="https://github.com/mik-laj" target="_blank">Kamil</a> and <a href="https://github.com/turbaszek" target="_blank">Tomek</a>
+have been very supportive and I appreciate them for their support and amazing code reviews.</p>
+
+<p>Thanks to <a href="https://github.com/leahecole" target="_blank">Leah E. Cole</a> and <a href="https://github.com/mschickensoup" target="_blank">Karolina Rosół</a>, for their
+wonderful reviews. I&rsquo;m grateful.</p>
+
+<p>Thanks for reading!</p>
+
+ </description>
+ </item>
+
+ <item>
<title>Blog: Apache Airflow 1.10.10</title>
<link>/blog/airflow-1.10.10/</link>
<pubDate>Thu, 09 Apr 2020 00:00:00 +0000</pubDate>
diff --git a/blog/its-a-breeze-to-develop-apache-airflow/index.html b/blog/its-a-breeze-to-develop-apache-airflow/index.html
index 574c47e..f618b0a 100644
--- a/blog/its-a-breeze-to-develop-apache-airflow/index.html
+++ b/blog/its-a-breeze-to-develop-apache-airflow/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-11-22T00:00:00+00:00" />
-<meta property="article:modified_time" content="2019-11-26T12:05:52+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="It's a "Breeze" to develop Apache Airflow">
<meta itemprop="description" content="A Principal Software Engineer's journey to developer productivity. Learn how Jarek and his team speeded up and simplified Airflow development for the community.">
<meta itemprop="datePublished" content="2019-11-22T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2019-11-26T12:05:52+01:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="127">
diff --git a/blog/tags/community/index.html b/blog/tags/community/index.html
index ad98635..82504f5 100644
--- a/blog/tags/community/index.html
+++ b/blog/tags/community/index.html
@@ -30,7 +30,7 @@
<meta name="msapplication-TileImage" content="/favicons/ms-icon-144x144.png">
<meta name="theme-color" content="#ffffff">
-<title>Community | Apache Airflow</title><meta property="og:title" content="Community" />
+<title>community | Apache Airflow</title><meta property="og:title" content="community" />
<meta property="og:description" content="Platform created by the community to programmatically author, schedule and monitor workflows." />
<meta property="og:type" content="website" />
<meta property="og:url" content="/blog/tags/community/" />
@@ -38,13 +38,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="og:updated_time" content="2019-12-11T00:00:00+00:00" /><meta property="og:site_name" content="Apache Airflow" />
-<meta itemprop="name" content="Community">
+<meta itemprop="name" content="community">
<meta itemprop="description" content="Platform created by the community to programmatically author, schedule and monitor workflows.">
<meta name="twitter:card" content="summary_large_image"/>
<meta name="twitter:image" content="/images/feature-image.png"/>
-<meta name="twitter:title" content="Community"/>
+<meta name="twitter:title" content="community"/>
<meta name="twitter:description" content="Platform created by the community to programmatically author, schedule and monitor workflows."/>
@@ -407,6 +407,11 @@ if (!doNotTrack) {
<a class="tag "
+ href="/blog/tags/rest-api/">Rest api</a>
+
+
+
+ <a class="tag "
href="/blog/tags/survey/">Survey</a>
diff --git a/blog/tags/community/index.xml b/blog/tags/community/index.xml
index e451c3e..39dc48f 100644
--- a/blog/tags/community/index.xml
+++ b/blog/tags/community/index.xml
@@ -1,8 +1,8 @@
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
- <title>Apache Airflow – Community</title>
+ <title>Apache Airflow – community</title>
<link>/blog/tags/community/</link>
- <description>Recent content in Community on Apache Airflow</description>
+ <description>Recent content in community on Apache Airflow</description>
<generator>Hugo -- gohugo.io</generator>
<lastBuildDate>Wed, 11 Dec 2019 00:00:00 +0000</lastBuildDate>
diff --git a/blog/tags/development/index.html b/blog/tags/development/index.html
index 259e0bd..4baef4b 100644
--- a/blog/tags/development/index.html
+++ b/blog/tags/development/index.html
@@ -407,6 +407,11 @@ if (!doNotTrack) {
<a class="tag "
+ href="/blog/tags/rest-api/">Rest api</a>
+
+
+
+ <a class="tag "
href="/blog/tags/survey/">Survey</a>
diff --git a/blog/tags/documentation/index.html b/blog/tags/documentation/index.html
index 6f1e41c..9c124a9 100644
--- a/blog/tags/documentation/index.html
+++ b/blog/tags/documentation/index.html
@@ -407,6 +407,11 @@ if (!doNotTrack) {
<a class="tag "
+ href="/blog/tags/rest-api/">Rest api</a>
+
+
+
+ <a class="tag "
href="/blog/tags/survey/">Survey</a>
diff --git a/blog/tags/release/index.html b/blog/tags/release/index.html
index f7f2c69..3497206 100644
--- a/blog/tags/release/index.html
+++ b/blog/tags/release/index.html
@@ -407,6 +407,11 @@ if (!doNotTrack) {
<a class="tag "
+ href="/blog/tags/rest-api/">Rest api</a>
+
+
+
+ <a class="tag "
href="/blog/tags/survey/">Survey</a>
diff --git a/blog/tags/documentation/index.html b/blog/tags/rest-api/index.html
similarity index 98%
copy from blog/tags/documentation/index.html
copy to blog/tags/rest-api/index.html
index 6f1e41c..e73181b 100644
--- a/blog/tags/documentation/index.html
+++ b/blog/tags/rest-api/index.html
@@ -9,7 +9,7 @@
<META NAME="ROBOTS" CONTENT="INDEX, FOLLOW">
-<link rel="alternate" type="application/rss+xml" href="/blog/tags/documentation/index.xml">
+<link rel="alternate" type="application/rss+xml" href="/blog/tags/rest-api/index.xml">
<link rel="apple-touch-icon" sizes="57x57" href="/favicons/apple-icon-57x57.png">
@@ -30,21 +30,21 @@
<meta name="msapplication-TileImage" content="/favicons/ms-icon-144x144.png">
<meta name="theme-color" content="#ffffff">
-<title>Documentation | Apache Airflow</title><meta property="og:title" content="Documentation" />
+<title>REST API | Apache Airflow</title><meta property="og:title" content="REST API" />
<meta property="og:description" content="Platform created by the community to programmatically author, schedule and monitor workflows." />
<meta property="og:type" content="website" />
-<meta property="og:url" content="/blog/tags/documentation/" />
+<meta property="og:url" content="/blog/tags/rest-api/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="og:updated_time" content="2019-12-20T00:00:00+00:00" /><meta property="og:site_name" content="Apache Airflow" />
-<meta itemprop="name" content="Documentation">
+<meta property="og:updated_time" content="2020-07-19T00:00:00+00:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta itemprop="name" content="REST API">
<meta itemprop="description" content="Platform created by the community to programmatically author, schedule and monitor workflows.">
<meta name="twitter:card" content="summary_large_image"/>
<meta name="twitter:image" content="/images/feature-image.png"/>
-<meta name="twitter:title" content="Documentation"/>
+<meta name="twitter:title" content="REST API"/>
<meta name="twitter:description" content="Platform created by the community to programmatically author, schedule and monitor workflows."/>
@@ -396,7 +396,7 @@ if (!doNotTrack) {
- <a class="tag active"
+ <a class="tag "
href="/blog/tags/documentation/">Documentation</a>
@@ -406,6 +406,11 @@ if (!doNotTrack) {
+ <a class="tag active"
+ href="/blog/tags/rest-api/">Rest api</a>
+
+
+
<a class="tag "
href="/blog/tags/survey/">Survey</a>
@@ -432,17 +437,15 @@ if (!doNotTrack) {
<div class="tags-container">
- <a class="tag" href="/blog/tags/documentation/">Documentation</a>
-
</div>
- <span class="bodytext__medium--brownish-grey box-event__blogpost--date">Fri, Dec 20, 2019</span>
+ <span class="bodytext__medium--brownish-grey box-event__blogpost--date">Sun, Jul 19, 2020</span>
</div>
- <p class="box-event__blogpost--header">Experience in Google Season of Docs 2019 with Apache Airflow</p>
- <p class="box-event__blogpost--author">Kartik Khare</p>
- <p class="box-event__blogpost--description"></p>
+ <p class="box-event__blogpost--header">Implementing Stable API for Apache Airflow</p>
+ <p class="box-event__blogpost--author">Ephraim Anierobi</p>
+ <p class="box-event__blogpost--description">An Outreachy intern's progress report on contributing to Apache Airflow REST API.</p>
<div class="mt-auto">
- <a href="/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/">
+ <a href="/blog/implementing-stable-api-for-apache-airflow/">
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read more</button>
@@ -463,7 +466,7 @@ if (!doNotTrack) {
<div class="base-layout--button">
- <a href=https://github.com/apache/airflow-site/edit/master/landing-pages/site/content/en/tags/documentation>
+ <a href=https://github.com/apache/airflow-site/edit/master/landing-pages/site/content/en/tags/rest-api>
diff --git a/blog/tags/rest-api/index.xml b/blog/tags/rest-api/index.xml
new file mode 100644
index 0000000..170c77e
--- /dev/null
+++ b/blog/tags/rest-api/index.xml
@@ -0,0 +1,119 @@
+<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
+ <channel>
+ <title>Apache Airflow – REST API</title>
+ <link>/blog/tags/rest-api/</link>
+ <description>Recent content in REST API on Apache Airflow</description>
+ <generator>Hugo -- gohugo.io</generator>
+ <lastBuildDate>Sun, 19 Jul 2020 00:00:00 +0000</lastBuildDate>
+
+ <atom:link href="/blog/tags/rest-api/index.xml" rel="self" type="application/rss+xml" />
+
+
+
+
+
+
+
+ <item>
+ <title>Blog: Implementing Stable API for Apache Airflow</title>
+ <link>/blog/implementing-stable-api-for-apache-airflow/</link>
+ <pubDate>Sun, 19 Jul 2020 00:00:00 +0000</pubDate>
+
+ <guid>/blog/implementing-stable-api-for-apache-airflow/</guid>
+ <description>
+
+
+
+
+<p>My <a href="https://outreachy.org" target="_blank">Outreachy internship</a> is coming to its ends which is also the best time to look back and
+reflect on the progress so far.</p>
+
+<p>The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post,
+I will be sharing my progress so far.</p>
+
+<p>We started a bit late implementing the REST API because it took time for the OpenAPI 3.0
+specification we were to use for the project to be merged. Thanks to <a href="https://github.com/mik-laj" target="_blank">Kamil</a>,
+who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I
+implemented and the challenges I encountered, including how I overcame them.</p>
+
+<h3 id="implementing-the-read-only-connection-endpoints">Implementing The Read-Only Connection Endpoints</h3>
+
+<p>The <a href="https://github.com/apache/airflow/pull/9095" target="_blank">read-only connection endpoints</a> were the first endpoint I implemented. Looking back,
+I can see how much I have improved.</p>
+
+<p>I started by implementing the database schema for the Connection table using <a href="https://marshmallow.readthedocs.io/en/2.x-line/" target="_blank">Marshmallow 2</a>.
+We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder
+is deeply integrated to Apache Airflow. This meant I had to unlearn Marshmallow 3 that I had
+ been studying before this realization, but thankfully, <a href="https://marshmallow.readthedocs.io/en/stable/index.html" target="_blank">Marshmallow 3</a> isn&rsquo;t too
+ different, so I was able to start using Marshmallow 2 in no time.</p>
+
+<p>This first PR would have been more difficult than it was unless there had been any reference
+endpoint to look at. <a href="https://github.com/mik-laj" target="_blank">Kamil</a> implemented a <a href="https://github.com/apache/airflow/pull/9045" target="_blank">draft PR</a> in which I took inspiration from.
+Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that
+ I learned using <a href="https://github.com/wolever/parameterized" target="_blank">parameterized</a> in unit tests :D.</p>
+
+<h3 id="implementing-the-read-only-dagruns-endpoints">Implementing The Read-Only DagRuns Endpoints</h3>
+
+<p>This <a href="https://github.com/apache/airflow/pull/9153" target="_blank">endpoint</a> came with its many challenges, especially on filtering with <code>datetimes</code>.
+This was because the <code>connexion</code> library we were using to build the REST API was not validating
+date-time format in OpenAPI 3.0 specification, what I eventually found out, was intentional.
+Connexion dropped <code>strict-rfc3339</code> because of the later license which is not compatible with
+Apache 2.0 license.</p>
+
+<p>I implemented a workaround on this, by defining a function called <code>conn_parse_datetime</code> in the
+API utils module. This was later refactored and thankfully, <a href="https://github.com/mik-laj" target="_blank">Kamil</a>
+ implemented a decorator that allowed us to have cleaner code on the views while using this function.</p>
+
+<p>Then we tried using <code>rfc3339-validator</code> whose license is compatible with Apache 2.0 licence but
+ later discarded this because with our custom date parser we were able to use duration and
+ not just date times.</p>
+
+<h3 id="other-endpoints">Other Endpoints</h3>
+
+<p>I implemented some different other endpoints. One peculiar issue I faced was because of Marshmallow 2
+not giving error when extra fields are in the request body. I implemented a <code>validate_unknown</code>
+method on the schema to handle this. Thankfully, Flask-AppBuilder updated to using Marshmallow 3,
+we quickly updated Flask-AppBuilder in Apache Airflow and started using Marshmallow 3 too.</p>
+
+<p>Here are some PRs I contributed that are related to the REST API:</p>
+
+<ol>
+<li><p><a href="https://github.com/apache/airflow/pull/9227" target="_blank">Add event log endpoints</a>
+The event log would help users get information on operations performed at the UI</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9266" target="_blank">Add CRUD endpoints for connection</a>
+This PR performs DELETE, PATCH and POST operations on <code>Connection</code></p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9331" target="_blank">Add log endpoint</a>
+This PR enables users to get Task Instances log entries</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9431" target="_blank">Move limit &amp; offset to kwargs in views plus work on a configurable maximum limit</a>
+This helped us in having a neat code on the views and added configurable maximum limit on query results.</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9648" target="_blank">Update FlaskAppBuilder to v3</a>
+This enabled Airflow to start using v3 of Flask App Builder and also made it possible for the API to use
+ a modern database serializer/deserializer</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9771" target="_blank">Add migration guide from the experimental REST API to the stable REST API</a>
+This would enable users to start using the stable REST API in less time.</p></li>
+</ol>
+
+<h3 id="follow-ups">Follow-Ups</h3>
+
+<p>There is still lots of works to be done on the REST API including writing helpful documentation.
+I still follow up on these and hopefully, we will complete the REST API before the internship ends.</p>
+
+<p>I am very grateful to my mentors, <a href="https://github.com/potiuk" target="_blank">Jarek</a> and <a href="https://github.com/kaxil" target="_blank">Kaxil</a> for their
+patience with me and for surviving my never-ending questions. <a href="https://github.com/mik-laj" target="_blank">Kamil</a> and <a href="https://github.com/turbaszek" target="_blank">Tomek</a>
+have been very supportive and I appreciate them for their support and amazing code reviews.</p>
+
+<p>Thanks to <a href="https://github.com/leahecole" target="_blank">Leah E. Cole</a> and <a href="https://github.com/mschickensoup" target="_blank">Karolina Rosół</a>, for their
+wonderful reviews. I&rsquo;m grateful.</p>
+
+<p>Thanks for reading!</p>
+
+ </description>
+ </item>
+
+ </channel>
+</rss>
diff --git a/blog/tags/survey/index.html b/blog/tags/survey/index.html
index 9b9c71b..03010e8 100644
--- a/blog/tags/survey/index.html
+++ b/blog/tags/survey/index.html
@@ -406,6 +406,11 @@ if (!doNotTrack) {
+ <a class="tag "
+ href="/blog/tags/rest-api/">Rest api</a>
+
+
+
<a class="tag active"
href="/blog/tags/survey/">Survey</a>
diff --git a/blog/tags/users/index.html b/blog/tags/users/index.html
index e900fb2..ec352a9 100644
--- a/blog/tags/users/index.html
+++ b/blog/tags/users/index.html
@@ -407,6 +407,11 @@ if (!doNotTrack) {
<a class="tag "
+ href="/blog/tags/rest-api/">Rest api</a>
+
+
+
+ <a class="tag "
href="/blog/tags/survey/">Survey</a>
diff --git a/community/index.html b/community/index.html
index be77c05..d0058d8 100644
--- a/community/index.html
+++ b/community/index.html
@@ -1241,16 +1241,65 @@ if (!doNotTrack) {
</div>
+ <div class="list-item">
+
+
+
+<div class="card">
+ <div class="box-event">
+ <img src="https://github.com/dimberman.png" alt="Committer Daniel Imberman" class="avatar"/>
+ <p class="box-event__committer--nick">@dimberman</p>
+ <p class="bodytext__medium--brownish-grey">Daniel Imberman</p>
+ <div class="box-event__committer--social-media-container">
+
+
+
+ <a href="https://github.com/dimberman" class="box-event__committer--social-media-icon">
+ <svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
+ <path d="M15.379 0a15.381 15.381 0 00-4.86 29.974c.769.141 1.05-.334 1.05-.741 0-.365-.013-1.332-.021-2.616-4.278.929-5.181-2.062-5.181-2.062a4.071 4.071 0 00-1.708-2.25c-1.4-.954.106-.935.106-.935a3.231 3.231 0 012.356 1.585 3.274 3.274 0 004.476 1.278 3.287 3.287 0 01.976-2.056c-3.415-.388-7.005-1.707-7.005-7.6a5.947 5.947 0 011.583-4.127 5.53 5.53 0 01.151-4.07s1.291-.413 4.229 1.577a14.577 14.577 0 017.7 0c2.936-1.99 4.225-1.577 4.225-1.577a5.523 5.523 0 01.153 4.07 5.937 5.937 0 [...]
+ data-name="Path 224" fill="#51504f" fill-rule="evenodd"/>
+</svg>
+
+ </a>
+
+
+
+
+ </div>
+ </div>
</div>
-<div class="show-more-button">
+
+ </div>
+ <div class="list-item">
+
-<button id="show-more-pmcs" class="btn-filled bodytext__medium--white ">Show more</button>
+<div class="card">
+ <div class="box-event">
+ <img src="https://github.com/mik-laj.png" alt="Committer Kamil Breguła" class="avatar"/>
+ <p class="box-event__committer--nick">@mik-laj</p>
+ <p class="bodytext__medium--brownish-grey">Kamil Breguła</p>
+ <div class="box-event__committer--social-media-container">
+
+
+
+ <a href="https://github.com/mik-laj" class="box-event__committer--social-media-icon">
+ <svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
+ <path d="M15.379 0a15.381 15.381 0 00-4.86 29.974c.769.141 1.05-.334 1.05-.741 0-.365-.013-1.332-.021-2.616-4.278.929-5.181-2.062-5.181-2.062a4.071 4.071 0 00-1.708-2.25c-1.4-.954.106-.935.106-.935a3.231 3.231 0 012.356 1.585 3.274 3.274 0 004.476 1.278 3.287 3.287 0 01.976-2.056c-3.415-.388-7.005-1.707-7.005-7.6a5.947 5.947 0 011.583-4.127 5.53 5.53 0 01.151-4.07s1.291-.413 4.229 1.577a14.577 14.577 0 017.7 0c2.936-1.99 4.225-1.577 4.225-1.577a5.523 5.523 0 01.153 4.07 5.937 5.937 0 [...]
+ data-name="Path 224" fill="#51504f" fill-rule="evenodd"/>
+</svg>
+
+ </a>
+
+
+
+
+ </div>
+ </div>
</div>
-<h5 class="community--committers-header">Committers</h5>
-<div id="committers-container" class="list-items">
+ </div>
<div class="list-item">
@@ -1258,14 +1307,14 @@ if (!doNotTrack) {
<div class="card">
<div class="box-event">
- <img src="https://github.com/BasPH.png" alt="Committer Bas Harenslak" class="avatar"/>
- <p class="box-event__committer--nick">@BasPH</p>
- <p class="bodytext__medium--brownish-grey">Bas Harenslak</p>
+ <img src="https://github.com/turbaszek.png" alt="Committer Tomek Urbaszek" class="avatar"/>
+ <p class="box-event__committer--nick">@turbaszek</p>
+ <p class="bodytext__medium--brownish-grey">Tomek Urbaszek</p>
<div class="box-event__committer--social-media-container">
- <a href="https://github.com/BasPH" class="box-event__committer--social-media-icon">
+ <a href="https://github.com/turbaszek" class="box-event__committer--social-media-icon">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
<path d="M15.379 0a15.381 15.381 0 00-4.86 29.974c.769.141 1.05-.334 1.05-.741 0-.365-.013-1.332-.021-2.616-4.278.929-5.181-2.062-5.181-2.062a4.071 4.071 0 00-1.708-2.25c-1.4-.954.106-.935.106-.935a3.231 3.231 0 012.356 1.585 3.274 3.274 0 004.476 1.278 3.287 3.287 0 01.976-2.056c-3.415-.388-7.005-1.707-7.005-7.6a5.947 5.947 0 011.583-4.127 5.53 5.53 0 01.151-4.07s1.291-.413 4.229 1.577a14.577 14.577 0 017.7 0c2.936-1.99 4.225-1.577 4.225-1.577a5.523 5.523 0 01.153 4.07 5.937 5.937 0 [...]
data-name="Path 224" fill="#51504f" fill-rule="evenodd"/>
@@ -1282,20 +1331,31 @@ if (!doNotTrack) {
</div>
+</div>
+<div class="show-more-button">
+
+
+<button id="show-more-pmcs" class="btn-filled bodytext__medium--white ">Show more</button>
+
+</div>
+
+<h5 class="community--committers-header">Committers</h5>
+<div id="committers-container" class="list-items">
+
<div class="list-item">
<div class="card">
<div class="box-event">
- <img src="https://github.com/milton0825.png" alt="Committer Chao-Han Tsai" class="avatar"/>
- <p class="box-event__committer--nick">@milton0825</p>
- <p class="bodytext__medium--brownish-grey">Chao-Han Tsai</p>
+ <img src="https://github.com/BasPH.png" alt="Committer Bas Harenslak" class="avatar"/>
+ <p class="box-event__committer--nick">@BasPH</p>
+ <p class="bodytext__medium--brownish-grey">Bas Harenslak</p>
<div class="box-event__committer--social-media-container">
- <a href="https://github.com/milton0825" class="box-event__committer--social-media-icon">
+ <a href="https://github.com/BasPH" class="box-event__committer--social-media-icon">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
<path d="M15.379 0a15.381 15.381 0 00-4.86 29.974c.769.141 1.05-.334 1.05-.741 0-.365-.013-1.332-.021-2.616-4.278.929-5.181-2.062-5.181-2.062a4.071 4.071 0 00-1.708-2.25c-1.4-.954.106-.935.106-.935a3.231 3.231 0 012.356 1.585 3.274 3.274 0 004.476 1.278 3.287 3.287 0 01.976-2.056c-3.415-.388-7.005-1.707-7.005-7.6a5.947 5.947 0 011.583-4.127 5.53 5.53 0 01.151-4.07s1.291-.413 4.229 1.577a14.577 14.577 0 017.7 0c2.936-1.99 4.225-1.577 4.225-1.577a5.523 5.523 0 01.153 4.07 5.937 5.937 0 [...]
data-name="Path 224" fill="#51504f" fill-rule="evenodd"/>
@@ -1318,14 +1378,14 @@ if (!doNotTrack) {
<div class="card">
<div class="box-event">
- <img src="https://github.com/dimberman.png" alt="Committer Daniel Imberman" class="avatar"/>
- <p class="box-event__committer--nick">@dimberman</p>
- <p class="bodytext__medium--brownish-grey">Daniel Imberman</p>
+ <img src="https://github.com/milton0825.png" alt="Committer Chao-Han Tsai" class="avatar"/>
+ <p class="box-event__committer--nick">@milton0825</p>
+ <p class="bodytext__medium--brownish-grey">Chao-Han Tsai</p>
<div class="box-event__committer--social-media-container">
- <a href="https://github.com/dimberman" class="box-event__committer--social-media-icon">
+ <a href="https://github.com/milton0825" class="box-event__committer--social-media-icon">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
<path d="M15.379 0a15.381 15.381 0 00-4.86 29.974c.769.141 1.05-.334 1.05-.741 0-.365-.013-1.332-.021-2.616-4.278.929-5.181-2.062-5.181-2.062a4.071 4.071 0 00-1.708-2.25c-1.4-.954.106-.935.106-.935a3.231 3.231 0 012.356 1.585 3.274 3.274 0 004.476 1.278 3.287 3.287 0 01.976-2.056c-3.415-.388-7.005-1.707-7.005-7.6a5.947 5.947 0 011.583-4.127 5.53 5.53 0 01.151-4.07s1.291-.413 4.229 1.577a14.577 14.577 0 017.7 0c2.936-1.99 4.225-1.577 4.225-1.577a5.523 5.523 0 01.153 4.07 5.937 5.937 0 [...]
data-name="Path 224" fill="#51504f" fill-rule="evenodd"/>
@@ -1438,14 +1498,33 @@ if (!doNotTrack) {
<div class="card">
<div class="box-event">
- <img src="https://github.com/mik-laj.png" alt="Committer Kamil Breguła" class="avatar"/>
- <p class="box-event__committer--nick">@mik-laj</p>
- <p class="bodytext__medium--brownish-grey">Kamil Breguła</p>
+ <img src="/images/committer-placeholder.svg" alt="Committer Patrick Leo Tardif" class="avatar"/>
+ <p class="box-event__committer--nick">@pltardif</p>
+ <p class="bodytext__medium--brownish-grey">Patrick Leo Tardif</p>
+ <div class="box-event__committer--social-media-container">
+
+
+
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item">
+
+
+
+<div class="card">
+ <div class="box-event">
+ <img src="https://github.com/XD-DENG.png" alt="Committer Xiaodong Deng" class="avatar"/>
+ <p class="box-event__committer--nick">@XD-DENG</p>
+ <p class="bodytext__medium--brownish-grey">Xiaodong Deng</p>
<div class="box-event__committer--social-media-container">
- <a href="https://github.com/mik-laj" class="box-event__committer--social-media-icon">
+ <a href="https://github.com/XD-DENG" class="box-event__committer--social-media-icon">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
<path d="M15.379 0a15.381 15.381 0 00-4.86 29.974c.769.141 1.05-.334 1.05-.741 0-.365-.013-1.332-.021-2.616-4.278.929-5.181-2.062-5.181-2.062a4.071 4.071 0 00-1.708-2.25c-1.4-.954.106-.935.106-.935a3.231 3.231 0 012.356 1.585 3.274 3.274 0 004.476 1.278 3.287 3.287 0 01.976-2.056c-3.415-.388-7.005-1.707-7.005-7.6a5.947 5.947 0 011.583-4.127 5.53 5.53 0 01.151-4.07s1.291-.413 4.229 1.577a14.577 14.577 0 017.7 0c2.936-1.99 4.225-1.577 4.225-1.577a5.523 5.523 0 01.153 4.07 5.937 5.937 0 [...]
data-name="Path 224" fill="#51504f" fill-rule="evenodd"/>
@@ -1468,11 +1547,22 @@ if (!doNotTrack) {
<div class="card">
<div class="box-event">
- <img src="/images/committer-placeholder.svg" alt="Committer Patrick Leo Tardif" class="avatar"/>
- <p class="box-event__committer--nick">@pltardif</p>
- <p class="bodytext__medium--brownish-grey">Patrick Leo Tardif</p>
+ <img src="https://github.com/houqp.png" alt="Committer Qingping Hou" class="avatar"/>
+ <p class="box-event__committer--nick">@houqp</p>
+ <p class="bodytext__medium--brownish-grey">Qingping Hou</p>
<div class="box-event__committer--social-media-container">
+
+
+ <a href="https://github.com/houqp" class="box-event__committer--social-media-icon">
+ <svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
+ <path d="M15.379 0a15.381 15.381 0 00-4.86 29.974c.769.141 1.05-.334 1.05-.741 0-.365-.013-1.332-.021-2.616-4.278.929-5.181-2.062-5.181-2.062a4.071 4.071 0 00-1.708-2.25c-1.4-.954.106-.935.106-.935a3.231 3.231 0 012.356 1.585 3.274 3.274 0 004.476 1.278 3.287 3.287 0 01.976-2.056c-3.415-.388-7.005-1.707-7.005-7.6a5.947 5.947 0 011.583-4.127 5.53 5.53 0 01.151-4.07s1.291-.413 4.229 1.577a14.577 14.577 0 017.7 0c2.936-1.99 4.225-1.577 4.225-1.577a5.523 5.523 0 01.153 4.07 5.937 5.937 0 [...]
+ data-name="Path 224" fill="#51504f" fill-rule="evenodd"/>
+</svg>
+
+ </a>
+
+
</div>
@@ -1487,14 +1577,44 @@ if (!doNotTrack) {
<div class="card">
<div class="box-event">
- <img src="https://github.com/XD-DENG.png" alt="Committer Xiaodong Deng" class="avatar"/>
- <p class="box-event__committer--nick">@XD-DENG</p>
- <p class="bodytext__medium--brownish-grey">Xiaodong Deng</p>
+ <img src="https://github.com/ryw.png" alt="Committer Ry Walker" class="avatar"/>
+ <p class="box-event__committer--nick">@ryw</p>
+ <p class="bodytext__medium--brownish-grey">Ry Walker</p>
<div class="box-event__committer--social-media-container">
- <a href="https://github.com/XD-DENG" class="box-event__committer--social-media-icon">
+ <a href="https://github.com/ryw" class="box-event__committer--social-media-icon">
+ <svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
+ <path d="M15.379 0a15.381 15.381 0 00-4.86 29.974c.769.141 1.05-.334 1.05-.741 0-.365-.013-1.332-.021-2.616-4.278.929-5.181-2.062-5.181-2.062a4.071 4.071 0 00-1.708-2.25c-1.4-.954.106-.935.106-.935a3.231 3.231 0 012.356 1.585 3.274 3.274 0 004.476 1.278 3.287 3.287 0 01.976-2.056c-3.415-.388-7.005-1.707-7.005-7.6a5.947 5.947 0 011.583-4.127 5.53 5.53 0 01.151-4.07s1.291-.413 4.229 1.577a14.577 14.577 0 017.7 0c2.936-1.99 4.225-1.577 4.225-1.577a5.523 5.523 0 01.153 4.07 5.937 5.937 0 [...]
+ data-name="Path 224" fill="#51504f" fill-rule="evenodd"/>
+</svg>
+
+ </a>
+
+
+
+
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item">
+
+
+
+<div class="card">
+ <div class="box-event">
+ <img src="https://github.com/leahecole.png" alt="Committer Leah Cole" class="avatar"/>
+ <p class="box-event__committer--nick">@leahecole</p>
+ <p class="bodytext__medium--brownish-grey">Leah Cole</p>
+ <div class="box-event__committer--social-media-container">
+
+
+
+ <a href="https://github.com/leahecole" class="box-event__committer--social-media-icon">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
<path d="M15.379 0a15.381 15.381 0 00-4.86 29.974c.769.141 1.05-.334 1.05-.741 0-.365-.013-1.332-.021-2.616-4.278.929-5.181-2.062-5.181-2.062a4.071 4.071 0 00-1.708-2.25c-1.4-.954.106-.935.106-.935a3.231 3.231 0 012.356 1.585 3.274 3.274 0 004.476 1.278 3.287 3.287 0 01.976-2.056c-3.415-.388-7.005-1.707-7.005-7.6a5.947 5.947 0 011.583-4.127 5.53 5.53 0 01.151-4.07s1.291-.413 4.229 1.577a14.577 14.577 0 017.7 0c2.936-1.99 4.225-1.577 4.225-1.577a5.523 5.523 0 01.153 4.07 5.937 5.937 0 [...]
data-name="Path 224" fill="#51504f" fill-rule="evenodd"/>
diff --git a/index.html b/index.html
index 26d661a..6e277d0 100644
--- a/index.html
+++ b/index.html
@@ -1170,12 +1170,12 @@ if (!doNotTrack) {
<div id="integrations-container" class="list-items">
- <a class="list-item" href="/docs/stable/integration.html#gcp-google-cloud-platform">
+ <a class="list-item" href="/docs/stable/integration.html#software-integrations">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Dataproc</span>
+ <span class="box-event__integration--name">Redis</span>
</div>
</div>
@@ -1183,12 +1183,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#software-integrations">
+ <a class="list-item" href="/docs/stable/integration.html#aws-amazon-web-services">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">SQLite</span>
+ <span class="box-event__integration--name">Amazon Simple Notification Service (SNS)</span>
</div>
</div>
@@ -1196,12 +1196,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#gcp-google-cloud-platform">
+ <a class="list-item" href="/docs/stable/integration.html#aws-amazon-web-services">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Cloud Vision</span>
+ <span class="box-event__integration--name">Amazon EMR</span>
</div>
</div>
@@ -1209,12 +1209,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#gcp-google-cloud-platform">
+ <a class="list-item" href="/docs/stable/integration.html#service-integrations">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Bigtable</span>
+ <span class="box-event__integration--name">Google Spreadsheet</span>
</div>
</div>
@@ -1222,12 +1222,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#protocol-integrations">
+ <a class="list-item" href="/docs/stable/integration.html#service-integrations">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Filesystem</span>
+ <span class="box-event__integration--name">Zendesk</span>
</div>
</div>
@@ -1240,7 +1240,7 @@ if (!doNotTrack) {
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Cloud Storage (GCS)</span>
+ <span class="box-event__integration--name">Cloud Functions</span>
</div>
</div>
@@ -1248,12 +1248,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#service-integrations">
+ <a class="list-item" href="/docs/stable/integration.html#gcp-google-cloud-platform">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Slack</span>
+ <span class="box-event__integration--name">BigQuery Data Transfer Service</span>
</div>
</div>
@@ -1261,12 +1261,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#software-integrations">
+ <a class="list-item" href="/docs/stable/integration.html#gcp-google-cloud-platform">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">MySQL</span>
+ <span class="box-event__integration--name">Cloud Spanner</span>
</div>
</div>
diff --git a/index.xml b/index.xml
index 7f88ca2..655ef10 100644
--- a/index.xml
+++ b/index.xml
@@ -13,6 +13,107 @@
<item>
+ <title>Blog: Implementing Stable API for Apache Airflow</title>
+ <link>/blog/implementing-stable-api-for-apache-airflow/</link>
+ <pubDate>Sun, 19 Jul 2020 00:00:00 +0000</pubDate>
+
+ <guid>/blog/implementing-stable-api-for-apache-airflow/</guid>
+ <description>
+
+
+
+
+<p>My <a href="https://outreachy.org" target="_blank">Outreachy internship</a> is coming to its ends which is also the best time to look back and
+reflect on the progress so far.</p>
+
+<p>The goal of my project is to Extend and Improve the Apache Airflow REST API. In this post,
+I will be sharing my progress so far.</p>
+
+<p>We started a bit late implementing the REST API because it took time for the OpenAPI 3.0
+specification we were to use for the project to be merged. Thanks to <a href="https://github.com/mik-laj" target="_blank">Kamil</a>,
+who paved the way for us to start implementing the REST API endpoints. Below are the endpoints I
+implemented and the challenges I encountered, including how I overcame them.</p>
+
+<h3 id="implementing-the-read-only-connection-endpoints">Implementing The Read-Only Connection Endpoints</h3>
+
+<p>The <a href="https://github.com/apache/airflow/pull/9095" target="_blank">read-only connection endpoints</a> were the first endpoint I implemented. Looking back,
+I can see how much I have improved.</p>
+
+<p>I started by implementing the database schema for the Connection table using <a href="https://marshmallow.readthedocs.io/en/2.x-line/" target="_blank">Marshmallow 2</a>.
+We had to use Marshmallow 2 because Flask-AppBuilder was still using it and Flask-AppBuilder
+is deeply integrated to Apache Airflow. This meant I had to unlearn Marshmallow 3 that I had
+ been studying before this realization, but thankfully, <a href="https://marshmallow.readthedocs.io/en/stable/index.html" target="_blank">Marshmallow 3</a> isn&rsquo;t too
+ different, so I was able to start using Marshmallow 2 in no time.</p>
+
+<p>This first PR would have been more difficult than it was unless there had been any reference
+endpoint to look at. <a href="https://github.com/mik-laj" target="_blank">Kamil</a> implemented a <a href="https://github.com/apache/airflow/pull/9045" target="_blank">draft PR</a> in which I took inspiration from.
+Thanks to this, It was easy for me to write the unit tests. It was also in this endpoint that
+ I learned using <a href="https://github.com/wolever/parameterized" target="_blank">parameterized</a> in unit tests :D.</p>
+
+<h3 id="implementing-the-read-only-dagruns-endpoints">Implementing The Read-Only DagRuns Endpoints</h3>
+
+<p>This <a href="https://github.com/apache/airflow/pull/9153" target="_blank">endpoint</a> came with its many challenges, especially on filtering with <code>datetimes</code>.
+This was because the <code>connexion</code> library we were using to build the REST API was not validating
+date-time format in OpenAPI 3.0 specification, what I eventually found out, was intentional.
+Connexion dropped <code>strict-rfc3339</code> because of the later license which is not compatible with
+Apache 2.0 license.</p>
+
+<p>I implemented a workaround on this, by defining a function called <code>conn_parse_datetime</code> in the
+API utils module. This was later refactored and thankfully, <a href="https://github.com/mik-laj" target="_blank">Kamil</a>
+ implemented a decorator that allowed us to have cleaner code on the views while using this function.</p>
+
+<p>Then we tried using <code>rfc3339-validator</code> whose license is compatible with Apache 2.0 licence but
+ later discarded this because with our custom date parser we were able to use duration and
+ not just date times.</p>
+
+<h3 id="other-endpoints">Other Endpoints</h3>
+
+<p>I implemented some different other endpoints. One peculiar issue I faced was because of Marshmallow 2
+not giving error when extra fields are in the request body. I implemented a <code>validate_unknown</code>
+method on the schema to handle this. Thankfully, Flask-AppBuilder updated to using Marshmallow 3,
+we quickly updated Flask-AppBuilder in Apache Airflow and started using Marshmallow 3 too.</p>
+
+<p>Here are some PRs I contributed that are related to the REST API:</p>
+
+<ol>
+<li><p><a href="https://github.com/apache/airflow/pull/9227" target="_blank">Add event log endpoints</a>
+The event log would help users get information on operations performed at the UI</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9266" target="_blank">Add CRUD endpoints for connection</a>
+This PR performs DELETE, PATCH and POST operations on <code>Connection</code></p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9331" target="_blank">Add log endpoint</a>
+This PR enables users to get Task Instances log entries</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9431" target="_blank">Move limit &amp; offset to kwargs in views plus work on a configurable maximum limit</a>
+This helped us in having a neat code on the views and added configurable maximum limit on query results.</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9648" target="_blank">Update FlaskAppBuilder to v3</a>
+This enabled Airflow to start using v3 of Flask App Builder and also made it possible for the API to use
+ a modern database serializer/deserializer</p></li>
+
+<li><p><a href="https://github.com/apache/airflow/pull/9771" target="_blank">Add migration guide from the experimental REST API to the stable REST API</a>
+This would enable users to start using the stable REST API in less time.</p></li>
+</ol>
+
+<h3 id="follow-ups">Follow-Ups</h3>
+
+<p>There is still lots of works to be done on the REST API including writing helpful documentation.
+I still follow up on these and hopefully, we will complete the REST API before the internship ends.</p>
+
+<p>I am very grateful to my mentors, <a href="https://github.com/potiuk" target="_blank">Jarek</a> and <a href="https://github.com/kaxil" target="_blank">Kaxil</a> for their
+patience with me and for surviving my never-ending questions. <a href="https://github.com/mik-laj" target="_blank">Kamil</a> and <a href="https://github.com/turbaszek" target="_blank">Tomek</a>
+have been very supportive and I appreciate them for their support and amazing code reviews.</p>
+
+<p>Thanks to <a href="https://github.com/leahecole" target="_blank">Leah E. Cole</a> and <a href="https://github.com/mschickensoup" target="_blank">Karolina Rosół</a>, for their
+wonderful reviews. I&rsquo;m grateful.</p>
+
+<p>Thanks for reading!</p>
+
+ </description>
+ </item>
+
+ <item>
<title>Blog: Apache Airflow 1.10.10</title>
<link>/blog/airflow-1.10.10/</link>
<pubDate>Thu, 09 Apr 2020 00:00:00 +0000</pubDate>
diff --git a/search/index.html b/search/index.html
index 2830e36..7b31737 100644
--- a/search/index.html
+++ b/search/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/search/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2019-10-11T18:35:02+02:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Search Results">
<meta itemprop="description" content="">
-<meta itemprop="dateModified" content="2019-10-11T18:35:02+02:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="0">
diff --git a/sitemap.xml b/sitemap.xml
index 92ffea2..3d878d0 100644
--- a/sitemap.xml
+++ b/sitemap.xml
@@ -4,187 +4,197 @@
<url>
<loc>/docs/overview/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/tasks/beds/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/tasks/ponycopters/configuring-ponycopters/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/getting-started/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/examples/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/tasks/ponycopters/launching-ponycopters/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/tutorials/multi-bear/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/tasks/porridge/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/concepts/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/tasks/task/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/tutorials/tutorial2/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/tasks/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/tutorials/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/reference/</loc>
- <lastmod>2019-10-09T17:36:20+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/contribution-guidelines/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/</loc>
- <lastmod>2019-10-30T01:28:49+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
- <loc>/blog/airflow-1.10.10/</loc>
- <lastmod>2020-04-10T18:44:33+01:00</lastmod>
+ <loc>/blog/implementing-stable-api-for-apache-airflow/</loc>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
- <loc>/blog/tags/release/</loc>
- <lastmod>2020-04-10T18:44:33+01:00</lastmod>
+ <loc>/blog/tags/rest-api/</loc>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/tags/</loc>
- <lastmod>2020-04-10T18:44:33+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
+ </url>
+
+ <url>
+ <loc>/blog/airflow-1.10.10/</loc>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
+ </url>
+
+ <url>
+ <loc>/blog/tags/release/</loc>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/airflow-1.10.8-1.10.9/</loc>
- <lastmod>2020-02-23T22:34:43+00:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/tags/documentation/</loc>
- <lastmod>2020-02-08T05:17:09+05:30</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/</loc>
- <lastmod>2020-02-08T05:17:09+05:30</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/airflow-survey/</loc>
- <lastmod>2020-02-19T10:34:11+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/tags/community/</loc>
- <lastmod>2020-02-19T10:34:11+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/announcing-new-website/</loc>
- <lastmod>2020-02-19T10:34:11+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/tags/survey/</loc>
- <lastmod>2020-02-19T10:34:11+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/tags/users/</loc>
- <lastmod>2020-02-19T10:34:11+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/</loc>
- <lastmod>2020-02-11T08:20:10+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/tags/development/</loc>
- <lastmod>2019-12-30T16:35:47+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/documenting-using-local-development-environments/</loc>
- <lastmod>2019-12-30T16:35:47+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/its-a-breeze-to-develop-apache-airflow/</loc>
- <lastmod>2019-11-26T12:05:52+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/getting-started/example-page/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/reference/parameter-reference/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/docs/tasks/ponycopters/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/use-cases/adobe/</loc>
- <lastmod>2019-11-29T14:27:18+11:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/use-cases/big-fish-games/</loc>
- <lastmod>2019-11-15T13:01:44+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/blog/</loc>
- <lastmod>2019-10-30T01:28:49+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
@@ -193,57 +203,57 @@
<url>
<loc>/community/</loc>
- <lastmod>2020-05-01T16:54:57+03:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/use-cases/dish/</loc>
- <lastmod>2019-11-15T13:01:44+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/use-cases/experity/</loc>
- <lastmod>2019-11-15T13:01:44+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/</loc>
- <lastmod>2019-11-18T23:44:20+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/install/</loc>
- <lastmod>2019-10-30T01:28:49+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/meetups/</loc>
- <lastmod>2019-10-30T01:28:49+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/use-cases/onefootball/</loc>
- <lastmod>2019-11-15T13:01:44+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/privacy-notice/</loc>
- <lastmod>2019-11-15T13:03:08+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/roadmap/</loc>
- <lastmod>2019-11-26T15:22:46+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/search/</loc>
- <lastmod>2019-10-11T18:35:02+02:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
<url>
<loc>/use-cases/</loc>
- <lastmod>2019-11-15T13:01:44+01:00</lastmod>
+ <lastmod>2020-08-06T09:52:15+02:00</lastmod>
</url>
</urlset>
\ No newline at end of file
diff --git a/tags/index.html b/tags/index.html
index 3456637..f6d065d 100644
--- a/tags/index.html
+++ b/tags/index.html
@@ -37,7 +37,7 @@
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="og:updated_time" content="2020-04-09T00:00:00+00:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="og:updated_time" content="2020-07-19T00:00:00+00:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Tags">
<meta itemprop="description" content="Platform created by the community to programmatically author, schedule and monitor workflows.">
diff --git a/tags/index.xml b/tags/index.xml
index c86dde0..4488a13 100644
--- a/tags/index.xml
+++ b/tags/index.xml
@@ -4,7 +4,7 @@
<link>/tags/</link>
<description>Recent content in Tags on Apache Airflow</description>
<generator>Hugo -- gohugo.io</generator>
- <lastBuildDate>Thu, 09 Apr 2020 00:00:00 +0000</lastBuildDate>
+ <lastBuildDate>Sun, 19 Jul 2020 00:00:00 +0000</lastBuildDate>
<atom:link href="/tags/index.xml" rel="self" type="application/rss+xml" />
diff --git a/use-cases/adobe/index.html b/use-cases/adobe/index.html
index 1244730..f3e8ed2 100644
--- a/use-cases/adobe/index.html
+++ b/use-cases/adobe/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/adobe/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2019-11-29T14:27:18+11:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Adobe">
<meta itemprop="description" content="What was the problem? Modern big data platforms need sophisticated data pipelines connecting to many backend services enabling complex workflows. These workflows need to be deployed, monitored, and run either on regular schedules or triggered by external events. Adobe Experience Platform component services architected and built an orchestration service to enable their users to author, schedule, and monitor complex hierarchical (including sequential a [...]
-<meta itemprop="dateModified" content="2019-11-29T14:27:18+11:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="251">
diff --git a/use-cases/big-fish-games/index.html b/use-cases/big-fish-games/index.html
index b3b120d..67c2e3f 100644
--- a/use-cases/big-fish-games/index.html
+++ b/use-cases/big-fish-games/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/big-fish-games/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2019-11-15T13:01:44+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Big Fish Games">
<meta itemprop="description" content="What was the problem? The main challenge is the lack of standardized ETL workflow orchestration tools. PowerShell and Python-based ETL frameworks built in-house are currently used for scheduling and running analytical workloads. However, there is no web UI through which we can monitor these workflows and it requires additional effort to maintain this framework. These scheduled jobs based on external dependencies are not well suited to modern Big Data [...]
-<meta itemprop="dateModified" content="2019-11-15T13:01:44+01:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="336">
diff --git a/use-cases/dish/index.html b/use-cases/dish/index.html
index e872549..6db828b 100644
--- a/use-cases/dish/index.html
+++ b/use-cases/dish/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/dish/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2019-11-15T13:01:44+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Dish">
<meta itemprop="description" content="What was the problem? We faced increasing complexity managing lengthy crontabs with scheduling being an issue, this required carefully planning timing due to resource constraints, usage patterns, and especially custom code needed for retry logic. In the last case, having to verify success of previous jobs and/or steps prior to running the next. Furthermore, time to results is important, but we were increasingly relying on buffers for processing, wher [...]
-<meta itemprop="dateModified" content="2019-11-15T13:01:44+01:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="249">
diff --git a/use-cases/experity/index.html b/use-cases/experity/index.html
index 7cf02cf..7b3694f 100644
--- a/use-cases/experity/index.html
+++ b/use-cases/experity/index.html
@@ -36,13 +36,13 @@ How did Apache Airflow help to solve this problem? Ultimately we decided flexibl
<meta property="og:url" content="/use-cases/experity/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2019-11-15T13:01:44+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Experity">
<meta itemprop="description" content="What was the problem? We had to deploy our complex, flagship app to multiple nodes in multiple ways. This required tasks to communicate across Windows nodes and coordinate timing perfectly. We did not want to buy an expensive enterprise scheduling tool and needed ultimate flexibility.
How did Apache Airflow help to solve this problem? Ultimately we decided flexible, multi-node, DAG capable tooling was key and airflow was one of the few tools that fit that bill.">
-<meta itemprop="dateModified" content="2019-11-15T13:01:44+01:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="191">
diff --git a/use-cases/onefootball/index.html b/use-cases/onefootball/index.html
index f128e84..a15298a 100644
--- a/use-cases/onefootball/index.html
+++ b/use-cases/onefootball/index.html
@@ -36,13 +36,13 @@ On top of that, new data tools appear each month: third party data sources, clou
<meta property="og:url" content="/use-cases/onefootball/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2019-11-15T13:01:44+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-06T09:52:15+02:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Onefootball">
<meta itemprop="description" content="What was the problem? With millions of daily active users, managing the complexity of data engineering at Onefootball is a constant challenge. Lengthy crontabs, multiplication of custom API clients, erosion of confidence in the analytics served, increasing heroism (“only one person can solve this issue”). Those are the challenges that most teams face unless they consciously invest in their tools and processes.
On top of that, new data tools appear each month: third party data sources, cloud providers solutions, different storage technologies… Managing all those integrations is costly and brittle, especially for small data engineering teams that are trying to do more with less.">
-<meta itemprop="dateModified" content="2019-11-15T13:01:44+01:00" />
+<meta itemprop="dateModified" content="2020-08-06T09:52:15+02:00" />
<meta itemprop="wordCount" content="294">