You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ma...@apache.org on 2016/06/05 05:23:54 UTC
[04/34] incubator-airflow-site git commit: Initial commit
http://git-wip-us.apache.org/repos/asf/incubator-airflow-site/blob/9e19165c/faq.html
----------------------------------------------------------------------
diff --git a/faq.html b/faq.html
new file mode 100644
index 0000000..d5e001d
--- /dev/null
+++ b/faq.html
@@ -0,0 +1,293 @@
+
+
+<!DOCTYPE html>
+<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
+<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
+<head>
+ <meta charset="utf-8">
+
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <title>FAQ — Airflow Documentation</title>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
+
+
+
+
+
+ <link rel="top" title="Airflow Documentation" href="index.html"/>
+ <link rel="next" title="API Reference" href="code.html"/>
+ <link rel="prev" title="Security" href="security.html"/>
+
+
+ <script src="_static/js/modernizr.min.js"></script>
+
+</head>
+
+<body class="wy-body-for-nav" role="document">
+
+ <div class="wy-grid-for-nav">
+
+
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-scroll">
+ <div class="wy-side-nav-search">
+
+
+
+ <a href="index.html" class="icon icon-home"> Airflow
+
+
+
+ </a>
+
+
+
+
+
+
+
+<div role="search">
+ <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+
+
+ </div>
+
+ <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
+
+
+
+ <ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="project.html">Project</a></li>
+<li class="toctree-l1"><a class="reference internal" href="license.html">License</a></li>
+<li class="toctree-l1"><a class="reference internal" href="start.html">Quick Start</a></li>
+<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial</a></li>
+<li class="toctree-l1"><a class="reference internal" href="configuration.html">Configuration</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ui.html">UI / Screenshots</a></li>
+<li class="toctree-l1"><a class="reference internal" href="concepts.html">Concepts</a></li>
+<li class="toctree-l1"><a class="reference internal" href="profiling.html">Data Profiling</a></li>
+<li class="toctree-l1"><a class="reference internal" href="cli.html">Command Line Interface</a></li>
+<li class="toctree-l1"><a class="reference internal" href="scheduler.html">Scheduling & Triggers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="plugins.html">Plugins</a></li>
+<li class="toctree-l1"><a class="reference internal" href="security.html">Security</a></li>
+<li class="toctree-l1 current"><a class="current reference internal" href="#">FAQ</a></li>
+<li class="toctree-l1"><a class="reference internal" href="code.html">API Reference</a></li>
+</ul>
+
+
+
+ </div>
+ </div>
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
+
+
+ <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">Airflow</a>
+ </nav>
+
+
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+
+
+
+
+
+
+<div role="navigation" aria-label="breadcrumbs navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html">Docs</a> »</li>
+
+ <li>FAQ</li>
+ <li class="wy-breadcrumbs-aside">
+
+
+ <a href="_sources/faq.txt" rel="nofollow"> View page source</a>
+
+
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+ <div itemprop="articleBody">
+
+ <div class="section" id="faq">
+<h1>FAQ<a class="headerlink" href="#faq" title="Permalink to this headline">�</a></h1>
+<p><strong>Why isn’t my task getting scheduled?</strong></p>
+<p>There are very many reasons why your task might not be getting scheduled.
+Here are some of the common causes:</p>
+<ul class="simple">
+<li>Does your script “compile”, can the Airflow engine parse it and find your
+DAG object. To test this, you can run <code class="docutils literal"><span class="pre">airflow</span> <span class="pre">list_dags</span></code> and
+confirm that your DAG shows up in the list. You can also run
+<code class="docutils literal"><span class="pre">airflow</span> <span class="pre">list_tasks</span> <span class="pre">foo_dag_id</span> <span class="pre">--tree</span></code> and confirm that your task
+shows up in the list as expected. If you use the CeleryExecutor, you
+may way to confirm that this works both where the scheduler runs as well
+as where the worker runs.</li>
+<li>Is your <code class="docutils literal"><span class="pre">start_date</span></code> set properly? The Airflow scheduler triggers the
+task soon after the <code class="docutils literal"><span class="pre">start_date</span> <span class="pre">+</span> <span class="pre">scheduler_interval</span></code> is passed.</li>
+<li>Is your <code class="docutils literal"><span class="pre">start_date</span></code> beyond where you can see it in the UI? If you
+set your it to some time say 3 months ago, you won’t be able to see
+it in the main view in the UI, but you should be able to see it in the
+<code class="docutils literal"><span class="pre">Menu</span> <span class="pre">-></span> <span class="pre">Browse</span> <span class="pre">->Task</span> <span class="pre">Instances</span></code>.</li>
+<li>Are the dependencies for the task met. The task instances directly
+upstream from the task need to be in a <code class="docutils literal"><span class="pre">success</span></code> state. Also,
+if you have set <code class="docutils literal"><span class="pre">depends_on_past=True</span></code>, the previous task instance
+needs to have succeeded (except if it is the first run for that task).
+Also, if <code class="docutils literal"><span class="pre">wait_for_downstream=True</span></code>, make sure you understand
+what it means.
+You can view how these properties are set from the <code class="docutils literal"><span class="pre">Task</span> <span class="pre">Details</span></code>
+page for your task.</li>
+<li>Are the DagRuns you need created and active? A DagRun represents a specific
+execution of an entire DAG and has a state (running, success, failed, ...).
+The scheduler creates new DagRun as it moves forward, but never goes back
+in time to create new ones. The scheduler only evaluates <code class="docutils literal"><span class="pre">running</span></code> DagRuns
+to see what task instances it can trigger. Note that clearing tasks
+instances (from the UI or CLI) does set the state of a DagRun back to
+running. You can bulk view the list of DagRuns and alter states by clicking
+on the schedule tag for a DAG.</li>
+<li>Is the <code class="docutils literal"><span class="pre">concurrency</span></code> parameter of your DAG reached? <code class="docutils literal"><span class="pre">concurency</span></code> defines
+how many <code class="docutils literal"><span class="pre">running</span></code> task instances a DAG is allowed to have, beyond which
+point things get queued.</li>
+<li>Is the <code class="docutils literal"><span class="pre">max_active_runs</span></code> parameter of your DAG reached? <code class="docutils literal"><span class="pre">max_active_runs</span></code> defines
+how many <code class="docutils literal"><span class="pre">running</span></code> concurrent instances of a DAG there are allowed to be.</li>
+</ul>
+<p>You may also want to read the Scheduler section of the docs and make
+sure you fully understand how it proceeds.</p>
+<p><strong>How do I trigger tasks based on another task’s failure?</strong></p>
+<p>Check out the <code class="docutils literal"><span class="pre">Trigger</span> <span class="pre">Rule</span></code> section in the Concepts section of the
+documentation</p>
+<p><strong>Why are connection passwords still not encrypted in the metadata db after I installed airflow[crypto]</strong>?</p>
+<ul class="simple">
+<li>Verify that the <code class="docutils literal"><span class="pre">fernet_key</span></code> defined in <code class="docutils literal"><span class="pre">$AIRFLOW_HOME/airflow.cfg</span></code> is a valid Fernet key. It must be a base64-encoded 32-byte key. You need to restart the webserver after you update the key</li>
+<li>For existing connections (the ones that you had defined before installing <code class="docutils literal"><span class="pre">airflow[crypto]</span></code> and creating a Fernet key), you need to open each connection in the connection admin UI, re-type the password, and save it</li>
+</ul>
+<p><strong>What’s the deal with ``start_date``?</strong></p>
+<p><code class="docutils literal"><span class="pre">start_date</span></code> is partly legacy from the pre-DagRun era, but it is still
+relevant in many ways. When creating a new DAG, you probably want to set
+a global <code class="docutils literal"><span class="pre">start_date</span></code> for your tasks using <code class="docutils literal"><span class="pre">default_args</span></code>. The first
+DagRun to be created will be based on the <code class="docutils literal"><span class="pre">min(start_date)</span></code> for all your
+task. From that point on, the scheduler creates new DagRuns based on
+your <code class="docutils literal"><span class="pre">schedule_interval</span></code> and the corresponding task instances run as your
+dependencies are met. When introducing new tasks to your DAG, you need to
+pay special attention to <code class="docutils literal"><span class="pre">start_date</span></code>, and may want to reactivate
+inactive DagRuns to get the new task to get onboarded properly.</p>
+<p>We recommend against using dynamic values as <code class="docutils literal"><span class="pre">start_date</span></code>, especially
+<code class="docutils literal"><span class="pre">datetime.now()</span></code> as it can be quite confusing. The task is triggered
+once the period closes, and in theory an <code class="docutils literal"><span class="pre">@hourly</span></code> DAG would never get to
+an hour after now as <code class="docutils literal"><span class="pre">now()</span></code> moves along.</p>
+<p>We also recommend using rounded <code class="docutils literal"><span class="pre">start_date</span></code> in relation to your
+<code class="docutils literal"><span class="pre">schedule_interval</span></code>. This means an <code class="docutils literal"><span class="pre">@hourly</span></code> would be at <code class="docutils literal"><span class="pre">00:00</span></code>
+minutes:seconds, a <code class="docutils literal"><span class="pre">@daily</span></code> job at midnight, a <code class="docutils literal"><span class="pre">@monthly</span></code> job on the
+first of the month. You can use any sensor or a <code class="docutils literal"><span class="pre">TimeDeltaSensor</span></code> to delay
+the execution of tasks within that period. While <code class="docutils literal"><span class="pre">schedule_interval</span></code>
+does allow specifying a <code class="docutils literal"><span class="pre">datetime.timedelta</span></code>
+object, we recommend using the macros or cron expressions instead, as
+it enforces this idea of rounded schedules.</p>
+<p>When using <code class="docutils literal"><span class="pre">depends_on_past=True</span></code> it’s important to pay special attention
+to <code class="docutils literal"><span class="pre">start_date</span></code> as the past dependency is not enforced only on the specific
+schedule of the <code class="docutils literal"><span class="pre">start_date</span></code> specified for the task. It’ also
+important to watch DagRun activity status in time when introducing
+new <code class="docutils literal"><span class="pre">depends_on_past=True</span></code>, unless you are planning on running a backfill
+for the new task(s).</p>
+<p>Also important to note is that the tasks <code class="docutils literal"><span class="pre">start_date</span></code>, in the context of a
+backfill CLI command, get overridden by the backfill’s command <code class="docutils literal"><span class="pre">start_date</span></code>.
+This allows for a backfill on tasks that have <code class="docutils literal"><span class="pre">depends_on_past=True</span></code> to
+actually start, if it wasn’t the case, the backfill just wouldn’t start.</p>
+</div>
+
+
+ </div>
+ </div>
+ <footer>
+
+ <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
+
+ <a href="code.html" class="btn btn-neutral float-right" title="API Reference" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a>
+
+
+ <a href="security.html" class="btn btn-neutral" title="Security" accesskey="p"><span class="fa fa-arrow-circle-left"></span> Previous</a>
+
+ </div>
+
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>
+ © Copyright 2014, Maxime Beauchemin, Airbnb.
+
+ </p>
+ </div>
+ Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+</footer>
+
+ </div>
+ </div>
+
+ </section>
+
+ </div>
+
+
+
+
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT:'./',
+ VERSION:'',
+ COLLAPSE_INDEX:false,
+ FILE_SUFFIX:'.html',
+ HAS_SOURCE: true
+ };
+ </script>
+ <script type="text/javascript" src="_static/jquery.js"></script>
+ <script type="text/javascript" src="_static/underscore.js"></script>
+ <script type="text/javascript" src="_static/doctools.js"></script>
+
+
+
+
+
+ <script type="text/javascript" src="_static/js/theme.js"></script>
+
+
+
+
+ <script type="text/javascript">
+ jQuery(function () {
+ SphinxRtdTheme.StickyNav.enable();
+ });
+ </script>
+
+
+</body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-airflow-site/blob/9e19165c/genindex.html
----------------------------------------------------------------------
diff --git a/genindex.html b/genindex.html
new file mode 100644
index 0000000..eea3c14
--- /dev/null
+++ b/genindex.html
@@ -0,0 +1,1258 @@
+
+
+
+<!DOCTYPE html>
+<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
+<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
+<head>
+ <meta charset="utf-8">
+
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <title>Index — Airflow Documentation</title>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
+
+
+
+
+
+ <link rel="top" title="Airflow Documentation" href="index.html"/>
+
+
+ <script src="_static/js/modernizr.min.js"></script>
+
+</head>
+
+<body class="wy-body-for-nav" role="document">
+
+ <div class="wy-grid-for-nav">
+
+
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-scroll">
+ <div class="wy-side-nav-search">
+
+
+
+ <a href="index.html" class="icon icon-home"> Airflow
+
+
+
+ </a>
+
+
+
+
+
+
+
+<div role="search">
+ <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+
+
+ </div>
+
+ <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
+
+
+
+ <ul>
+<li class="toctree-l1"><a class="reference internal" href="project.html">Project</a></li>
+<li class="toctree-l1"><a class="reference internal" href="license.html">License</a></li>
+<li class="toctree-l1"><a class="reference internal" href="start.html">Quick Start</a></li>
+<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial</a></li>
+<li class="toctree-l1"><a class="reference internal" href="configuration.html">Configuration</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ui.html">UI / Screenshots</a></li>
+<li class="toctree-l1"><a class="reference internal" href="concepts.html">Concepts</a></li>
+<li class="toctree-l1"><a class="reference internal" href="profiling.html">Data Profiling</a></li>
+<li class="toctree-l1"><a class="reference internal" href="cli.html">Command Line Interface</a></li>
+<li class="toctree-l1"><a class="reference internal" href="scheduler.html">Scheduling & Triggers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="plugins.html">Plugins</a></li>
+<li class="toctree-l1"><a class="reference internal" href="security.html">Security</a></li>
+<li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li>
+<li class="toctree-l1"><a class="reference internal" href="code.html">API Reference</a></li>
+</ul>
+
+
+
+ </div>
+ </div>
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
+
+
+ <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">Airflow</a>
+ </nav>
+
+
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+
+
+
+
+
+
+<div role="navigation" aria-label="breadcrumbs navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html">Docs</a> »</li>
+
+ <li></li>
+ <li class="wy-breadcrumbs-aside">
+
+
+
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+ <div itemprop="articleBody">
+
+
+<h1 id="index">Index</h1>
+
+<div class="genindex-jumpbox">
+ <a href="#A"><strong>A</strong></a>
+ | <a href="#B"><strong>B</strong></a>
+ | <a href="#C"><strong>C</strong></a>
+ | <a href="#D"><strong>D</strong></a>
+ | <a href="#E"><strong>E</strong></a>
+ | <a href="#F"><strong>F</strong></a>
+ | <a href="#G"><strong>G</strong></a>
+ | <a href="#H"><strong>H</strong></a>
+ | <a href="#I"><strong>I</strong></a>
+ | <a href="#K"><strong>K</strong></a>
+ | <a href="#L"><strong>L</strong></a>
+ | <a href="#M"><strong>M</strong></a>
+ | <a href="#O"><strong>O</strong></a>
+ | <a href="#P"><strong>P</strong></a>
+ | <a href="#R"><strong>R</strong></a>
+ | <a href="#S"><strong>S</strong></a>
+ | <a href="#T"><strong>T</strong></a>
+ | <a href="#U"><strong>U</strong></a>
+ | <a href="#V"><strong>V</strong></a>
+ | <a href="#W"><strong>W</strong></a>
+ | <a href="#X"><strong>X</strong></a>
+
+</div>
+<h2 id="A">A</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.DAG.add_task">add_task() (airflow.models.DAG method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.add_tasks">add_tasks() (airflow.models.DAG method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#module-airflow.contrib.executors">airflow.contrib.executors (module)</a>
+ </dt>
+
+
+ <dt><a href="code.html#module-airflow.contrib.hooks">airflow.contrib.hooks (module)</a>
+ </dt>
+
+
+ <dt><a href="code.html#module-airflow.contrib.operators">airflow.contrib.operators (module)</a>
+ </dt>
+
+
+ <dt><a href="code.html#module-airflow.executors">airflow.executors (module)</a>
+ </dt>
+
+
+ <dt><a href="code.html#module-airflow.hooks">airflow.hooks (module)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#module-airflow.macros">airflow.macros (module)</a>
+ </dt>
+
+
+ <dt><a href="code.html#module-airflow.macros.hive">airflow.macros.hive (module)</a>
+ </dt>
+
+
+ <dt><a href="code.html#module-airflow.models">airflow.models (module)</a>
+ </dt>
+
+
+ <dt><a href="code.html#module-airflow.operators">airflow.operators (module)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.are_dependencies_met">are_dependencies_met() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.are_dependents_done">are_dependents_done() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="B">B</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.DagBag.bag_dag">bag_dag() (airflow.models.DagBag method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator">BaseOperator (class in airflow.models)</a>, <a href="code.html#airflow.models.BaseOperator">[1]</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.sensors.BaseSensorOperator">BaseSensorOperator (class in airflow.operators.sensors)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.BashOperator">BashOperator (class in airflow.operators)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.operators.BranchPythonOperator">BranchPythonOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DbApiHook.bulk_dump">bulk_dump() (airflow.hooks.DbApiHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DbApiHook.bulk_load">bulk_load() (airflow.hooks.DbApiHook method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.hooks.MySqlHook.bulk_load">(airflow.hooks.MySqlHook method)</a>
+ </dt>
+
+ </dl></dd>
+ </dl></td>
+</tr></table>
+
+<h2 id="C">C</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.executors.CeleryExecutor">CeleryExecutor (class in airflow.executors)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.check_for_bucket">check_for_bucket() (airflow.hooks.S3Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.check_for_key">check_for_key() (airflow.hooks.S3Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.HiveMetastoreHook.check_for_partition">check_for_partition() (airflow.hooks.HiveMetastoreHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.WebHDFSHook.check_for_path">check_for_path() (airflow.hooks.WebHDFSHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.check_for_prefix">check_for_prefix() (airflow.hooks.S3Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.check_for_wildcard_key">check_for_wildcard_key() (airflow.hooks.S3Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.SSHHook.check_output">check_output() (airflow.contrib.hooks.SSHHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.clear">clear() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.clear_xcom_data">clear_xcom_data() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.cli">cli() (airflow.models.DAG method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.FTPHook.close_conn">close_conn() (airflow.contrib.hooks.FTPHook method)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.macros.hive.closest_ds_partition">closest_ds_partition() (in module airflow.macros.hive)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.CloudantHook">CloudantHook (class in airflow.contrib.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DagBag.collect_dags">collect_dags() (airflow.models.DagBag method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.command">command() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.concurrency_reached">concurrency_reached (airflow.models.DAG attribute)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.Connection">Connection (class in airflow.models)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.SlackAPIOperator.construct_api_call_params">construct_api_call_params() (airflow.operators.SlackAPIOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DruidHook.construct_ingest_query">construct_ingest_query() (airflow.hooks.DruidHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.crawl_for_tasks">crawl_for_tasks() (airflow.models.DAG method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.create_dagrun">create_dagrun() (airflow.models.DAG method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.FTPHook.create_directory">create_directory() (airflow.contrib.hooks.FTPHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.current_state">current_state() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="D">D</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.BaseOperator.dag">dag (airflow.models.BaseOperator attribute)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG">DAG (class in airflow.models)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DagBag">DagBag (class in airflow.models)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DagBag.dagbag_report">dagbag_report() (airflow.models.DagBag method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.CloudantHook.db">db() (airflow.contrib.hooks.CloudantHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DbApiHook">DbApiHook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.FTPHook.delete_directory">delete_directory() (airflow.contrib.hooks.FTPHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.FTPHook.delete_file">delete_file() (airflow.contrib.hooks.FTPHook method)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.contrib.hooks.FTPHook.describe_directory">describe_directory() (airflow.contrib.hooks.FTPHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.detect_downstream_cycle">detect_downstream_cycle() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.docker_operator.DockerOperator">DockerOperator (class in airflow.operators.docker_operator)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.downstream_list">downstream_list (airflow.models.BaseOperator attribute)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DruidHook">DruidHook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.macros.ds_add">ds_add() (in module airflow.macros)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.macros.ds_format">ds_format() (in module airflow.macros)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.DummyOperator">DummyOperator (class in airflow.operators)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="E">E</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.operators.EmailOperator">EmailOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.error">error() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.evaluate_trigger_rule">evaluate_trigger_rule() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.BaseOperator.execute">execute() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.operators.BashOperator.execute">(airflow.operators.BashOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.SlackAPIOperator.execute">(airflow.operators.SlackAPIOperator method)</a>
+ </dt>
+
+ </dl></dd>
+
+ <dt><a href="code.html#airflow.operators.ExternalTaskSensor">ExternalTaskSensor (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.Connection.extra_dejson">extra_dejson (airflow.models.Connection attribute)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="F">F</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.DAG.filepath">filepath (airflow.models.DAG attribute)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.folder">folder (airflow.models.DAG attribute)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.contrib.hooks.FTPHook">FTPHook (class in airflow.contrib.hooks)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="G">G</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.operators.GenericTransfer">GenericTransfer (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.get_bucket">get_bucket() (airflow.hooks.S3Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.FTPHook.get_conn">get_conn() (airflow.contrib.hooks.FTPHook method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.contrib.hooks.VerticaHook.get_conn">(airflow.contrib.hooks.VerticaHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DbApiHook.get_conn">(airflow.hooks.DbApiHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DruidHook.get_conn">(airflow.hooks.DruidHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.HttpHook.get_conn">(airflow.hooks.HttpHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.MsSqlHook.get_conn">(airflow.hooks.MsSqlHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.MySqlHook.get_conn">(airflow.hooks.MySqlHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.PrestoHook.get_conn">(airflow.hooks.PrestoHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.get_conn">(airflow.hooks.S3Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.SqliteHook.get_conn">(airflow.hooks.SqliteHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.WebHDFSHook.get_conn">(airflow.hooks.WebHDFSHook method)</a>
+ </dt>
+
+ </dl></dd>
+
+ <dt><a href="code.html#airflow.hooks.DbApiHook.get_cursor">get_cursor() (airflow.hooks.DbApiHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DagBag.get_dag">get_dag() (airflow.models.DagBag method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.HiveMetastoreHook.get_databases">get_databases() (airflow.hooks.HiveMetastoreHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.get_direct_relatives">get_direct_relatives() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DbApiHook.get_first">get_first() (airflow.hooks.DbApiHook method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.hooks.PrestoHook.get_first">(airflow.hooks.PrestoHook method)</a>
+ </dt>
+
+ </dl></dd>
+
+ <dt><a href="code.html#airflow.models.BaseOperator.get_flat_relatives">get_flat_relatives() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.get_key">get_key() (airflow.hooks.S3Hook method)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.hooks.HiveMetastoreHook.get_metastore_client">get_metastore_client() (airflow.hooks.HiveMetastoreHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DbApiHook.get_pandas_df">get_pandas_df() (airflow.hooks.DbApiHook method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.hooks.HiveServer2Hook.get_pandas_df">(airflow.hooks.HiveServer2Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.PrestoHook.get_pandas_df">(airflow.hooks.PrestoHook method)</a>
+ </dt>
+
+ </dl></dd>
+
+ <dt><a href="code.html#airflow.hooks.HiveMetastoreHook.get_partitions">get_partitions() (airflow.hooks.HiveMetastoreHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DbApiHook.get_records">get_records() (airflow.hooks.DbApiHook method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.hooks.HiveServer2Hook.get_records">(airflow.hooks.HiveServer2Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.PrestoHook.get_records">(airflow.hooks.PrestoHook method)</a>
+ </dt>
+
+ </dl></dd>
+
+ <dt><a href="code.html#airflow.hooks.HiveMetastoreHook.get_table">get_table() (airflow.hooks.HiveMetastoreHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.HiveMetastoreHook.get_tables">get_tables() (airflow.hooks.HiveMetastoreHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.get_task_instances">get_task_instances() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.get_template_env">get_template_env() (airflow.models.DAG method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.get_wildcard_key">get_wildcard_key() (airflow.hooks.S3Hook method)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="H">H</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.BaseOperator.has_dag">has_dag() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.HdfsSensor">HdfsSensor (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.operators.hipchat_operator.HipChatAPIOperator">HipChatAPIOperator (class in airflow.contrib.operators.hipchat_operator)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.operators.hipchat_operator.HipChatAPISendRoomNotificationOperator">HipChatAPISendRoomNotificationOperator (class in airflow.contrib.operators.hipchat_operator)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.Hive2SambaOperator">Hive2SambaOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.HiveCliHook">HiveCliHook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.HiveMetastoreHook">HiveMetastoreHook (class in airflow.hooks)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.operators.HiveOperator">HiveOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.HivePartitionSensor">HivePartitionSensor (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.HiveServer2Hook">HiveServer2Hook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.HiveToDruidTransfer">HiveToDruidTransfer (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.HiveToMySqlTransfer">HiveToMySqlTransfer (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.HttpHook">HttpHook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.HttpSensor">HttpSensor (class in airflow.operators)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="I">I</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.hooks.DbApiHook.insert_rows">insert_rows() (airflow.hooks.DbApiHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.macros.integrate_plugins">integrate_plugins() (in module airflow.macros)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.is_paused">is_paused (airflow.models.DAG attribute)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.TaskInstance.is_premature">is_premature() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.is_queueable">is_queueable() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.is_runnable">is_runnable() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="K">K</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.TaskInstance.key">key (airflow.models.TaskInstance attribute)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.DagBag.kill_zombies">kill_zombies() (airflow.models.DagBag method)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="L">L</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.DAG.latest_execution_date">latest_execution_date (airflow.models.DAG attribute)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.FTPHook.list_directory">list_directory() (airflow.contrib.hooks.FTPHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.list_keys">list_keys() (airflow.hooks.S3Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.list_prefixes">list_prefixes() (airflow.hooks.S3Hook method)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.hooks.HiveCliHook.load_file">load_file() (airflow.hooks.HiveCliHook method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.load_file">(airflow.hooks.S3Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.WebHDFSHook.load_file">(airflow.hooks.WebHDFSHook method)</a>
+ </dt>
+
+ </dl></dd>
+
+ <dt><a href="code.html#airflow.hooks.DruidHook.load_from_hdfs">load_from_hdfs() (airflow.hooks.DruidHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.S3Hook.load_string">load_string() (airflow.hooks.S3Hook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.executors.LocalExecutor">LocalExecutor (class in airflow.executors)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="M">M</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.hooks.HiveMetastoreHook.max_partition">max_partition() (airflow.hooks.HiveMetastoreHook method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.macros.hive.max_partition">(in module airflow.macros.hive)</a>
+ </dt>
+
+ </dl></dd>
+
+ <dt><a href="code.html#airflow.operators.MetastorePartitionSensor">MetastorePartitionSensor (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.MsSqlHook">MsSqlHook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.MsSqlOperator">MsSqlOperator (class in airflow.operators)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.operators.MsSqlToHiveTransfer">MsSqlToHiveTransfer (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.MySqlHook">MySqlHook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.MySqlOperator">MySqlOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.MySqlToHiveTransfer">MySqlToHiveTransfer (class in airflow.operators)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="O">O</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.BaseOperator.on_kill">on_kill() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="P">P</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.TaskInstance.pool_full">pool_full() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.SSHHook.Popen">Popen() (airflow.contrib.hooks.SSHHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.post_execute">post_execute() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.PostgresHook">PostgresHook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.PostgresOperator">PostgresOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.pre_execute">pre_execute() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.prepare_template">prepare_template() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.operators.PrestoCheckOperator">PrestoCheckOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.PrestoHook">PrestoHook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.PrestoIntervalCheckOperator">PrestoIntervalCheckOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.PrestoValueCheckOperator">PrestoValueCheckOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DagBag.process_file">process_file() (airflow.models.DagBag method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.PythonOperator">PythonOperator (class in airflow.operators)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="R">R</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.macros.random">random() (in module airflow.macros)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.ready_for_retry">ready_for_retry() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.refresh_from_db">refresh_from_db() (airflow.models.TaskInstance method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.render_template">render_template() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.render_template_from_field">render_template_from_field() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.contrib.hooks.FTPHook.retrieve_file">retrieve_file() (airflow.contrib.hooks.FTPHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.DbApiHook.run">run() (airflow.hooks.DbApiHook method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.hooks.HttpHook.run">(airflow.hooks.HttpHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.PrestoHook.run">(airflow.hooks.PrestoHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.run">(airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.run">(airflow.models.DAG method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance.run">(airflow.models.TaskInstance method)</a>
+ </dt>
+
+ </dl></dd>
+
+ <dt><a href="code.html#airflow.hooks.HttpHook.run_and_check">run_and_check() (airflow.hooks.HttpHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.HiveCliHook.run_cli">run_cli() (airflow.hooks.HiveCliHook method)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="S">S</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.hooks.S3Hook">S3Hook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.S3KeySensor">S3KeySensor (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.S3ToHiveTransfer">S3ToHiveTransfer (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.schedule_interval">schedule_interval (airflow.models.BaseOperator attribute)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.executors.SequentialExecutor">SequentialExecutor (class in airflow.executors)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.set_dependency">set_dependency() (airflow.models.DAG method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.set_downstream">set_downstream() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.BaseOperator.set_upstream">set_upstream() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.ShortCircuitOperator">ShortCircuitOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.SimpleHttpOperator">SimpleHttpOperator (class in airflow.operators)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.DagBag.size">size() (airflow.models.DagBag method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.SlackAPIOperator">SlackAPIOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.SlackAPIPostOperator">SlackAPIPostOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.SqliteHook">SqliteHook (class in airflow.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.SqlSensor">SqlSensor (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.operators.SSHExecuteOperator">SSHExecuteOperator (class in airflow.contrib.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.SSHHook">SSHHook (class in airflow.contrib.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.FTPHook.store_file">store_file() (airflow.contrib.hooks.FTPHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.sub_dag">sub_dag() (airflow.models.DAG method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.DAG.subdags">subdags (airflow.models.DAG attribute)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="T">T</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.hooks.HiveMetastoreHook.table_exists">table_exists() (airflow.hooks.HiveMetastoreHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.models.TaskInstance">TaskInstance (class in airflow.models)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.hooks.HiveCliHook.test_hql">test_hql() (airflow.hooks.HiveCliHook method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.TimeSensor">TimeSensor (class in airflow.operators)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.DAG.tree_view">tree_view() (airflow.models.DAG method)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.operators.TriggerDagRunOperator">TriggerDagRunOperator (class in airflow.operators)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.hooks.SSHHook.tunnel">tunnel() (airflow.contrib.hooks.SSHHook method)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="U">U</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.BaseOperator.upstream_list">upstream_list (airflow.models.BaseOperator attribute)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="V">V</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.contrib.hooks.VerticaHook">VerticaHook (class in airflow.contrib.hooks)</a>
+ </dt>
+
+
+ <dt><a href="code.html#airflow.contrib.operators.VerticaOperator">VerticaOperator (class in airflow.contrib.operators)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.contrib.operators.VerticaToHiveTransfer">VerticaToHiveTransfer (class in airflow.contrib.operators)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="W">W</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.hooks.WebHDFSHook">WebHDFSHook (class in airflow.hooks)</a>
+ </dt>
+
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.operators.WebHdfsSensor">WebHdfsSensor (class in airflow.operators)</a>
+ </dt>
+
+ </dl></td>
+</tr></table>
+
+<h2 id="X">X</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.BaseOperator.xcom_pull">xcom_pull() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.models.TaskInstance.xcom_pull">(airflow.models.TaskInstance method)</a>
+ </dt>
+
+ </dl></dd>
+ </dl></td>
+ <td style="width: 33%" valign="top"><dl>
+
+ <dt><a href="code.html#airflow.models.BaseOperator.xcom_push">xcom_push() (airflow.models.BaseOperator method)</a>
+ </dt>
+
+ <dd><dl>
+
+ <dt><a href="code.html#airflow.models.TaskInstance.xcom_push">(airflow.models.TaskInstance method)</a>
+ </dt>
+
+ </dl></dd>
+ </dl></td>
+</tr></table>
+
+
+
+ </div>
+ </div>
+ <footer>
+
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>
+ © Copyright 2014, Maxime Beauchemin, Airbnb.
+
+ </p>
+ </div>
+ Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+</footer>
+
+ </div>
+ </div>
+
+ </section>
+
+ </div>
+
+
+
+
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT:'./',
+ VERSION:'',
+ COLLAPSE_INDEX:false,
+ FILE_SUFFIX:'.html',
+ HAS_SOURCE: true
+ };
+ </script>
+ <script type="text/javascript" src="_static/jquery.js"></script>
+ <script type="text/javascript" src="_static/underscore.js"></script>
+ <script type="text/javascript" src="_static/doctools.js"></script>
+
+
+
+
+
+ <script type="text/javascript" src="_static/js/theme.js"></script>
+
+
+
+
+ <script type="text/javascript">
+ jQuery(function () {
+ SphinxRtdTheme.StickyNav.enable();
+ });
+ </script>
+
+
+</body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-airflow-site/blob/9e19165c/index.html
----------------------------------------------------------------------
diff --git a/index.html b/index.html
new file mode 100644
index 0000000..c764349
--- /dev/null
+++ b/index.html
@@ -0,0 +1,417 @@
+
+
+<!DOCTYPE html>
+<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
+<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
+<head>
+ <meta charset="utf-8">
+
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <title>Apache Airflow (incubating) Documentation — Airflow Documentation</title>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
+
+
+
+
+
+ <link rel="top" title="Airflow Documentation" href="#"/>
+ <link rel="next" title="Project" href="project.html"/>
+
+
+ <script src="_static/js/modernizr.min.js"></script>
+
+</head>
+
+<body class="wy-body-for-nav" role="document">
+
+ <div class="wy-grid-for-nav">
+
+
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-scroll">
+ <div class="wy-side-nav-search">
+
+
+
+ <a href="#" class="icon icon-home"> Airflow
+
+
+
+ </a>
+
+
+
+
+
+
+
+<div role="search">
+ <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+
+
+ </div>
+
+ <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
+
+
+
+ <ul>
+<li class="toctree-l1"><a class="reference internal" href="project.html">Project</a></li>
+<li class="toctree-l1"><a class="reference internal" href="license.html">License</a></li>
+<li class="toctree-l1"><a class="reference internal" href="start.html">Quick Start</a></li>
+<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial</a></li>
+<li class="toctree-l1"><a class="reference internal" href="configuration.html">Configuration</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ui.html">UI / Screenshots</a></li>
+<li class="toctree-l1"><a class="reference internal" href="concepts.html">Concepts</a></li>
+<li class="toctree-l1"><a class="reference internal" href="profiling.html">Data Profiling</a></li>
+<li class="toctree-l1"><a class="reference internal" href="cli.html">Command Line Interface</a></li>
+<li class="toctree-l1"><a class="reference internal" href="scheduler.html">Scheduling & Triggers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="plugins.html">Plugins</a></li>
+<li class="toctree-l1"><a class="reference internal" href="security.html">Security</a></li>
+<li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li>
+<li class="toctree-l1"><a class="reference internal" href="code.html">API Reference</a></li>
+</ul>
+
+
+
+ </div>
+ </div>
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
+
+
+ <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="#">Airflow</a>
+ </nav>
+
+
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+
+
+
+
+
+
+<div role="navigation" aria-label="breadcrumbs navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="#">Docs</a> »</li>
+
+ <li>Apache Airflow (incubating) Documentation</li>
+ <li class="wy-breadcrumbs-aside">
+
+
+ <a href="_sources/index.txt" rel="nofollow"> View page source</a>
+
+
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+ <div itemprop="articleBody">
+
+ <a class="reference internal image-reference" href="_images/pin_large.png"><img alt="_images/pin_large.png" src="_images/pin_large.png" style="width: 100px;" /></a>
+<a class="reference internal image-reference" href="_images/incubator.jpg"><img alt="_images/incubator.jpg" src="_images/incubator.jpg" style="width: 150px;" /></a>
+<div class="section" id="apache-airflow-incubating-documentation">
+<h1>Apache Airflow (incubating) Documentation<a class="headerlink" href="#apache-airflow-incubating-documentation" title="Permalink to this headline">�</a></h1>
+<p>Airflow is a platform to programmatically author, schedule and monitor
+workflows.</p>
+<p>Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks.
+The airflow scheduler executes your tasks on an array of workers while
+following the specified dependencies. Rich command line utilities make
+performing complex surgeries on DAGs a snap. The rich user interface
+makes it easy to visualize pipelines running in production,
+monitor progress, and troubleshoot issues when needed.</p>
+<p>When workflows are defined as code, they become more maintainable,
+versionable, testable, and collaborative.</p>
+<hr class="docutils" />
+<img alt="_images/airflow.gif" src="_images/airflow.gif" />
+<hr class="docutils" />
+<div class="section" id="principles">
+<h2>Principles<a class="headerlink" href="#principles" title="Permalink to this headline">�</a></h2>
+<ul class="simple">
+<li><strong>Dynamic</strong>: Airflow pipelines are configuration as code (Python), allowing for dynamic pipeline generation. This allows for writing code that instantiates pipelines dynamically.</li>
+<li><strong>Extensible</strong>: Easily define your own operators, executors and extend the library so that it fits the level of abstraction that suits your environment.</li>
+<li><strong>Elegant</strong>: Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful <strong>Jinja</strong> templating engine.</li>
+<li><strong>Scalable</strong>: Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers. Airflow is ready to scale to infinity.</li>
+</ul>
+</div>
+<div class="section" id="beyond-the-horizon">
+<h2>Beyond the Horizon<a class="headerlink" href="#beyond-the-horizon" title="Permalink to this headline">�</a></h2>
+<p>Airflow <strong>is not</strong> a data streaming solution. Tasks do not move data from
+one to the other (though tasks can exchange metadata!). Airflow is not
+in the <a class="reference external" href="http://spark.apache.org/streaming/">Spark Streaming</a>
+or <a class="reference external" href="https://storm.apache.org/">Storm</a> space, it is more comparable to
+<a class="reference external" href="http://oozie.apache.org/">Oozie</a> or
+<a class="reference external" href="http://data.linkedin.com/opensource/azkaban">Azkaban</a>.</p>
+<p>Workflows are expected to be mostly static or slowly changing. You can think
+of the structure of the tasks in your workflow as slightly more dynamic
+than a database structure would be. Airflow workflows are expected to look
+similar from a run to the next, this allows for clarity around
+unit of work and continuity.</p>
+</div>
+<div class="section" id="content">
+<h2>Content<a class="headerlink" href="#content" title="Permalink to this headline">�</a></h2>
+<div class="toctree-wrapper compound">
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="project.html">Project</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="project.html#history">History</a></li>
+<li class="toctree-l2"><a class="reference internal" href="project.html#committers">Committers</a></li>
+<li class="toctree-l2"><a class="reference internal" href="project.html#resources-links">Resources & links</a></li>
+<li class="toctree-l2"><a class="reference internal" href="project.html#roadmap">Roadmap</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="license.html">License</a></li>
+<li class="toctree-l1"><a class="reference internal" href="start.html">Quick Start</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="start.html#what-s-next">What’s Next?</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="installation.html#getting-airflow">Getting Airflow</a></li>
+<li class="toctree-l2"><a class="reference internal" href="installation.html#extra-packages">Extra Packages</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#example-pipeline-definition">Example Pipeline definition</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#it-s-a-dag-definition-file">It’s a DAG definition file</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#importing-modules">Importing Modules</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#default-arguments">Default Arguments</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#instantiate-a-dag">Instantiate a DAG</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#tasks">Tasks</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#templating-with-jinja">Templating with Jinja</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#setting-up-dependencies">Setting up Dependencies</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#recap">Recap</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#testing">Testing</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="tutorial.html#running-the-script">Running the Script</a></li>
+<li class="toctree-l3"><a class="reference internal" href="tutorial.html#command-line-metadata-validation">Command Line Metadata Validation</a></li>
+<li class="toctree-l3"><a class="reference internal" href="tutorial.html#id1">Testing</a></li>
+<li class="toctree-l3"><a class="reference internal" href="tutorial.html#backfill">Backfill</a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="tutorial.html#what-s-next">What’s Next?</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="configuration.html">Configuration</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="configuration.html#setting-configuration-options">Setting Configuration Options</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration.html#setting-up-a-backend">Setting up a Backend</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration.html#connections">Connections</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration.html#scaling-out-with-celery">Scaling Out with Celery</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration.html#logs">Logs</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration.html#scaling-out-on-mesos-community-contributed">Scaling Out on Mesos (community contributed)</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration.html#integration-with-systemd">Integration with systemd</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration.html#integration-with-upstart">Integration with upstart</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="ui.html">UI / Screenshots</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="ui.html#dags-view">DAGs View</a></li>
+<li class="toctree-l2"><a class="reference internal" href="ui.html#tree-view">Tree View</a></li>
+<li class="toctree-l2"><a class="reference internal" href="ui.html#graph-view">Graph View</a></li>
+<li class="toctree-l2"><a class="reference internal" href="ui.html#variable-view">Variable View</a></li>
+<li class="toctree-l2"><a class="reference internal" href="ui.html#gantt-chart">Gantt Chart</a></li>
+<li class="toctree-l2"><a class="reference internal" href="ui.html#task-duration">Task Duration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="ui.html#code-view">Code View</a></li>
+<li class="toctree-l2"><a class="reference internal" href="ui.html#task-instance-context-menu">Task Instance Context Menu</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="concepts.html">Concepts</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="concepts.html#core-ideas">Core Ideas</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#dags">DAGs</a><ul>
+<li class="toctree-l4"><a class="reference internal" href="concepts.html#scope">Scope</a></li>
+<li class="toctree-l4"><a class="reference internal" href="concepts.html#default-arguments">Default Arguments</a></li>
+<li class="toctree-l4"><a class="reference internal" href="concepts.html#context-manager">Context Manager</a></li>
+</ul>
+</li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#operators">Operators</a><ul>
+<li class="toctree-l4"><a class="reference internal" href="concepts.html#dag-assignment">DAG Assignment</a></li>
+<li class="toctree-l4"><a class="reference internal" href="concepts.html#bitshift-composition">Bitshift Composition</a></li>
+</ul>
+</li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#tasks">Tasks</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#task-instances">Task Instances</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#workflows">Workflows</a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="concepts.html#additional-functionality">Additional Functionality</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#hooks">Hooks</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#pools">Pools</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#connections">Connections</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#queues">Queues</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#xcoms">XComs</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#variables">Variables</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#branching">Branching</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#subdags">SubDAGs</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#slas">SLAs</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#trigger-rules">Trigger Rules</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#zombies-undeads">Zombies & Undeads</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#cluster-policy">Cluster Policy</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#documentation-notes">Documentation & Notes</a></li>
+<li class="toctree-l3"><a class="reference internal" href="concepts.html#jinja-templating">Jinja Templating</a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="concepts.html#packaged-dags">Packaged dags</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="profiling.html">Data Profiling</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="profiling.html#adhoc-queries">Adhoc Queries</a></li>
+<li class="toctree-l2"><a class="reference internal" href="profiling.html#charts">Charts</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="profiling.html#chart-screenshot">Chart Screenshot</a></li>
+<li class="toctree-l3"><a class="reference internal" href="profiling.html#chart-form-screenshot">Chart Form Screenshot</a></li>
+</ul>
+</li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="cli.html">Command Line Interface</a></li>
+<li class="toctree-l1"><a class="reference internal" href="scheduler.html">Scheduling & Triggers</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="scheduler.html#dag-runs">DAG Runs</a></li>
+<li class="toctree-l2"><a class="reference internal" href="scheduler.html#external-triggers">External Triggers</a></li>
+<li class="toctree-l2"><a class="reference internal" href="scheduler.html#to-keep-in-mind">To Keep in Mind</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="plugins.html">Plugins</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="plugins.html#what-for">What for?</a></li>
+<li class="toctree-l2"><a class="reference internal" href="plugins.html#why-build-on-top-of-airflow">Why build on top of Airflow?</a></li>
+<li class="toctree-l2"><a class="reference internal" href="plugins.html#interface">Interface</a></li>
+<li class="toctree-l2"><a class="reference internal" href="plugins.html#example">Example</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="security.html">Security</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="security.html#web-authentication">Web Authentication</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="security.html#password">Password</a></li>
+<li class="toctree-l3"><a class="reference internal" href="security.html#ldap">LDAP</a></li>
+<li class="toctree-l3"><a class="reference internal" href="security.html#roll-your-own">Roll your own</a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="security.html#multi-tenancy">Multi-tenancy</a></li>
+<li class="toctree-l2"><a class="reference internal" href="security.html#kerberos">Kerberos</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="security.html#limitations">Limitations</a></li>
+<li class="toctree-l3"><a class="reference internal" href="security.html#enabling-kerberos">Enabling kerberos</a></li>
+<li class="toctree-l3"><a class="reference internal" href="security.html#using-kerberos-authentication">Using kerberos authentication</a></li>
+<li class="toctree-l3"><a class="reference internal" href="security.html#github-enterprise-ghe-authentication">GitHub Enterprise (GHE) Authentication</a></li>
+<li class="toctree-l3"><a class="reference internal" href="security.html#setting-up-ghe-authentication">Setting up GHE Authentication</a></li>
+</ul>
+</li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li>
+<li class="toctree-l1"><a class="reference internal" href="code.html">API Reference</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="code.html#operators">Operators</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="code.html#baseoperator">BaseOperator</a></li>
+<li class="toctree-l3"><a class="reference internal" href="code.html#basesensoroperator">BaseSensorOperator</a></li>
+<li class="toctree-l3"><a class="reference internal" href="code.html#module-airflow.operators">Operator API</a></li>
+<li class="toctree-l3"><a class="reference internal" href="code.html#community-contributed-operators">Community-contributed Operators</a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="code.html#macros">Macros</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="code.html#default-variables">Default Variables</a></li>
+<li class="toctree-l3"><a class="reference internal" href="code.html#id2">Macros</a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="code.html#models">Models</a></li>
+<li class="toctree-l2"><a class="reference internal" href="code.html#module-airflow.hooks">Hooks</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="code.html#community-contributed-hooks">Community contributed hooks</a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="code.html#executors">Executors</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="code.html#community-contributed-executors">Community-contributed executors</a></li>
+</ul>
+</li>
+</ul>
+</li>
+</ul>
+</div>
+</div>
+</div>
+
+
+ </div>
+ </div>
+ <footer>
+
+ <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
+
+ <a href="project.html" class="btn btn-neutral float-right" title="Project" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a>
+
+
+ </div>
+
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>
+ © Copyright 2014, Maxime Beauchemin, Airbnb.
+
+ </p>
+ </div>
+ Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+</footer>
+
+ </div>
+ </div>
+
+ </section>
+
+ </div>
+
+
+
+
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT:'./',
+ VERSION:'',
+ COLLAPSE_INDEX:false,
+ FILE_SUFFIX:'.html',
+ HAS_SOURCE: true
+ };
+ </script>
+ <script type="text/javascript" src="_static/jquery.js"></script>
+ <script type="text/javascript" src="_static/underscore.js"></script>
+ <script type="text/javascript" src="_static/doctools.js"></script>
+
+
+
+
+
+ <script type="text/javascript" src="_static/js/theme.js"></script>
+
+
+
+
+ <script type="text/javascript">
+ jQuery(function () {
+ SphinxRtdTheme.StickyNav.enable();
+ });
+ </script>
+
+
+</body>
+</html>
\ No newline at end of file