You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by airflowuser <ai...@protonmail.com.INVALID> on 2019/06/25 17:59:19 UTC
Re: 1.10.4beta2 "snapshot" for available testing
Why [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365) listed under features?
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, June 25, 2019 1:40 PM, Ash Berlin-Taylor <as...@apache.org> wrote:
> Hi Everyone,
>
> I'm excited to annoucne that 1.10.4 is almost ready for release.
>
> The release branch already contains 216 commits since 1.10.3 this time around.
>
> In order to distinguish it from an actual (apache) release it is:
>
> 1. Marked as beta (python package managers do not install beta versions by default - PEP 440)
> 2. It is not at an official Apache distribution location
>
> It can be installed with:
>
> pip install 'apache-airflow==1.10.4b2'
>
> or downloaded from https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/ https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/
>
>
> (Don't worry, without asking for`--pre` or specifying the version `pip install apache-airflow` will still get 1.10.3)
>
> Included below is the changelog of this release so far. I'd like to try and get this out before the end of the month, so will create an RC in a day or two baring any major problems reported.
>
> Cheers,
> Ash
>
> Airflow 1.10.4
>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> New Features
> """"""""""""
>
> - [AIRFLOW-4456] Add sub-classable BaseBranchOperator (#5231)
> - [AIRFLOW-4343] Show warning in UI if scheduler is not running (#5127)
> - [AIRFLOW-4739] Add ability to arbitrarily define kubernetes worker pod labels (#5376)
> - [AIRFLOW-4348] Add GCP console link in BigQueryOperator (#5195)
> - [AIRFLOW-4306] Global operator extra links (#5094)
> - [AIRFLOW-4812] Add batch images annotation (#5433)
> - [AIRFLOW-4135] Add Google Cloud Build operator and hook (#5251)
> - [AIRFLOW-4781] Add the ability to specify ports in kubernetesOperator (#5410)
> - [AIRFLOW-4521] Pause dag also pause its subdags (#5283)
> - [AIRFLOW-4738] Enforce exampleinclude for example DAGs (#5375)
> - [AIRFLOW-4326] Airflow AWS SQS Operator (#5110)
> - [AIRFLOW-3729] Support "DownwardAPI" in env variables for KubernetesPodOperator (#4554)
> - [AIRFLOW-4585] Implement Kubernetes Pod Mutation Hook (#5359)
> - [AIRFLOW-161] New redirect route and extra links (#5059)
> - [AIRFLOW-4420] Backfill respects task_concurrency (#5221)
> - [AIRFLOW-4147] Add Operator to publish event to Redis (#4967)
> - [AIRFLOW-3359] Add option to pass customer encryption keys to Dataproc (#4200)
> - [AIRFLOW-4318] Create Google Cloud Translate Speech Operator (#5102)
> - [AIRFLOW-3960] Adds Google Cloud Speech operators (#4780)
> - [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365)
> - [AIRFLOW-1501] Add GoogleCloudStorageDeleteOperator (#5230)
> - [AIRFLOW-3672] Add support for Mongo DB DNS Seedlist Connection Format (#4481)
> - [AIRFLOW-4397] add integrations docs manually for gcs sensors (#5204)
> - [AIRFLOW-4251] Instrument DagRun schedule delay (#5050)
> - [AIRFLOW-4118] instrument DagRun duration (#4946)
> - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5182)
> - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5140)
> - [AIRFLOW-4168] Create Google Cloud Video Intelligence Operators (#4985)
> - [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor (#5166)
> - [AIRFLOW-4335] Add default num_retries to GCP connection (#5117)
> - [AIRFLOW-3808] Add cluster_fields to BigQueryHook's create_empty_table (#4654)
> - [AIRFLOW-4362] Fix test_execution_limited_parallelism (#5141)
> - [AIRFLOW-4307] Backfill respects concurrency limit (#5128)
> - [AIRFLOW-4268] Add MsSqlToGoogleCloudStorageOperator (#5077)
> - [AIRFLOW-4169] Add Google Cloud Vision Detect Operators (#4986)
> - [AIRFLOW-XXX] Fix WS-2019-0032 (#5384)
> - [AIRFLOW-XXX] Fix CVE-2019-11358 (#5197)
> - [AIRFLOW-XXX] Change allowed version of Jinja2 to fix CVE-2019-10906 (#5075)
>
> Improvement
> """""""""""
>
> - [AIRFLOW-4805] Add py_file as templated field in DataflowPythonOperator (#5451)
> - [AIRFLOW-4838] Surface Athena errors in AWSAthenaOperator (#5467)
> - [AIRFLOW-4831] conf.has_option no longer throws if section is missing. (#5455)
> - [AIRFLOW-4829] More descriptive exceptions for EMR sensors (#5452)
> - [AIRFLOW-4414] AWSAthenaOperator: Push QueryExecutionID to XCom (#5276)
> - [AIRFLOW-4791] add "schema" keyword arg to SnowflakeOperator (#5415)
> - [AIRFLOW-4759] Don't error when marking sucessful run as failed (#5435)
> - [AIRFLOW-4716] Instrument dag loading time duration (#5350)
> - [AIRFLOW-3958] Support list tasks as upstream in chain (#4779)
> - [AIRFLOW-4409] Prevent task duration break by null value (#5178)
> - [AIRFLOW-4418] Add "failed only" option to task modal (#5193)
> - [AIRFLOW-4740] Accept string `end_date` in DAG default_args (#5381)
> - [AIRFLOW-4423] Improve date handling in mysql to gcs operator. (#5196)
> - [AIRFLOW-4447] Display task duration as human friendly format in UI (#5218)
> - [AIRFLOW-4377] Remove needless object conversion in DAG.owner() (#5144)
> - [AIRFLOW-4766] Add autoscaling option for DataprocClusterCreateOperator (#5425)
> - [AIRFLOW-4795] Upgrade alembic to latest release. (#5411)
> - [AIRFLOW-4793] Add signature_name to mlengine operator (#5417)
> - [AIRFLOW-3211] Reattach to GCP Dataproc jobs upon Airflow restart (#4083)
> - [AIRFLOW-4750] Log identified zombie task instances (#5389)
> - [AIRFLOW-3870] STFPOperator: Update log level and return value (#4355)
> - [AIRFLOW-4759] Batch queries in set_state API. (#5403)
> - [AIRFLOW-2737] Restore original license header to airflow.api.auth.backend.kerberos_auth
> - [AIRFLOW-3635] Fix incorrect logic in detele_dag (introduced in PR#4406) (#4445)
> - [AIRFLOW-3599] Removed Dagbag from delete dag (#4406)
> - [AIRFLOW-4737] Increase and document celery queue name limit (#5383)
> - [AIRFLOW-4505] Correct Tag ALL for PY3 (#5275)
> - [AIRFLOW-4743] Add environment variables support to SSHOperator (#5385)
> - [AIRFLOW-4725] Fix setup.py PEP440 & Sphinx-PyPI-upload dependency (#5363)
> - [AIRFLOW-3370] Add stdout output options to Elasticsearch task log handler (#5048)
> - [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. (#5164)
> - [AIRFLOW-1381] Allow setting host temporary directory in DockerOperator (#5369)
> - [AIRFLOW-4598] Task retries are not exhausted for K8s executor (#5347)
> - [AIRFLOW-4218] Support to Provide http args to K8executor while calling k8 python client lib apis (#5060)
> - [AIRFLOW-4159] Add support for additional static pod labels for K8sExecutor (#5134)
> - [AIRFLOW-4720] Allow comments in .airflowignore files. (#5355)
> - [AIRFLOW-4486] Add AWS IAM authenication in MySqlHook (#5334)
> - [AIRFLOW-4417] Add AWS IAM authenication for PostgresHook (#5223)
> - [AIRFLOW-3990] Compile regular expressions. (#4813)
> - [AIRFLOW-4572] Rename prepare_classpath() to prepare_syspath() (#5328)
> - [AIRFLOW-3869] Raise consistent exception in AirflowConfigParser.getboolean (#4692)
> - [AIRFLOW-4571] Add headers to templated field for SimpleHttpOperator (#5326)
> - [AIRFLOW-3867] Rename GCP's subpackage (#4690)
> - [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df (#4549)
> - [AIRFLOW-4546] Upgrade google-cloud-bigtable. (#5307)
> - [AIRFLOW-4519] Optimise operator classname sorting in views (#5282)
> - [AIRFLOW-4503] Support fully pig options (#5271)
> - [AIRFLOW-4468] add sql_alchemy_max_overflow parameter (#5249)
> - [AIRFLOW-4467] Add dataproc_jars to templated fields in Dataproc oper… (#5248)
> - [AIRFLOW-4381] Use get_direct_relative_ids get task relatives (#5147)
> - [AIRFLOW-3624] Add masterType parameter to MLEngineTrainingOperator (#4428)
> - [AIRFLOW-3143] Support Auto-Zone in DataprocClusterCreateOperator (#5169)
> - [AIRFLOW-3874] Improve BigQueryHook.run_with_configuration's location support (#4695)
> - [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag (#5165)
> - [AIRFLOW-4031] Allow for key pair auth in snowflake hook (#4875)
> - [AIRFLOW-3901] add role as optional config parameter for SnowflakeHook (#4721)
> - [AIRFLOW-3455] add region in snowflake connector (#4285)
> - [AIRFLOW-4073] add template_ext for AWS Athena operator (#4907)
> - [AIRFLOW-4093] AWSAthenaOperator: Throw exception if job failed/cancelled/reach max retries (#4919)
> - [AIRFLOW-4356] Add extra RuntimeEnvironment keys to DataFlowHook (#5149)
> - [AIRFLOW-4337] Fix docker-compose deprecation warning in CI (#5119)
> - [AIRFLOW-3603] QuboleOperator: Remove SQLCommand from SparkCmd documentation (#4411)
> - [AIRFLOW-4328] Fix link to task instances from Pool page (#5124)
> - [AIRFLOW-4255] Make GCS Hook Backwards compatible (#5089)
> - [AIRFLOW-4103] Allow uppercase letters in dataflow job names (#4925)
> - [AIRFLOW-4255] Replace Discovery based api with client based for GCS (#5054)
> - [AIRFLOW-4311] Remove sleep in localexecutor (#5096)
> - [AIRFLOW-2836] Minor improvement-contrib.sensors.FileSensor (#3674)
> - [AIRFLOW-4104] Add type annotations to common classes. (#4926)
> - [AIRFLOW-3910] Raise exception explicitly in Connection.get_hook() (#4728)
> - [AIRFLOW-3322] Update QuboleHook to fetch args dynamically from qds_sdk (#4165)
> - [AIRFLOW-4565] instrument celery executor (#5321)
> - [AIRFLOW-4573] Import airflow_local_settings after prepare_classpath (#5330)
> - [AIRFLOW-4448] Don't bake ENV and _cmd into tmp config for non-sudo (#4050)
> - [AIRFLOW-4295] Make `method` attribute case insensitive in HttpHook (#5313)
> - [AIRFLOW-3703] Add dnsPolicy option for KubernetesPodOperator (#4520)
> - [AIRFLOW-3057] add prev_*_date_success to template context (#5372)
> - [AIRFLOW-4336] Stop showing entire GCS files bytes in log for gcs_download_operator (#5151)
> - [AIRFLOW-4528] Cancel DataProc task on timeout (#5293)
>
> Bug fixes
> """""""""
>
> - [AIRFLOW-4587] Replace self.conn with self.get_conn() in AWSAthenaHook (#5462)
> - [AIRFLOW-4516] K8s runAsUser and fsGroup cannot be strings (#5429)
> - [AIRFLOW-4298] Stop Scheduler repeatedly warning "connection invalidated" (#5470)
> - [AIRFLOW-4559] JenkinsJobTriggerOperator bugfix (#5318)
> - [AIRFLOW-4841] Pin Sphinx AutoApi to 1.0.0 (#5468)
> - [AIRFLOW-4479] Include s3_overwrite kwarg in load_bytes method (#5312)
> - [AIRFLOW-3746] Fix DockerOperator missing container exit (#4583)
> - [AIRFLOW-4233] Remove Template Extension from Bq to GCS Operator (#5456)
> - [AIRFLOW-2141][AIRFLOW-3157][AIRFLOW-4170] Serialize non-str value by JSON when importing Variables (#4991)
> - [AIRFLOW-4826] Remove warning from `airflow resetdb` command (#5447)
> - [AIRFLOW-4148] Fix editing DagRuns when clicking state column (#5436)
> - [AIRFLOW-4455] dag_details broken for subdags in RBAC UI (#5234)
> - [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods (#4551)
> - [AIRFLOW-4459] Fix wrong DAG count in /home page when DAG count is zero (#5235)
> - [AIRFLOW-3876] AttributeError: module 'distutils' has no attribute 'util'
> - [AIRFLOW-4146] Fix CgroupTaskRunner errors (#5224)
> - [AIRFLOW-4524] Fix bug with "Ignore \*" toggles in RBAC mode (#5378)
> - [AIRFLOW-4765] Fix DataProcPigOperator execute method (#5426)
> - [AIRFLOW-4798] obviate interdependencies for dagbag and TI tests (#5422)
> - [AIRFLOW-4800] fix GKEClusterHook ctor calls (#5424)
> - [AIRFLOW-4799] don't mutate self.env in BashOperator execute method (#5421)
> - [AIRFLOW-4393] Add retry logic when fetching pod status and/or logs in KubernetesPodOperator (#5284)
> - [AIRFLOW-4174] Fix HttpHook run with backoff (#5213)
> - [AIRFLOW-4463] Handle divide-by-zero errors in short retry intervals (#5243)
> - [AIRFLOW-2614] Speed up trigger_dag API call when lots of DAGs in system
> - [AIRFLOW-4756] add ti.state to ti.start_date as criteria for gantt (#5399)
> - [AIRFLOW-4760] Fix zip-packaged DAGs disappearing from DagBag when reloaded (#5404)
> - [AIRFLOW-4731] Fix GCS hook with google-storage-client 1.16 (#5368)
> - [AIRFLOW-3506] use match_phrase to query log_id in elasticsearch (#4342)
> - [AIRFLOW-4084] fix ElasticSearch log download (#5177)
> - [AIRFLOW-4501] Register pendulum datetime converter for sqla+pymysql (#5190)
> - [AIRFLOW-986] HiveCliHook ignores 'proxy_user' value in a connection's extra parameter (#5305)
> - [AIRFLOW-4442] fix hive_tblproperties in HiveToDruidTransfer (#5211)
> - [AIRFLOW-4557] Add gcp_conn_id parameter to get_sqlproxy_runner() of CloudSqlDatabaseHook (#5314)
> - [AIRFLOW-4545] Upgrade FAB to latest version (#4955)
> - [AIRFLOW-4492] Change Dataproc Cluster operators to poll Operations (#5269)
> - [AIRFLOW-4452] Webserver and Scheduler keep crashing because of slackclient update (#5225)
> - [AIRFLOW-4450] Fix request arguments in has_dag_access (#5220)
> - [AIRFLOW-4434] Support Impala with the HiveServer2Hook (#5206)
> - [AIRFLOW-3449] Write local dag parsing logs when remote logging enabled. (#5175)
> - [AIRFLOW-4300] Fix graph modal call when DAG has not yet run (#5185)
> - [AIRFLOW-4401] Use managers for Queue synchronization (#5200)
> - [AIRFLOW-3626] Fixed triggering DAGs contained within zip files (#4439)
> - [AIRFLOW-3720] Fix missmatch while comparing GCS and S3 files (#4766)
> - [AIRFLOW-4403] search by `dag_id` or `owners` in UI (#5184)
> - [AIRFLOW-4308] Fix TZ-loop around DST on python 3.6+ (#5095)
> - [AIRFLOW-4324] fix DAG fuzzy search in RBAC UI (#5131)
> - [AIRFLOW-4297] Temporary hot fix on manage_slas() for 1.10.4 release (#5150)
> - [AIRFLOW-4299] Upgrade to Celery 4.3.0 to fix crashing workers (#5116)
> - [AIRFLOW-4291] Correctly render doc_md in DAG graph page (#5121)
> - [AIRFLOW-4310] Fix incorrect link on Dag Details page (#5122)
> - [AIRFLOW-4331] Correct filter for Null-state runs from Dag Detail page (#5123)
> - [AIRFLOW-4294] Fix missing dag & task runs in UI dag_id contains a dot (#5111)
> - [AIRFLOW-4332] Upgrade sqlalchemy to remove security Vulnerability (#5113)
> - [AIRFLOW-4312] Add template_fields & template_ext to BigQueryCheckO… (#5097)
> - [AIRFLOW-4293] Fix downgrade in d4ecb8fbee3_add_schedule_interval_to_dag.py (#5086)
> - [AIRFLOW-4267] Fix TI duration in Graph View (#5071)
> - [AIRFLOW-4163] IntervalCheckOperator supports relative diff and not ignore 0 (#4983)
> - [AIRFLOW-3938] QuboleOperator Fixes and Support for SqlCommand (#4832)
> - [AIRFLOW-2903] Change default owner to `airflow` (#4151)
> - [aIRFLOW-4136] Fix overwrite of key_file by constructor (#5155)
> - [AIRFLOW-3241] Remove Invalid template ext in GCS Sensors (#4076)
>
> Misc/Internal
> """""""""""""
>
> - [AIRFLOW-4211] Add tests for WebHDFSHook (#5015)
> - [AIRFLOW-4320] Add tests for SegmentTrackEventOperator (#5104)
> - [AIRFLOW-4319] Add tests for Bigquery related Operators (#5101)
> - [AIRFLOW-4014] Change DatastoreHook and add tests (#4842)
> - [AIRFLOW-4322] Add test for VerticaOperator (#5107)
> - [AIRFLOW-4323] Add 2 tests for WinRMOperator (#5108)
> - [AIRFLOW-3677] Improve CheckOperator test coverage (#4756)
> - [AIRFLOW-4659] Fix pylint problems for api module (#5398)
> - [AIRFLOW-4358] Speed up test_jobs by not running tasks (#5162)
> - [AIRFLOW-4394] Don't test behaviour of BackfillJob from CLI tests (#5160)
> - [AIRFLOW-3471] Move XCom out of models.py (#4629)
> - [AIRFLOW-4379] Remove duplicate code & Add validation in gcs_to_gcs.py (#5145)
> - [AIRFLOW-4259] Move models out of models.py (#5056)
> - [AIRFLOW-XXX] Speed up building of Cassanda module on Travis (#5233)
> - [AIRFLOW-4535] Break jobs.py into multiple files (#5303)
> - [AIRFLOW-1464] Batch update task_instance state (#5323)
> - [AIRFLOW-4554] Test for sudo command, add some other test docs (#5310)
> - [AIRFLOW-4419] Refine concurrency check in scheduler (#5194)
> - [AIRFLOW-4269] Minor acceleration of jobs._process_task_instances() (#5076)
> - [AIRFLOW-4341] Remove `View.render()` already exists in fab.BaseView (#5125)
> - [AIRFLOW-4342] Use @cached_property instead of re-implementing it each time (#5126)
> - [AIRFLOW-4256] Remove noqa from migrations (#5055)
> - [AIRFLOW-4034] Remove unnecessary string formatting with `**locals()` (#4861)
> - [AIRFLOW-3944] Remove code smells (#4762)
>
> Doc-only changes
> """"""""""""""""
>
> - [AIRFLOW-3341] FAQ return DAG object example (#4605)
> - [AIRFLOW-4433] Add missing type in DockerOperator doc string (#5205)
> - [AIRFLOW-4321] Replace incorrect info of Max Size limit of GCS Object Size (#5106)
> - [AIRFLOW-XXX] Add information about user list (#5341)
> - [AIRFLOW-XXX] Clarify documentation related to autodetect parameter in GCS_to_BQ Op (#5294)
> - [AIRFLOW-XXX] Remove mention of pytz compatibility from timezone documentation (#5316)
> - [AIRFLOW-XXX] Add missing docs for GoogleCloudStorageDeleteOperator (#5274)
> - [AIRFLOW-XXX] Remove incorrect note about Scopes of GCP connection (#5242)
> - [AIRFLOW-XXX] Fix mistakes in docs of Dataproc operators (#5192)
> - [AIRFLOW-XXX] Link to correct class for timedelta in macros.rst (#5226)
> - [AIRFLOW-XXX] Add Kamil as committer (#5216)
> - [AIRFLOW-XXX] Add Joshua and Kevin as committer (#5207)
> - [AIRFLOW-XXX] Reduce log spam in tests (#5174)
> - [AIRFLOW-XXX] Speed up tests for PythonSensor (#5158)
> - [AIRFLOW-XXX] Add Bas Harenslak to committer list (#5157)
> - [AIRFLOW-XXX] Add Jarek Potiuk to commiter list (#5132)
> - [AIRFLOW-XXX] Update docstring for SchedulerJob (#5105)
> - [AIRFLOW-XXX] Fix docstrings for CassandraToGoogleCloudStorageOperator (#5103)
> - [AIRFLOW-XXX] update SlackWebhookHook and SlackWebhookOperator docstring (#5074)
> - [AIRFLOW-XXX] Ignore python files under node_modules in docs (#5063)
> - [AIRFLOW-XXX] Build a universal wheel with LICNESE files (#5052)
> - [AIRFLOW-XXX] Fix docstrings of SQSHook (#5099)
> - [AIRFLOW-XXX] Use Py3.7 on readthedocs
> - [AIRFLOW-4446] Fix typos (#5217)
Re: 1.10.4beta2 "snapshot" for available testing
Posted by airflowuser <ai...@protonmail.com.INVALID>.
This line is empty in V10 test and V10 stable as well :)
not sure what's going on.
I'll try to reinstall and check this again
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, June 26, 2019 1:39 PM, Ash Berlin-Taylor <as...@apache.org> wrote:
> Are you sure you are on 1.10.4b2? The stack trace points at line 113 which in 1.10.4b2 is an empty line https://github.com/apache/airflow/blob/1.10.4b2/airflow/www/views.py#L113 https://github.com/apache/airflow/blob/1.10.4b2/airflow/www/views.py#L113
>
> The stack trace matches 1.10.3 thoughhttps://github.com/apache/airflow/blob/1.10.3/airflow/www/views.py#L113
>
> > On 26 Jun 2019, at 11:34, airflowuser airflowuser@protonmail.com.INVALID wrote:
> > http://localhost:8080/admin/dagmodel/
> > Sent with ProtonMail Secure Email.
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Wednesday, June 26, 2019 1:32 PM, Ash Berlin-Taylor ash@apache.org wrote:
> >
> > > What URL does that end up on? I can't reproduce that.
> > > -a
> > >
> > > > On 26 Jun 2019, at 10:52, airflowuser airflowuser@protonmail.com.INVALID wrote:
> > > > Bug:
> > > > On the old UI:
> > > >
> > > > 1. Click on edit DAG
> > > > 2. Enter Description
> > > > 3.Click Save
> > > >
> > > >
> > > > You'll get error:
> > > > Traceback (most recent call last):
> > > > File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2311, in wsgi_app
> > > > response = self.full_dispatch_request()
> > > > ....
> > > > packages/flask_admin/model/base.py", line 1861, in get_list_value
> > > > self.column_type_formatters,
> > > > File "/usr/local/lib/python3.6/site-packages/flask_admin/model/base.py", line 1826, in _get_list_value
> > > > value = column_fmt(self, context, model, name)
> > > > File "/usr/local/lib/python3.6/site-packages/airflow/www/views.py", line 113, in dag_link
> > > > execution_date=m.execution_date)
> > > > AttributeError: 'DagModel' object has no attribute 'execution_date'
> > > > Didn't happen to me on 1.10.3
> > > > Not really sure why execution_date is relevant for when you want to edit details on the DAG that is unrelated to a specific DAG run.
> > > > Sent with ProtonMail Secure Email.
> > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > On Wednesday, June 26, 2019 11:09 AM, Ash Berlin-Taylor ash@apache.org wrote:
> > > >
> > > > > Because the list was autogenerated from the issue type in the jira ticket, and I didn't check all 200+ items in the change log :)
> > > > > -a
> > > > >
> > > > > > On 25 Jun 2019, at 18:59, airflowuser airflowuser@protonmail.com.INVALID wrote:
> > > > > > Why [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365) listed under features?
> > > > > > Sent with ProtonMail Secure Email.
> > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > > > On Tuesday, June 25, 2019 1:40 PM, Ash Berlin-Taylor ash@apache.org wrote:
> > > > > >
> > > > > > > Hi Everyone,
> > > > > > > I'm excited to annoucne that 1.10.4 is almost ready for release.
> > > > > > > The release branch already contains 216 commits since 1.10.3 this time around.
> > > > > > > In order to distinguish it from an actual (apache) release it is:
> > > > > > >
> > > > > > > 1. Marked as beta (python package managers do not install beta versions by default - PEP 440)
> > > > > > > 2. It is not at an official Apache distribution location
> > > > > > >
> > > > > > > It can be installed with:
> > > > > > > pip install 'apache-airflow==1.10.4b2'
> > > > > > > or downloaded from https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/ https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/
> > > > > > > (Don't worry, without asking for`--pre` or specifying the version `pip install apache-airflow` will still get 1.10.3)
> > > > > > > Included below is the changelog of this release so far. I'd like to try and get this out before the end of the month, so will create an RC in a day or two baring any major problems reported.
> > > > > > > Cheers,
> > > > > > > Ash
> > > > > > > Airflow 1.10.4
> > > > > > > New Features
> > > > > > > """"""""""""
> > > > > > >
> > > > > > > - [AIRFLOW-4456] Add sub-classable BaseBranchOperator (#5231)
> > > > > > > - [AIRFLOW-4343] Show warning in UI if scheduler is not running (#5127)
> > > > > > > - [AIRFLOW-4739] Add ability to arbitrarily define kubernetes worker pod labels (#5376)
> > > > > > > - [AIRFLOW-4348] Add GCP console link in BigQueryOperator (#5195)
> > > > > > > - [AIRFLOW-4306] Global operator extra links (#5094)
> > > > > > > - [AIRFLOW-4812] Add batch images annotation (#5433)
> > > > > > > - [AIRFLOW-4135] Add Google Cloud Build operator and hook (#5251)
> > > > > > > - [AIRFLOW-4781] Add the ability to specify ports in kubernetesOperator (#5410)
> > > > > > > - [AIRFLOW-4521] Pause dag also pause its subdags (#5283)
> > > > > > > - [AIRFLOW-4738] Enforce exampleinclude for example DAGs (#5375)
> > > > > > > - [AIRFLOW-4326] Airflow AWS SQS Operator (#5110)
> > > > > > > - [AIRFLOW-3729] Support "DownwardAPI" in env variables for KubernetesPodOperator (#4554)
> > > > > > > - [AIRFLOW-4585] Implement Kubernetes Pod Mutation Hook (#5359)
> > > > > > > - [AIRFLOW-161] New redirect route and extra links (#5059)
> > > > > > > - [AIRFLOW-4420] Backfill respects task_concurrency (#5221)
> > > > > > > - [AIRFLOW-4147] Add Operator to publish event to Redis (#4967)
> > > > > > > - [AIRFLOW-3359] Add option to pass customer encryption keys to Dataproc (#4200)
> > > > > > > - [AIRFLOW-4318] Create Google Cloud Translate Speech Operator (#5102)
> > > > > > > - [AIRFLOW-3960] Adds Google Cloud Speech operators (#4780)
> > > > > > > - [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365)
> > > > > > > - [AIRFLOW-1501] Add GoogleCloudStorageDeleteOperator (#5230)
> > > > > > > - [AIRFLOW-3672] Add support for Mongo DB DNS Seedlist Connection Format (#4481)
> > > > > > > - [AIRFLOW-4397] add integrations docs manually for gcs sensors (#5204)
> > > > > > > - [AIRFLOW-4251] Instrument DagRun schedule delay (#5050)
> > > > > > > - [AIRFLOW-4118] instrument DagRun duration (#4946)
> > > > > > > - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5182)
> > > > > > > - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5140)
> > > > > > > - [AIRFLOW-4168] Create Google Cloud Video Intelligence Operators (#4985)
> > > > > > > - [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor (#5166)
> > > > > > > - [AIRFLOW-4335] Add default num_retries to GCP connection (#5117)
> > > > > > > - [AIRFLOW-3808] Add cluster_fields to BigQueryHook's create_empty_table (#4654)
> > > > > > > - [AIRFLOW-4362] Fix test_execution_limited_parallelism (#5141)
> > > > > > > - [AIRFLOW-4307] Backfill respects concurrency limit (#5128)
> > > > > > > - [AIRFLOW-4268] Add MsSqlToGoogleCloudStorageOperator (#5077)
> > > > > > > - [AIRFLOW-4169] Add Google Cloud Vision Detect Operators (#4986)
> > > > > > > - [AIRFLOW-XXX] Fix WS-2019-0032 (#5384)
> > > > > > > - [AIRFLOW-XXX] Fix CVE-2019-11358 (#5197)
> > > > > > > - [AIRFLOW-XXX] Change allowed version of Jinja2 to fix CVE-2019-10906 (#5075)
> > > > > > >
> > > > > > > Improvement
> > > > > > > """""""""""
> > > > > > >
> > > > > > > - [AIRFLOW-4805] Add py_file as templated field in DataflowPythonOperator (#5451)
> > > > > > > - [AIRFLOW-4838] Surface Athena errors in AWSAthenaOperator (#5467)
> > > > > > > - [AIRFLOW-4831] conf.has_option no longer throws if section is missing. (#5455)
> > > > > > > - [AIRFLOW-4829] More descriptive exceptions for EMR sensors (#5452)
> > > > > > > - [AIRFLOW-4414] AWSAthenaOperator: Push QueryExecutionID to XCom (#5276)
> > > > > > > - [AIRFLOW-4791] add "schema" keyword arg to SnowflakeOperator (#5415)
> > > > > > > - [AIRFLOW-4759] Don't error when marking sucessful run as failed (#5435)
> > > > > > > - [AIRFLOW-4716] Instrument dag loading time duration (#5350)
> > > > > > > - [AIRFLOW-3958] Support list tasks as upstream in chain (#4779)
> > > > > > > - [AIRFLOW-4409] Prevent task duration break by null value (#5178)
> > > > > > > - [AIRFLOW-4418] Add "failed only" option to task modal (#5193)
> > > > > > > - [AIRFLOW-4740] Accept string `end_date` in DAG default_args (#5381)
> > > > > > > - [AIRFLOW-4423] Improve date handling in mysql to gcs operator. (#5196)
> > > > > > > - [AIRFLOW-4447] Display task duration as human friendly format in UI (#5218)
> > > > > > > - [AIRFLOW-4377] Remove needless object conversion in DAG.owner() (#5144)
> > > > > > > - [AIRFLOW-4766] Add autoscaling option for DataprocClusterCreateOperator (#5425)
> > > > > > > - [AIRFLOW-4795] Upgrade alembic to latest release. (#5411)
> > > > > > > - [AIRFLOW-4793] Add signature_name to mlengine operator (#5417)
> > > > > > > - [AIRFLOW-3211] Reattach to GCP Dataproc jobs upon Airflow restart (#4083)
> > > > > > > - [AIRFLOW-4750] Log identified zombie task instances (#5389)
> > > > > > > - [AIRFLOW-3870] STFPOperator: Update log level and return value (#4355)
> > > > > > > - [AIRFLOW-4759] Batch queries in set_state API. (#5403)
> > > > > > > - [AIRFLOW-2737] Restore original license header to airflow.api.auth.backend.kerberos_auth
> > > > > > > - [AIRFLOW-3635] Fix incorrect logic in detele_dag (introduced in PR#4406) (#4445)
> > > > > > > - [AIRFLOW-3599] Removed Dagbag from delete dag (#4406)
> > > > > > > - [AIRFLOW-4737] Increase and document celery queue name limit (#5383)
> > > > > > > - [AIRFLOW-4505] Correct Tag ALL for PY3 (#5275)
> > > > > > > - [AIRFLOW-4743] Add environment variables support to SSHOperator (#5385)
> > > > > > > - [AIRFLOW-4725] Fix setup.py PEP440 & Sphinx-PyPI-upload dependency (#5363)
> > > > > > > - [AIRFLOW-3370] Add stdout output options to Elasticsearch task log handler (#5048)
> > > > > > > - [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. (#5164)
> > > > > > > - [AIRFLOW-1381] Allow setting host temporary directory in DockerOperator (#5369)
> > > > > > > - [AIRFLOW-4598] Task retries are not exhausted for K8s executor (#5347)
> > > > > > > - [AIRFLOW-4218] Support to Provide http args to K8executor while calling k8 python client lib apis (#5060)
> > > > > > > - [AIRFLOW-4159] Add support for additional static pod labels for K8sExecutor (#5134)
> > > > > > > - [AIRFLOW-4720] Allow comments in .airflowignore files. (#5355)
> > > > > > > - [AIRFLOW-4486] Add AWS IAM authenication in MySqlHook (#5334)
> > > > > > > - [AIRFLOW-4417] Add AWS IAM authenication for PostgresHook (#5223)
> > > > > > > - [AIRFLOW-3990] Compile regular expressions. (#4813)
> > > > > > > - [AIRFLOW-4572] Rename prepare_classpath() to prepare_syspath() (#5328)
> > > > > > > - [AIRFLOW-3869] Raise consistent exception in AirflowConfigParser.getboolean (#4692)
> > > > > > > - [AIRFLOW-4571] Add headers to templated field for SimpleHttpOperator (#5326)
> > > > > > > - [AIRFLOW-3867] Rename GCP's subpackage (#4690)
> > > > > > > - [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df (#4549)
> > > > > > > - [AIRFLOW-4546] Upgrade google-cloud-bigtable. (#5307)
> > > > > > > - [AIRFLOW-4519] Optimise operator classname sorting in views (#5282)
> > > > > > > - [AIRFLOW-4503] Support fully pig options (#5271)
> > > > > > > - [AIRFLOW-4468] add sql_alchemy_max_overflow parameter (#5249)
> > > > > > > - [AIRFLOW-4467] Add dataproc_jars to templated fields in Dataproc oper… (#5248)
> > > > > > > - [AIRFLOW-4381] Use get_direct_relative_ids get task relatives (#5147)
> > > > > > > - [AIRFLOW-3624] Add masterType parameter to MLEngineTrainingOperator (#4428)
> > > > > > > - [AIRFLOW-3143] Support Auto-Zone in DataprocClusterCreateOperator (#5169)
> > > > > > > - [AIRFLOW-3874] Improve BigQueryHook.run_with_configuration's location support (#4695)
> > > > > > > - [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag (#5165)
> > > > > > > - [AIRFLOW-4031] Allow for key pair auth in snowflake hook (#4875)
> > > > > > > - [AIRFLOW-3901] add role as optional config parameter for SnowflakeHook (#4721)
> > > > > > > - [AIRFLOW-3455] add region in snowflake connector (#4285)
> > > > > > > - [AIRFLOW-4073] add template_ext for AWS Athena operator (#4907)
> > > > > > > - [AIRFLOW-4093] AWSAthenaOperator: Throw exception if job failed/cancelled/reach max retries (#4919)
> > > > > > > - [AIRFLOW-4356] Add extra RuntimeEnvironment keys to DataFlowHook (#5149)
> > > > > > > - [AIRFLOW-4337] Fix docker-compose deprecation warning in CI (#5119)
> > > > > > > - [AIRFLOW-3603] QuboleOperator: Remove SQLCommand from SparkCmd documentation (#4411)
> > > > > > > - [AIRFLOW-4328] Fix link to task instances from Pool page (#5124)
> > > > > > > - [AIRFLOW-4255] Make GCS Hook Backwards compatible (#5089)
> > > > > > > - [AIRFLOW-4103] Allow uppercase letters in dataflow job names (#4925)
> > > > > > > - [AIRFLOW-4255] Replace Discovery based api with client based for GCS (#5054)
> > > > > > > - [AIRFLOW-4311] Remove sleep in localexecutor (#5096)
> > > > > > > - [AIRFLOW-2836] Minor improvement-contrib.sensors.FileSensor (#3674)
> > > > > > > - [AIRFLOW-4104] Add type annotations to common classes. (#4926)
> > > > > > > - [AIRFLOW-3910] Raise exception explicitly in Connection.get_hook() (#4728)
> > > > > > > - [AIRFLOW-3322] Update QuboleHook to fetch args dynamically from qds_sdk (#4165)
> > > > > > > - [AIRFLOW-4565] instrument celery executor (#5321)
> > > > > > > - [AIRFLOW-4573] Import airflow_local_settings after prepare_classpath (#5330)
> > > > > > > - [AIRFLOW-4448] Don't bake ENV and _cmd into tmp config for non-sudo (#4050)
> > > > > > > - [AIRFLOW-4295] Make `method` attribute case insensitive in HttpHook (#5313)
> > > > > > > - [AIRFLOW-3703] Add dnsPolicy option for KubernetesPodOperator (#4520)
> > > > > > > - [AIRFLOW-3057] add prev_*_date_success to template context (#5372)
> > > > > > > - [AIRFLOW-4336] Stop showing entire GCS files bytes in log for gcs_download_operator (#5151)
> > > > > > > - [AIRFLOW-4528] Cancel DataProc task on timeout (#5293)
> > > > > > >
> > > > > > > Bug fixes
> > > > > > > """""""""
> > > > > > >
> > > > > > > - [AIRFLOW-4587] Replace self.conn with self.get_conn() in AWSAthenaHook (#5462)
> > > > > > > - [AIRFLOW-4516] K8s runAsUser and fsGroup cannot be strings (#5429)
> > > > > > > - [AIRFLOW-4298] Stop Scheduler repeatedly warning "connection invalidated" (#5470)
> > > > > > > - [AIRFLOW-4559] JenkinsJobTriggerOperator bugfix (#5318)
> > > > > > > - [AIRFLOW-4841] Pin Sphinx AutoApi to 1.0.0 (#5468)
> > > > > > > - [AIRFLOW-4479] Include s3_overwrite kwarg in load_bytes method (#5312)
> > > > > > > - [AIRFLOW-3746] Fix DockerOperator missing container exit (#4583)
> > > > > > > - [AIRFLOW-4233] Remove Template Extension from Bq to GCS Operator (#5456)
> > > > > > > - [AIRFLOW-2141][AIRFLOW-3157][AIRFLOW-4170] Serialize non-str value by JSON when importing Variables (#4991)
> > > > > > > - [AIRFLOW-4826] Remove warning from `airflow resetdb` command (#5447)
> > > > > > > - [AIRFLOW-4148] Fix editing DagRuns when clicking state column (#5436)
> > > > > > > - [AIRFLOW-4455] dag_details broken for subdags in RBAC UI (#5234)
> > > > > > > - [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods (#4551)
> > > > > > > - [AIRFLOW-4459] Fix wrong DAG count in /home page when DAG count is zero (#5235)
> > > > > > > - [AIRFLOW-3876] AttributeError: module 'distutils' has no attribute 'util'
> > > > > > > - [AIRFLOW-4146] Fix CgroupTaskRunner errors (#5224)
> > > > > > > - [AIRFLOW-4524] Fix bug with "Ignore \*" toggles in RBAC mode (#5378)
> > > > > > > - [AIRFLOW-4765] Fix DataProcPigOperator execute method (#5426)
> > > > > > > - [AIRFLOW-4798] obviate interdependencies for dagbag and TI tests (#5422)
> > > > > > > - [AIRFLOW-4800] fix GKEClusterHook ctor calls (#5424)
> > > > > > > - [AIRFLOW-4799] don't mutate self.env in BashOperator execute method (#5421)
> > > > > > > - [AIRFLOW-4393] Add retry logic when fetching pod status and/or logs in KubernetesPodOperator (#5284)
> > > > > > > - [AIRFLOW-4174] Fix HttpHook run with backoff (#5213)
> > > > > > > - [AIRFLOW-4463] Handle divide-by-zero errors in short retry intervals (#5243)
> > > > > > > - [AIRFLOW-2614] Speed up trigger_dag API call when lots of DAGs in system
> > > > > > > - [AIRFLOW-4756] add ti.state to ti.start_date as criteria for gantt (#5399)
> > > > > > > - [AIRFLOW-4760] Fix zip-packaged DAGs disappearing from DagBag when reloaded (#5404)
> > > > > > > - [AIRFLOW-4731] Fix GCS hook with google-storage-client 1.16 (#5368)
> > > > > > > - [AIRFLOW-3506] use match_phrase to query log_id in elasticsearch (#4342)
> > > > > > > - [AIRFLOW-4084] fix ElasticSearch log download (#5177)
> > > > > > > - [AIRFLOW-4501] Register pendulum datetime converter for sqla+pymysql (#5190)
> > > > > > > - [AIRFLOW-986] HiveCliHook ignores 'proxy_user' value in a connection's extra parameter (#5305)
> > > > > > > - [AIRFLOW-4442] fix hive_tblproperties in HiveToDruidTransfer (#5211)
> > > > > > > - [AIRFLOW-4557] Add gcp_conn_id parameter to get_sqlproxy_runner() of CloudSqlDatabaseHook (#5314)
> > > > > > > - [AIRFLOW-4545] Upgrade FAB to latest version (#4955)
> > > > > > > - [AIRFLOW-4492] Change Dataproc Cluster operators to poll Operations (#5269)
> > > > > > > - [AIRFLOW-4452] Webserver and Scheduler keep crashing because of slackclient update (#5225)
> > > > > > > - [AIRFLOW-4450] Fix request arguments in has_dag_access (#5220)
> > > > > > > - [AIRFLOW-4434] Support Impala with the HiveServer2Hook (#5206)
> > > > > > > - [AIRFLOW-3449] Write local dag parsing logs when remote logging enabled. (#5175)
> > > > > > > - [AIRFLOW-4300] Fix graph modal call when DAG has not yet run (#5185)
> > > > > > > - [AIRFLOW-4401] Use managers for Queue synchronization (#5200)
> > > > > > > - [AIRFLOW-3626] Fixed triggering DAGs contained within zip files (#4439)
> > > > > > > - [AIRFLOW-3720] Fix missmatch while comparing GCS and S3 files (#4766)
> > > > > > > - [AIRFLOW-4403] search by `dag_id` or `owners` in UI (#5184)
> > > > > > > - [AIRFLOW-4308] Fix TZ-loop around DST on python 3.6+ (#5095)
> > > > > > > - [AIRFLOW-4324] fix DAG fuzzy search in RBAC UI (#5131)
> > > > > > > - [AIRFLOW-4297] Temporary hot fix on manage_slas() for 1.10.4 release (#5150)
> > > > > > > - [AIRFLOW-4299] Upgrade to Celery 4.3.0 to fix crashing workers (#5116)
> > > > > > > - [AIRFLOW-4291] Correctly render doc_md in DAG graph page (#5121)
> > > > > > > - [AIRFLOW-4310] Fix incorrect link on Dag Details page (#5122)
> > > > > > > - [AIRFLOW-4331] Correct filter for Null-state runs from Dag Detail page (#5123)
> > > > > > > - [AIRFLOW-4294] Fix missing dag & task runs in UI dag_id contains a dot (#5111)
> > > > > > > - [AIRFLOW-4332] Upgrade sqlalchemy to remove security Vulnerability (#5113)
> > > > > > > - [AIRFLOW-4312] Add template_fields & template_ext to BigQueryCheckO… (#5097)
> > > > > > > - [AIRFLOW-4293] Fix downgrade in d4ecb8fbee3_add_schedule_interval_to_dag.py (#5086)
> > > > > > > - [AIRFLOW-4267] Fix TI duration in Graph View (#5071)
> > > > > > > - [AIRFLOW-4163] IntervalCheckOperator supports relative diff and not ignore 0 (#4983)
> > > > > > > - [AIRFLOW-3938] QuboleOperator Fixes and Support for SqlCommand (#4832)
> > > > > > > - [AIRFLOW-2903] Change default owner to `airflow` (#4151)
> > > > > > > - [aIRFLOW-4136] Fix overwrite of key_file by constructor (#5155)
> > > > > > > - [AIRFLOW-3241] Remove Invalid template ext in GCS Sensors (#4076)
> > > > > > >
> > > > > > > Misc/Internal
> > > > > > > """""""""""""
> > > > > > >
> > > > > > > - [AIRFLOW-4211] Add tests for WebHDFSHook (#5015)
> > > > > > > - [AIRFLOW-4320] Add tests for SegmentTrackEventOperator (#5104)
> > > > > > > - [AIRFLOW-4319] Add tests for Bigquery related Operators (#5101)
> > > > > > > - [AIRFLOW-4014] Change DatastoreHook and add tests (#4842)
> > > > > > > - [AIRFLOW-4322] Add test for VerticaOperator (#5107)
> > > > > > > - [AIRFLOW-4323] Add 2 tests for WinRMOperator (#5108)
> > > > > > > - [AIRFLOW-3677] Improve CheckOperator test coverage (#4756)
> > > > > > > - [AIRFLOW-4659] Fix pylint problems for api module (#5398)
> > > > > > > - [AIRFLOW-4358] Speed up test_jobs by not running tasks (#5162)
> > > > > > > - [AIRFLOW-4394] Don't test behaviour of BackfillJob from CLI tests (#5160)
> > > > > > > - [AIRFLOW-3471] Move XCom out of models.py (#4629)
> > > > > > > - [AIRFLOW-4379] Remove duplicate code & Add validation in gcs_to_gcs.py (#5145)
> > > > > > > - [AIRFLOW-4259] Move models out of models.py (#5056)
> > > > > > > - [AIRFLOW-XXX] Speed up building of Cassanda module on Travis (#5233)
> > > > > > > - [AIRFLOW-4535] Break jobs.py into multiple files (#5303)
> > > > > > > - [AIRFLOW-1464] Batch update task_instance state (#5323)
> > > > > > > - [AIRFLOW-4554] Test for sudo command, add some other test docs (#5310)
> > > > > > > - [AIRFLOW-4419] Refine concurrency check in scheduler (#5194)
> > > > > > > - [AIRFLOW-4269] Minor acceleration of jobs._process_task_instances() (#5076)
> > > > > > > - [AIRFLOW-4341] Remove `View.render()` already exists in fab.BaseView (#5125)
> > > > > > > - [AIRFLOW-4342] Use @cached_property instead of re-implementing it each time (#5126)
> > > > > > > - [AIRFLOW-4256] Remove noqa from migrations (#5055)
> > > > > > > - [AIRFLOW-4034] Remove unnecessary string formatting with `**locals()` (#4861)
> > > > > > > - [AIRFLOW-3944] Remove code smells (#4762)
> > > > > > >
> > > > > > > Doc-only changes
> > > > > > > """"""""""""""""
> > > > > > >
> > > > > > > - [AIRFLOW-3341] FAQ return DAG object example (#4605)
> > > > > > > - [AIRFLOW-4433] Add missing type in DockerOperator doc string (#5205)
> > > > > > > - [AIRFLOW-4321] Replace incorrect info of Max Size limit of GCS Object Size (#5106)
> > > > > > > - [AIRFLOW-XXX] Add information about user list (#5341)
> > > > > > > - [AIRFLOW-XXX] Clarify documentation related to autodetect parameter in GCS_to_BQ Op (#5294)
> > > > > > > - [AIRFLOW-XXX] Remove mention of pytz compatibility from timezone documentation (#5316)
> > > > > > > - [AIRFLOW-XXX] Add missing docs for GoogleCloudStorageDeleteOperator (#5274)
> > > > > > > - [AIRFLOW-XXX] Remove incorrect note about Scopes of GCP connection (#5242)
> > > > > > > - [AIRFLOW-XXX] Fix mistakes in docs of Dataproc operators (#5192)
> > > > > > > - [AIRFLOW-XXX] Link to correct class for timedelta in macros.rst (#5226)
> > > > > > > - [AIRFLOW-XXX] Add Kamil as committer (#5216)
> > > > > > > - [AIRFLOW-XXX] Add Joshua and Kevin as committer (#5207)
> > > > > > > - [AIRFLOW-XXX] Reduce log spam in tests (#5174)
> > > > > > > - [AIRFLOW-XXX] Speed up tests for PythonSensor (#5158)
> > > > > > > - [AIRFLOW-XXX] Add Bas Harenslak to committer list (#5157)
> > > > > > > - [AIRFLOW-XXX] Add Jarek Potiuk to commiter list (#5132)
> > > > > > > - [AIRFLOW-XXX] Update docstring for SchedulerJob (#5105)
> > > > > > > - [AIRFLOW-XXX] Fix docstrings for CassandraToGoogleCloudStorageOperator (#5103)
> > > > > > > - [AIRFLOW-XXX] update SlackWebhookHook and SlackWebhookOperator docstring (#5074)
> > > > > > > - [AIRFLOW-XXX] Ignore python files under node_modules in docs (#5063)
> > > > > > > - [AIRFLOW-XXX] Build a universal wheel with LICNESE files (#5052)
> > > > > > > - [AIRFLOW-XXX] Fix docstrings of SQSHook (#5099)
> > > > > > > - [AIRFLOW-XXX] Use Py3.7 on readthedocs
> > > > > > > - [AIRFLOW-4446] Fix typos (#5217)
Re: 1.10.4beta2 "snapshot" for available testing
Posted by Ash Berlin-Taylor <as...@apache.org>.
Are you sure you are on 1.10.4b2? The stack trace points at line 113 which in 1.10.4b2 is an empty line https://github.com/apache/airflow/blob/1.10.4b2/airflow/www/views.py#L113 <https://github.com/apache/airflow/blob/1.10.4b2/airflow/www/views.py#L113>
The stack trace matches 1.10.3 though https://github.com/apache/airflow/blob/1.10.3/airflow/www/views.py#L113
> On 26 Jun 2019, at 11:34, airflowuser <ai...@protonmail.com.INVALID> wrote:
>
> http://localhost:8080/admin/dagmodel/
>
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Wednesday, June 26, 2019 1:32 PM, Ash Berlin-Taylor <as...@apache.org> wrote:
>
>> What URL does that end up on? I can't reproduce that.
>>
>> -a
>>
>>> On 26 Jun 2019, at 10:52, airflowuser airflowuser@protonmail.com.INVALID wrote:
>>> Bug:
>>> On the old UI:
>>>
>>> 1. Click on edit DAG
>>> 2. Enter Description
>>> 3.Click Save
>>>
>>>
>>> You'll get error:
>>> Traceback (most recent call last):
>>> File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2311, in wsgi_app
>>> response = self.full_dispatch_request()
>>> ....
>>> packages/flask_admin/model/base.py", line 1861, in get_list_value
>>> self.column_type_formatters,
>>> File "/usr/local/lib/python3.6/site-packages/flask_admin/model/base.py", line 1826, in _get_list_value
>>> value = column_fmt(self, context, model, name)
>>> File "/usr/local/lib/python3.6/site-packages/airflow/www/views.py", line 113, in dag_link
>>> execution_date=m.execution_date)
>>> AttributeError: 'DagModel' object has no attribute 'execution_date'
>>> Didn't happen to me on 1.10.3
>>> Not really sure why execution_date is relevant for when you want to edit details on the DAG that is unrelated to a specific DAG run.
>>> Sent with ProtonMail Secure Email.
>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>> On Wednesday, June 26, 2019 11:09 AM, Ash Berlin-Taylor ash@apache.org wrote:
>>>
>>>> Because the list was autogenerated from the issue type in the jira ticket, and I didn't check all 200+ items in the change log :)
>>>> -a
>>>>
>>>>> On 25 Jun 2019, at 18:59, airflowuser airflowuser@protonmail.com.INVALID wrote:
>>>>> Why [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365) listed under features?
>>>>> Sent with ProtonMail Secure Email.
>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>>> On Tuesday, June 25, 2019 1:40 PM, Ash Berlin-Taylor ash@apache.org wrote:
>>>>>
>>>>>> Hi Everyone,
>>>>>> I'm excited to annoucne that 1.10.4 is almost ready for release.
>>>>>> The release branch already contains 216 commits since 1.10.3 this time around.
>>>>>> In order to distinguish it from an actual (apache) release it is:
>>>>>>
>>>>>> 1. Marked as beta (python package managers do not install beta versions by default - PEP 440)
>>>>>> 2. It is not at an official Apache distribution location
>>>>>>
>>>>>> It can be installed with:
>>>>>> pip install 'apache-airflow==1.10.4b2'
>>>>>> or downloaded from https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/ https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/
>>>>>> (Don't worry, without asking for`--pre` or specifying the version `pip install apache-airflow` will still get 1.10.3)
>>>>>> Included below is the changelog of this release so far. I'd like to try and get this out before the end of the month, so will create an RC in a day or two baring any major problems reported.
>>>>>> Cheers,
>>>>>> Ash
>>>>>> Airflow 1.10.4
>>>>>> New Features
>>>>>> """"""""""""
>>>>>>
>>>>>> - [AIRFLOW-4456] Add sub-classable BaseBranchOperator (#5231)
>>>>>> - [AIRFLOW-4343] Show warning in UI if scheduler is not running (#5127)
>>>>>> - [AIRFLOW-4739] Add ability to arbitrarily define kubernetes worker pod labels (#5376)
>>>>>> - [AIRFLOW-4348] Add GCP console link in BigQueryOperator (#5195)
>>>>>> - [AIRFLOW-4306] Global operator extra links (#5094)
>>>>>> - [AIRFLOW-4812] Add batch images annotation (#5433)
>>>>>> - [AIRFLOW-4135] Add Google Cloud Build operator and hook (#5251)
>>>>>> - [AIRFLOW-4781] Add the ability to specify ports in kubernetesOperator (#5410)
>>>>>> - [AIRFLOW-4521] Pause dag also pause its subdags (#5283)
>>>>>> - [AIRFLOW-4738] Enforce exampleinclude for example DAGs (#5375)
>>>>>> - [AIRFLOW-4326] Airflow AWS SQS Operator (#5110)
>>>>>> - [AIRFLOW-3729] Support "DownwardAPI" in env variables for KubernetesPodOperator (#4554)
>>>>>> - [AIRFLOW-4585] Implement Kubernetes Pod Mutation Hook (#5359)
>>>>>> - [AIRFLOW-161] New redirect route and extra links (#5059)
>>>>>> - [AIRFLOW-4420] Backfill respects task_concurrency (#5221)
>>>>>> - [AIRFLOW-4147] Add Operator to publish event to Redis (#4967)
>>>>>> - [AIRFLOW-3359] Add option to pass customer encryption keys to Dataproc (#4200)
>>>>>> - [AIRFLOW-4318] Create Google Cloud Translate Speech Operator (#5102)
>>>>>> - [AIRFLOW-3960] Adds Google Cloud Speech operators (#4780)
>>>>>> - [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365)
>>>>>> - [AIRFLOW-1501] Add GoogleCloudStorageDeleteOperator (#5230)
>>>>>> - [AIRFLOW-3672] Add support for Mongo DB DNS Seedlist Connection Format (#4481)
>>>>>> - [AIRFLOW-4397] add integrations docs manually for gcs sensors (#5204)
>>>>>> - [AIRFLOW-4251] Instrument DagRun schedule delay (#5050)
>>>>>> - [AIRFLOW-4118] instrument DagRun duration (#4946)
>>>>>> - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5182)
>>>>>> - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5140)
>>>>>> - [AIRFLOW-4168] Create Google Cloud Video Intelligence Operators (#4985)
>>>>>> - [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor (#5166)
>>>>>> - [AIRFLOW-4335] Add default num_retries to GCP connection (#5117)
>>>>>> - [AIRFLOW-3808] Add cluster_fields to BigQueryHook's create_empty_table (#4654)
>>>>>> - [AIRFLOW-4362] Fix test_execution_limited_parallelism (#5141)
>>>>>> - [AIRFLOW-4307] Backfill respects concurrency limit (#5128)
>>>>>> - [AIRFLOW-4268] Add MsSqlToGoogleCloudStorageOperator (#5077)
>>>>>> - [AIRFLOW-4169] Add Google Cloud Vision Detect Operators (#4986)
>>>>>> - [AIRFLOW-XXX] Fix WS-2019-0032 (#5384)
>>>>>> - [AIRFLOW-XXX] Fix CVE-2019-11358 (#5197)
>>>>>> - [AIRFLOW-XXX] Change allowed version of Jinja2 to fix CVE-2019-10906 (#5075)
>>>>>>
>>>>>> Improvement
>>>>>> """""""""""
>>>>>>
>>>>>> - [AIRFLOW-4805] Add py_file as templated field in DataflowPythonOperator (#5451)
>>>>>> - [AIRFLOW-4838] Surface Athena errors in AWSAthenaOperator (#5467)
>>>>>> - [AIRFLOW-4831] conf.has_option no longer throws if section is missing. (#5455)
>>>>>> - [AIRFLOW-4829] More descriptive exceptions for EMR sensors (#5452)
>>>>>> - [AIRFLOW-4414] AWSAthenaOperator: Push QueryExecutionID to XCom (#5276)
>>>>>> - [AIRFLOW-4791] add "schema" keyword arg to SnowflakeOperator (#5415)
>>>>>> - [AIRFLOW-4759] Don't error when marking sucessful run as failed (#5435)
>>>>>> - [AIRFLOW-4716] Instrument dag loading time duration (#5350)
>>>>>> - [AIRFLOW-3958] Support list tasks as upstream in chain (#4779)
>>>>>> - [AIRFLOW-4409] Prevent task duration break by null value (#5178)
>>>>>> - [AIRFLOW-4418] Add "failed only" option to task modal (#5193)
>>>>>> - [AIRFLOW-4740] Accept string `end_date` in DAG default_args (#5381)
>>>>>> - [AIRFLOW-4423] Improve date handling in mysql to gcs operator. (#5196)
>>>>>> - [AIRFLOW-4447] Display task duration as human friendly format in UI (#5218)
>>>>>> - [AIRFLOW-4377] Remove needless object conversion in DAG.owner() (#5144)
>>>>>> - [AIRFLOW-4766] Add autoscaling option for DataprocClusterCreateOperator (#5425)
>>>>>> - [AIRFLOW-4795] Upgrade alembic to latest release. (#5411)
>>>>>> - [AIRFLOW-4793] Add signature_name to mlengine operator (#5417)
>>>>>> - [AIRFLOW-3211] Reattach to GCP Dataproc jobs upon Airflow restart (#4083)
>>>>>> - [AIRFLOW-4750] Log identified zombie task instances (#5389)
>>>>>> - [AIRFLOW-3870] STFPOperator: Update log level and return value (#4355)
>>>>>> - [AIRFLOW-4759] Batch queries in set_state API. (#5403)
>>>>>> - [AIRFLOW-2737] Restore original license header to airflow.api.auth.backend.kerberos_auth
>>>>>> - [AIRFLOW-3635] Fix incorrect logic in detele_dag (introduced in PR#4406) (#4445)
>>>>>> - [AIRFLOW-3599] Removed Dagbag from delete dag (#4406)
>>>>>> - [AIRFLOW-4737] Increase and document celery queue name limit (#5383)
>>>>>> - [AIRFLOW-4505] Correct Tag ALL for PY3 (#5275)
>>>>>> - [AIRFLOW-4743] Add environment variables support to SSHOperator (#5385)
>>>>>> - [AIRFLOW-4725] Fix setup.py PEP440 & Sphinx-PyPI-upload dependency (#5363)
>>>>>> - [AIRFLOW-3370] Add stdout output options to Elasticsearch task log handler (#5048)
>>>>>> - [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. (#5164)
>>>>>> - [AIRFLOW-1381] Allow setting host temporary directory in DockerOperator (#5369)
>>>>>> - [AIRFLOW-4598] Task retries are not exhausted for K8s executor (#5347)
>>>>>> - [AIRFLOW-4218] Support to Provide http args to K8executor while calling k8 python client lib apis (#5060)
>>>>>> - [AIRFLOW-4159] Add support for additional static pod labels for K8sExecutor (#5134)
>>>>>> - [AIRFLOW-4720] Allow comments in .airflowignore files. (#5355)
>>>>>> - [AIRFLOW-4486] Add AWS IAM authenication in MySqlHook (#5334)
>>>>>> - [AIRFLOW-4417] Add AWS IAM authenication for PostgresHook (#5223)
>>>>>> - [AIRFLOW-3990] Compile regular expressions. (#4813)
>>>>>> - [AIRFLOW-4572] Rename prepare_classpath() to prepare_syspath() (#5328)
>>>>>> - [AIRFLOW-3869] Raise consistent exception in AirflowConfigParser.getboolean (#4692)
>>>>>> - [AIRFLOW-4571] Add headers to templated field for SimpleHttpOperator (#5326)
>>>>>> - [AIRFLOW-3867] Rename GCP's subpackage (#4690)
>>>>>> - [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df (#4549)
>>>>>> - [AIRFLOW-4546] Upgrade google-cloud-bigtable. (#5307)
>>>>>> - [AIRFLOW-4519] Optimise operator classname sorting in views (#5282)
>>>>>> - [AIRFLOW-4503] Support fully pig options (#5271)
>>>>>> - [AIRFLOW-4468] add sql_alchemy_max_overflow parameter (#5249)
>>>>>> - [AIRFLOW-4467] Add dataproc_jars to templated fields in Dataproc oper… (#5248)
>>>>>> - [AIRFLOW-4381] Use get_direct_relative_ids get task relatives (#5147)
>>>>>> - [AIRFLOW-3624] Add masterType parameter to MLEngineTrainingOperator (#4428)
>>>>>> - [AIRFLOW-3143] Support Auto-Zone in DataprocClusterCreateOperator (#5169)
>>>>>> - [AIRFLOW-3874] Improve BigQueryHook.run_with_configuration's location support (#4695)
>>>>>> - [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag (#5165)
>>>>>> - [AIRFLOW-4031] Allow for key pair auth in snowflake hook (#4875)
>>>>>> - [AIRFLOW-3901] add role as optional config parameter for SnowflakeHook (#4721)
>>>>>> - [AIRFLOW-3455] add region in snowflake connector (#4285)
>>>>>> - [AIRFLOW-4073] add template_ext for AWS Athena operator (#4907)
>>>>>> - [AIRFLOW-4093] AWSAthenaOperator: Throw exception if job failed/cancelled/reach max retries (#4919)
>>>>>> - [AIRFLOW-4356] Add extra RuntimeEnvironment keys to DataFlowHook (#5149)
>>>>>> - [AIRFLOW-4337] Fix docker-compose deprecation warning in CI (#5119)
>>>>>> - [AIRFLOW-3603] QuboleOperator: Remove SQLCommand from SparkCmd documentation (#4411)
>>>>>> - [AIRFLOW-4328] Fix link to task instances from Pool page (#5124)
>>>>>> - [AIRFLOW-4255] Make GCS Hook Backwards compatible (#5089)
>>>>>> - [AIRFLOW-4103] Allow uppercase letters in dataflow job names (#4925)
>>>>>> - [AIRFLOW-4255] Replace Discovery based api with client based for GCS (#5054)
>>>>>> - [AIRFLOW-4311] Remove sleep in localexecutor (#5096)
>>>>>> - [AIRFLOW-2836] Minor improvement-contrib.sensors.FileSensor (#3674)
>>>>>> - [AIRFLOW-4104] Add type annotations to common classes. (#4926)
>>>>>> - [AIRFLOW-3910] Raise exception explicitly in Connection.get_hook() (#4728)
>>>>>> - [AIRFLOW-3322] Update QuboleHook to fetch args dynamically from qds_sdk (#4165)
>>>>>> - [AIRFLOW-4565] instrument celery executor (#5321)
>>>>>> - [AIRFLOW-4573] Import airflow_local_settings after prepare_classpath (#5330)
>>>>>> - [AIRFLOW-4448] Don't bake ENV and _cmd into tmp config for non-sudo (#4050)
>>>>>> - [AIRFLOW-4295] Make `method` attribute case insensitive in HttpHook (#5313)
>>>>>> - [AIRFLOW-3703] Add dnsPolicy option for KubernetesPodOperator (#4520)
>>>>>> - [AIRFLOW-3057] add prev_*_date_success to template context (#5372)
>>>>>> - [AIRFLOW-4336] Stop showing entire GCS files bytes in log for gcs_download_operator (#5151)
>>>>>> - [AIRFLOW-4528] Cancel DataProc task on timeout (#5293)
>>>>>>
>>>>>> Bug fixes
>>>>>> """""""""
>>>>>>
>>>>>> - [AIRFLOW-4587] Replace self.conn with self.get_conn() in AWSAthenaHook (#5462)
>>>>>> - [AIRFLOW-4516] K8s runAsUser and fsGroup cannot be strings (#5429)
>>>>>> - [AIRFLOW-4298] Stop Scheduler repeatedly warning "connection invalidated" (#5470)
>>>>>> - [AIRFLOW-4559] JenkinsJobTriggerOperator bugfix (#5318)
>>>>>> - [AIRFLOW-4841] Pin Sphinx AutoApi to 1.0.0 (#5468)
>>>>>> - [AIRFLOW-4479] Include s3_overwrite kwarg in load_bytes method (#5312)
>>>>>> - [AIRFLOW-3746] Fix DockerOperator missing container exit (#4583)
>>>>>> - [AIRFLOW-4233] Remove Template Extension from Bq to GCS Operator (#5456)
>>>>>> - [AIRFLOW-2141][AIRFLOW-3157][AIRFLOW-4170] Serialize non-str value by JSON when importing Variables (#4991)
>>>>>> - [AIRFLOW-4826] Remove warning from `airflow resetdb` command (#5447)
>>>>>> - [AIRFLOW-4148] Fix editing DagRuns when clicking state column (#5436)
>>>>>> - [AIRFLOW-4455] dag_details broken for subdags in RBAC UI (#5234)
>>>>>> - [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods (#4551)
>>>>>> - [AIRFLOW-4459] Fix wrong DAG count in /home page when DAG count is zero (#5235)
>>>>>> - [AIRFLOW-3876] AttributeError: module 'distutils' has no attribute 'util'
>>>>>> - [AIRFLOW-4146] Fix CgroupTaskRunner errors (#5224)
>>>>>> - [AIRFLOW-4524] Fix bug with "Ignore \*" toggles in RBAC mode (#5378)
>>>>>> - [AIRFLOW-4765] Fix DataProcPigOperator execute method (#5426)
>>>>>> - [AIRFLOW-4798] obviate interdependencies for dagbag and TI tests (#5422)
>>>>>> - [AIRFLOW-4800] fix GKEClusterHook ctor calls (#5424)
>>>>>> - [AIRFLOW-4799] don't mutate self.env in BashOperator execute method (#5421)
>>>>>> - [AIRFLOW-4393] Add retry logic when fetching pod status and/or logs in KubernetesPodOperator (#5284)
>>>>>> - [AIRFLOW-4174] Fix HttpHook run with backoff (#5213)
>>>>>> - [AIRFLOW-4463] Handle divide-by-zero errors in short retry intervals (#5243)
>>>>>> - [AIRFLOW-2614] Speed up trigger_dag API call when lots of DAGs in system
>>>>>> - [AIRFLOW-4756] add ti.state to ti.start_date as criteria for gantt (#5399)
>>>>>> - [AIRFLOW-4760] Fix zip-packaged DAGs disappearing from DagBag when reloaded (#5404)
>>>>>> - [AIRFLOW-4731] Fix GCS hook with google-storage-client 1.16 (#5368)
>>>>>> - [AIRFLOW-3506] use match_phrase to query log_id in elasticsearch (#4342)
>>>>>> - [AIRFLOW-4084] fix ElasticSearch log download (#5177)
>>>>>> - [AIRFLOW-4501] Register pendulum datetime converter for sqla+pymysql (#5190)
>>>>>> - [AIRFLOW-986] HiveCliHook ignores 'proxy_user' value in a connection's extra parameter (#5305)
>>>>>> - [AIRFLOW-4442] fix hive_tblproperties in HiveToDruidTransfer (#5211)
>>>>>> - [AIRFLOW-4557] Add gcp_conn_id parameter to get_sqlproxy_runner() of CloudSqlDatabaseHook (#5314)
>>>>>> - [AIRFLOW-4545] Upgrade FAB to latest version (#4955)
>>>>>> - [AIRFLOW-4492] Change Dataproc Cluster operators to poll Operations (#5269)
>>>>>> - [AIRFLOW-4452] Webserver and Scheduler keep crashing because of slackclient update (#5225)
>>>>>> - [AIRFLOW-4450] Fix request arguments in has_dag_access (#5220)
>>>>>> - [AIRFLOW-4434] Support Impala with the HiveServer2Hook (#5206)
>>>>>> - [AIRFLOW-3449] Write local dag parsing logs when remote logging enabled. (#5175)
>>>>>> - [AIRFLOW-4300] Fix graph modal call when DAG has not yet run (#5185)
>>>>>> - [AIRFLOW-4401] Use managers for Queue synchronization (#5200)
>>>>>> - [AIRFLOW-3626] Fixed triggering DAGs contained within zip files (#4439)
>>>>>> - [AIRFLOW-3720] Fix missmatch while comparing GCS and S3 files (#4766)
>>>>>> - [AIRFLOW-4403] search by `dag_id` or `owners` in UI (#5184)
>>>>>> - [AIRFLOW-4308] Fix TZ-loop around DST on python 3.6+ (#5095)
>>>>>> - [AIRFLOW-4324] fix DAG fuzzy search in RBAC UI (#5131)
>>>>>> - [AIRFLOW-4297] Temporary hot fix on manage_slas() for 1.10.4 release (#5150)
>>>>>> - [AIRFLOW-4299] Upgrade to Celery 4.3.0 to fix crashing workers (#5116)
>>>>>> - [AIRFLOW-4291] Correctly render doc_md in DAG graph page (#5121)
>>>>>> - [AIRFLOW-4310] Fix incorrect link on Dag Details page (#5122)
>>>>>> - [AIRFLOW-4331] Correct filter for Null-state runs from Dag Detail page (#5123)
>>>>>> - [AIRFLOW-4294] Fix missing dag & task runs in UI dag_id contains a dot (#5111)
>>>>>> - [AIRFLOW-4332] Upgrade sqlalchemy to remove security Vulnerability (#5113)
>>>>>> - [AIRFLOW-4312] Add template_fields & template_ext to BigQueryCheckO… (#5097)
>>>>>> - [AIRFLOW-4293] Fix downgrade in d4ecb8fbee3_add_schedule_interval_to_dag.py (#5086)
>>>>>> - [AIRFLOW-4267] Fix TI duration in Graph View (#5071)
>>>>>> - [AIRFLOW-4163] IntervalCheckOperator supports relative diff and not ignore 0 (#4983)
>>>>>> - [AIRFLOW-3938] QuboleOperator Fixes and Support for SqlCommand (#4832)
>>>>>> - [AIRFLOW-2903] Change default owner to `airflow` (#4151)
>>>>>> - [aIRFLOW-4136] Fix overwrite of key_file by constructor (#5155)
>>>>>> - [AIRFLOW-3241] Remove Invalid template ext in GCS Sensors (#4076)
>>>>>>
>>>>>> Misc/Internal
>>>>>> """""""""""""
>>>>>>
>>>>>> - [AIRFLOW-4211] Add tests for WebHDFSHook (#5015)
>>>>>> - [AIRFLOW-4320] Add tests for SegmentTrackEventOperator (#5104)
>>>>>> - [AIRFLOW-4319] Add tests for Bigquery related Operators (#5101)
>>>>>> - [AIRFLOW-4014] Change DatastoreHook and add tests (#4842)
>>>>>> - [AIRFLOW-4322] Add test for VerticaOperator (#5107)
>>>>>> - [AIRFLOW-4323] Add 2 tests for WinRMOperator (#5108)
>>>>>> - [AIRFLOW-3677] Improve CheckOperator test coverage (#4756)
>>>>>> - [AIRFLOW-4659] Fix pylint problems for api module (#5398)
>>>>>> - [AIRFLOW-4358] Speed up test_jobs by not running tasks (#5162)
>>>>>> - [AIRFLOW-4394] Don't test behaviour of BackfillJob from CLI tests (#5160)
>>>>>> - [AIRFLOW-3471] Move XCom out of models.py (#4629)
>>>>>> - [AIRFLOW-4379] Remove duplicate code & Add validation in gcs_to_gcs.py (#5145)
>>>>>> - [AIRFLOW-4259] Move models out of models.py (#5056)
>>>>>> - [AIRFLOW-XXX] Speed up building of Cassanda module on Travis (#5233)
>>>>>> - [AIRFLOW-4535] Break jobs.py into multiple files (#5303)
>>>>>> - [AIRFLOW-1464] Batch update task_instance state (#5323)
>>>>>> - [AIRFLOW-4554] Test for sudo command, add some other test docs (#5310)
>>>>>> - [AIRFLOW-4419] Refine concurrency check in scheduler (#5194)
>>>>>> - [AIRFLOW-4269] Minor acceleration of jobs._process_task_instances() (#5076)
>>>>>> - [AIRFLOW-4341] Remove `View.render()` already exists in fab.BaseView (#5125)
>>>>>> - [AIRFLOW-4342] Use @cached_property instead of re-implementing it each time (#5126)
>>>>>> - [AIRFLOW-4256] Remove noqa from migrations (#5055)
>>>>>> - [AIRFLOW-4034] Remove unnecessary string formatting with `**locals()` (#4861)
>>>>>> - [AIRFLOW-3944] Remove code smells (#4762)
>>>>>>
>>>>>> Doc-only changes
>>>>>> """"""""""""""""
>>>>>>
>>>>>> - [AIRFLOW-3341] FAQ return DAG object example (#4605)
>>>>>> - [AIRFLOW-4433] Add missing type in DockerOperator doc string (#5205)
>>>>>> - [AIRFLOW-4321] Replace incorrect info of Max Size limit of GCS Object Size (#5106)
>>>>>> - [AIRFLOW-XXX] Add information about user list (#5341)
>>>>>> - [AIRFLOW-XXX] Clarify documentation related to autodetect parameter in GCS_to_BQ Op (#5294)
>>>>>> - [AIRFLOW-XXX] Remove mention of pytz compatibility from timezone documentation (#5316)
>>>>>> - [AIRFLOW-XXX] Add missing docs for GoogleCloudStorageDeleteOperator (#5274)
>>>>>> - [AIRFLOW-XXX] Remove incorrect note about Scopes of GCP connection (#5242)
>>>>>> - [AIRFLOW-XXX] Fix mistakes in docs of Dataproc operators (#5192)
>>>>>> - [AIRFLOW-XXX] Link to correct class for timedelta in macros.rst (#5226)
>>>>>> - [AIRFLOW-XXX] Add Kamil as committer (#5216)
>>>>>> - [AIRFLOW-XXX] Add Joshua and Kevin as committer (#5207)
>>>>>> - [AIRFLOW-XXX] Reduce log spam in tests (#5174)
>>>>>> - [AIRFLOW-XXX] Speed up tests for PythonSensor (#5158)
>>>>>> - [AIRFLOW-XXX] Add Bas Harenslak to committer list (#5157)
>>>>>> - [AIRFLOW-XXX] Add Jarek Potiuk to commiter list (#5132)
>>>>>> - [AIRFLOW-XXX] Update docstring for SchedulerJob (#5105)
>>>>>> - [AIRFLOW-XXX] Fix docstrings for CassandraToGoogleCloudStorageOperator (#5103)
>>>>>> - [AIRFLOW-XXX] update SlackWebhookHook and SlackWebhookOperator docstring (#5074)
>>>>>> - [AIRFLOW-XXX] Ignore python files under node_modules in docs (#5063)
>>>>>> - [AIRFLOW-XXX] Build a universal wheel with LICNESE files (#5052)
>>>>>> - [AIRFLOW-XXX] Fix docstrings of SQSHook (#5099)
>>>>>> - [AIRFLOW-XXX] Use Py3.7 on readthedocs
>>>>>> - [AIRFLOW-4446] Fix typos (#5217)
>
>
Re: 1.10.4beta2 "snapshot" for available testing
Posted by airflowuser <ai...@protonmail.com.INVALID>.
http://localhost:8080/admin/dagmodel/
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, June 26, 2019 1:32 PM, Ash Berlin-Taylor <as...@apache.org> wrote:
> What URL does that end up on? I can't reproduce that.
>
> -a
>
> > On 26 Jun 2019, at 10:52, airflowuser airflowuser@protonmail.com.INVALID wrote:
> > Bug:
> > On the old UI:
> >
> > 1. Click on edit DAG
> > 2. Enter Description
> > 3.Click Save
> >
> >
> > You'll get error:
> > Traceback (most recent call last):
> > File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2311, in wsgi_app
> > response = self.full_dispatch_request()
> > ....
> > packages/flask_admin/model/base.py", line 1861, in get_list_value
> > self.column_type_formatters,
> > File "/usr/local/lib/python3.6/site-packages/flask_admin/model/base.py", line 1826, in _get_list_value
> > value = column_fmt(self, context, model, name)
> > File "/usr/local/lib/python3.6/site-packages/airflow/www/views.py", line 113, in dag_link
> > execution_date=m.execution_date)
> > AttributeError: 'DagModel' object has no attribute 'execution_date'
> > Didn't happen to me on 1.10.3
> > Not really sure why execution_date is relevant for when you want to edit details on the DAG that is unrelated to a specific DAG run.
> > Sent with ProtonMail Secure Email.
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Wednesday, June 26, 2019 11:09 AM, Ash Berlin-Taylor ash@apache.org wrote:
> >
> > > Because the list was autogenerated from the issue type in the jira ticket, and I didn't check all 200+ items in the change log :)
> > > -a
> > >
> > > > On 25 Jun 2019, at 18:59, airflowuser airflowuser@protonmail.com.INVALID wrote:
> > > > Why [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365) listed under features?
> > > > Sent with ProtonMail Secure Email.
> > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > On Tuesday, June 25, 2019 1:40 PM, Ash Berlin-Taylor ash@apache.org wrote:
> > > >
> > > > > Hi Everyone,
> > > > > I'm excited to annoucne that 1.10.4 is almost ready for release.
> > > > > The release branch already contains 216 commits since 1.10.3 this time around.
> > > > > In order to distinguish it from an actual (apache) release it is:
> > > > >
> > > > > 1. Marked as beta (python package managers do not install beta versions by default - PEP 440)
> > > > > 2. It is not at an official Apache distribution location
> > > > >
> > > > > It can be installed with:
> > > > > pip install 'apache-airflow==1.10.4b2'
> > > > > or downloaded from https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/ https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/
> > > > > (Don't worry, without asking for`--pre` or specifying the version `pip install apache-airflow` will still get 1.10.3)
> > > > > Included below is the changelog of this release so far. I'd like to try and get this out before the end of the month, so will create an RC in a day or two baring any major problems reported.
> > > > > Cheers,
> > > > > Ash
> > > > > Airflow 1.10.4
> > > > > New Features
> > > > > """"""""""""
> > > > >
> > > > > - [AIRFLOW-4456] Add sub-classable BaseBranchOperator (#5231)
> > > > > - [AIRFLOW-4343] Show warning in UI if scheduler is not running (#5127)
> > > > > - [AIRFLOW-4739] Add ability to arbitrarily define kubernetes worker pod labels (#5376)
> > > > > - [AIRFLOW-4348] Add GCP console link in BigQueryOperator (#5195)
> > > > > - [AIRFLOW-4306] Global operator extra links (#5094)
> > > > > - [AIRFLOW-4812] Add batch images annotation (#5433)
> > > > > - [AIRFLOW-4135] Add Google Cloud Build operator and hook (#5251)
> > > > > - [AIRFLOW-4781] Add the ability to specify ports in kubernetesOperator (#5410)
> > > > > - [AIRFLOW-4521] Pause dag also pause its subdags (#5283)
> > > > > - [AIRFLOW-4738] Enforce exampleinclude for example DAGs (#5375)
> > > > > - [AIRFLOW-4326] Airflow AWS SQS Operator (#5110)
> > > > > - [AIRFLOW-3729] Support "DownwardAPI" in env variables for KubernetesPodOperator (#4554)
> > > > > - [AIRFLOW-4585] Implement Kubernetes Pod Mutation Hook (#5359)
> > > > > - [AIRFLOW-161] New redirect route and extra links (#5059)
> > > > > - [AIRFLOW-4420] Backfill respects task_concurrency (#5221)
> > > > > - [AIRFLOW-4147] Add Operator to publish event to Redis (#4967)
> > > > > - [AIRFLOW-3359] Add option to pass customer encryption keys to Dataproc (#4200)
> > > > > - [AIRFLOW-4318] Create Google Cloud Translate Speech Operator (#5102)
> > > > > - [AIRFLOW-3960] Adds Google Cloud Speech operators (#4780)
> > > > > - [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365)
> > > > > - [AIRFLOW-1501] Add GoogleCloudStorageDeleteOperator (#5230)
> > > > > - [AIRFLOW-3672] Add support for Mongo DB DNS Seedlist Connection Format (#4481)
> > > > > - [AIRFLOW-4397] add integrations docs manually for gcs sensors (#5204)
> > > > > - [AIRFLOW-4251] Instrument DagRun schedule delay (#5050)
> > > > > - [AIRFLOW-4118] instrument DagRun duration (#4946)
> > > > > - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5182)
> > > > > - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5140)
> > > > > - [AIRFLOW-4168] Create Google Cloud Video Intelligence Operators (#4985)
> > > > > - [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor (#5166)
> > > > > - [AIRFLOW-4335] Add default num_retries to GCP connection (#5117)
> > > > > - [AIRFLOW-3808] Add cluster_fields to BigQueryHook's create_empty_table (#4654)
> > > > > - [AIRFLOW-4362] Fix test_execution_limited_parallelism (#5141)
> > > > > - [AIRFLOW-4307] Backfill respects concurrency limit (#5128)
> > > > > - [AIRFLOW-4268] Add MsSqlToGoogleCloudStorageOperator (#5077)
> > > > > - [AIRFLOW-4169] Add Google Cloud Vision Detect Operators (#4986)
> > > > > - [AIRFLOW-XXX] Fix WS-2019-0032 (#5384)
> > > > > - [AIRFLOW-XXX] Fix CVE-2019-11358 (#5197)
> > > > > - [AIRFLOW-XXX] Change allowed version of Jinja2 to fix CVE-2019-10906 (#5075)
> > > > >
> > > > > Improvement
> > > > > """""""""""
> > > > >
> > > > > - [AIRFLOW-4805] Add py_file as templated field in DataflowPythonOperator (#5451)
> > > > > - [AIRFLOW-4838] Surface Athena errors in AWSAthenaOperator (#5467)
> > > > > - [AIRFLOW-4831] conf.has_option no longer throws if section is missing. (#5455)
> > > > > - [AIRFLOW-4829] More descriptive exceptions for EMR sensors (#5452)
> > > > > - [AIRFLOW-4414] AWSAthenaOperator: Push QueryExecutionID to XCom (#5276)
> > > > > - [AIRFLOW-4791] add "schema" keyword arg to SnowflakeOperator (#5415)
> > > > > - [AIRFLOW-4759] Don't error when marking sucessful run as failed (#5435)
> > > > > - [AIRFLOW-4716] Instrument dag loading time duration (#5350)
> > > > > - [AIRFLOW-3958] Support list tasks as upstream in chain (#4779)
> > > > > - [AIRFLOW-4409] Prevent task duration break by null value (#5178)
> > > > > - [AIRFLOW-4418] Add "failed only" option to task modal (#5193)
> > > > > - [AIRFLOW-4740] Accept string `end_date` in DAG default_args (#5381)
> > > > > - [AIRFLOW-4423] Improve date handling in mysql to gcs operator. (#5196)
> > > > > - [AIRFLOW-4447] Display task duration as human friendly format in UI (#5218)
> > > > > - [AIRFLOW-4377] Remove needless object conversion in DAG.owner() (#5144)
> > > > > - [AIRFLOW-4766] Add autoscaling option for DataprocClusterCreateOperator (#5425)
> > > > > - [AIRFLOW-4795] Upgrade alembic to latest release. (#5411)
> > > > > - [AIRFLOW-4793] Add signature_name to mlengine operator (#5417)
> > > > > - [AIRFLOW-3211] Reattach to GCP Dataproc jobs upon Airflow restart (#4083)
> > > > > - [AIRFLOW-4750] Log identified zombie task instances (#5389)
> > > > > - [AIRFLOW-3870] STFPOperator: Update log level and return value (#4355)
> > > > > - [AIRFLOW-4759] Batch queries in set_state API. (#5403)
> > > > > - [AIRFLOW-2737] Restore original license header to airflow.api.auth.backend.kerberos_auth
> > > > > - [AIRFLOW-3635] Fix incorrect logic in detele_dag (introduced in PR#4406) (#4445)
> > > > > - [AIRFLOW-3599] Removed Dagbag from delete dag (#4406)
> > > > > - [AIRFLOW-4737] Increase and document celery queue name limit (#5383)
> > > > > - [AIRFLOW-4505] Correct Tag ALL for PY3 (#5275)
> > > > > - [AIRFLOW-4743] Add environment variables support to SSHOperator (#5385)
> > > > > - [AIRFLOW-4725] Fix setup.py PEP440 & Sphinx-PyPI-upload dependency (#5363)
> > > > > - [AIRFLOW-3370] Add stdout output options to Elasticsearch task log handler (#5048)
> > > > > - [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. (#5164)
> > > > > - [AIRFLOW-1381] Allow setting host temporary directory in DockerOperator (#5369)
> > > > > - [AIRFLOW-4598] Task retries are not exhausted for K8s executor (#5347)
> > > > > - [AIRFLOW-4218] Support to Provide http args to K8executor while calling k8 python client lib apis (#5060)
> > > > > - [AIRFLOW-4159] Add support for additional static pod labels for K8sExecutor (#5134)
> > > > > - [AIRFLOW-4720] Allow comments in .airflowignore files. (#5355)
> > > > > - [AIRFLOW-4486] Add AWS IAM authenication in MySqlHook (#5334)
> > > > > - [AIRFLOW-4417] Add AWS IAM authenication for PostgresHook (#5223)
> > > > > - [AIRFLOW-3990] Compile regular expressions. (#4813)
> > > > > - [AIRFLOW-4572] Rename prepare_classpath() to prepare_syspath() (#5328)
> > > > > - [AIRFLOW-3869] Raise consistent exception in AirflowConfigParser.getboolean (#4692)
> > > > > - [AIRFLOW-4571] Add headers to templated field for SimpleHttpOperator (#5326)
> > > > > - [AIRFLOW-3867] Rename GCP's subpackage (#4690)
> > > > > - [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df (#4549)
> > > > > - [AIRFLOW-4546] Upgrade google-cloud-bigtable. (#5307)
> > > > > - [AIRFLOW-4519] Optimise operator classname sorting in views (#5282)
> > > > > - [AIRFLOW-4503] Support fully pig options (#5271)
> > > > > - [AIRFLOW-4468] add sql_alchemy_max_overflow parameter (#5249)
> > > > > - [AIRFLOW-4467] Add dataproc_jars to templated fields in Dataproc oper… (#5248)
> > > > > - [AIRFLOW-4381] Use get_direct_relative_ids get task relatives (#5147)
> > > > > - [AIRFLOW-3624] Add masterType parameter to MLEngineTrainingOperator (#4428)
> > > > > - [AIRFLOW-3143] Support Auto-Zone in DataprocClusterCreateOperator (#5169)
> > > > > - [AIRFLOW-3874] Improve BigQueryHook.run_with_configuration's location support (#4695)
> > > > > - [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag (#5165)
> > > > > - [AIRFLOW-4031] Allow for key pair auth in snowflake hook (#4875)
> > > > > - [AIRFLOW-3901] add role as optional config parameter for SnowflakeHook (#4721)
> > > > > - [AIRFLOW-3455] add region in snowflake connector (#4285)
> > > > > - [AIRFLOW-4073] add template_ext for AWS Athena operator (#4907)
> > > > > - [AIRFLOW-4093] AWSAthenaOperator: Throw exception if job failed/cancelled/reach max retries (#4919)
> > > > > - [AIRFLOW-4356] Add extra RuntimeEnvironment keys to DataFlowHook (#5149)
> > > > > - [AIRFLOW-4337] Fix docker-compose deprecation warning in CI (#5119)
> > > > > - [AIRFLOW-3603] QuboleOperator: Remove SQLCommand from SparkCmd documentation (#4411)
> > > > > - [AIRFLOW-4328] Fix link to task instances from Pool page (#5124)
> > > > > - [AIRFLOW-4255] Make GCS Hook Backwards compatible (#5089)
> > > > > - [AIRFLOW-4103] Allow uppercase letters in dataflow job names (#4925)
> > > > > - [AIRFLOW-4255] Replace Discovery based api with client based for GCS (#5054)
> > > > > - [AIRFLOW-4311] Remove sleep in localexecutor (#5096)
> > > > > - [AIRFLOW-2836] Minor improvement-contrib.sensors.FileSensor (#3674)
> > > > > - [AIRFLOW-4104] Add type annotations to common classes. (#4926)
> > > > > - [AIRFLOW-3910] Raise exception explicitly in Connection.get_hook() (#4728)
> > > > > - [AIRFLOW-3322] Update QuboleHook to fetch args dynamically from qds_sdk (#4165)
> > > > > - [AIRFLOW-4565] instrument celery executor (#5321)
> > > > > - [AIRFLOW-4573] Import airflow_local_settings after prepare_classpath (#5330)
> > > > > - [AIRFLOW-4448] Don't bake ENV and _cmd into tmp config for non-sudo (#4050)
> > > > > - [AIRFLOW-4295] Make `method` attribute case insensitive in HttpHook (#5313)
> > > > > - [AIRFLOW-3703] Add dnsPolicy option for KubernetesPodOperator (#4520)
> > > > > - [AIRFLOW-3057] add prev_*_date_success to template context (#5372)
> > > > > - [AIRFLOW-4336] Stop showing entire GCS files bytes in log for gcs_download_operator (#5151)
> > > > > - [AIRFLOW-4528] Cancel DataProc task on timeout (#5293)
> > > > >
> > > > > Bug fixes
> > > > > """""""""
> > > > >
> > > > > - [AIRFLOW-4587] Replace self.conn with self.get_conn() in AWSAthenaHook (#5462)
> > > > > - [AIRFLOW-4516] K8s runAsUser and fsGroup cannot be strings (#5429)
> > > > > - [AIRFLOW-4298] Stop Scheduler repeatedly warning "connection invalidated" (#5470)
> > > > > - [AIRFLOW-4559] JenkinsJobTriggerOperator bugfix (#5318)
> > > > > - [AIRFLOW-4841] Pin Sphinx AutoApi to 1.0.0 (#5468)
> > > > > - [AIRFLOW-4479] Include s3_overwrite kwarg in load_bytes method (#5312)
> > > > > - [AIRFLOW-3746] Fix DockerOperator missing container exit (#4583)
> > > > > - [AIRFLOW-4233] Remove Template Extension from Bq to GCS Operator (#5456)
> > > > > - [AIRFLOW-2141][AIRFLOW-3157][AIRFLOW-4170] Serialize non-str value by JSON when importing Variables (#4991)
> > > > > - [AIRFLOW-4826] Remove warning from `airflow resetdb` command (#5447)
> > > > > - [AIRFLOW-4148] Fix editing DagRuns when clicking state column (#5436)
> > > > > - [AIRFLOW-4455] dag_details broken for subdags in RBAC UI (#5234)
> > > > > - [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods (#4551)
> > > > > - [AIRFLOW-4459] Fix wrong DAG count in /home page when DAG count is zero (#5235)
> > > > > - [AIRFLOW-3876] AttributeError: module 'distutils' has no attribute 'util'
> > > > > - [AIRFLOW-4146] Fix CgroupTaskRunner errors (#5224)
> > > > > - [AIRFLOW-4524] Fix bug with "Ignore \*" toggles in RBAC mode (#5378)
> > > > > - [AIRFLOW-4765] Fix DataProcPigOperator execute method (#5426)
> > > > > - [AIRFLOW-4798] obviate interdependencies for dagbag and TI tests (#5422)
> > > > > - [AIRFLOW-4800] fix GKEClusterHook ctor calls (#5424)
> > > > > - [AIRFLOW-4799] don't mutate self.env in BashOperator execute method (#5421)
> > > > > - [AIRFLOW-4393] Add retry logic when fetching pod status and/or logs in KubernetesPodOperator (#5284)
> > > > > - [AIRFLOW-4174] Fix HttpHook run with backoff (#5213)
> > > > > - [AIRFLOW-4463] Handle divide-by-zero errors in short retry intervals (#5243)
> > > > > - [AIRFLOW-2614] Speed up trigger_dag API call when lots of DAGs in system
> > > > > - [AIRFLOW-4756] add ti.state to ti.start_date as criteria for gantt (#5399)
> > > > > - [AIRFLOW-4760] Fix zip-packaged DAGs disappearing from DagBag when reloaded (#5404)
> > > > > - [AIRFLOW-4731] Fix GCS hook with google-storage-client 1.16 (#5368)
> > > > > - [AIRFLOW-3506] use match_phrase to query log_id in elasticsearch (#4342)
> > > > > - [AIRFLOW-4084] fix ElasticSearch log download (#5177)
> > > > > - [AIRFLOW-4501] Register pendulum datetime converter for sqla+pymysql (#5190)
> > > > > - [AIRFLOW-986] HiveCliHook ignores 'proxy_user' value in a connection's extra parameter (#5305)
> > > > > - [AIRFLOW-4442] fix hive_tblproperties in HiveToDruidTransfer (#5211)
> > > > > - [AIRFLOW-4557] Add gcp_conn_id parameter to get_sqlproxy_runner() of CloudSqlDatabaseHook (#5314)
> > > > > - [AIRFLOW-4545] Upgrade FAB to latest version (#4955)
> > > > > - [AIRFLOW-4492] Change Dataproc Cluster operators to poll Operations (#5269)
> > > > > - [AIRFLOW-4452] Webserver and Scheduler keep crashing because of slackclient update (#5225)
> > > > > - [AIRFLOW-4450] Fix request arguments in has_dag_access (#5220)
> > > > > - [AIRFLOW-4434] Support Impala with the HiveServer2Hook (#5206)
> > > > > - [AIRFLOW-3449] Write local dag parsing logs when remote logging enabled. (#5175)
> > > > > - [AIRFLOW-4300] Fix graph modal call when DAG has not yet run (#5185)
> > > > > - [AIRFLOW-4401] Use managers for Queue synchronization (#5200)
> > > > > - [AIRFLOW-3626] Fixed triggering DAGs contained within zip files (#4439)
> > > > > - [AIRFLOW-3720] Fix missmatch while comparing GCS and S3 files (#4766)
> > > > > - [AIRFLOW-4403] search by `dag_id` or `owners` in UI (#5184)
> > > > > - [AIRFLOW-4308] Fix TZ-loop around DST on python 3.6+ (#5095)
> > > > > - [AIRFLOW-4324] fix DAG fuzzy search in RBAC UI (#5131)
> > > > > - [AIRFLOW-4297] Temporary hot fix on manage_slas() for 1.10.4 release (#5150)
> > > > > - [AIRFLOW-4299] Upgrade to Celery 4.3.0 to fix crashing workers (#5116)
> > > > > - [AIRFLOW-4291] Correctly render doc_md in DAG graph page (#5121)
> > > > > - [AIRFLOW-4310] Fix incorrect link on Dag Details page (#5122)
> > > > > - [AIRFLOW-4331] Correct filter for Null-state runs from Dag Detail page (#5123)
> > > > > - [AIRFLOW-4294] Fix missing dag & task runs in UI dag_id contains a dot (#5111)
> > > > > - [AIRFLOW-4332] Upgrade sqlalchemy to remove security Vulnerability (#5113)
> > > > > - [AIRFLOW-4312] Add template_fields & template_ext to BigQueryCheckO… (#5097)
> > > > > - [AIRFLOW-4293] Fix downgrade in d4ecb8fbee3_add_schedule_interval_to_dag.py (#5086)
> > > > > - [AIRFLOW-4267] Fix TI duration in Graph View (#5071)
> > > > > - [AIRFLOW-4163] IntervalCheckOperator supports relative diff and not ignore 0 (#4983)
> > > > > - [AIRFLOW-3938] QuboleOperator Fixes and Support for SqlCommand (#4832)
> > > > > - [AIRFLOW-2903] Change default owner to `airflow` (#4151)
> > > > > - [aIRFLOW-4136] Fix overwrite of key_file by constructor (#5155)
> > > > > - [AIRFLOW-3241] Remove Invalid template ext in GCS Sensors (#4076)
> > > > >
> > > > > Misc/Internal
> > > > > """""""""""""
> > > > >
> > > > > - [AIRFLOW-4211] Add tests for WebHDFSHook (#5015)
> > > > > - [AIRFLOW-4320] Add tests for SegmentTrackEventOperator (#5104)
> > > > > - [AIRFLOW-4319] Add tests for Bigquery related Operators (#5101)
> > > > > - [AIRFLOW-4014] Change DatastoreHook and add tests (#4842)
> > > > > - [AIRFLOW-4322] Add test for VerticaOperator (#5107)
> > > > > - [AIRFLOW-4323] Add 2 tests for WinRMOperator (#5108)
> > > > > - [AIRFLOW-3677] Improve CheckOperator test coverage (#4756)
> > > > > - [AIRFLOW-4659] Fix pylint problems for api module (#5398)
> > > > > - [AIRFLOW-4358] Speed up test_jobs by not running tasks (#5162)
> > > > > - [AIRFLOW-4394] Don't test behaviour of BackfillJob from CLI tests (#5160)
> > > > > - [AIRFLOW-3471] Move XCom out of models.py (#4629)
> > > > > - [AIRFLOW-4379] Remove duplicate code & Add validation in gcs_to_gcs.py (#5145)
> > > > > - [AIRFLOW-4259] Move models out of models.py (#5056)
> > > > > - [AIRFLOW-XXX] Speed up building of Cassanda module on Travis (#5233)
> > > > > - [AIRFLOW-4535] Break jobs.py into multiple files (#5303)
> > > > > - [AIRFLOW-1464] Batch update task_instance state (#5323)
> > > > > - [AIRFLOW-4554] Test for sudo command, add some other test docs (#5310)
> > > > > - [AIRFLOW-4419] Refine concurrency check in scheduler (#5194)
> > > > > - [AIRFLOW-4269] Minor acceleration of jobs._process_task_instances() (#5076)
> > > > > - [AIRFLOW-4341] Remove `View.render()` already exists in fab.BaseView (#5125)
> > > > > - [AIRFLOW-4342] Use @cached_property instead of re-implementing it each time (#5126)
> > > > > - [AIRFLOW-4256] Remove noqa from migrations (#5055)
> > > > > - [AIRFLOW-4034] Remove unnecessary string formatting with `**locals()` (#4861)
> > > > > - [AIRFLOW-3944] Remove code smells (#4762)
> > > > >
> > > > > Doc-only changes
> > > > > """"""""""""""""
> > > > >
> > > > > - [AIRFLOW-3341] FAQ return DAG object example (#4605)
> > > > > - [AIRFLOW-4433] Add missing type in DockerOperator doc string (#5205)
> > > > > - [AIRFLOW-4321] Replace incorrect info of Max Size limit of GCS Object Size (#5106)
> > > > > - [AIRFLOW-XXX] Add information about user list (#5341)
> > > > > - [AIRFLOW-XXX] Clarify documentation related to autodetect parameter in GCS_to_BQ Op (#5294)
> > > > > - [AIRFLOW-XXX] Remove mention of pytz compatibility from timezone documentation (#5316)
> > > > > - [AIRFLOW-XXX] Add missing docs for GoogleCloudStorageDeleteOperator (#5274)
> > > > > - [AIRFLOW-XXX] Remove incorrect note about Scopes of GCP connection (#5242)
> > > > > - [AIRFLOW-XXX] Fix mistakes in docs of Dataproc operators (#5192)
> > > > > - [AIRFLOW-XXX] Link to correct class for timedelta in macros.rst (#5226)
> > > > > - [AIRFLOW-XXX] Add Kamil as committer (#5216)
> > > > > - [AIRFLOW-XXX] Add Joshua and Kevin as committer (#5207)
> > > > > - [AIRFLOW-XXX] Reduce log spam in tests (#5174)
> > > > > - [AIRFLOW-XXX] Speed up tests for PythonSensor (#5158)
> > > > > - [AIRFLOW-XXX] Add Bas Harenslak to committer list (#5157)
> > > > > - [AIRFLOW-XXX] Add Jarek Potiuk to commiter list (#5132)
> > > > > - [AIRFLOW-XXX] Update docstring for SchedulerJob (#5105)
> > > > > - [AIRFLOW-XXX] Fix docstrings for CassandraToGoogleCloudStorageOperator (#5103)
> > > > > - [AIRFLOW-XXX] update SlackWebhookHook and SlackWebhookOperator docstring (#5074)
> > > > > - [AIRFLOW-XXX] Ignore python files under node_modules in docs (#5063)
> > > > > - [AIRFLOW-XXX] Build a universal wheel with LICNESE files (#5052)
> > > > > - [AIRFLOW-XXX] Fix docstrings of SQSHook (#5099)
> > > > > - [AIRFLOW-XXX] Use Py3.7 on readthedocs
> > > > > - [AIRFLOW-4446] Fix typos (#5217)
Re: 1.10.4beta2 "snapshot" for available testing
Posted by Ash Berlin-Taylor <as...@apache.org>.
What URL does that end up on? I can't reproduce that.
-a
> On 26 Jun 2019, at 10:52, airflowuser <ai...@protonmail.com.INVALID> wrote:
>
> Bug:
> On the old UI:
> 1. Click on edit DAG
> 2. Enter Description
> 3.Click Save
>
> You'll get error:
> Traceback (most recent call last):
> File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2311, in wsgi_app
> response = self.full_dispatch_request()
>
> ....
>
> packages/flask_admin/model/base.py", line 1861, in get_list_value
> self.column_type_formatters,
> File "/usr/local/lib/python3.6/site-packages/flask_admin/model/base.py", line 1826, in _get_list_value
> value = column_fmt(self, context, model, name)
> File "/usr/local/lib/python3.6/site-packages/airflow/www/views.py", line 113, in dag_link
> execution_date=m.execution_date)
> AttributeError: 'DagModel' object has no attribute 'execution_date'
>
>
> Didn't happen to me on 1.10.3
>
> Not really sure why execution_date is relevant for when you want to edit details on the DAG that is unrelated to a specific DAG run.
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Wednesday, June 26, 2019 11:09 AM, Ash Berlin-Taylor <as...@apache.org> wrote:
>
>> Because the list was autogenerated from the issue type in the jira ticket, and I didn't check all 200+ items in the change log :)
>>
>> -a
>>
>>> On 25 Jun 2019, at 18:59, airflowuser airflowuser@protonmail.com.INVALID wrote:
>>> Why [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365) listed under features?
>>> Sent with ProtonMail Secure Email.
>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>> On Tuesday, June 25, 2019 1:40 PM, Ash Berlin-Taylor ash@apache.org wrote:
>>>
>>>> Hi Everyone,
>>>> I'm excited to annoucne that 1.10.4 is almost ready for release.
>>>> The release branch already contains 216 commits since 1.10.3 this time around.
>>>> In order to distinguish it from an actual (apache) release it is:
>>>>
>>>> 1. Marked as beta (python package managers do not install beta versions by default - PEP 440)
>>>> 2. It is not at an official Apache distribution location
>>>>
>>>> It can be installed with:
>>>> pip install 'apache-airflow==1.10.4b2'
>>>> or downloaded from https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/ https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/
>>>> (Don't worry, without asking for`--pre` or specifying the version `pip install apache-airflow` will still get 1.10.3)
>>>> Included below is the changelog of this release so far. I'd like to try and get this out before the end of the month, so will create an RC in a day or two baring any major problems reported.
>>>> Cheers,
>>>> Ash
>>>> Airflow 1.10.4
>>>>
>>>> New Features
>>>> """"""""""""
>>>>
>>>> - [AIRFLOW-4456] Add sub-classable BaseBranchOperator (#5231)
>>>> - [AIRFLOW-4343] Show warning in UI if scheduler is not running (#5127)
>>>> - [AIRFLOW-4739] Add ability to arbitrarily define kubernetes worker pod labels (#5376)
>>>> - [AIRFLOW-4348] Add GCP console link in BigQueryOperator (#5195)
>>>> - [AIRFLOW-4306] Global operator extra links (#5094)
>>>> - [AIRFLOW-4812] Add batch images annotation (#5433)
>>>> - [AIRFLOW-4135] Add Google Cloud Build operator and hook (#5251)
>>>> - [AIRFLOW-4781] Add the ability to specify ports in kubernetesOperator (#5410)
>>>> - [AIRFLOW-4521] Pause dag also pause its subdags (#5283)
>>>> - [AIRFLOW-4738] Enforce exampleinclude for example DAGs (#5375)
>>>> - [AIRFLOW-4326] Airflow AWS SQS Operator (#5110)
>>>> - [AIRFLOW-3729] Support "DownwardAPI" in env variables for KubernetesPodOperator (#4554)
>>>> - [AIRFLOW-4585] Implement Kubernetes Pod Mutation Hook (#5359)
>>>> - [AIRFLOW-161] New redirect route and extra links (#5059)
>>>> - [AIRFLOW-4420] Backfill respects task_concurrency (#5221)
>>>> - [AIRFLOW-4147] Add Operator to publish event to Redis (#4967)
>>>> - [AIRFLOW-3359] Add option to pass customer encryption keys to Dataproc (#4200)
>>>> - [AIRFLOW-4318] Create Google Cloud Translate Speech Operator (#5102)
>>>> - [AIRFLOW-3960] Adds Google Cloud Speech operators (#4780)
>>>> - [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365)
>>>> - [AIRFLOW-1501] Add GoogleCloudStorageDeleteOperator (#5230)
>>>> - [AIRFLOW-3672] Add support for Mongo DB DNS Seedlist Connection Format (#4481)
>>>> - [AIRFLOW-4397] add integrations docs manually for gcs sensors (#5204)
>>>> - [AIRFLOW-4251] Instrument DagRun schedule delay (#5050)
>>>> - [AIRFLOW-4118] instrument DagRun duration (#4946)
>>>> - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5182)
>>>> - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5140)
>>>> - [AIRFLOW-4168] Create Google Cloud Video Intelligence Operators (#4985)
>>>> - [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor (#5166)
>>>> - [AIRFLOW-4335] Add default num_retries to GCP connection (#5117)
>>>> - [AIRFLOW-3808] Add cluster_fields to BigQueryHook's create_empty_table (#4654)
>>>> - [AIRFLOW-4362] Fix test_execution_limited_parallelism (#5141)
>>>> - [AIRFLOW-4307] Backfill respects concurrency limit (#5128)
>>>> - [AIRFLOW-4268] Add MsSqlToGoogleCloudStorageOperator (#5077)
>>>> - [AIRFLOW-4169] Add Google Cloud Vision Detect Operators (#4986)
>>>> - [AIRFLOW-XXX] Fix WS-2019-0032 (#5384)
>>>> - [AIRFLOW-XXX] Fix CVE-2019-11358 (#5197)
>>>> - [AIRFLOW-XXX] Change allowed version of Jinja2 to fix CVE-2019-10906 (#5075)
>>>>
>>>> Improvement
>>>> """""""""""
>>>>
>>>> - [AIRFLOW-4805] Add py_file as templated field in DataflowPythonOperator (#5451)
>>>> - [AIRFLOW-4838] Surface Athena errors in AWSAthenaOperator (#5467)
>>>> - [AIRFLOW-4831] conf.has_option no longer throws if section is missing. (#5455)
>>>> - [AIRFLOW-4829] More descriptive exceptions for EMR sensors (#5452)
>>>> - [AIRFLOW-4414] AWSAthenaOperator: Push QueryExecutionID to XCom (#5276)
>>>> - [AIRFLOW-4791] add "schema" keyword arg to SnowflakeOperator (#5415)
>>>> - [AIRFLOW-4759] Don't error when marking sucessful run as failed (#5435)
>>>> - [AIRFLOW-4716] Instrument dag loading time duration (#5350)
>>>> - [AIRFLOW-3958] Support list tasks as upstream in chain (#4779)
>>>> - [AIRFLOW-4409] Prevent task duration break by null value (#5178)
>>>> - [AIRFLOW-4418] Add "failed only" option to task modal (#5193)
>>>> - [AIRFLOW-4740] Accept string `end_date` in DAG default_args (#5381)
>>>> - [AIRFLOW-4423] Improve date handling in mysql to gcs operator. (#5196)
>>>> - [AIRFLOW-4447] Display task duration as human friendly format in UI (#5218)
>>>> - [AIRFLOW-4377] Remove needless object conversion in DAG.owner() (#5144)
>>>> - [AIRFLOW-4766] Add autoscaling option for DataprocClusterCreateOperator (#5425)
>>>> - [AIRFLOW-4795] Upgrade alembic to latest release. (#5411)
>>>> - [AIRFLOW-4793] Add signature_name to mlengine operator (#5417)
>>>> - [AIRFLOW-3211] Reattach to GCP Dataproc jobs upon Airflow restart (#4083)
>>>> - [AIRFLOW-4750] Log identified zombie task instances (#5389)
>>>> - [AIRFLOW-3870] STFPOperator: Update log level and return value (#4355)
>>>> - [AIRFLOW-4759] Batch queries in set_state API. (#5403)
>>>> - [AIRFLOW-2737] Restore original license header to airflow.api.auth.backend.kerberos_auth
>>>> - [AIRFLOW-3635] Fix incorrect logic in detele_dag (introduced in PR#4406) (#4445)
>>>> - [AIRFLOW-3599] Removed Dagbag from delete dag (#4406)
>>>> - [AIRFLOW-4737] Increase and document celery queue name limit (#5383)
>>>> - [AIRFLOW-4505] Correct Tag ALL for PY3 (#5275)
>>>> - [AIRFLOW-4743] Add environment variables support to SSHOperator (#5385)
>>>> - [AIRFLOW-4725] Fix setup.py PEP440 & Sphinx-PyPI-upload dependency (#5363)
>>>> - [AIRFLOW-3370] Add stdout output options to Elasticsearch task log handler (#5048)
>>>> - [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. (#5164)
>>>> - [AIRFLOW-1381] Allow setting host temporary directory in DockerOperator (#5369)
>>>> - [AIRFLOW-4598] Task retries are not exhausted for K8s executor (#5347)
>>>> - [AIRFLOW-4218] Support to Provide http args to K8executor while calling k8 python client lib apis (#5060)
>>>> - [AIRFLOW-4159] Add support for additional static pod labels for K8sExecutor (#5134)
>>>> - [AIRFLOW-4720] Allow comments in .airflowignore files. (#5355)
>>>> - [AIRFLOW-4486] Add AWS IAM authenication in MySqlHook (#5334)
>>>> - [AIRFLOW-4417] Add AWS IAM authenication for PostgresHook (#5223)
>>>> - [AIRFLOW-3990] Compile regular expressions. (#4813)
>>>> - [AIRFLOW-4572] Rename prepare_classpath() to prepare_syspath() (#5328)
>>>> - [AIRFLOW-3869] Raise consistent exception in AirflowConfigParser.getboolean (#4692)
>>>> - [AIRFLOW-4571] Add headers to templated field for SimpleHttpOperator (#5326)
>>>> - [AIRFLOW-3867] Rename GCP's subpackage (#4690)
>>>> - [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df (#4549)
>>>> - [AIRFLOW-4546] Upgrade google-cloud-bigtable. (#5307)
>>>> - [AIRFLOW-4519] Optimise operator classname sorting in views (#5282)
>>>> - [AIRFLOW-4503] Support fully pig options (#5271)
>>>> - [AIRFLOW-4468] add sql_alchemy_max_overflow parameter (#5249)
>>>> - [AIRFLOW-4467] Add dataproc_jars to templated fields in Dataproc oper… (#5248)
>>>> - [AIRFLOW-4381] Use get_direct_relative_ids get task relatives (#5147)
>>>> - [AIRFLOW-3624] Add masterType parameter to MLEngineTrainingOperator (#4428)
>>>> - [AIRFLOW-3143] Support Auto-Zone in DataprocClusterCreateOperator (#5169)
>>>> - [AIRFLOW-3874] Improve BigQueryHook.run_with_configuration's location support (#4695)
>>>> - [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag (#5165)
>>>> - [AIRFLOW-4031] Allow for key pair auth in snowflake hook (#4875)
>>>> - [AIRFLOW-3901] add role as optional config parameter for SnowflakeHook (#4721)
>>>> - [AIRFLOW-3455] add region in snowflake connector (#4285)
>>>> - [AIRFLOW-4073] add template_ext for AWS Athena operator (#4907)
>>>> - [AIRFLOW-4093] AWSAthenaOperator: Throw exception if job failed/cancelled/reach max retries (#4919)
>>>> - [AIRFLOW-4356] Add extra RuntimeEnvironment keys to DataFlowHook (#5149)
>>>> - [AIRFLOW-4337] Fix docker-compose deprecation warning in CI (#5119)
>>>> - [AIRFLOW-3603] QuboleOperator: Remove SQLCommand from SparkCmd documentation (#4411)
>>>> - [AIRFLOW-4328] Fix link to task instances from Pool page (#5124)
>>>> - [AIRFLOW-4255] Make GCS Hook Backwards compatible (#5089)
>>>> - [AIRFLOW-4103] Allow uppercase letters in dataflow job names (#4925)
>>>> - [AIRFLOW-4255] Replace Discovery based api with client based for GCS (#5054)
>>>> - [AIRFLOW-4311] Remove sleep in localexecutor (#5096)
>>>> - [AIRFLOW-2836] Minor improvement-contrib.sensors.FileSensor (#3674)
>>>> - [AIRFLOW-4104] Add type annotations to common classes. (#4926)
>>>> - [AIRFLOW-3910] Raise exception explicitly in Connection.get_hook() (#4728)
>>>> - [AIRFLOW-3322] Update QuboleHook to fetch args dynamically from qds_sdk (#4165)
>>>> - [AIRFLOW-4565] instrument celery executor (#5321)
>>>> - [AIRFLOW-4573] Import airflow_local_settings after prepare_classpath (#5330)
>>>> - [AIRFLOW-4448] Don't bake ENV and _cmd into tmp config for non-sudo (#4050)
>>>> - [AIRFLOW-4295] Make `method` attribute case insensitive in HttpHook (#5313)
>>>> - [AIRFLOW-3703] Add dnsPolicy option for KubernetesPodOperator (#4520)
>>>> - [AIRFLOW-3057] add prev_*_date_success to template context (#5372)
>>>> - [AIRFLOW-4336] Stop showing entire GCS files bytes in log for gcs_download_operator (#5151)
>>>> - [AIRFLOW-4528] Cancel DataProc task on timeout (#5293)
>>>>
>>>> Bug fixes
>>>> """""""""
>>>>
>>>> - [AIRFLOW-4587] Replace self.conn with self.get_conn() in AWSAthenaHook (#5462)
>>>> - [AIRFLOW-4516] K8s runAsUser and fsGroup cannot be strings (#5429)
>>>> - [AIRFLOW-4298] Stop Scheduler repeatedly warning "connection invalidated" (#5470)
>>>> - [AIRFLOW-4559] JenkinsJobTriggerOperator bugfix (#5318)
>>>> - [AIRFLOW-4841] Pin Sphinx AutoApi to 1.0.0 (#5468)
>>>> - [AIRFLOW-4479] Include s3_overwrite kwarg in load_bytes method (#5312)
>>>> - [AIRFLOW-3746] Fix DockerOperator missing container exit (#4583)
>>>> - [AIRFLOW-4233] Remove Template Extension from Bq to GCS Operator (#5456)
>>>> - [AIRFLOW-2141][AIRFLOW-3157][AIRFLOW-4170] Serialize non-str value by JSON when importing Variables (#4991)
>>>> - [AIRFLOW-4826] Remove warning from `airflow resetdb` command (#5447)
>>>> - [AIRFLOW-4148] Fix editing DagRuns when clicking state column (#5436)
>>>> - [AIRFLOW-4455] dag_details broken for subdags in RBAC UI (#5234)
>>>> - [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods (#4551)
>>>> - [AIRFLOW-4459] Fix wrong DAG count in /home page when DAG count is zero (#5235)
>>>> - [AIRFLOW-3876] AttributeError: module 'distutils' has no attribute 'util'
>>>> - [AIRFLOW-4146] Fix CgroupTaskRunner errors (#5224)
>>>> - [AIRFLOW-4524] Fix bug with "Ignore \*" toggles in RBAC mode (#5378)
>>>> - [AIRFLOW-4765] Fix DataProcPigOperator execute method (#5426)
>>>> - [AIRFLOW-4798] obviate interdependencies for dagbag and TI tests (#5422)
>>>> - [AIRFLOW-4800] fix GKEClusterHook ctor calls (#5424)
>>>> - [AIRFLOW-4799] don't mutate self.env in BashOperator execute method (#5421)
>>>> - [AIRFLOW-4393] Add retry logic when fetching pod status and/or logs in KubernetesPodOperator (#5284)
>>>> - [AIRFLOW-4174] Fix HttpHook run with backoff (#5213)
>>>> - [AIRFLOW-4463] Handle divide-by-zero errors in short retry intervals (#5243)
>>>> - [AIRFLOW-2614] Speed up trigger_dag API call when lots of DAGs in system
>>>> - [AIRFLOW-4756] add ti.state to ti.start_date as criteria for gantt (#5399)
>>>> - [AIRFLOW-4760] Fix zip-packaged DAGs disappearing from DagBag when reloaded (#5404)
>>>> - [AIRFLOW-4731] Fix GCS hook with google-storage-client 1.16 (#5368)
>>>> - [AIRFLOW-3506] use match_phrase to query log_id in elasticsearch (#4342)
>>>> - [AIRFLOW-4084] fix ElasticSearch log download (#5177)
>>>> - [AIRFLOW-4501] Register pendulum datetime converter for sqla+pymysql (#5190)
>>>> - [AIRFLOW-986] HiveCliHook ignores 'proxy_user' value in a connection's extra parameter (#5305)
>>>> - [AIRFLOW-4442] fix hive_tblproperties in HiveToDruidTransfer (#5211)
>>>> - [AIRFLOW-4557] Add gcp_conn_id parameter to get_sqlproxy_runner() of CloudSqlDatabaseHook (#5314)
>>>> - [AIRFLOW-4545] Upgrade FAB to latest version (#4955)
>>>> - [AIRFLOW-4492] Change Dataproc Cluster operators to poll Operations (#5269)
>>>> - [AIRFLOW-4452] Webserver and Scheduler keep crashing because of slackclient update (#5225)
>>>> - [AIRFLOW-4450] Fix request arguments in has_dag_access (#5220)
>>>> - [AIRFLOW-4434] Support Impala with the HiveServer2Hook (#5206)
>>>> - [AIRFLOW-3449] Write local dag parsing logs when remote logging enabled. (#5175)
>>>> - [AIRFLOW-4300] Fix graph modal call when DAG has not yet run (#5185)
>>>> - [AIRFLOW-4401] Use managers for Queue synchronization (#5200)
>>>> - [AIRFLOW-3626] Fixed triggering DAGs contained within zip files (#4439)
>>>> - [AIRFLOW-3720] Fix missmatch while comparing GCS and S3 files (#4766)
>>>> - [AIRFLOW-4403] search by `dag_id` or `owners` in UI (#5184)
>>>> - [AIRFLOW-4308] Fix TZ-loop around DST on python 3.6+ (#5095)
>>>> - [AIRFLOW-4324] fix DAG fuzzy search in RBAC UI (#5131)
>>>> - [AIRFLOW-4297] Temporary hot fix on manage_slas() for 1.10.4 release (#5150)
>>>> - [AIRFLOW-4299] Upgrade to Celery 4.3.0 to fix crashing workers (#5116)
>>>> - [AIRFLOW-4291] Correctly render doc_md in DAG graph page (#5121)
>>>> - [AIRFLOW-4310] Fix incorrect link on Dag Details page (#5122)
>>>> - [AIRFLOW-4331] Correct filter for Null-state runs from Dag Detail page (#5123)
>>>> - [AIRFLOW-4294] Fix missing dag & task runs in UI dag_id contains a dot (#5111)
>>>> - [AIRFLOW-4332] Upgrade sqlalchemy to remove security Vulnerability (#5113)
>>>> - [AIRFLOW-4312] Add template_fields & template_ext to BigQueryCheckO… (#5097)
>>>> - [AIRFLOW-4293] Fix downgrade in d4ecb8fbee3_add_schedule_interval_to_dag.py (#5086)
>>>> - [AIRFLOW-4267] Fix TI duration in Graph View (#5071)
>>>> - [AIRFLOW-4163] IntervalCheckOperator supports relative diff and not ignore 0 (#4983)
>>>> - [AIRFLOW-3938] QuboleOperator Fixes and Support for SqlCommand (#4832)
>>>> - [AIRFLOW-2903] Change default owner to `airflow` (#4151)
>>>> - [aIRFLOW-4136] Fix overwrite of key_file by constructor (#5155)
>>>> - [AIRFLOW-3241] Remove Invalid template ext in GCS Sensors (#4076)
>>>>
>>>> Misc/Internal
>>>> """""""""""""
>>>>
>>>> - [AIRFLOW-4211] Add tests for WebHDFSHook (#5015)
>>>> - [AIRFLOW-4320] Add tests for SegmentTrackEventOperator (#5104)
>>>> - [AIRFLOW-4319] Add tests for Bigquery related Operators (#5101)
>>>> - [AIRFLOW-4014] Change DatastoreHook and add tests (#4842)
>>>> - [AIRFLOW-4322] Add test for VerticaOperator (#5107)
>>>> - [AIRFLOW-4323] Add 2 tests for WinRMOperator (#5108)
>>>> - [AIRFLOW-3677] Improve CheckOperator test coverage (#4756)
>>>> - [AIRFLOW-4659] Fix pylint problems for api module (#5398)
>>>> - [AIRFLOW-4358] Speed up test_jobs by not running tasks (#5162)
>>>> - [AIRFLOW-4394] Don't test behaviour of BackfillJob from CLI tests (#5160)
>>>> - [AIRFLOW-3471] Move XCom out of models.py (#4629)
>>>> - [AIRFLOW-4379] Remove duplicate code & Add validation in gcs_to_gcs.py (#5145)
>>>> - [AIRFLOW-4259] Move models out of models.py (#5056)
>>>> - [AIRFLOW-XXX] Speed up building of Cassanda module on Travis (#5233)
>>>> - [AIRFLOW-4535] Break jobs.py into multiple files (#5303)
>>>> - [AIRFLOW-1464] Batch update task_instance state (#5323)
>>>> - [AIRFLOW-4554] Test for sudo command, add some other test docs (#5310)
>>>> - [AIRFLOW-4419] Refine concurrency check in scheduler (#5194)
>>>> - [AIRFLOW-4269] Minor acceleration of jobs._process_task_instances() (#5076)
>>>> - [AIRFLOW-4341] Remove `View.render()` already exists in fab.BaseView (#5125)
>>>> - [AIRFLOW-4342] Use @cached_property instead of re-implementing it each time (#5126)
>>>> - [AIRFLOW-4256] Remove noqa from migrations (#5055)
>>>> - [AIRFLOW-4034] Remove unnecessary string formatting with `**locals()` (#4861)
>>>> - [AIRFLOW-3944] Remove code smells (#4762)
>>>>
>>>> Doc-only changes
>>>> """"""""""""""""
>>>>
>>>> - [AIRFLOW-3341] FAQ return DAG object example (#4605)
>>>> - [AIRFLOW-4433] Add missing type in DockerOperator doc string (#5205)
>>>> - [AIRFLOW-4321] Replace incorrect info of Max Size limit of GCS Object Size (#5106)
>>>> - [AIRFLOW-XXX] Add information about user list (#5341)
>>>> - [AIRFLOW-XXX] Clarify documentation related to autodetect parameter in GCS_to_BQ Op (#5294)
>>>> - [AIRFLOW-XXX] Remove mention of pytz compatibility from timezone documentation (#5316)
>>>> - [AIRFLOW-XXX] Add missing docs for GoogleCloudStorageDeleteOperator (#5274)
>>>> - [AIRFLOW-XXX] Remove incorrect note about Scopes of GCP connection (#5242)
>>>> - [AIRFLOW-XXX] Fix mistakes in docs of Dataproc operators (#5192)
>>>> - [AIRFLOW-XXX] Link to correct class for timedelta in macros.rst (#5226)
>>>> - [AIRFLOW-XXX] Add Kamil as committer (#5216)
>>>> - [AIRFLOW-XXX] Add Joshua and Kevin as committer (#5207)
>>>> - [AIRFLOW-XXX] Reduce log spam in tests (#5174)
>>>> - [AIRFLOW-XXX] Speed up tests for PythonSensor (#5158)
>>>> - [AIRFLOW-XXX] Add Bas Harenslak to committer list (#5157)
>>>> - [AIRFLOW-XXX] Add Jarek Potiuk to commiter list (#5132)
>>>> - [AIRFLOW-XXX] Update docstring for SchedulerJob (#5105)
>>>> - [AIRFLOW-XXX] Fix docstrings for CassandraToGoogleCloudStorageOperator (#5103)
>>>> - [AIRFLOW-XXX] update SlackWebhookHook and SlackWebhookOperator docstring (#5074)
>>>> - [AIRFLOW-XXX] Ignore python files under node_modules in docs (#5063)
>>>> - [AIRFLOW-XXX] Build a universal wheel with LICNESE files (#5052)
>>>> - [AIRFLOW-XXX] Fix docstrings of SQSHook (#5099)
>>>> - [AIRFLOW-XXX] Use Py3.7 on readthedocs
>>>> - [AIRFLOW-4446] Fix typos (#5217)
>
>
Re: 1.10.4beta2 "snapshot" for available testing
Posted by airflowuser <ai...@protonmail.com.INVALID>.
Bug:
On the old UI:
1. Click on edit DAG
2. Enter Description
3.Click Save
You'll get error:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2311, in wsgi_app
response = self.full_dispatch_request()
....
packages/flask_admin/model/base.py", line 1861, in get_list_value
self.column_type_formatters,
File "/usr/local/lib/python3.6/site-packages/flask_admin/model/base.py", line 1826, in _get_list_value
value = column_fmt(self, context, model, name)
File "/usr/local/lib/python3.6/site-packages/airflow/www/views.py", line 113, in dag_link
execution_date=m.execution_date)
AttributeError: 'DagModel' object has no attribute 'execution_date'
Didn't happen to me on 1.10.3
Not really sure why execution_date is relevant for when you want to edit details on the DAG that is unrelated to a specific DAG run.
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, June 26, 2019 11:09 AM, Ash Berlin-Taylor <as...@apache.org> wrote:
> Because the list was autogenerated from the issue type in the jira ticket, and I didn't check all 200+ items in the change log :)
>
> -a
>
> > On 25 Jun 2019, at 18:59, airflowuser airflowuser@protonmail.com.INVALID wrote:
> > Why [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365) listed under features?
> > Sent with ProtonMail Secure Email.
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Tuesday, June 25, 2019 1:40 PM, Ash Berlin-Taylor ash@apache.org wrote:
> >
> > > Hi Everyone,
> > > I'm excited to annoucne that 1.10.4 is almost ready for release.
> > > The release branch already contains 216 commits since 1.10.3 this time around.
> > > In order to distinguish it from an actual (apache) release it is:
> > >
> > > 1. Marked as beta (python package managers do not install beta versions by default - PEP 440)
> > > 2. It is not at an official Apache distribution location
> > >
> > > It can be installed with:
> > > pip install 'apache-airflow==1.10.4b2'
> > > or downloaded from https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/ https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/
> > > (Don't worry, without asking for`--pre` or specifying the version `pip install apache-airflow` will still get 1.10.3)
> > > Included below is the changelog of this release so far. I'd like to try and get this out before the end of the month, so will create an RC in a day or two baring any major problems reported.
> > > Cheers,
> > > Ash
> > > Airflow 1.10.4
> > >
> > > New Features
> > > """"""""""""
> > >
> > > - [AIRFLOW-4456] Add sub-classable BaseBranchOperator (#5231)
> > > - [AIRFLOW-4343] Show warning in UI if scheduler is not running (#5127)
> > > - [AIRFLOW-4739] Add ability to arbitrarily define kubernetes worker pod labels (#5376)
> > > - [AIRFLOW-4348] Add GCP console link in BigQueryOperator (#5195)
> > > - [AIRFLOW-4306] Global operator extra links (#5094)
> > > - [AIRFLOW-4812] Add batch images annotation (#5433)
> > > - [AIRFLOW-4135] Add Google Cloud Build operator and hook (#5251)
> > > - [AIRFLOW-4781] Add the ability to specify ports in kubernetesOperator (#5410)
> > > - [AIRFLOW-4521] Pause dag also pause its subdags (#5283)
> > > - [AIRFLOW-4738] Enforce exampleinclude for example DAGs (#5375)
> > > - [AIRFLOW-4326] Airflow AWS SQS Operator (#5110)
> > > - [AIRFLOW-3729] Support "DownwardAPI" in env variables for KubernetesPodOperator (#4554)
> > > - [AIRFLOW-4585] Implement Kubernetes Pod Mutation Hook (#5359)
> > > - [AIRFLOW-161] New redirect route and extra links (#5059)
> > > - [AIRFLOW-4420] Backfill respects task_concurrency (#5221)
> > > - [AIRFLOW-4147] Add Operator to publish event to Redis (#4967)
> > > - [AIRFLOW-3359] Add option to pass customer encryption keys to Dataproc (#4200)
> > > - [AIRFLOW-4318] Create Google Cloud Translate Speech Operator (#5102)
> > > - [AIRFLOW-3960] Adds Google Cloud Speech operators (#4780)
> > > - [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365)
> > > - [AIRFLOW-1501] Add GoogleCloudStorageDeleteOperator (#5230)
> > > - [AIRFLOW-3672] Add support for Mongo DB DNS Seedlist Connection Format (#4481)
> > > - [AIRFLOW-4397] add integrations docs manually for gcs sensors (#5204)
> > > - [AIRFLOW-4251] Instrument DagRun schedule delay (#5050)
> > > - [AIRFLOW-4118] instrument DagRun duration (#4946)
> > > - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5182)
> > > - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5140)
> > > - [AIRFLOW-4168] Create Google Cloud Video Intelligence Operators (#4985)
> > > - [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor (#5166)
> > > - [AIRFLOW-4335] Add default num_retries to GCP connection (#5117)
> > > - [AIRFLOW-3808] Add cluster_fields to BigQueryHook's create_empty_table (#4654)
> > > - [AIRFLOW-4362] Fix test_execution_limited_parallelism (#5141)
> > > - [AIRFLOW-4307] Backfill respects concurrency limit (#5128)
> > > - [AIRFLOW-4268] Add MsSqlToGoogleCloudStorageOperator (#5077)
> > > - [AIRFLOW-4169] Add Google Cloud Vision Detect Operators (#4986)
> > > - [AIRFLOW-XXX] Fix WS-2019-0032 (#5384)
> > > - [AIRFLOW-XXX] Fix CVE-2019-11358 (#5197)
> > > - [AIRFLOW-XXX] Change allowed version of Jinja2 to fix CVE-2019-10906 (#5075)
> > >
> > > Improvement
> > > """""""""""
> > >
> > > - [AIRFLOW-4805] Add py_file as templated field in DataflowPythonOperator (#5451)
> > > - [AIRFLOW-4838] Surface Athena errors in AWSAthenaOperator (#5467)
> > > - [AIRFLOW-4831] conf.has_option no longer throws if section is missing. (#5455)
> > > - [AIRFLOW-4829] More descriptive exceptions for EMR sensors (#5452)
> > > - [AIRFLOW-4414] AWSAthenaOperator: Push QueryExecutionID to XCom (#5276)
> > > - [AIRFLOW-4791] add "schema" keyword arg to SnowflakeOperator (#5415)
> > > - [AIRFLOW-4759] Don't error when marking sucessful run as failed (#5435)
> > > - [AIRFLOW-4716] Instrument dag loading time duration (#5350)
> > > - [AIRFLOW-3958] Support list tasks as upstream in chain (#4779)
> > > - [AIRFLOW-4409] Prevent task duration break by null value (#5178)
> > > - [AIRFLOW-4418] Add "failed only" option to task modal (#5193)
> > > - [AIRFLOW-4740] Accept string `end_date` in DAG default_args (#5381)
> > > - [AIRFLOW-4423] Improve date handling in mysql to gcs operator. (#5196)
> > > - [AIRFLOW-4447] Display task duration as human friendly format in UI (#5218)
> > > - [AIRFLOW-4377] Remove needless object conversion in DAG.owner() (#5144)
> > > - [AIRFLOW-4766] Add autoscaling option for DataprocClusterCreateOperator (#5425)
> > > - [AIRFLOW-4795] Upgrade alembic to latest release. (#5411)
> > > - [AIRFLOW-4793] Add signature_name to mlengine operator (#5417)
> > > - [AIRFLOW-3211] Reattach to GCP Dataproc jobs upon Airflow restart (#4083)
> > > - [AIRFLOW-4750] Log identified zombie task instances (#5389)
> > > - [AIRFLOW-3870] STFPOperator: Update log level and return value (#4355)
> > > - [AIRFLOW-4759] Batch queries in set_state API. (#5403)
> > > - [AIRFLOW-2737] Restore original license header to airflow.api.auth.backend.kerberos_auth
> > > - [AIRFLOW-3635] Fix incorrect logic in detele_dag (introduced in PR#4406) (#4445)
> > > - [AIRFLOW-3599] Removed Dagbag from delete dag (#4406)
> > > - [AIRFLOW-4737] Increase and document celery queue name limit (#5383)
> > > - [AIRFLOW-4505] Correct Tag ALL for PY3 (#5275)
> > > - [AIRFLOW-4743] Add environment variables support to SSHOperator (#5385)
> > > - [AIRFLOW-4725] Fix setup.py PEP440 & Sphinx-PyPI-upload dependency (#5363)
> > > - [AIRFLOW-3370] Add stdout output options to Elasticsearch task log handler (#5048)
> > > - [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. (#5164)
> > > - [AIRFLOW-1381] Allow setting host temporary directory in DockerOperator (#5369)
> > > - [AIRFLOW-4598] Task retries are not exhausted for K8s executor (#5347)
> > > - [AIRFLOW-4218] Support to Provide http args to K8executor while calling k8 python client lib apis (#5060)
> > > - [AIRFLOW-4159] Add support for additional static pod labels for K8sExecutor (#5134)
> > > - [AIRFLOW-4720] Allow comments in .airflowignore files. (#5355)
> > > - [AIRFLOW-4486] Add AWS IAM authenication in MySqlHook (#5334)
> > > - [AIRFLOW-4417] Add AWS IAM authenication for PostgresHook (#5223)
> > > - [AIRFLOW-3990] Compile regular expressions. (#4813)
> > > - [AIRFLOW-4572] Rename prepare_classpath() to prepare_syspath() (#5328)
> > > - [AIRFLOW-3869] Raise consistent exception in AirflowConfigParser.getboolean (#4692)
> > > - [AIRFLOW-4571] Add headers to templated field for SimpleHttpOperator (#5326)
> > > - [AIRFLOW-3867] Rename GCP's subpackage (#4690)
> > > - [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df (#4549)
> > > - [AIRFLOW-4546] Upgrade google-cloud-bigtable. (#5307)
> > > - [AIRFLOW-4519] Optimise operator classname sorting in views (#5282)
> > > - [AIRFLOW-4503] Support fully pig options (#5271)
> > > - [AIRFLOW-4468] add sql_alchemy_max_overflow parameter (#5249)
> > > - [AIRFLOW-4467] Add dataproc_jars to templated fields in Dataproc oper… (#5248)
> > > - [AIRFLOW-4381] Use get_direct_relative_ids get task relatives (#5147)
> > > - [AIRFLOW-3624] Add masterType parameter to MLEngineTrainingOperator (#4428)
> > > - [AIRFLOW-3143] Support Auto-Zone in DataprocClusterCreateOperator (#5169)
> > > - [AIRFLOW-3874] Improve BigQueryHook.run_with_configuration's location support (#4695)
> > > - [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag (#5165)
> > > - [AIRFLOW-4031] Allow for key pair auth in snowflake hook (#4875)
> > > - [AIRFLOW-3901] add role as optional config parameter for SnowflakeHook (#4721)
> > > - [AIRFLOW-3455] add region in snowflake connector (#4285)
> > > - [AIRFLOW-4073] add template_ext for AWS Athena operator (#4907)
> > > - [AIRFLOW-4093] AWSAthenaOperator: Throw exception if job failed/cancelled/reach max retries (#4919)
> > > - [AIRFLOW-4356] Add extra RuntimeEnvironment keys to DataFlowHook (#5149)
> > > - [AIRFLOW-4337] Fix docker-compose deprecation warning in CI (#5119)
> > > - [AIRFLOW-3603] QuboleOperator: Remove SQLCommand from SparkCmd documentation (#4411)
> > > - [AIRFLOW-4328] Fix link to task instances from Pool page (#5124)
> > > - [AIRFLOW-4255] Make GCS Hook Backwards compatible (#5089)
> > > - [AIRFLOW-4103] Allow uppercase letters in dataflow job names (#4925)
> > > - [AIRFLOW-4255] Replace Discovery based api with client based for GCS (#5054)
> > > - [AIRFLOW-4311] Remove sleep in localexecutor (#5096)
> > > - [AIRFLOW-2836] Minor improvement-contrib.sensors.FileSensor (#3674)
> > > - [AIRFLOW-4104] Add type annotations to common classes. (#4926)
> > > - [AIRFLOW-3910] Raise exception explicitly in Connection.get_hook() (#4728)
> > > - [AIRFLOW-3322] Update QuboleHook to fetch args dynamically from qds_sdk (#4165)
> > > - [AIRFLOW-4565] instrument celery executor (#5321)
> > > - [AIRFLOW-4573] Import airflow_local_settings after prepare_classpath (#5330)
> > > - [AIRFLOW-4448] Don't bake ENV and _cmd into tmp config for non-sudo (#4050)
> > > - [AIRFLOW-4295] Make `method` attribute case insensitive in HttpHook (#5313)
> > > - [AIRFLOW-3703] Add dnsPolicy option for KubernetesPodOperator (#4520)
> > > - [AIRFLOW-3057] add prev_*_date_success to template context (#5372)
> > > - [AIRFLOW-4336] Stop showing entire GCS files bytes in log for gcs_download_operator (#5151)
> > > - [AIRFLOW-4528] Cancel DataProc task on timeout (#5293)
> > >
> > > Bug fixes
> > > """""""""
> > >
> > > - [AIRFLOW-4587] Replace self.conn with self.get_conn() in AWSAthenaHook (#5462)
> > > - [AIRFLOW-4516] K8s runAsUser and fsGroup cannot be strings (#5429)
> > > - [AIRFLOW-4298] Stop Scheduler repeatedly warning "connection invalidated" (#5470)
> > > - [AIRFLOW-4559] JenkinsJobTriggerOperator bugfix (#5318)
> > > - [AIRFLOW-4841] Pin Sphinx AutoApi to 1.0.0 (#5468)
> > > - [AIRFLOW-4479] Include s3_overwrite kwarg in load_bytes method (#5312)
> > > - [AIRFLOW-3746] Fix DockerOperator missing container exit (#4583)
> > > - [AIRFLOW-4233] Remove Template Extension from Bq to GCS Operator (#5456)
> > > - [AIRFLOW-2141][AIRFLOW-3157][AIRFLOW-4170] Serialize non-str value by JSON when importing Variables (#4991)
> > > - [AIRFLOW-4826] Remove warning from `airflow resetdb` command (#5447)
> > > - [AIRFLOW-4148] Fix editing DagRuns when clicking state column (#5436)
> > > - [AIRFLOW-4455] dag_details broken for subdags in RBAC UI (#5234)
> > > - [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods (#4551)
> > > - [AIRFLOW-4459] Fix wrong DAG count in /home page when DAG count is zero (#5235)
> > > - [AIRFLOW-3876] AttributeError: module 'distutils' has no attribute 'util'
> > > - [AIRFLOW-4146] Fix CgroupTaskRunner errors (#5224)
> > > - [AIRFLOW-4524] Fix bug with "Ignore \*" toggles in RBAC mode (#5378)
> > > - [AIRFLOW-4765] Fix DataProcPigOperator execute method (#5426)
> > > - [AIRFLOW-4798] obviate interdependencies for dagbag and TI tests (#5422)
> > > - [AIRFLOW-4800] fix GKEClusterHook ctor calls (#5424)
> > > - [AIRFLOW-4799] don't mutate self.env in BashOperator execute method (#5421)
> > > - [AIRFLOW-4393] Add retry logic when fetching pod status and/or logs in KubernetesPodOperator (#5284)
> > > - [AIRFLOW-4174] Fix HttpHook run with backoff (#5213)
> > > - [AIRFLOW-4463] Handle divide-by-zero errors in short retry intervals (#5243)
> > > - [AIRFLOW-2614] Speed up trigger_dag API call when lots of DAGs in system
> > > - [AIRFLOW-4756] add ti.state to ti.start_date as criteria for gantt (#5399)
> > > - [AIRFLOW-4760] Fix zip-packaged DAGs disappearing from DagBag when reloaded (#5404)
> > > - [AIRFLOW-4731] Fix GCS hook with google-storage-client 1.16 (#5368)
> > > - [AIRFLOW-3506] use match_phrase to query log_id in elasticsearch (#4342)
> > > - [AIRFLOW-4084] fix ElasticSearch log download (#5177)
> > > - [AIRFLOW-4501] Register pendulum datetime converter for sqla+pymysql (#5190)
> > > - [AIRFLOW-986] HiveCliHook ignores 'proxy_user' value in a connection's extra parameter (#5305)
> > > - [AIRFLOW-4442] fix hive_tblproperties in HiveToDruidTransfer (#5211)
> > > - [AIRFLOW-4557] Add gcp_conn_id parameter to get_sqlproxy_runner() of CloudSqlDatabaseHook (#5314)
> > > - [AIRFLOW-4545] Upgrade FAB to latest version (#4955)
> > > - [AIRFLOW-4492] Change Dataproc Cluster operators to poll Operations (#5269)
> > > - [AIRFLOW-4452] Webserver and Scheduler keep crashing because of slackclient update (#5225)
> > > - [AIRFLOW-4450] Fix request arguments in has_dag_access (#5220)
> > > - [AIRFLOW-4434] Support Impala with the HiveServer2Hook (#5206)
> > > - [AIRFLOW-3449] Write local dag parsing logs when remote logging enabled. (#5175)
> > > - [AIRFLOW-4300] Fix graph modal call when DAG has not yet run (#5185)
> > > - [AIRFLOW-4401] Use managers for Queue synchronization (#5200)
> > > - [AIRFLOW-3626] Fixed triggering DAGs contained within zip files (#4439)
> > > - [AIRFLOW-3720] Fix missmatch while comparing GCS and S3 files (#4766)
> > > - [AIRFLOW-4403] search by `dag_id` or `owners` in UI (#5184)
> > > - [AIRFLOW-4308] Fix TZ-loop around DST on python 3.6+ (#5095)
> > > - [AIRFLOW-4324] fix DAG fuzzy search in RBAC UI (#5131)
> > > - [AIRFLOW-4297] Temporary hot fix on manage_slas() for 1.10.4 release (#5150)
> > > - [AIRFLOW-4299] Upgrade to Celery 4.3.0 to fix crashing workers (#5116)
> > > - [AIRFLOW-4291] Correctly render doc_md in DAG graph page (#5121)
> > > - [AIRFLOW-4310] Fix incorrect link on Dag Details page (#5122)
> > > - [AIRFLOW-4331] Correct filter for Null-state runs from Dag Detail page (#5123)
> > > - [AIRFLOW-4294] Fix missing dag & task runs in UI dag_id contains a dot (#5111)
> > > - [AIRFLOW-4332] Upgrade sqlalchemy to remove security Vulnerability (#5113)
> > > - [AIRFLOW-4312] Add template_fields & template_ext to BigQueryCheckO… (#5097)
> > > - [AIRFLOW-4293] Fix downgrade in d4ecb8fbee3_add_schedule_interval_to_dag.py (#5086)
> > > - [AIRFLOW-4267] Fix TI duration in Graph View (#5071)
> > > - [AIRFLOW-4163] IntervalCheckOperator supports relative diff and not ignore 0 (#4983)
> > > - [AIRFLOW-3938] QuboleOperator Fixes and Support for SqlCommand (#4832)
> > > - [AIRFLOW-2903] Change default owner to `airflow` (#4151)
> > > - [aIRFLOW-4136] Fix overwrite of key_file by constructor (#5155)
> > > - [AIRFLOW-3241] Remove Invalid template ext in GCS Sensors (#4076)
> > >
> > > Misc/Internal
> > > """""""""""""
> > >
> > > - [AIRFLOW-4211] Add tests for WebHDFSHook (#5015)
> > > - [AIRFLOW-4320] Add tests for SegmentTrackEventOperator (#5104)
> > > - [AIRFLOW-4319] Add tests for Bigquery related Operators (#5101)
> > > - [AIRFLOW-4014] Change DatastoreHook and add tests (#4842)
> > > - [AIRFLOW-4322] Add test for VerticaOperator (#5107)
> > > - [AIRFLOW-4323] Add 2 tests for WinRMOperator (#5108)
> > > - [AIRFLOW-3677] Improve CheckOperator test coverage (#4756)
> > > - [AIRFLOW-4659] Fix pylint problems for api module (#5398)
> > > - [AIRFLOW-4358] Speed up test_jobs by not running tasks (#5162)
> > > - [AIRFLOW-4394] Don't test behaviour of BackfillJob from CLI tests (#5160)
> > > - [AIRFLOW-3471] Move XCom out of models.py (#4629)
> > > - [AIRFLOW-4379] Remove duplicate code & Add validation in gcs_to_gcs.py (#5145)
> > > - [AIRFLOW-4259] Move models out of models.py (#5056)
> > > - [AIRFLOW-XXX] Speed up building of Cassanda module on Travis (#5233)
> > > - [AIRFLOW-4535] Break jobs.py into multiple files (#5303)
> > > - [AIRFLOW-1464] Batch update task_instance state (#5323)
> > > - [AIRFLOW-4554] Test for sudo command, add some other test docs (#5310)
> > > - [AIRFLOW-4419] Refine concurrency check in scheduler (#5194)
> > > - [AIRFLOW-4269] Minor acceleration of jobs._process_task_instances() (#5076)
> > > - [AIRFLOW-4341] Remove `View.render()` already exists in fab.BaseView (#5125)
> > > - [AIRFLOW-4342] Use @cached_property instead of re-implementing it each time (#5126)
> > > - [AIRFLOW-4256] Remove noqa from migrations (#5055)
> > > - [AIRFLOW-4034] Remove unnecessary string formatting with `**locals()` (#4861)
> > > - [AIRFLOW-3944] Remove code smells (#4762)
> > >
> > > Doc-only changes
> > > """"""""""""""""
> > >
> > > - [AIRFLOW-3341] FAQ return DAG object example (#4605)
> > > - [AIRFLOW-4433] Add missing type in DockerOperator doc string (#5205)
> > > - [AIRFLOW-4321] Replace incorrect info of Max Size limit of GCS Object Size (#5106)
> > > - [AIRFLOW-XXX] Add information about user list (#5341)
> > > - [AIRFLOW-XXX] Clarify documentation related to autodetect parameter in GCS_to_BQ Op (#5294)
> > > - [AIRFLOW-XXX] Remove mention of pytz compatibility from timezone documentation (#5316)
> > > - [AIRFLOW-XXX] Add missing docs for GoogleCloudStorageDeleteOperator (#5274)
> > > - [AIRFLOW-XXX] Remove incorrect note about Scopes of GCP connection (#5242)
> > > - [AIRFLOW-XXX] Fix mistakes in docs of Dataproc operators (#5192)
> > > - [AIRFLOW-XXX] Link to correct class for timedelta in macros.rst (#5226)
> > > - [AIRFLOW-XXX] Add Kamil as committer (#5216)
> > > - [AIRFLOW-XXX] Add Joshua and Kevin as committer (#5207)
> > > - [AIRFLOW-XXX] Reduce log spam in tests (#5174)
> > > - [AIRFLOW-XXX] Speed up tests for PythonSensor (#5158)
> > > - [AIRFLOW-XXX] Add Bas Harenslak to committer list (#5157)
> > > - [AIRFLOW-XXX] Add Jarek Potiuk to commiter list (#5132)
> > > - [AIRFLOW-XXX] Update docstring for SchedulerJob (#5105)
> > > - [AIRFLOW-XXX] Fix docstrings for CassandraToGoogleCloudStorageOperator (#5103)
> > > - [AIRFLOW-XXX] update SlackWebhookHook and SlackWebhookOperator docstring (#5074)
> > > - [AIRFLOW-XXX] Ignore python files under node_modules in docs (#5063)
> > > - [AIRFLOW-XXX] Build a universal wheel with LICNESE files (#5052)
> > > - [AIRFLOW-XXX] Fix docstrings of SQSHook (#5099)
> > > - [AIRFLOW-XXX] Use Py3.7 on readthedocs
> > > - [AIRFLOW-4446] Fix typos (#5217)
Re: 1.10.4beta2 "snapshot" for available testing
Posted by Ash Berlin-Taylor <as...@apache.org>.
Because the list was autogenerated from the issue type in the jira ticket, and I didn't check all 200+ items in the change log :)
-a
> On 25 Jun 2019, at 18:59, airflowuser <ai...@protonmail.com.INVALID> wrote:
>
> Why [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365) listed under features?
>
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Tuesday, June 25, 2019 1:40 PM, Ash Berlin-Taylor <as...@apache.org> wrote:
>
>> Hi Everyone,
>>
>> I'm excited to annoucne that 1.10.4 is almost ready for release.
>>
>> The release branch already contains 216 commits since 1.10.3 this time around.
>>
>> In order to distinguish it from an actual (apache) release it is:
>>
>> 1. Marked as beta (python package managers do not install beta versions by default - PEP 440)
>> 2. It is not at an official Apache distribution location
>>
>> It can be installed with:
>>
>> pip install 'apache-airflow==1.10.4b2'
>>
>> or downloaded from https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/ https://dist.apache.org/repos/dist/dev/airflow/1.10.4b2/
>>
>>
>> (Don't worry, without asking for`--pre` or specifying the version `pip install apache-airflow` will still get 1.10.3)
>>
>> Included below is the changelog of this release so far. I'd like to try and get this out before the end of the month, so will create an RC in a day or two baring any major problems reported.
>>
>> Cheers,
>> Ash
>>
>> Airflow 1.10.4
>>
>> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> New Features
>> """"""""""""
>>
>> - [AIRFLOW-4456] Add sub-classable BaseBranchOperator (#5231)
>> - [AIRFLOW-4343] Show warning in UI if scheduler is not running (#5127)
>> - [AIRFLOW-4739] Add ability to arbitrarily define kubernetes worker pod labels (#5376)
>> - [AIRFLOW-4348] Add GCP console link in BigQueryOperator (#5195)
>> - [AIRFLOW-4306] Global operator extra links (#5094)
>> - [AIRFLOW-4812] Add batch images annotation (#5433)
>> - [AIRFLOW-4135] Add Google Cloud Build operator and hook (#5251)
>> - [AIRFLOW-4781] Add the ability to specify ports in kubernetesOperator (#5410)
>> - [AIRFLOW-4521] Pause dag also pause its subdags (#5283)
>> - [AIRFLOW-4738] Enforce exampleinclude for example DAGs (#5375)
>> - [AIRFLOW-4326] Airflow AWS SQS Operator (#5110)
>> - [AIRFLOW-3729] Support "DownwardAPI" in env variables for KubernetesPodOperator (#4554)
>> - [AIRFLOW-4585] Implement Kubernetes Pod Mutation Hook (#5359)
>> - [AIRFLOW-161] New redirect route and extra links (#5059)
>> - [AIRFLOW-4420] Backfill respects task_concurrency (#5221)
>> - [AIRFLOW-4147] Add Operator to publish event to Redis (#4967)
>> - [AIRFLOW-3359] Add option to pass customer encryption keys to Dataproc (#4200)
>> - [AIRFLOW-4318] Create Google Cloud Translate Speech Operator (#5102)
>> - [AIRFLOW-3960] Adds Google Cloud Speech operators (#4780)
>> - [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365)
>> - [AIRFLOW-1501] Add GoogleCloudStorageDeleteOperator (#5230)
>> - [AIRFLOW-3672] Add support for Mongo DB DNS Seedlist Connection Format (#4481)
>> - [AIRFLOW-4397] add integrations docs manually for gcs sensors (#5204)
>> - [AIRFLOW-4251] Instrument DagRun schedule delay (#5050)
>> - [AIRFLOW-4118] instrument DagRun duration (#4946)
>> - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5182)
>> - [AIRFLOW-4361] Fix flaky test_integration_run_dag_with_scheduler_failure (#5140)
>> - [AIRFLOW-4168] Create Google Cloud Video Intelligence Operators (#4985)
>> - [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor (#5166)
>> - [AIRFLOW-4335] Add default num_retries to GCP connection (#5117)
>> - [AIRFLOW-3808] Add cluster_fields to BigQueryHook's create_empty_table (#4654)
>> - [AIRFLOW-4362] Fix test_execution_limited_parallelism (#5141)
>> - [AIRFLOW-4307] Backfill respects concurrency limit (#5128)
>> - [AIRFLOW-4268] Add MsSqlToGoogleCloudStorageOperator (#5077)
>> - [AIRFLOW-4169] Add Google Cloud Vision Detect Operators (#4986)
>> - [AIRFLOW-XXX] Fix WS-2019-0032 (#5384)
>> - [AIRFLOW-XXX] Fix CVE-2019-11358 (#5197)
>> - [AIRFLOW-XXX] Change allowed version of Jinja2 to fix CVE-2019-10906 (#5075)
>>
>> Improvement
>> """""""""""
>>
>> - [AIRFLOW-4805] Add py_file as templated field in DataflowPythonOperator (#5451)
>> - [AIRFLOW-4838] Surface Athena errors in AWSAthenaOperator (#5467)
>> - [AIRFLOW-4831] conf.has_option no longer throws if section is missing. (#5455)
>> - [AIRFLOW-4829] More descriptive exceptions for EMR sensors (#5452)
>> - [AIRFLOW-4414] AWSAthenaOperator: Push QueryExecutionID to XCom (#5276)
>> - [AIRFLOW-4791] add "schema" keyword arg to SnowflakeOperator (#5415)
>> - [AIRFLOW-4759] Don't error when marking sucessful run as failed (#5435)
>> - [AIRFLOW-4716] Instrument dag loading time duration (#5350)
>> - [AIRFLOW-3958] Support list tasks as upstream in chain (#4779)
>> - [AIRFLOW-4409] Prevent task duration break by null value (#5178)
>> - [AIRFLOW-4418] Add "failed only" option to task modal (#5193)
>> - [AIRFLOW-4740] Accept string `end_date` in DAG default_args (#5381)
>> - [AIRFLOW-4423] Improve date handling in mysql to gcs operator. (#5196)
>> - [AIRFLOW-4447] Display task duration as human friendly format in UI (#5218)
>> - [AIRFLOW-4377] Remove needless object conversion in DAG.owner() (#5144)
>> - [AIRFLOW-4766] Add autoscaling option for DataprocClusterCreateOperator (#5425)
>> - [AIRFLOW-4795] Upgrade alembic to latest release. (#5411)
>> - [AIRFLOW-4793] Add signature_name to mlengine operator (#5417)
>> - [AIRFLOW-3211] Reattach to GCP Dataproc jobs upon Airflow restart (#4083)
>> - [AIRFLOW-4750] Log identified zombie task instances (#5389)
>> - [AIRFLOW-3870] STFPOperator: Update log level and return value (#4355)
>> - [AIRFLOW-4759] Batch queries in set_state API. (#5403)
>> - [AIRFLOW-2737] Restore original license header to airflow.api.auth.backend.kerberos_auth
>> - [AIRFLOW-3635] Fix incorrect logic in detele_dag (introduced in PR#4406) (#4445)
>> - [AIRFLOW-3599] Removed Dagbag from delete dag (#4406)
>> - [AIRFLOW-4737] Increase and document celery queue name limit (#5383)
>> - [AIRFLOW-4505] Correct Tag ALL for PY3 (#5275)
>> - [AIRFLOW-4743] Add environment variables support to SSHOperator (#5385)
>> - [AIRFLOW-4725] Fix setup.py PEP440 & Sphinx-PyPI-upload dependency (#5363)
>> - [AIRFLOW-3370] Add stdout output options to Elasticsearch task log handler (#5048)
>> - [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. (#5164)
>> - [AIRFLOW-1381] Allow setting host temporary directory in DockerOperator (#5369)
>> - [AIRFLOW-4598] Task retries are not exhausted for K8s executor (#5347)
>> - [AIRFLOW-4218] Support to Provide http args to K8executor while calling k8 python client lib apis (#5060)
>> - [AIRFLOW-4159] Add support for additional static pod labels for K8sExecutor (#5134)
>> - [AIRFLOW-4720] Allow comments in .airflowignore files. (#5355)
>> - [AIRFLOW-4486] Add AWS IAM authenication in MySqlHook (#5334)
>> - [AIRFLOW-4417] Add AWS IAM authenication for PostgresHook (#5223)
>> - [AIRFLOW-3990] Compile regular expressions. (#4813)
>> - [AIRFLOW-4572] Rename prepare_classpath() to prepare_syspath() (#5328)
>> - [AIRFLOW-3869] Raise consistent exception in AirflowConfigParser.getboolean (#4692)
>> - [AIRFLOW-4571] Add headers to templated field for SimpleHttpOperator (#5326)
>> - [AIRFLOW-3867] Rename GCP's subpackage (#4690)
>> - [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df (#4549)
>> - [AIRFLOW-4546] Upgrade google-cloud-bigtable. (#5307)
>> - [AIRFLOW-4519] Optimise operator classname sorting in views (#5282)
>> - [AIRFLOW-4503] Support fully pig options (#5271)
>> - [AIRFLOW-4468] add sql_alchemy_max_overflow parameter (#5249)
>> - [AIRFLOW-4467] Add dataproc_jars to templated fields in Dataproc oper… (#5248)
>> - [AIRFLOW-4381] Use get_direct_relative_ids get task relatives (#5147)
>> - [AIRFLOW-3624] Add masterType parameter to MLEngineTrainingOperator (#4428)
>> - [AIRFLOW-3143] Support Auto-Zone in DataprocClusterCreateOperator (#5169)
>> - [AIRFLOW-3874] Improve BigQueryHook.run_with_configuration's location support (#4695)
>> - [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag (#5165)
>> - [AIRFLOW-4031] Allow for key pair auth in snowflake hook (#4875)
>> - [AIRFLOW-3901] add role as optional config parameter for SnowflakeHook (#4721)
>> - [AIRFLOW-3455] add region in snowflake connector (#4285)
>> - [AIRFLOW-4073] add template_ext for AWS Athena operator (#4907)
>> - [AIRFLOW-4093] AWSAthenaOperator: Throw exception if job failed/cancelled/reach max retries (#4919)
>> - [AIRFLOW-4356] Add extra RuntimeEnvironment keys to DataFlowHook (#5149)
>> - [AIRFLOW-4337] Fix docker-compose deprecation warning in CI (#5119)
>> - [AIRFLOW-3603] QuboleOperator: Remove SQLCommand from SparkCmd documentation (#4411)
>> - [AIRFLOW-4328] Fix link to task instances from Pool page (#5124)
>> - [AIRFLOW-4255] Make GCS Hook Backwards compatible (#5089)
>> - [AIRFLOW-4103] Allow uppercase letters in dataflow job names (#4925)
>> - [AIRFLOW-4255] Replace Discovery based api with client based for GCS (#5054)
>> - [AIRFLOW-4311] Remove sleep in localexecutor (#5096)
>> - [AIRFLOW-2836] Minor improvement-contrib.sensors.FileSensor (#3674)
>> - [AIRFLOW-4104] Add type annotations to common classes. (#4926)
>> - [AIRFLOW-3910] Raise exception explicitly in Connection.get_hook() (#4728)
>> - [AIRFLOW-3322] Update QuboleHook to fetch args dynamically from qds_sdk (#4165)
>> - [AIRFLOW-4565] instrument celery executor (#5321)
>> - [AIRFLOW-4573] Import airflow_local_settings after prepare_classpath (#5330)
>> - [AIRFLOW-4448] Don't bake ENV and _cmd into tmp config for non-sudo (#4050)
>> - [AIRFLOW-4295] Make `method` attribute case insensitive in HttpHook (#5313)
>> - [AIRFLOW-3703] Add dnsPolicy option for KubernetesPodOperator (#4520)
>> - [AIRFLOW-3057] add prev_*_date_success to template context (#5372)
>> - [AIRFLOW-4336] Stop showing entire GCS files bytes in log for gcs_download_operator (#5151)
>> - [AIRFLOW-4528] Cancel DataProc task on timeout (#5293)
>>
>> Bug fixes
>> """""""""
>>
>> - [AIRFLOW-4587] Replace self.conn with self.get_conn() in AWSAthenaHook (#5462)
>> - [AIRFLOW-4516] K8s runAsUser and fsGroup cannot be strings (#5429)
>> - [AIRFLOW-4298] Stop Scheduler repeatedly warning "connection invalidated" (#5470)
>> - [AIRFLOW-4559] JenkinsJobTriggerOperator bugfix (#5318)
>> - [AIRFLOW-4841] Pin Sphinx AutoApi to 1.0.0 (#5468)
>> - [AIRFLOW-4479] Include s3_overwrite kwarg in load_bytes method (#5312)
>> - [AIRFLOW-3746] Fix DockerOperator missing container exit (#4583)
>> - [AIRFLOW-4233] Remove Template Extension from Bq to GCS Operator (#5456)
>> - [AIRFLOW-2141][AIRFLOW-3157][AIRFLOW-4170] Serialize non-str value by JSON when importing Variables (#4991)
>> - [AIRFLOW-4826] Remove warning from `airflow resetdb` command (#5447)
>> - [AIRFLOW-4148] Fix editing DagRuns when clicking state column (#5436)
>> - [AIRFLOW-4455] dag_details broken for subdags in RBAC UI (#5234)
>> - [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods (#4551)
>> - [AIRFLOW-4459] Fix wrong DAG count in /home page when DAG count is zero (#5235)
>> - [AIRFLOW-3876] AttributeError: module 'distutils' has no attribute 'util'
>> - [AIRFLOW-4146] Fix CgroupTaskRunner errors (#5224)
>> - [AIRFLOW-4524] Fix bug with "Ignore \*" toggles in RBAC mode (#5378)
>> - [AIRFLOW-4765] Fix DataProcPigOperator execute method (#5426)
>> - [AIRFLOW-4798] obviate interdependencies for dagbag and TI tests (#5422)
>> - [AIRFLOW-4800] fix GKEClusterHook ctor calls (#5424)
>> - [AIRFLOW-4799] don't mutate self.env in BashOperator execute method (#5421)
>> - [AIRFLOW-4393] Add retry logic when fetching pod status and/or logs in KubernetesPodOperator (#5284)
>> - [AIRFLOW-4174] Fix HttpHook run with backoff (#5213)
>> - [AIRFLOW-4463] Handle divide-by-zero errors in short retry intervals (#5243)
>> - [AIRFLOW-2614] Speed up trigger_dag API call when lots of DAGs in system
>> - [AIRFLOW-4756] add ti.state to ti.start_date as criteria for gantt (#5399)
>> - [AIRFLOW-4760] Fix zip-packaged DAGs disappearing from DagBag when reloaded (#5404)
>> - [AIRFLOW-4731] Fix GCS hook with google-storage-client 1.16 (#5368)
>> - [AIRFLOW-3506] use match_phrase to query log_id in elasticsearch (#4342)
>> - [AIRFLOW-4084] fix ElasticSearch log download (#5177)
>> - [AIRFLOW-4501] Register pendulum datetime converter for sqla+pymysql (#5190)
>> - [AIRFLOW-986] HiveCliHook ignores 'proxy_user' value in a connection's extra parameter (#5305)
>> - [AIRFLOW-4442] fix hive_tblproperties in HiveToDruidTransfer (#5211)
>> - [AIRFLOW-4557] Add gcp_conn_id parameter to get_sqlproxy_runner() of CloudSqlDatabaseHook (#5314)
>> - [AIRFLOW-4545] Upgrade FAB to latest version (#4955)
>> - [AIRFLOW-4492] Change Dataproc Cluster operators to poll Operations (#5269)
>> - [AIRFLOW-4452] Webserver and Scheduler keep crashing because of slackclient update (#5225)
>> - [AIRFLOW-4450] Fix request arguments in has_dag_access (#5220)
>> - [AIRFLOW-4434] Support Impala with the HiveServer2Hook (#5206)
>> - [AIRFLOW-3449] Write local dag parsing logs when remote logging enabled. (#5175)
>> - [AIRFLOW-4300] Fix graph modal call when DAG has not yet run (#5185)
>> - [AIRFLOW-4401] Use managers for Queue synchronization (#5200)
>> - [AIRFLOW-3626] Fixed triggering DAGs contained within zip files (#4439)
>> - [AIRFLOW-3720] Fix missmatch while comparing GCS and S3 files (#4766)
>> - [AIRFLOW-4403] search by `dag_id` or `owners` in UI (#5184)
>> - [AIRFLOW-4308] Fix TZ-loop around DST on python 3.6+ (#5095)
>> - [AIRFLOW-4324] fix DAG fuzzy search in RBAC UI (#5131)
>> - [AIRFLOW-4297] Temporary hot fix on manage_slas() for 1.10.4 release (#5150)
>> - [AIRFLOW-4299] Upgrade to Celery 4.3.0 to fix crashing workers (#5116)
>> - [AIRFLOW-4291] Correctly render doc_md in DAG graph page (#5121)
>> - [AIRFLOW-4310] Fix incorrect link on Dag Details page (#5122)
>> - [AIRFLOW-4331] Correct filter for Null-state runs from Dag Detail page (#5123)
>> - [AIRFLOW-4294] Fix missing dag & task runs in UI dag_id contains a dot (#5111)
>> - [AIRFLOW-4332] Upgrade sqlalchemy to remove security Vulnerability (#5113)
>> - [AIRFLOW-4312] Add template_fields & template_ext to BigQueryCheckO… (#5097)
>> - [AIRFLOW-4293] Fix downgrade in d4ecb8fbee3_add_schedule_interval_to_dag.py (#5086)
>> - [AIRFLOW-4267] Fix TI duration in Graph View (#5071)
>> - [AIRFLOW-4163] IntervalCheckOperator supports relative diff and not ignore 0 (#4983)
>> - [AIRFLOW-3938] QuboleOperator Fixes and Support for SqlCommand (#4832)
>> - [AIRFLOW-2903] Change default owner to `airflow` (#4151)
>> - [aIRFLOW-4136] Fix overwrite of key_file by constructor (#5155)
>> - [AIRFLOW-3241] Remove Invalid template ext in GCS Sensors (#4076)
>>
>> Misc/Internal
>> """""""""""""
>>
>> - [AIRFLOW-4211] Add tests for WebHDFSHook (#5015)
>> - [AIRFLOW-4320] Add tests for SegmentTrackEventOperator (#5104)
>> - [AIRFLOW-4319] Add tests for Bigquery related Operators (#5101)
>> - [AIRFLOW-4014] Change DatastoreHook and add tests (#4842)
>> - [AIRFLOW-4322] Add test for VerticaOperator (#5107)
>> - [AIRFLOW-4323] Add 2 tests for WinRMOperator (#5108)
>> - [AIRFLOW-3677] Improve CheckOperator test coverage (#4756)
>> - [AIRFLOW-4659] Fix pylint problems for api module (#5398)
>> - [AIRFLOW-4358] Speed up test_jobs by not running tasks (#5162)
>> - [AIRFLOW-4394] Don't test behaviour of BackfillJob from CLI tests (#5160)
>> - [AIRFLOW-3471] Move XCom out of models.py (#4629)
>> - [AIRFLOW-4379] Remove duplicate code & Add validation in gcs_to_gcs.py (#5145)
>> - [AIRFLOW-4259] Move models out of models.py (#5056)
>> - [AIRFLOW-XXX] Speed up building of Cassanda module on Travis (#5233)
>> - [AIRFLOW-4535] Break jobs.py into multiple files (#5303)
>> - [AIRFLOW-1464] Batch update task_instance state (#5323)
>> - [AIRFLOW-4554] Test for sudo command, add some other test docs (#5310)
>> - [AIRFLOW-4419] Refine concurrency check in scheduler (#5194)
>> - [AIRFLOW-4269] Minor acceleration of jobs._process_task_instances() (#5076)
>> - [AIRFLOW-4341] Remove `View.render()` already exists in fab.BaseView (#5125)
>> - [AIRFLOW-4342] Use @cached_property instead of re-implementing it each time (#5126)
>> - [AIRFLOW-4256] Remove noqa from migrations (#5055)
>> - [AIRFLOW-4034] Remove unnecessary string formatting with `**locals()` (#4861)
>> - [AIRFLOW-3944] Remove code smells (#4762)
>>
>> Doc-only changes
>> """"""""""""""""
>>
>> - [AIRFLOW-3341] FAQ return DAG object example (#4605)
>> - [AIRFLOW-4433] Add missing type in DockerOperator doc string (#5205)
>> - [AIRFLOW-4321] Replace incorrect info of Max Size limit of GCS Object Size (#5106)
>> - [AIRFLOW-XXX] Add information about user list (#5341)
>> - [AIRFLOW-XXX] Clarify documentation related to autodetect parameter in GCS_to_BQ Op (#5294)
>> - [AIRFLOW-XXX] Remove mention of pytz compatibility from timezone documentation (#5316)
>> - [AIRFLOW-XXX] Add missing docs for GoogleCloudStorageDeleteOperator (#5274)
>> - [AIRFLOW-XXX] Remove incorrect note about Scopes of GCP connection (#5242)
>> - [AIRFLOW-XXX] Fix mistakes in docs of Dataproc operators (#5192)
>> - [AIRFLOW-XXX] Link to correct class for timedelta in macros.rst (#5226)
>> - [AIRFLOW-XXX] Add Kamil as committer (#5216)
>> - [AIRFLOW-XXX] Add Joshua and Kevin as committer (#5207)
>> - [AIRFLOW-XXX] Reduce log spam in tests (#5174)
>> - [AIRFLOW-XXX] Speed up tests for PythonSensor (#5158)
>> - [AIRFLOW-XXX] Add Bas Harenslak to committer list (#5157)
>> - [AIRFLOW-XXX] Add Jarek Potiuk to commiter list (#5132)
>> - [AIRFLOW-XXX] Update docstring for SchedulerJob (#5105)
>> - [AIRFLOW-XXX] Fix docstrings for CassandraToGoogleCloudStorageOperator (#5103)
>> - [AIRFLOW-XXX] update SlackWebhookHook and SlackWebhookOperator docstring (#5074)
>> - [AIRFLOW-XXX] Ignore python files under node_modules in docs (#5063)
>> - [AIRFLOW-XXX] Build a universal wheel with LICNESE files (#5052)
>> - [AIRFLOW-XXX] Fix docstrings of SQSHook (#5099)
>> - [AIRFLOW-XXX] Use Py3.7 on readthedocs
>> - [AIRFLOW-4446] Fix typos (#5217)
>
>