You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Ash Berlin-Taylor <as...@apache.org> on 2018/11/18 20:45:28 UTC
[RESULT][VOTE] Airflow 1.10.1 RC2
Hello,
Apache Airflow (incubating) 1.10.1 (based on RC2) has been accepted.
3 “+1” binding votes received:
- Ash Berlin-Taylor (binding)
- Kaxil Naik (binding)
- Fokko Driesprong (binding)
2 "+1" non-binding votes received:
- Deng Xiaodong (non-binding)
- Ikar Pohorsky (non-binding)
My next step is to open a thread with the IPMC.
Cheers,
Ash
> On 18 Nov 2018, at 16:47, Driesprong, Fokko <fo...@driesprong.frl> wrote:
>
> A +1 from my side as well.
>
> Thanks for picking this up Ash. Just checked the new release using Docke
> <https://github.com/Fokko/docker-airflow/commit/eb904450ffbc38cee61421ad8c6ff7cfd28c42eb>r,
> everything seems to work.
>
> Cheers, Fokko
>
> Op za 17 nov. 2018 om 16:43 schreef Deng Xiaodong <xd...@gmail.com>:
>
>> Even though my vote is non-binding, I would like to change my vote to +1 as
>> well.
>> Reason being the both points I suggested earlier were not regressions from
>> 1.10.0, and they should not be blocking the release.
>>
>> Cheers.
>>
>> XD
>>
>> On Sat, Nov 17, 2018 at 8:11 PM Naik Kaxil <k....@reply.com> wrote:
>>
>>> +1 (binding) . I am convinced, we should follow up with 1.10.2 with fixes
>>> soon with small number of commits avoiding a huge gap again between minor
>>> releases.
>>>
>>> Regards,
>>> Kaxil
>>>
>>> On 17/11/2018, 11:53, "Ash Berlin-Taylor" <as...@apache.org> wrote:
>>>
>>> The RBAC UI is still marked as experimental and this isn't a
>>> regression from 1.10.0, so could you be convinced to change this to a +1?
>>>
>>> There are other more critical changes I would like to get out, and I
>>> will follow up straight away with a 1.10.2 that addresses this and XD's
>>> points.
>>>
>>> (I feel Bolke's pain :) I'm now moderately annoyed at the Apache
>>> release process and how long it takes, it means each release ends up
>>> getting big)
>>>
>>> -ash
>>>
>>>>
>>>
>>> Kaxil Naik
>>>
>>> Data Reply
>>> Nova South
>>> 160 Victoria Street, Westminster
>>> London SW1E 5LB - UK
>>> phone: +44 (0)20 7730 6000
>>> k.naik@reply.com
>>> www.reply.com
>>> On 17 Nov 2018, at 01:01, Naik Kaxil <k....@reply.com> wrote:
>>>>
>>>> -1 (binding) . Tested it on Python 2.7.14, got expected result but
>>> had 1 security concern that I want to get in the release.
>>>>
>>>> Even when 'expose_config'=False, RBAC you still shows the configs
>>> which can contain sensitive information like airflow metadb passwords,
>> etc.
>>>>
>>>> If we can get that in +1 from me. The PR with this fixed has been
>>> merged in the master, commit:
>>>
>> https://github.com/apache/incubator-airflow/commit/85abd44e241e17338a800e37a3c2e85ef346898d
>>> <
>>>
>> https://github.com/apache/incubator-airflow/commit/85abd44e241e17338a800e37a3c2e85ef346898d
>>>>
>>>>
>>>> PR: https://github.com/apache/incubator-airflow/pull/4194 <
>>> https://github.com/apache/incubator-airflow/pull/4194>
>>>>
>>>> Regards,
>>>> Kaxil
>>>>
>>>> On 16/11/2018, 13:41, "Deng Xiaodong" <xd.deng.r@gmail.com
>> <mailto:
>>> xd.deng.r@gmail.com>> wrote:
>>>>
>>>> Hi Ash,
>>>>
>>>> I would like to give -1 (non-binding), due to two reasons we
>>> discussed
>>>> earlier on Slack:
>>>>
>>>> - there is an issue with the new “delete DAG” button in UI. It’s
>>> a great
>>>> feature, so let’s try to release it “bug-less”. The fix is in PR
>>>> https://github.com/apache/incubator-airflow/pull/4069 (But
>>> understand your
>>>> concern is that this PR comes with no test yet).
>>>>
>>>> - it may be good to pin all dependencies to a specific version
>> to
>>> avoid the
>>>> incident caused by dependency breaking change (like what happens
>>> to Redis
>>>> yesterday)
>>>>
>>>>
>>>> Last but not least: nice job! Thanks for your works!
>>>>
>>>>
>>>> XD
>>>>
>>>>
>>>> On Fri, Nov 16, 2018 at 21:13 Ash Berlin-Taylor <ash@apache.org
>>>
>>> wrote:
>>>>
>>>>> Friendly reminder for people (and especially committers) to test
>>> this out
>>>>> and vote on it please!
>>>>>
>>>>> -ash
>>>>>
>>>>>>
>>>>
>>>> Kaxil Naik
>>>>
>>>> Data Reply
>>>> Nova South
>>>> 160 Victoria Street, Westminster
>>>> London SW1E 5LB - UK
>>>> phone: +44 (0)20 7730 6000
>>>> k.naik@reply.com <ma...@reply.com>
>>>> www.reply.com <http://www.reply.com/>
>>>> On 14 Nov 2018, at 22:31, Ash Berlin-Taylor <ash@apache.org
>> <mailto:
>>> ash@apache.org>> wrote:
>>>>>>
>>>>>> Hey all,
>>>>>>
>>>>>> I have cut Airflow 1.10.1 RC2. This email is calling a vote on
>> the
>>>>> release, which will last for 72 hours. Consider this my (binding)
>>> +1.
>>>>>>
>>>>>> Airflow 1.10.1 RC2 is available at:
>>>>>>
>>>>>>
>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.1rc2/
>>>>>>
>>>>>> apache-airflow-1.10.1rc2+incubating-source.tar.gz is a source
>>> release
>>>>> that comes with INSTALL instructions.
>>>>>> apache-airflow-1.10.1rc2+incubating-bin.tar.gz is the binary
>> Python
>>>>> "sdist" release.
>>>>>>
>>>>>> Public keys are available at:
>>>>>>
>>>>>>
>> https://dist.apache.org/repos/dist/release/incubator/airflow/KEYS
>>>>>>
>>>>>> This release candidate has been published to PyPi as a convince
>> for
>>>>> testing, but the vote is against the published artefacts at the
>>> above URL,
>>>>> and not this. To install from PyPI run `pip install --pre
>>> apache-airflow`
>>>>>>
>>>>>> Only votes from PMC members are binding, but members of the
>>> community
>>>>> are encouraged to test the release and vote with "(non-binding)".
>>>>>>
>>>>>> Changes since 1.10.1rc1:
>>>>>>
>>>>>> [AIRFLOW-3343] Update DockerOperator for Docker-py 3.0.0 API
>>> changes
>>>>> (#4187)
>>>>>> [AIRFLOW-XXX] Include 3193 in the changelog
>>>>>> [AIRFLOW-XXX] Remove duplicated line in Changelog (#4181)
>>>>>> [AIRFLOW-3339] Correctly get DAG timezone when start_date in
>>>>> default_args (#4186)
>>>>>>
>>>>>> Changes since 1.10.1b1:
>>>>>>
>>>>>> [AIRFLOW-XXX] Correct date and version in Changelog
>>>>>> [AIRFLOW-2779] Add license headers to doc files (#4178)
>>>>>> [AIRFLOW-XXX] Changelog and version for 1.10.1
>>>>>> [AIRFLOW-2779] Add license headers to doc files (#4178)
>>>>>> [AIRFLOW-2779] Add project version to license (#4177)
>>>>>> [AIRFLOW-XXX] Sync changelog between release and master branch
>>>>>> [AIRFLOW-XXX] Add missing docs for SNS classes (#4155)
>>>>>> [AIRFLOW-3072] Assign permission get_logs_with_metadata to viewer
>>> role
>>>>> (#4175)
>>>>>> [AIRFLOW-2723] Update lxml dependancy to >= 4.0.0
>>>>>> [AIRFLOW-3325] Fix UI Page DAGs-column 'Recent Tasks' display
>> issue
>>>>> (#4173)
>>>>>> [AIRFLOW-XXX] Update Updating instructions for changes in 1.10.1
>>>>>> [AIRFLOW-XXX] Fix a few typos in CHANGELOG (#4169)
>>>>>>
>>>>>>
>>>>>> Full changelog is below:
>>>>>>
>>>>>> New features:
>>>>>>
>>>>>> [AIRFLOW-2524] Airflow integration with AWS Sagemaker
>>>>>> [AIRFLOW-2657] Add ability to delete DAG from web ui
>>>>>> [AIRFLOW-2780] Adds IMAP Hook to interact with a mail server
>>>>>> [AIRFLOW-2794] Add delete support for Azure blob
>>>>>> [AIRFLOW-2912] Add operators for Google Cloud Functions
>>>>>> [AIRFLOW-2974] Add Start/Restart/Terminate methods Databricks
>> Hook
>>>>>> [AIRFLOW-2989] No Parameter to change bootDiskType for
>>>>> DataprocClusterCreateOperator
>>>>>> [AIRFLOW-3078] Basic operators for Google Compute Engine
>>>>>> [AIRFLOW-3147] Update Flask-AppBuilder version
>>>>>> [AIRFLOW-3231] Basic operators for Google Cloud SQL (deploy /
>>> patch /
>>>>> delete)
>>>>>> [AIRFLOW-3276] Google Cloud SQL database create / patch / delete
>>>>> operators
>>>>>>
>>>>>> Improvements:
>>>>>>
>>>>>> [AIRFLOW-393] Add progress callbacks for FTP downloads
>>>>>> [AIRFLOW-520] Show Airflow version on web page
>>>>>> [AIRFLOW-843] Exceptions now available in context during
>>>>> on_failure_callback
>>>>>> [AIRFLOW-2476] Update tabulate dependency to v0.8.2
>>>>>> [AIRFLOW-2592] Bump Bleach dependency
>>>>>> [AIRFLOW-2622] Add "confirm=False" option to SFTPOperator
>>>>>> [AIRFLOW-2662] support affinity & nodeSelector policies for
>>> kubernetes
>>>>> executor/operator
>>>>>> [AIRFLOW-2709] Improve error handling in Databricks hook
>>>>>> [AIRFLOW-2723] Update lxml dependancy to >= 4.0.
>>>>>> [AIRFLOW-2763] No precheck mechanism in place during worker
>>>>> initialisation for the connection to metadata database
>>>>>> [AIRFLOW-2789] Add ability to create single node cluster to
>>>>> DataprocClusterCreateOperator
>>>>>> [AIRFLOW-2797] Add ability to create Google Dataproc cluster with
>>> custom
>>>>> image
>>>>>> [AIRFLOW-2854] kubernetes_pod_operator add more configuration
>> items
>>>>>> [AIRFLOW-2855] Need to Check Validity of Cron Expression When
>>> Process
>>>>> DAG File/Zip File
>>>>>> [AIRFLOW-2904] Clean an unnecessary line in
>>>>> airflow/executors/celery_executor.py
>>>>>> [AIRFLOW-2921] A trivial incorrectness in CeleryExecutor()
>>>>>> [AIRFLOW-2922] Potential deal-lock bug in CeleryExecutor()
>>>>>> [AIRFLOW-2932] GoogleCloudStorageHook - allow compression of file
>>>>>> [AIRFLOW-2949] Syntax Highlight for Single Quote
>>>>>> [AIRFLOW-2951] dag_run end_date Null after a dag is finished
>>>>>> [AIRFLOW-2956] Kubernetes tolerations for pod operator
>>>>>> [AIRFLOW-2997] Support for clustered tables in Bigquery
>>> hooks/operators
>>>>>> [AIRFLOW-3006] Fix error when schedule_interval="None"
>>>>>> [AIRFLOW-3008] Move Kubernetes related example DAGs to
>>>>> contrib/example_dags
>>>>>> [AIRFLOW-3025] Allow to specify dns and dns-search parameters for
>>>>> DockerOperator
>>>>>> [AIRFLOW-3067] (www_rbac) Flask flash messages are not displayed
>>>>> properly (no background color)
>>>>>> [AIRFLOW-3069] Decode output of S3 file transform operator
>>>>>> [AIRFLOW-3072] Assign permission get_logs_with_metadata to viewer
>>> role
>>>>>> [AIRFLOW-3090] INFO logs are too verbose
>>>>>> [AIRFLOW-3103] Update Flask-Login
>>>>>> [AIRFLOW-3112] Align SFTP hook with SSH hook
>>>>>> [AIRFLOW-3119] Enable loglevel on celery worker and inherit from
>>>>> airflow.cfg
>>>>>> [AIRFLOW-3137] Make ProxyFix middleware optional
>>>>>> [AIRFLOW-3173] Add _cmd options for more password config options
>>>>>> [AIRFLOW-3177] Change scheduler_heartbeat metric from gauge to
>>> counter
>>>>>> [AIRFLOW-3195] Druid Hook: Log ingestion spec and task id
>>>>>> [AIRFLOW-3197] EMR Hook is missing some parameters to valid on
>> the
>>> AWS
>>>>> API
>>>>>> [AIRFLOW-3232] Make documentation for GCF Functions operator more
>>>>> readable
>>>>>> [AIRFLOW-3262] Can't get log containing Response when using
>>>>> SimpleHttpOperator
>>>>>> [AIRFLOW-3265] Add support for "unix_socket" in connection extra
>>> for
>>>>> Mysql Hook
>>>>>>
>>>>>> Doc-only changes:
>>>>>>
>>>>>> [AIRFLOW-1441] Tutorial Inconsistencies Between Example Pipeline
>>>>> Definition and Recap
>>>>>> [AIRFLOW-2682] Add how-to guide(s) for how to use basic operators
>>> like
>>>>> BashOperator and PythonOperator
>>>>>> [AIRFLOW-3104] .airflowignore feature is not mentioned at all in
>>>>> documentation
>>>>>> [AIRFLOW-3237] Refactor example DAGs
>>>>>> [AIRFLOW-3187] Update airflow.gif file with a slower version
>>>>>> [AIRFLOW-3159] Update Airflow documentation on GCP Logging
>>>>>> [AIRFLOW-3030] Command Line docs incorrect subdir
>>>>>> [AIRFLOW-2990] Docstrings for Hooks/Operators are in incorrect
>>> format
>>>>>> [AIRFLOW-3127] Celery SSL Documentation is out-dated
>>>>>> [AIRFLOW-2779] Add license headers to doc files
>>>>>> [AIRFLOW-2779] Add project version to license
>>>>>>
>>>>>> Bug fixes:
>>>>>>
>>>>>> [AIRFLOW-839] docker_operator.py attempts to log status key
>> without
>>>>> first checking existence
>>>>>> [AIRFLOW-1104] Concurrency check in scheduler should count queued
>>> tasks
>>>>> as well as running
>>>>>> [AIRFLOW-1163] Add support for x-forwarded-* headers to support
>>> access
>>>>> behind AWS ELB
>>>>>> [AIRFLOW-1195] Cleared tasks in SubDagOperator do not trigger
>>> Parent
>>>>> dag_runs
>>>>>> [AIRFLOW-1508] Skipped state not part of State.task_states
>>>>>> [AIRFLOW-1762] Use key_file in SSHHook.create_tunnel()
>>>>>> [AIRFLOW-1837] Differing start_dates on tasks not respected by
>>> scheduler.
>>>>>> [AIRFLOW-1874] Support standard SQL in Check, ValueCheck and
>>>>> IntervalCheck BigQuery operators
>>>>>> [AIRFLOW-1917] print() from python operators end up with extra
>> new
>>> line
>>>>>> [AIRFLOW-1970] Database cannot be initialized if an invalid
>> fernet
>>> key
>>>>> is provided
>>>>>> [AIRFLOW-2145] Deadlock after clearing a running task
>>>>>> [AIRFLOW-2216] Cannot specify a profile for AWS Hook to load with
>>> s3
>>>>> config file
>>>>>> [AIRFLOW-2574] initdb fails when mysql password contains percent
>>> sign
>>>>>> [AIRFLOW-2707] Error accessing log files from web UI
>>>>>> [AIRFLOW-2716] Replace new Python 3.7 keywords
>>>>>> [AIRFLOW-2744] RBAC app doesn't integrate plugins (blueprints
>> etc)
>>>>>> [AIRFLOW-2772] BigQuery hook does not allow specifying both the
>>>>> partition field name and table name at the same time
>>>>>> [AIRFLOW-2778] Bad Import in collect_dag in DagBag
>>>>>> [AIRFLOW-2786] Variables view fails to render if a variable has
>> an
>>> empty
>>>>> key
>>>>>> [AIRFLOW-2799] Filtering UI objects by datetime is broken
>>>>>> [AIRFLOW-2800] Remove airflow/ low-hanging linting errors
>>>>>> [AIRFLOW-2825] S3ToHiveTransfer operator may not may able to
>>> handle GZIP
>>>>> file with uppercase ext in S3
>>>>>> [AIRFLOW-2848] dag_id is missing in metadata table "job" for
>>> LocalTaskJob
>>>>>> [AIRFLOW-2860] DruidHook: time variable is not updated correctly
>>> when
>>>>> checking for timeout
>>>>>> [AIRFLOW-2865] Race condition between on_success_callback and
>>>>> LocalTaskJob's cleanup
>>>>>> [AIRFLOW-2893] Stuck dataflow job due to jobName mismatch.
>>>>>> [AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs
>>>>>> [AIRFLOW-2900] Code not visible for Packaged DAGs
>>>>>> [AIRFLOW-2905] Switch to regional dataflow job service.
>>>>>> [AIRFLOW-2907] Sendgrid - Attachments - ERROR - Object of type
>>> 'bytes'
>>>>> is not JSON serializable
>>>>>> [AIRFLOW-2938] Invalid 'extra' field in connection can raise an
>>>>> AttributeError when attempting to edit
>>>>>> [AIRFLOW-2979] Deprecated Celery Option not in Options list
>>>>>> [AIRFLOW-2981] TypeError in dataflow operators when using GCS jar
>>> or
>>>>> py_file
>>>>>> [AIRFLOW-2984] Cannot convert naive_datetime when task has a
>> naive
>>>>> start_date/end_date
>>>>>> [AIRFLOW-2994] flatten_results in BigQueryOperator/BigQueryHook
>>> should
>>>>> default to None
>>>>>> [AIRFLOW-3002] ValueError in dataflow operators when using GCS
>> jar
>>> or
>>>>> py_file
>>>>>> [AIRFLOW-3012] Email on sla miss is send only to first address on
>>> the
>>>>> list
>>>>>> [AIRFLOW-3046] ECS Operator mistakenly reports success when task
>> is
>>>>> killed due to EC2 host termination
>>>>>> [AIRFLOW-3064] No output from `airflow test` due to default
>> logging
>>>>> config
>>>>>> [AIRFLOW-3072] Only admin can view logs in RBAC UI
>>>>>> [AIRFLOW-3079] Improve initdb to support MSSQL Server
>>>>>> [AIRFLOW-3089] Google auth doesn't work under http
>>>>>> [AIRFLOW-3099] Errors raised when some blocs are missing in
>>> airflow.cfg
>>>>>> [AIRFLOW-3109] Default user permission should contain 'can_clear'
>>>>>> [AIRFLOW-3111] Confusing comments and instructions for log
>>> templates in
>>>>> UPDATING.md and default_airflow.cfg
>>>>>> [AIRFLOW-3124] Broken webserver debug mode (RBAC)
>>>>>> [AIRFLOW-3136] Scheduler Failing the Task retries run while
>>> processing
>>>>> Executor Events
>>>>>> [AIRFLOW-3138] Migration cc1e65623dc7 creates issues with
>> postgres
>>>>>> [AIRFLOW-3161] Log Url link does not link to task instance logs
>> in
>>> RBAC
>>>>> UI
>>>>>> [AIRFLOW-3162] HttpHook fails to parse URL when port is specified
>>>>>> [AIRFLOW-3183] Potential Bug in
>>>>> utils/dag_processing/DagFileProcessorManager.max_runs_reached()
>>>>>> [AIRFLOW-3203] Bugs in DockerOperator & Some operator test
>> scripts
>>> were
>>>>> named incorrectly
>>>>>> [AIRFLOW-3238] Dags, removed from the filesystem, are not
>>> deactivated on
>>>>> initdb
>>>>>> [AIRFLOW-3268] Cannot pass SSL dictionary to mysql connection via
>>> URL
>>>>>> [AIRFLOW-3277] Invalid timezone transition handling for cron
>>> schedules
>>>>>> [AIRFLOW-3295] Require encryption in DaskExecutor when
>>> certificates are
>>>>> configured.
>>>>>> [AIRFLOW-3297] EmrStepSensor marks cancelled step as successful
>>>
>>>
>>>
>>>
>>