You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Kaxil Naik <ka...@gmail.com> on 2018/09/21 22:34:32 UTC
Fwd: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
are missing in airflow.cfg
Hi Holden Karau,
Not sure on what is happening, but seems like the dashboard you are working
on is setting assignee. Pinging you just in-case if you aren't aware :)
---------- Forwarded message ---------
From: Holden Karau's magical unicorn (JIRA) <ji...@apache.org>
Date: Fri, Sep 21, 2018 at 11:31 PM
Subject: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs are
missing in airflow.cfg
To: <ka...@gmail.com>
[
https://issues.apache.org/jira/browse/AIRFLOW-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Holden Karau's magical unicorn reassigned AIRFLOW-3099:
-------------------------------------------------------
Assignee: Holden Karau's magical unicorn (was: Kaxil Naik)
> Errors raised when some blocs are missing in airflow.cfg
> --------------------------------------------------------
>
> Key: AIRFLOW-3099
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3099
> Project: Apache Airflow
> Issue Type: Bug
> Components: configuration
> Affects Versions: 1.10.0
> Reporter: Christophe
> Assignee: Holden Karau's magical unicorn
> Priority: Minor
> Fix For: 1.10.1
>
>
> When we upgrade from a version of airflow to another one and new config
bloc are available or if we delete useless blocs, lots of errors are raised
if we don't have some blocs in airflow.cfg file.
> We need to avoid these errors for non-required blocs.
>
> For the record logs (not exhaustive):
>
> {noformat}
> [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
sasl_enabled does not exist in the config!
> [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option host
does not exist in the config!
> [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option port
does not exist in the config!
> [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
username does not exist in the config!
> [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
password does not exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section hive Option
default_hive_mapred_queue does not exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
master does not exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
framework_name does not exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
task_cpu does not exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
task_memory does not exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
checkpoint does not exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
authenticate does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
Option elasticsearch_host does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
Option elasticsearch_log_id_template does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
Option elasticsearch_end_of_log_mark does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
worker_container_repository does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
worker_container_tag does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
delete_worker_pods does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
namespace does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
airflow_configmap does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
dags_volume_subpath does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
dags_volume_claim does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
logs_volume_subpath does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
logs_volume_claim does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
git_repo does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
git_branch does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
git_user does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
git_password does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
git_subpath does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
git_sync_container_repository does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
git_sync_container_tag does not exist in the config!
> [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option
git_sync_init_container_name does not exist in the config!
> [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option
worker_service_account_name does not exist in the config!
> [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option
image_pull_secrets does not exist in the config!
> [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option
gcp_service_account_keys does not exist in the config!
> [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option
in_cluster does not exist in the config!{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
--
*Kaxil Naik*
*Big Data Consultant *@ *Data Reply UK*
*Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
Developer
*Phone: *+44 (0) 74820 88992
*LinkedIn*: https://www.linkedin.com/in/kaxil
Re: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
are missing in airflow.cfg
Posted by Holden Karau <ho...@pigscanfly.ca>.
Thanks for the link and sorry for the spam.. For some reason it went into a
loop on that. I've turned off updating JIRA issues progress so the
dashboard will just be a read-only service and not do any of the updating.
On Sun, Sep 23, 2018 at 1:11 AM Deng Xiaodong <xd...@gmail.com> wrote:
> Hi Holden,
>
> This is the ticket link for your information,
> https://issues.apache.org/jira/browse/AIRFLOW-3104 .
> The magical unicorn changes Assignee to the "magical unicorn" then back
> to me, for already >20 times [and still going on. I may need to mute my
> email for now ;-) ].
>
> Thanks.
>
> XD
>
> On Sun, Sep 23, 2018 at 4:01 PM Deng Xiaodong <xd...@gmail.com> wrote:
>
>> Hi Holden,
>>
>> May you have a check at your JIRA magical unicorn? It's keeping changing
>> Assignee in my JIRA ticket (and I keep receiving notifications), for
>> more than 10 times within <10 minutes, and seems it's still going on.
>>
>> Thanks!
>>
>> XD
>>
>> On Sun, Sep 23, 2018 at 2:57 AM Holden Karau <ho...@pigscanfly.ca>
>> wrote:
>>
>>> Great, thanks! I’ll send a longer description email in a few days (I’m
>>> going to watch it over the weekend to make sure it doesn’t do anything
>>> surprising).
>>>
>>> On Sat, Sep 22, 2018 at 11:00 AM Kaxil Naik <ka...@gmail.com> wrote:
>>>
>>> > Yes, it is restoring assignees. It's fine with me :)
>>> >
>>> > On Sat, Sep 22, 2018 at 1:03 AM Holden Karau <ho...@pigscanfly.ca>
>>> wrote:
>>> >
>>> >> Thanks for the heads up :) So it should automatically restore the
>>> >> assignee, which it looks like it did, and it's set up to only do this
>>> for
>>> >> new PRs (no back-fills). I can turn it all the way off if it's
>>> annoying to
>>> >> folks but otherwise it will automatically switch JIRAs to in-progress
>>> as
>>> >> folks make PRs for them so when we go to look for JIRAs which are not
>>> being
>>> >> worked on with the filters in JIRA it works :)
>>> >>
>>> >> On Fri, Sep 21, 2018 at 3:34 PM Kaxil Naik <ka...@gmail.com>
>>> wrote:
>>> >>
>>> >>> Hi Holden Karau,
>>> >>>
>>> >>> Not sure on what is happening, but seems like the dashboard you are
>>> >>> working on is setting assignee. Pinging you just in-case if you
>>> aren't
>>> >>> aware :)
>>> >>>
>>> >>> ---------- Forwarded message ---------
>>> >>> From: Holden Karau's magical unicorn (JIRA) <ji...@apache.org>
>>> >>> Date: Fri, Sep 21, 2018 at 11:31 PM
>>> >>> Subject: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some
>>> blocs
>>> >>> are missing in airflow.cfg
>>> >>> To: <ka...@gmail.com>
>>> >>>
>>> >>>
>>> >>>
>>> >>> [
>>> >>>
>>> https://issues.apache.org/jira/browse/AIRFLOW-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>>> >>> ]
>>> >>>
>>> >>> Holden Karau's magical unicorn reassigned AIRFLOW-3099:
>>> >>> -------------------------------------------------------
>>> >>>
>>> >>> Assignee: Holden Karau's magical unicorn (was: Kaxil Naik)
>>> >>>
>>> >>> > Errors raised when some blocs are missing in airflow.cfg
>>> >>> > --------------------------------------------------------
>>> >>> >
>>> >>> > Key: AIRFLOW-3099
>>> >>> > URL:
>>> >>> https://issues.apache.org/jira/browse/AIRFLOW-3099
>>> >>> > Project: Apache Airflow
>>> >>> > Issue Type: Bug
>>> >>> > Components: configuration
>>> >>> > Affects Versions: 1.10.0
>>> >>> > Reporter: Christophe
>>> >>> > Assignee: Holden Karau's magical unicorn
>>> >>> > Priority: Minor
>>> >>> > Fix For: 1.10.1
>>> >>> >
>>> >>> >
>>> >>> > When we upgrade from a version of airflow to another one and new
>>> >>> config bloc are available or if we delete useless blocs, lots of
>>> errors are
>>> >>> raised if we don't have some blocs in airflow.cfg file.
>>> >>> > We need to avoid these errors for non-required blocs.
>>> >>> >
>>> >>> > For the record logs (not exhaustive):
>>> >>> >
>>> >>> > {noformat}
>>> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>>> >>> sasl_enabled does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>>> >>> host does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>>> >>> port does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>>> >>> username does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>>> >>> password does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section hive Option
>>> >>> default_hive_mapred_queue does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> >>> master does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> >>> framework_name does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> >>> task_cpu does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> >>> task_memory does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> >>> checkpoint does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> >>> authenticate does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section
>>> elasticsearch
>>> >>> Option elasticsearch_host does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section
>>> elasticsearch
>>> >>> Option elasticsearch_log_id_template does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section
>>> elasticsearch
>>> >>> Option elasticsearch_end_of_log_mark does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option worker_container_repository does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option worker_container_tag does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option delete_worker_pods does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option namespace does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option airflow_configmap does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option dags_volume_subpath does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option dags_volume_claim does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option logs_volume_subpath does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option logs_volume_claim does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option git_repo does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option git_branch does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option git_user does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option git_password does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option git_subpath does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option git_sync_container_repository does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option git_sync_container_tag does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option git_sync_init_container_name does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option worker_service_account_name does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option image_pull_secrets does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option gcp_service_account_keys does not exist in the config!
>>> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>>> >>> Option in_cluster does not exist in the config!{noformat}
>>> >>> >
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> This message was sent by Atlassian JIRA
>>> >>> (v7.6.3#76005)
>>> >>>
>>> >>>
>>> >>> --
>>> >>> *Kaxil Naik*
>>> >>> *Big Data Consultant *@ *Data Reply UK*
>>> >>> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark &
>>> >>> Neo4j Developer
>>> >>> *Phone: *+44 (0) 74820 88992
>>> >>> *LinkedIn*: https://www.linkedin.com/in/kaxil
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> Twitter: https://twitter.com/holdenkarau
>>> >> Books (Learning Spark, High Performance Spark, etc.):
>>> >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>>> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>> >>
>>> >
>>> >
>>> > --
>>> > *Kaxil Naik*
>>> > *Big Data Consultant *@ *Data Reply UK*
>>> > *Certified *Google Cloud Data Engineer | *Certified* Apache Spark &
>>> Neo4j
>>> > Developer
>>> > *Phone: *+44 (0) 74820 88992
>>> > *LinkedIn*: https://www.linkedin.com/in/kaxil
>>> >
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>> Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>>
--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Re: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
are missing in airflow.cfg
Posted by Deng Xiaodong <xd...@gmail.com>.
Hi Holden,
This is the ticket link for your information,
https://issues.apache.org/jira/browse/AIRFLOW-3104 .
The magical unicorn changes Assignee to the "magical unicorn" then back to
me, for already >20 times [and still going on. I may need to mute my email
for now ;-) ].
Thanks.
XD
On Sun, Sep 23, 2018 at 4:01 PM Deng Xiaodong <xd...@gmail.com> wrote:
> Hi Holden,
>
> May you have a check at your JIRA magical unicorn? It's keeping changing
> Assignee in my JIRA ticket (and I keep receiving notifications), for more
> than 10 times within <10 minutes, and seems it's still going on.
>
> Thanks!
>
> XD
>
> On Sun, Sep 23, 2018 at 2:57 AM Holden Karau <ho...@pigscanfly.ca> wrote:
>
>> Great, thanks! I’ll send a longer description email in a few days (I’m
>> going to watch it over the weekend to make sure it doesn’t do anything
>> surprising).
>>
>> On Sat, Sep 22, 2018 at 11:00 AM Kaxil Naik <ka...@gmail.com> wrote:
>>
>> > Yes, it is restoring assignees. It's fine with me :)
>> >
>> > On Sat, Sep 22, 2018 at 1:03 AM Holden Karau <ho...@pigscanfly.ca>
>> wrote:
>> >
>> >> Thanks for the heads up :) So it should automatically restore the
>> >> assignee, which it looks like it did, and it's set up to only do this
>> for
>> >> new PRs (no back-fills). I can turn it all the way off if it's
>> annoying to
>> >> folks but otherwise it will automatically switch JIRAs to in-progress
>> as
>> >> folks make PRs for them so when we go to look for JIRAs which are not
>> being
>> >> worked on with the filters in JIRA it works :)
>> >>
>> >> On Fri, Sep 21, 2018 at 3:34 PM Kaxil Naik <ka...@gmail.com>
>> wrote:
>> >>
>> >>> Hi Holden Karau,
>> >>>
>> >>> Not sure on what is happening, but seems like the dashboard you are
>> >>> working on is setting assignee. Pinging you just in-case if you aren't
>> >>> aware :)
>> >>>
>> >>> ---------- Forwarded message ---------
>> >>> From: Holden Karau's magical unicorn (JIRA) <ji...@apache.org>
>> >>> Date: Fri, Sep 21, 2018 at 11:31 PM
>> >>> Subject: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some
>> blocs
>> >>> are missing in airflow.cfg
>> >>> To: <ka...@gmail.com>
>> >>>
>> >>>
>> >>>
>> >>> [
>> >>>
>> https://issues.apache.org/jira/browse/AIRFLOW-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> >>> ]
>> >>>
>> >>> Holden Karau's magical unicorn reassigned AIRFLOW-3099:
>> >>> -------------------------------------------------------
>> >>>
>> >>> Assignee: Holden Karau's magical unicorn (was: Kaxil Naik)
>> >>>
>> >>> > Errors raised when some blocs are missing in airflow.cfg
>> >>> > --------------------------------------------------------
>> >>> >
>> >>> > Key: AIRFLOW-3099
>> >>> > URL:
>> >>> https://issues.apache.org/jira/browse/AIRFLOW-3099
>> >>> > Project: Apache Airflow
>> >>> > Issue Type: Bug
>> >>> > Components: configuration
>> >>> > Affects Versions: 1.10.0
>> >>> > Reporter: Christophe
>> >>> > Assignee: Holden Karau's magical unicorn
>> >>> > Priority: Minor
>> >>> > Fix For: 1.10.1
>> >>> >
>> >>> >
>> >>> > When we upgrade from a version of airflow to another one and new
>> >>> config bloc are available or if we delete useless blocs, lots of
>> errors are
>> >>> raised if we don't have some blocs in airflow.cfg file.
>> >>> > We need to avoid these errors for non-required blocs.
>> >>> >
>> >>> > For the record logs (not exhaustive):
>> >>> >
>> >>> > {noformat}
>> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>> >>> sasl_enabled does not exist in the config!
>> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>> >>> host does not exist in the config!
>> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>> >>> port does not exist in the config!
>> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>> >>> username does not exist in the config!
>> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>> >>> password does not exist in the config!
>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section hive Option
>> >>> default_hive_mapred_queue does not exist in the config!
>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> >>> master does not exist in the config!
>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> >>> framework_name does not exist in the config!
>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> >>> task_cpu does not exist in the config!
>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> >>> task_memory does not exist in the config!
>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> >>> checkpoint does not exist in the config!
>> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> >>> authenticate does not exist in the config!
>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
>> >>> Option elasticsearch_host does not exist in the config!
>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
>> >>> Option elasticsearch_log_id_template does not exist in the config!
>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
>> >>> Option elasticsearch_end_of_log_mark does not exist in the config!
>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> >>> Option worker_container_repository does not exist in the config!
>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> >>> Option worker_container_tag does not exist in the config!
>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> >>> Option delete_worker_pods does not exist in the config!
>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> >>> Option namespace does not exist in the config!
>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> >>> Option airflow_configmap does not exist in the config!
>> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> >>> Option dags_volume_subpath does not exist in the config!
>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> >>> Option dags_volume_claim does not exist in the config!
>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> >>> Option logs_volume_subpath does not exist in the config!
>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> >>> Option logs_volume_claim does not exist in the config!
>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> >>> Option git_repo does not exist in the config!
>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> >>> Option git_branch does not exist in the config!
>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> >>> Option git_user does not exist in the config!
>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> >>> Option git_password does not exist in the config!
>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> >>> Option git_subpath does not exist in the config!
>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> >>> Option git_sync_container_repository does not exist in the config!
>> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> >>> Option git_sync_container_tag does not exist in the config!
>> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>> >>> Option git_sync_init_container_name does not exist in the config!
>> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>> >>> Option worker_service_account_name does not exist in the config!
>> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>> >>> Option image_pull_secrets does not exist in the config!
>> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>> >>> Option gcp_service_account_keys does not exist in the config!
>> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>> >>> Option in_cluster does not exist in the config!{noformat}
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> This message was sent by Atlassian JIRA
>> >>> (v7.6.3#76005)
>> >>>
>> >>>
>> >>> --
>> >>> *Kaxil Naik*
>> >>> *Big Data Consultant *@ *Data Reply UK*
>> >>> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark &
>> >>> Neo4j Developer
>> >>> *Phone: *+44 (0) 74820 88992
>> >>> *LinkedIn*: https://www.linkedin.com/in/kaxil
>> >>>
>> >>
>> >>
>> >> --
>> >> Twitter: https://twitter.com/holdenkarau
>> >> Books (Learning Spark, High Performance Spark, etc.):
>> >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>> >>
>> >
>> >
>> > --
>> > *Kaxil Naik*
>> > *Big Data Consultant *@ *Data Reply UK*
>> > *Certified *Google Cloud Data Engineer | *Certified* Apache Spark &
>> Neo4j
>> > Developer
>> > *Phone: *+44 (0) 74820 88992
>> > *LinkedIn*: https://www.linkedin.com/in/kaxil
>> >
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>
Re: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
are missing in airflow.cfg
Posted by Deng Xiaodong <xd...@gmail.com>.
Hi Holden,
May you have a check at your JIRA magical unicorn? It's keeping changing
Assignee in my JIRA ticket (and I keep receiving notifications), for more
than 10 times within <10 minutes, and seems it's still going on.
Thanks!
XD
On Sun, Sep 23, 2018 at 2:57 AM Holden Karau <ho...@pigscanfly.ca> wrote:
> Great, thanks! I’ll send a longer description email in a few days (I’m
> going to watch it over the weekend to make sure it doesn’t do anything
> surprising).
>
> On Sat, Sep 22, 2018 at 11:00 AM Kaxil Naik <ka...@gmail.com> wrote:
>
> > Yes, it is restoring assignees. It's fine with me :)
> >
> > On Sat, Sep 22, 2018 at 1:03 AM Holden Karau <ho...@pigscanfly.ca>
> wrote:
> >
> >> Thanks for the heads up :) So it should automatically restore the
> >> assignee, which it looks like it did, and it's set up to only do this
> for
> >> new PRs (no back-fills). I can turn it all the way off if it's annoying
> to
> >> folks but otherwise it will automatically switch JIRAs to in-progress as
> >> folks make PRs for them so when we go to look for JIRAs which are not
> being
> >> worked on with the filters in JIRA it works :)
> >>
> >> On Fri, Sep 21, 2018 at 3:34 PM Kaxil Naik <ka...@gmail.com> wrote:
> >>
> >>> Hi Holden Karau,
> >>>
> >>> Not sure on what is happening, but seems like the dashboard you are
> >>> working on is setting assignee. Pinging you just in-case if you aren't
> >>> aware :)
> >>>
> >>> ---------- Forwarded message ---------
> >>> From: Holden Karau's magical unicorn (JIRA) <ji...@apache.org>
> >>> Date: Fri, Sep 21, 2018 at 11:31 PM
> >>> Subject: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
> >>> are missing in airflow.cfg
> >>> To: <ka...@gmail.com>
> >>>
> >>>
> >>>
> >>> [
> >>>
> https://issues.apache.org/jira/browse/AIRFLOW-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> >>> ]
> >>>
> >>> Holden Karau's magical unicorn reassigned AIRFLOW-3099:
> >>> -------------------------------------------------------
> >>>
> >>> Assignee: Holden Karau's magical unicorn (was: Kaxil Naik)
> >>>
> >>> > Errors raised when some blocs are missing in airflow.cfg
> >>> > --------------------------------------------------------
> >>> >
> >>> > Key: AIRFLOW-3099
> >>> > URL:
> >>> https://issues.apache.org/jira/browse/AIRFLOW-3099
> >>> > Project: Apache Airflow
> >>> > Issue Type: Bug
> >>> > Components: configuration
> >>> > Affects Versions: 1.10.0
> >>> > Reporter: Christophe
> >>> > Assignee: Holden Karau's magical unicorn
> >>> > Priority: Minor
> >>> > Fix For: 1.10.1
> >>> >
> >>> >
> >>> > When we upgrade from a version of airflow to another one and new
> >>> config bloc are available or if we delete useless blocs, lots of
> errors are
> >>> raised if we don't have some blocs in airflow.cfg file.
> >>> > We need to avoid these errors for non-required blocs.
> >>> >
> >>> > For the record logs (not exhaustive):
> >>> >
> >>> > {noformat}
> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
> >>> sasl_enabled does not exist in the config!
> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
> >>> host does not exist in the config!
> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
> >>> port does not exist in the config!
> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
> >>> username does not exist in the config!
> >>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
> >>> password does not exist in the config!
> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section hive Option
> >>> default_hive_mapred_queue does not exist in the config!
> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> >>> master does not exist in the config!
> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> >>> framework_name does not exist in the config!
> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> >>> task_cpu does not exist in the config!
> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> >>> task_memory does not exist in the config!
> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> >>> checkpoint does not exist in the config!
> >>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> >>> authenticate does not exist in the config!
> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
> >>> Option elasticsearch_host does not exist in the config!
> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
> >>> Option elasticsearch_log_id_template does not exist in the config!
> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
> >>> Option elasticsearch_end_of_log_mark does not exist in the config!
> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
> >>> Option worker_container_repository does not exist in the config!
> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
> >>> Option worker_container_tag does not exist in the config!
> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
> >>> Option delete_worker_pods does not exist in the config!
> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
> >>> Option namespace does not exist in the config!
> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
> >>> Option airflow_configmap does not exist in the config!
> >>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
> >>> Option dags_volume_subpath does not exist in the config!
> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
> >>> Option dags_volume_claim does not exist in the config!
> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
> >>> Option logs_volume_subpath does not exist in the config!
> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
> >>> Option logs_volume_claim does not exist in the config!
> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
> >>> Option git_repo does not exist in the config!
> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
> >>> Option git_branch does not exist in the config!
> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
> >>> Option git_user does not exist in the config!
> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
> >>> Option git_password does not exist in the config!
> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
> >>> Option git_subpath does not exist in the config!
> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
> >>> Option git_sync_container_repository does not exist in the config!
> >>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
> >>> Option git_sync_container_tag does not exist in the config!
> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
> >>> Option git_sync_init_container_name does not exist in the config!
> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
> >>> Option worker_service_account_name does not exist in the config!
> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
> >>> Option image_pull_secrets does not exist in the config!
> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
> >>> Option gcp_service_account_keys does not exist in the config!
> >>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
> >>> Option in_cluster does not exist in the config!{noformat}
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> This message was sent by Atlassian JIRA
> >>> (v7.6.3#76005)
> >>>
> >>>
> >>> --
> >>> *Kaxil Naik*
> >>> *Big Data Consultant *@ *Data Reply UK*
> >>> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark &
> >>> Neo4j Developer
> >>> *Phone: *+44 (0) 74820 88992
> >>> *LinkedIn*: https://www.linkedin.com/in/kaxil
> >>>
> >>
> >>
> >> --
> >> Twitter: https://twitter.com/holdenkarau
> >> Books (Learning Spark, High Performance Spark, etc.):
> >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
> >>
> >
> >
> > --
> > *Kaxil Naik*
> > *Big Data Consultant *@ *Data Reply UK*
> > *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> > Developer
> > *Phone: *+44 (0) 74820 88992
> > *LinkedIn*: https://www.linkedin.com/in/kaxil
> >
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
Re: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
are missing in airflow.cfg
Posted by Holden Karau <ho...@pigscanfly.ca>.
Great, thanks! I’ll send a longer description email in a few days (I’m
going to watch it over the weekend to make sure it doesn’t do anything
surprising).
On Sat, Sep 22, 2018 at 11:00 AM Kaxil Naik <ka...@gmail.com> wrote:
> Yes, it is restoring assignees. It's fine with me :)
>
> On Sat, Sep 22, 2018 at 1:03 AM Holden Karau <ho...@pigscanfly.ca> wrote:
>
>> Thanks for the heads up :) So it should automatically restore the
>> assignee, which it looks like it did, and it's set up to only do this for
>> new PRs (no back-fills). I can turn it all the way off if it's annoying to
>> folks but otherwise it will automatically switch JIRAs to in-progress as
>> folks make PRs for them so when we go to look for JIRAs which are not being
>> worked on with the filters in JIRA it works :)
>>
>> On Fri, Sep 21, 2018 at 3:34 PM Kaxil Naik <ka...@gmail.com> wrote:
>>
>>> Hi Holden Karau,
>>>
>>> Not sure on what is happening, but seems like the dashboard you are
>>> working on is setting assignee. Pinging you just in-case if you aren't
>>> aware :)
>>>
>>> ---------- Forwarded message ---------
>>> From: Holden Karau's magical unicorn (JIRA) <ji...@apache.org>
>>> Date: Fri, Sep 21, 2018 at 11:31 PM
>>> Subject: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
>>> are missing in airflow.cfg
>>> To: <ka...@gmail.com>
>>>
>>>
>>>
>>> [
>>> https://issues.apache.org/jira/browse/AIRFLOW-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>>> ]
>>>
>>> Holden Karau's magical unicorn reassigned AIRFLOW-3099:
>>> -------------------------------------------------------
>>>
>>> Assignee: Holden Karau's magical unicorn (was: Kaxil Naik)
>>>
>>> > Errors raised when some blocs are missing in airflow.cfg
>>> > --------------------------------------------------------
>>> >
>>> > Key: AIRFLOW-3099
>>> > URL:
>>> https://issues.apache.org/jira/browse/AIRFLOW-3099
>>> > Project: Apache Airflow
>>> > Issue Type: Bug
>>> > Components: configuration
>>> > Affects Versions: 1.10.0
>>> > Reporter: Christophe
>>> > Assignee: Holden Karau's magical unicorn
>>> > Priority: Minor
>>> > Fix For: 1.10.1
>>> >
>>> >
>>> > When we upgrade from a version of airflow to another one and new
>>> config bloc are available or if we delete useless blocs, lots of errors are
>>> raised if we don't have some blocs in airflow.cfg file.
>>> > We need to avoid these errors for non-required blocs.
>>> >
>>> > For the record logs (not exhaustive):
>>> >
>>> > {noformat}
>>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>>> sasl_enabled does not exist in the config!
>>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>>> host does not exist in the config!
>>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>>> port does not exist in the config!
>>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>>> username does not exist in the config!
>>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>>> password does not exist in the config!
>>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section hive Option
>>> default_hive_mapred_queue does not exist in the config!
>>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> master does not exist in the config!
>>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> framework_name does not exist in the config!
>>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> task_cpu does not exist in the config!
>>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> task_memory does not exist in the config!
>>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> checkpoint does not exist in the config!
>>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>>> authenticate does not exist in the config!
>>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
>>> Option elasticsearch_host does not exist in the config!
>>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
>>> Option elasticsearch_log_id_template does not exist in the config!
>>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
>>> Option elasticsearch_end_of_log_mark does not exist in the config!
>>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> Option worker_container_repository does not exist in the config!
>>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> Option worker_container_tag does not exist in the config!
>>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> Option delete_worker_pods does not exist in the config!
>>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> Option namespace does not exist in the config!
>>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> Option airflow_configmap does not exist in the config!
>>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>>> Option dags_volume_subpath does not exist in the config!
>>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> Option dags_volume_claim does not exist in the config!
>>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> Option logs_volume_subpath does not exist in the config!
>>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> Option logs_volume_claim does not exist in the config!
>>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> Option git_repo does not exist in the config!
>>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> Option git_branch does not exist in the config!
>>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> Option git_user does not exist in the config!
>>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> Option git_password does not exist in the config!
>>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> Option git_subpath does not exist in the config!
>>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> Option git_sync_container_repository does not exist in the config!
>>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>>> Option git_sync_container_tag does not exist in the config!
>>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>>> Option git_sync_init_container_name does not exist in the config!
>>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>>> Option worker_service_account_name does not exist in the config!
>>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>>> Option image_pull_secrets does not exist in the config!
>>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>>> Option gcp_service_account_keys does not exist in the config!
>>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>>> Option in_cluster does not exist in the config!{noformat}
>>> >
>>>
>>>
>>>
>>> --
>>> This message was sent by Atlassian JIRA
>>> (v7.6.3#76005)
>>>
>>>
>>> --
>>> *Kaxil Naik*
>>> *Big Data Consultant *@ *Data Reply UK*
>>> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark &
>>> Neo4j Developer
>>> *Phone: *+44 (0) 74820 88992
>>> *LinkedIn*: https://www.linkedin.com/in/kaxil
>>>
>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>
>
> --
> *Kaxil Naik*
> *Big Data Consultant *@ *Data Reply UK*
> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> Developer
> *Phone: *+44 (0) 74820 88992
> *LinkedIn*: https://www.linkedin.com/in/kaxil
>
--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Re: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
are missing in airflow.cfg
Posted by Kaxil Naik <ka...@gmail.com>.
Yes, it is restoring assignees. It's fine with me :)
On Sat, Sep 22, 2018 at 1:03 AM Holden Karau <ho...@pigscanfly.ca> wrote:
> Thanks for the heads up :) So it should automatically restore the
> assignee, which it looks like it did, and it's set up to only do this for
> new PRs (no back-fills). I can turn it all the way off if it's annoying to
> folks but otherwise it will automatically switch JIRAs to in-progress as
> folks make PRs for them so when we go to look for JIRAs which are not being
> worked on with the filters in JIRA it works :)
>
> On Fri, Sep 21, 2018 at 3:34 PM Kaxil Naik <ka...@gmail.com> wrote:
>
>> Hi Holden Karau,
>>
>> Not sure on what is happening, but seems like the dashboard you are
>> working on is setting assignee. Pinging you just in-case if you aren't
>> aware :)
>>
>> ---------- Forwarded message ---------
>> From: Holden Karau's magical unicorn (JIRA) <ji...@apache.org>
>> Date: Fri, Sep 21, 2018 at 11:31 PM
>> Subject: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
>> are missing in airflow.cfg
>> To: <ka...@gmail.com>
>>
>>
>>
>> [
>> https://issues.apache.org/jira/browse/AIRFLOW-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> ]
>>
>> Holden Karau's magical unicorn reassigned AIRFLOW-3099:
>> -------------------------------------------------------
>>
>> Assignee: Holden Karau's magical unicorn (was: Kaxil Naik)
>>
>> > Errors raised when some blocs are missing in airflow.cfg
>> > --------------------------------------------------------
>> >
>> > Key: AIRFLOW-3099
>> > URL: https://issues.apache.org/jira/browse/AIRFLOW-3099
>> > Project: Apache Airflow
>> > Issue Type: Bug
>> > Components: configuration
>> > Affects Versions: 1.10.0
>> > Reporter: Christophe
>> > Assignee: Holden Karau's magical unicorn
>> > Priority: Minor
>> > Fix For: 1.10.1
>> >
>> >
>> > When we upgrade from a version of airflow to another one and new config
>> bloc are available or if we delete useless blocs, lots of errors are raised
>> if we don't have some blocs in airflow.cfg file.
>> > We need to avoid these errors for non-required blocs.
>> >
>> > For the record logs (not exhaustive):
>> >
>> > {noformat}
>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>> sasl_enabled does not exist in the config!
>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>> host does not exist in the config!
>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>> port does not exist in the config!
>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>> username does not exist in the config!
>> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
>> password does not exist in the config!
>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section hive Option
>> default_hive_mapred_queue does not exist in the config!
>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> master does not exist in the config!
>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> framework_name does not exist in the config!
>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> task_cpu does not exist in the config!
>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> task_memory does not exist in the config!
>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> checkpoint does not exist in the config!
>> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
>> authenticate does not exist in the config!
>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
>> Option elasticsearch_host does not exist in the config!
>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
>> Option elasticsearch_log_id_template does not exist in the config!
>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
>> Option elasticsearch_end_of_log_mark does not exist in the config!
>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> Option worker_container_repository does not exist in the config!
>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> Option worker_container_tag does not exist in the config!
>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> Option delete_worker_pods does not exist in the config!
>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> Option namespace does not exist in the config!
>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> Option airflow_configmap does not exist in the config!
>> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes
>> Option dags_volume_subpath does not exist in the config!
>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> Option dags_volume_claim does not exist in the config!
>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> Option logs_volume_subpath does not exist in the config!
>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> Option logs_volume_claim does not exist in the config!
>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> Option git_repo does not exist in the config!
>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> Option git_branch does not exist in the config!
>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> Option git_user does not exist in the config!
>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> Option git_password does not exist in the config!
>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> Option git_subpath does not exist in the config!
>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> Option git_sync_container_repository does not exist in the config!
>> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes
>> Option git_sync_container_tag does not exist in the config!
>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>> Option git_sync_init_container_name does not exist in the config!
>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>> Option worker_service_account_name does not exist in the config!
>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>> Option image_pull_secrets does not exist in the config!
>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>> Option gcp_service_account_keys does not exist in the config!
>> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes
>> Option in_cluster does not exist in the config!{noformat}
>> >
>>
>>
>>
>> --
>> This message was sent by Atlassian JIRA
>> (v7.6.3#76005)
>>
>>
>> --
>> *Kaxil Naik*
>> *Big Data Consultant *@ *Data Reply UK*
>> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark &
>> Neo4j Developer
>> *Phone: *+44 (0) 74820 88992
>> *LinkedIn*: https://www.linkedin.com/in/kaxil
>>
>
>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
--
*Kaxil Naik*
*Big Data Consultant *@ *Data Reply UK*
*Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
Developer
*Phone: *+44 (0) 74820 88992
*LinkedIn*: https://www.linkedin.com/in/kaxil
Re: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
are missing in airflow.cfg
Posted by Holden Karau <ho...@pigscanfly.ca>.
Thanks for the heads up :) So it should automatically restore the assignee,
which it looks like it did, and it's set up to only do this for new PRs (no
back-fills). I can turn it all the way off if it's annoying to folks but
otherwise it will automatically switch JIRAs to in-progress as folks make
PRs for them so when we go to look for JIRAs which are not being worked on
with the filters in JIRA it works :)
On Fri, Sep 21, 2018 at 3:34 PM Kaxil Naik <ka...@gmail.com> wrote:
> Hi Holden Karau,
>
> Not sure on what is happening, but seems like the dashboard you are
> working on is setting assignee. Pinging you just in-case if you aren't
> aware :)
>
> ---------- Forwarded message ---------
> From: Holden Karau's magical unicorn (JIRA) <ji...@apache.org>
> Date: Fri, Sep 21, 2018 at 11:31 PM
> Subject: [jira] [Assigned] (AIRFLOW-3099) Errors raised when some blocs
> are missing in airflow.cfg
> To: <ka...@gmail.com>
>
>
>
> [
> https://issues.apache.org/jira/browse/AIRFLOW-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
>
> Holden Karau's magical unicorn reassigned AIRFLOW-3099:
> -------------------------------------------------------
>
> Assignee: Holden Karau's magical unicorn (was: Kaxil Naik)
>
> > Errors raised when some blocs are missing in airflow.cfg
> > --------------------------------------------------------
> >
> > Key: AIRFLOW-3099
> > URL: https://issues.apache.org/jira/browse/AIRFLOW-3099
> > Project: Apache Airflow
> > Issue Type: Bug
> > Components: configuration
> > Affects Versions: 1.10.0
> > Reporter: Christophe
> > Assignee: Holden Karau's magical unicorn
> > Priority: Minor
> > Fix For: 1.10.1
> >
> >
> > When we upgrade from a version of airflow to another one and new config
> bloc are available or if we delete useless blocs, lots of errors are raised
> if we don't have some blocs in airflow.cfg file.
> > We need to avoid these errors for non-required blocs.
> >
> > For the record logs (not exhaustive):
> >
> > {noformat}
> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
> sasl_enabled does not exist in the config!
> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option host
> does not exist in the config!
> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option port
> does not exist in the config!
> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
> username does not exist in the config!
> > [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option
> password does not exist in the config!
> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section hive Option
> default_hive_mapred_queue does not exist in the config!
> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> master does not exist in the config!
> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> framework_name does not exist in the config!
> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> task_cpu does not exist in the config!
> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> task_memory does not exist in the config!
> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> checkpoint does not exist in the config!
> > [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option
> authenticate does not exist in the config!
> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
> Option elasticsearch_host does not exist in the config!
> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
> Option elasticsearch_log_id_template does not exist in the config!
> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch
> Option elasticsearch_end_of_log_mark does not exist in the config!
> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
> worker_container_repository does not exist in the config!
> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
> worker_container_tag does not exist in the config!
> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
> delete_worker_pods does not exist in the config!
> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
> namespace does not exist in the config!
> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
> airflow_configmap does not exist in the config!
> > [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option
> dags_volume_subpath does not exist in the config!
> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
> dags_volume_claim does not exist in the config!
> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
> logs_volume_subpath does not exist in the config!
> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
> logs_volume_claim does not exist in the config!
> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
> git_repo does not exist in the config!
> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
> git_branch does not exist in the config!
> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
> git_user does not exist in the config!
> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
> git_password does not exist in the config!
> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
> git_subpath does not exist in the config!
> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
> git_sync_container_repository does not exist in the config!
> > [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option
> git_sync_container_tag does not exist in the config!
> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option
> git_sync_init_container_name does not exist in the config!
> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option
> worker_service_account_name does not exist in the config!
> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option
> image_pull_secrets does not exist in the config!
> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option
> gcp_service_account_keys does not exist in the config!
> > [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option
> in_cluster does not exist in the config!{noformat}
> >
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)
>
>
> --
> *Kaxil Naik*
> *Big Data Consultant *@ *Data Reply UK*
> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> Developer
> *Phone: *+44 (0) 74820 88992
> *LinkedIn*: https://www.linkedin.com/in/kaxil
>
--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau