You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by "Gunvaldson, John" <jg...@cox.net> on 2022/03/30 00:26:02 UTC

Upgrade 2.0.1 to 2.2.4 - New Connection Error

Hello All, 
I just completed an upgrade from 2.0.1 to 2.2.4 and was pretty uneventful. Some missing dependencies I corrected, and Airflow web server, 
scheduler (Celery), workers and flower are working great. Many of the hello world, and example DAGs run fine - and of course they are not using a connection.
~~~~
However, previous DAGs that utilize SimpleHttpOperator, e.g.  —>
    ping_ambari = SimpleHttpOperator(
        task_id='ping_ambari',
        extra_options={"verify": dag.params.get("verify_ssl")},
        http_conn_id=dag.params.get("nifi_http_conn_id"),
        endpoint=<long http address>,
        method='GET',
        headers={"X-Requested-By": "ambari"},
        email_on_failure=True,
        dag=dag,
    )
Have started (and continue after reboots and restarts) to throw the error —> AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined”)

(Which I tracked down to ../airflow/models/connection.py )

~~~~

This is new, and moments before the upgrade this type of task ran fine

Any ideas what is new and changed that I need to adjust to attend to this? 

DB upgrade was uneventful and reported success. Config changes were easy, only like 4 to attend to.

All of my connections are in the GUI using the Admin Connections page (I don’t use any other type of connections)


Best Regards
John

Re: Upgrade 2.0.1 to 2.2.4 - New Connection Error

Posted by "Gunvaldson, John" <jg...@cox.net>.
Thank you, and understand, np

I started diffing all of the classes associated with connections and have identified a couple of good candidates to explore, I will find it

I can jump on Github if further questions, think I am close!

Best Regards
John



> On Mar 30, 2022, at 3:35 AM, Jarek Potiuk <ja...@potiuk.com> wrote:
> 
> Hej John,
> 
> I suggest you add more details and post it on slack or Github
> Discussions https://airflow.apache.org/community/ . Devlist is more
> for "project development" discussion rather than troubleshooting.
> BTW the issue seem to come from your usage of "params" field that was
> slightly modified in 2.2 I think and you should look there.
> 
> But posting more details/logs/your dag details in Slack or in Github
> Discussion might lead to someone being able to help better.
> 
> J,
> 
> On Wed, Mar 30, 2022 at 2:26 AM Gunvaldson, John <jg...@cox.net> wrote:

Re: Upgrade 2.0.1 to 2.2.4 - New Connection Error

Posted by Jarek Potiuk <ja...@potiuk.com>.
Hej John,

I suggest you add more details and post it on slack or Github
Discussions https://airflow.apache.org/community/ . Devlist is more
for "project development" discussion rather than troubleshooting.
BTW the issue seem to come from your usage of "params" field that was
slightly modified in 2.2 I think and you should look there.

But posting more details/logs/your dag details in Slack or in Github
Discussion might lead to someone being able to help better.

J,

On Wed, Mar 30, 2022 at 2:26 AM Gunvaldson, John <jg...@cox.net> wrote:
>
> Hello All,
>
> I just completed an upgrade from 2.0.1 to 2.2.4 and was pretty uneventful. Some missing dependencies I corrected, and Airflow web server,
>
> scheduler (Celery), workers and flower are working great. Many of the hello world, and example DAGs run fine - and of course they are not using a connection.
>
> ~~~~
>
> However, previous DAGs that utilize SimpleHttpOperator, e.g.  —>
>
>     ping_ambari = SimpleHttpOperator(
>
>         task_id='ping_ambari',
>         extra_options={"verify": dag.params.get("verify_ssl")},
>         http_conn_id=dag.params.get("nifi_http_conn_id"),
>         endpoint=<long http address>,
>         method='GET',
>         headers={"X-Requested-By": "ambari"},
>         email_on_failure=True,
>         dag=dag,
>     )
>
> Have started (and continue after reboots and restarts) to throw the error —> AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined”)
>
> (Which I tracked down to ../airflow/models/connection.py )
>
> ~~~~
>
> This is new, and moments before the upgrade this type of task ran fine
>
> Any ideas what is new and changed that I need to adjust to attend to this?
>
> DB upgrade was uneventful and reported success. Config changes were easy, only like 4 to attend to.
>
> All of my connections are in the GUI using the Admin Connections page (I don’t use any other type of connections)
>
>
> Best Regards
> John