[jira] [Updated] (AIRFLOW-5518) Mix between scheme and schema for HTTP connections

["Mickael V (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-5518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mickael V updated AIRFLOW-5518:
-------------------------------
    Description: 
There is an inconsistence in the usage of *scheme* and *schema* when using an HTTP connection.

If the connection is made through the UI or imported through config files (or to sum up, if it's kept in the Airflow DB), the *schema* represents the *scheme* ({{http}}, {{https}}).

But if the connection is parsed from a URI (for example if it's overloaded through an environment variable), the {{schema}} is the {{path}} or the URI.

This is wrong because then the {{HttpHook}} uses the {{schema}} to prefix the {{base_url}}. It is particularly visible when using https, which gets transformed to default http. As a result the URLs are built with HTTP instead of HTTPS.

 

There are two possibilities that I see to fix this :
 * At the {{Connection}} level, in {{parse_from_uri()}}, implement a special treatment for http connection to have {{conn_type='http'}} and {{schema=scheme}}
 * At the {{HttpHook}} level, in {{get_conn}}, look up {{conn_type}} to get the real {{scheme}}.

 

I propose using the first solution, as it is more consistent with what happens when a connection is added through other ways (UI and config files).

  was:
There is an inconsistence in the usage of *scheme* and *schema* when using an HTTP connection.

If the connection is made through the UI or imported through config files (or to sum up, if it's kept in the Airflow DB), the *schema* represents the *scheme* ({{http}}, {{https}}).

But if the connection is parsed from a URI (for example if it's overloaded through an environment variable), the {{schema}} is the {{path}} or the URI.

This is wrong because then the {{HttpHook}} uses the {{schema}} to prefix the {{base_url}}.

 

There are two possibilities that I see to fix this :
 * At the {{Connection}} level, in {{parse_from_uri()}}, implement a special treatment for http connection to have {{conn_type='http'}} and {{schema=scheme}}
 * At the {{HttpHook}} level, in {{get_conn}}, look up {{conn_type}} to get the real {{scheme}}.

 

I propose using the first solution, as it is more consistent with what happens when a connection is added through other ways (UI and config files).


> Mix between scheme and schema for HTTP connections
> --------------------------------------------------
>
>                 Key: AIRFLOW-5518
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5518
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: hooks
>    Affects Versions: 1.10.5
>            Reporter: Mickael V
>            Priority: Minor
>             Fix For: 1.10.6
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> There is an inconsistence in the usage of *scheme* and *schema* when using an HTTP connection.
> If the connection is made through the UI or imported through config files (or to sum up, if it's kept in the Airflow DB), the *schema* represents the *scheme* ({{http}}, {{https}}).
> But if the connection is parsed from a URI (for example if it's overloaded through an environment variable), the {{schema}} is the {{path}} or the URI.
> This is wrong because then the {{HttpHook}} uses the {{schema}} to prefix the {{base_url}}. It is particularly visible when using https, which gets transformed to default http. As a result the URLs are built with HTTP instead of HTTPS.
>  
> There are two possibilities that I see to fix this :
>  * At the {{Connection}} level, in {{parse_from_uri()}}, implement a special treatment for http connection to have {{conn_type='http'}} and {{schema=scheme}}
>  * At the {{HttpHook}} level, in {{get_conn}}, look up {{conn_type}} to get the real {{scheme}}.
>  
> I propose using the first solution, as it is more consistent with what happens when a connection is added through other ways (UI and config files).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)