[jira] [Comment Edited] (AIRFLOW-3616) Connection parsed from URI - unacceptable underscore in schema part

["Kamil (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732598#comment-16732598 ] 

Kamil edited comment on AIRFLOW-3616 at 1/3/19 2:30 AM:
--------------------------------------------------------

I looked at RFC 3986. It sys that scheme part must be in form:

{{scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )}}

The underscore is not allowed.

Therefore, I think that introducing aliases is more beneficial. I propose to create a substitution of an underline character to a minus sign.

This will allow you to define a connection using:

{{google-cloud-platform://user:pass@hostname/path}}

which connection type will be created as _google_cloud_platform_

_Reference:_

_https://www.ietf.org/rfc/rfc3986.txt_


was (Author: dzakus13):
I looked at RFC 3986. It sys that scheme part must be in form:

scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )

The underscore is not allowed.

Therefore, I think that introducing aliases is more beneficial. I propose to create a substitution of an underline character to a minus sign.

This will allow you to define a connection using:

{\{ google-cloud-platform://user:pass@hostname/path }}

which connection type will be created as \{{ google_cloud_platform }}

> Connection parsed from URI  - unacceptable underscore in schema part
> --------------------------------------------------------------------
>
>                 Key: AIRFLOW-3616
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: Kamil Bregula
>            Priority: Major
>
> There is a problem with creating a new connection based on the state of the environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we must create an environment variable. The name of the environment must be given according to the scheme:
>  {{AIRFLOW_CONN_[CONN_ID]}}
>  where {{[CONN_ID]}} is the connection identifier to be used, written in upper case.
>  The content of the variable defines the connection and is saved in the form of URI.
> Defining a URI is complex, but the key is that the connection type is given as the schema. There are many possible values to give. but the sample values are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately, the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately, this module does not support the uppercase character in schema port of URI - see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for parsing non-URL URIs - we should likely use different parser which handles more generic URIs. 
>  Especially that it also creates other problems:
>  https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant that does not contain an unacceptable character. For example {{google_cloud_platform => gcp}}. It is worth noting that one alias is currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]: urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='', path='google_cloud_platform://user:pass@hostname/path', params='', query='extra_param=extra_param_value', fragment='')
> In [3]: urlparse("gcp://user:pass@hostname/path?extra_param=extra_param_value")
> Out[3]: ParseResult(scheme='gcp', netloc='user:pass@hostname', path='/path', params='', query='extra_param=extra_param_value', fragment='')
> Connection parsed from URI - unacceptable underscore in schema part



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)