[jira] [Commented] (AIRFLOW-3264) Only '/' is URL-decoded when parsing hostname in the Connection

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16747056#comment-16747056 ] 

ASF subversion and git services commented on AIRFLOW-3264:
----------------------------------------------------------

Commit 1f0cace1f0dd1b7cffb0b3be9ce651cf7927ce52 in airflow's branch refs/heads/v1-10-test from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=1f0cace ]

[AIRFLOW-3264] URL decoding when parsing URI for connection (#4109)

Full URL decoding is performed now when parsing different
components of URI for connection. This enables to configure
paths to sockets including (for example ":") - so far
only '/' (%2f) was hard-coded in hostname. This change introduces
full decoding for all components of the URI.

Note that this is potentially breaking change if someone uses
% in some of their AIRFLOW_CONN_ defined connections.

> Only '/' is URL-decoded when parsing hostname in the Connection
> ---------------------------------------------------------------
>
>                 Key: AIRFLOW-3264
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3264
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: database
>    Affects Versions: 1.10.0
>            Reporter: Jarek Potiuk
>            Assignee: Jarek Potiuk
>            Priority: Major
>             Fix For: 1.10.1, 1.10.2
>
>
> Parsing of the URL for connection(using AIRFLOW_CONN_ environment variables) does not perform full URL-decode on the URL. It only handles hard-coded %2f encode to support "/" in hostname. However there are valid cases where the hostname, login, password, and query parameters can contain url-encoded values. For example in cloud-sql-proxy, generated socket path contains ":" (for example {{/cloudsql/myProject:us-central1:myInstance)}}
> We need to URL-encode ":" because otherwise urlparse will treat those ":" as separator for port number. Similarly user/password can contain url-encoded characters.
> I think we should fully URL-decode all relevant URL fields (including query parameters, user, password, hostname, path). However it is potentially breaking change (if someone has a user/password/hostname with % ) so maybe we should do some compromises around that (for example not decode the password - which are likely to contain '%' characters) although that would violate URL encoding/decoding specification.
> I will provide proposed fix shortly



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)