[jira] [Commented] (AIRFLOW-6072) aws_hook: Ability to set outbound proxy

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-6072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991470#comment-16991470 ] 

ASF GitHub Bot commented on AIRFLOW-6072:
-----------------------------------------

potiuk commented on pull request #6686: [AIRFLOW-6072] aws_hook: Outbound http proxy setting and other enhancements
URL: https://github.com/apache/airflow/pull/6686
 
 
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> aws_hook: Ability to set outbound proxy
> ---------------------------------------
>
>                 Key: AIRFLOW-6072
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6072
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: aws
>    Affects Versions: 1.10.6
>            Reporter: Bjorn Olsen
>            Assignee: Bjorn Olsen
>            Priority: Minor
>
> The boto3 connection used by aws_hook does not respect outbound http_proxy settings (even if these are set in system wide). 
>  
> The way to configure a proxy is to pass a botocore.config.Config object to boto3 when creating a client (according to this SO post).
> [https://stackoverflow.com/questions/33480108/how-do-you-use-an-http-https-proxy-with-boto3]
> While the aws_hook get_client_type() method is used extensively by AWS Operators, the "config" argument is not used by any operator. 
> Adding a check to aws_hook for "config" in the "extra_config" of the Airflow Connection, could allow us to pass kwargs there that build the Config object automatically by the hook.
> Otherwise we have to update every AWS Operator to also take a "config" parameter.
>  
> To set an outbound proxy is then as simple as adding this to your extra_config:
> {code:java}
> { .. ,
>   "config":{ "proxies": {
>     "http": "http://myproxy:8080",
>     "https": "http://myproxy:8080" }},
>  .. }
> {code}
>  
> This needs to work both for the main boto3 clients that do task work, but also during the assume_role process which also uses a boto3 client.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)