You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@libcloud.apache.org by Cedric Lebrun <Ce...@asg.com> on 2014/07/08 08:23:30 UTC
[dev] Libcloud behind ISA Proxy
Hi All,
I try to use Libcloud to connect to EC2.
My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server that requires authentication (MS ISA Proxy Server).
I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY, but still doesn't work.
Is there a way to set a proxy and proxy credentials directly using Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 ProxyHandler ?
Thanks for your help,
CL
Re: [dev] Libcloud behind ISA Proxy
Posted by ph...@stfc.ac.uk.
Hi Tomaz,
Thanks very much for integrating this into the code base.
Cheers,
Phil
On 21 Aug 2014, at 11:26, Tomaz Muraus <to...@apache.org> wrote:
> Another "just FYI" - I've pushed some more changes to trunk and now HTTP
> proxy support in Libcloud also supports basic access authentication.
>
> You can find some examples of how to use it in the documentation -
> https://libcloud.readthedocs.org/en/latest/other/using-http-proxy.html
>
>
> On Tue, Aug 19, 2014 at 7:17 PM, Tomaz Muraus <to...@apache.org> wrote:
>
>> I just wanted to let you know that we also needed support for HTTP proxy
>> for some things we are working on at DivvyCloud so I have pushed a
>> preliminary support to trunk -
>> https://github.com/apache/libcloud/commit/f3f600028384bc19c0f7ff051306a401d57afd35
>>
>> There are some know limitations (doesn't support with Python 2.6 and it
>> doesn't work with proxies which require an authentication), but besides
>> that, I have tested it with 2.7, PyPy, 3.1, 3.2, 3.3 and 3.4 and everything
>> appears to be working fine.
>>
>> You can find some documentation on how to use it at
>> https://libcloud.readthedocs.org/en/latest/other/using-http-proxy.html
>>
>>
>> On Wed, Jul 16, 2014 at 2:39 PM, Cedric Lebrun <Ce...@asg.com>
>> wrote:
>>
>>> I have also installed CNTLM to bypass proxy authentication.
>>> Not the best solution, but a temporary trick to be able to progress on my
>>> prototype :-)
>>>
>>> Cheers
>>> Cedric
>>>
>>> -----Original Message-----
>>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>>> Sent: mercredi 16 juillet 2014 14:33
>>> To: dev@libcloud.apache.org
>>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>>
>>> Hi Cedric,
>>>
>>> That's great news.
>>>
>>> I thought you'd mentioned that the proxy you were connecting through also
>>> needed authentication credentials. - Is that the case and if so what
>>> additional changes did you need to make to the code?
>>>
>>> Cheers,
>>> Phil
>>>
>>> On 16 Jul 2014, at 13:29, Cedric Lebrun <Ce...@asg.com> wrote:
>>>
>>>> With Python 3.4 (on Windows), I have finally succeeded using your patch.
>>>>
>>>> I have just had to replace
>>>>
>>>> self._set_hostport(proxy_host, None)
>>>>
>>>> by
>>>>
>>>> self.host = proxyHost
>>>> self.port = proxyPort
>>>>
>>>>
>>>> With the appropriate values for proxyHost and proxyPort.
>>>>
>>>>
>>>> Cedric
>>>>
>>>> -----Original Message-----
>>>> From: Cedric Lebrun [mailto:Cedric.Lebrun@asg.com]
>>>> Sent: mercredi 16 juillet 2014 08:22
>>>> To: dev@libcloud.apache.org
>>>> Subject: RE: [dev] Libcloud behind ISA Proxy
>>>>
>>>> Hi Phil,
>>>>
>>>> Sorry for my late reply because of vacations :-) I've tried with Python
>>> 2.7 on a CentOS environment as well as with Python 3.4 on a Windows 7
>>> environment. But in both case, no success so far.
>>>> Will continue investigating.
>>>>
>>>> Thank you.
>>>> Cedric
>>>>
>>>> -----Original Message-----
>>>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>>>> Sent: mardi 8 juillet 2014 15:29
>>>> To: dev@libcloud.apache.org
>>>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>>>
>>>> Hi Cedric,
>>>>
>>>> I'm using Python 2.7.2. There seems to be differences between the
>>> two. I would adapt your code to do the job of _set_hostport and set host
>>> and port separately. They are public attributes in any case.
>>>>
>>>> Cheers,
>>>> Phil
>>>>
>>>> On 8 Jul 2014, at 13:50, Cedric Lebrun <Ce...@asg.com> wrote:
>>>>
>>>>> Hi Philip,
>>>>>
>>>>> I'm using Python 2.7 as well (2.7.7).
>>>>> And Libcloud 0.15.1
>>>>>
>>>>> Cedric
>>>>>
>>>>> -----Original Message-----
>>>>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>>>>> Sent: mardi 8 juillet 2014 12:03
>>>>> To: dev@libcloud.apache.org
>>>>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>>>>
>>>>> Hi Cedric,
>>>>>
>>>>> What version of Python are you using?
>>>>>
>>>>> In my Python 2.7 installation, _set_hostport is a method of
>>> httplib.HTTPConnection. It basically does some format checking and assigns
>>> host and port number.
>>>>>
>>>>> Clearly some more work will be needed for a generic solution across
>>>>> supported versions of Python :)
>>>>>
>>>>> Cheers,
>>>>> Phil
>>>>>
>>>>>
>>>>> def _set_hostport(self, host, port):
>>>>> if port is None:
>>>>> i = host.rfind(':')
>>>>> j = host.rfind(']') # ipv6 addresses have [...]
>>>>> if i > j:
>>>>> try:
>>>>> port = int(host[i+1:])
>>>>> except ValueError:
>>>>> raise InvalidURL("nonnumeric port: '%s'" %
>>> host[i+1:])
>>>>> host = host[:i]
>>>>> else:
>>>>> port = self.default_port
>>>>> if host and host[0] == '[' and host[-1] == ']':
>>>>> host = host[1:-1]
>>>>> self.host = host
>>>>> self.port = port
>>>>>
>>>>>
>>>>> On 8 Jul 2014, at 10:45, Cedric Lebrun <Ce...@asg.com> wrote:
>>>>>
>>>>>> Hi Philip,
>>>>>>
>>>>>> Thanks for your help.
>>>>>> But after patching httplib_ssl, I fall into now another error:
>>>>>>
>>>>>> AttributeError: LibcloudHTTPSConnection instance has no attribute
>>> '_set_hostport'
>>>>>>
>>>>>>
>>>>>> Cedric
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>>>>>> Sent: mardi 8 juillet 2014 09:56
>>>>>> To: dev@libcloud.apache.org
>>>>>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>>>>>
>>>>>> Hi Cedric,
>>>>>>
>>>>>> I ran into the same issue and patched libcloud.httplib_ssl. I've
>>> included the code snippets below. The proxy I use doesn't require
>>> authentication but hopefully you can adapt to make it do what you need. I
>>> tested with Python 2.7 - may need tweaking for Python 3.
>>>>>>
>>>>>> Cheers,
>>>>>> Phil
>>>>>>
>>>>>>
>>>>>> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy
>>> settings from the environment:
>>>>>>
>>>>>>
>>>>>> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
>>>>>> """
>>>>>> LibcloudHTTPSConnection
>>>>>>
>>>>>> Subclass of HTTPSConnection which verifies certificate names if
>>>>>> and only if CA certificates are available.
>>>>>> """
>>>>>> verify = True # verify by default
>>>>>> ca_cert = None # no default CA Certificate
>>>>>>
>>>>>> def __init__(self, *args, **kwargs):
>>>>>> """
>>>>>> Constructor
>>>>>> """
>>>>>> self._setup_verify()
>>>>>> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>>>>>>
>>>>>> # Support for HTTPS Proxy
>>>>>> if 'https_proxy' in os.environ:
>>>>>> from urlparse import urlparse
>>>>>>
>>>>>> self.set_tunnel(self.host, port=self.port)
>>>>>>
>>>>>> proxy_host = urlparse(os.environ['https_proxy']).netloc
>>>>>> self._set_hostport(proxy_host, None)
>>>>>>
>>>>>> .
>>>>>> .
>>>>>> .
>>>>>>
>>>>>> And then modified the connect call to use the tunnel:
>>>>>>
>>>>>>
>>>>>> def connect(self):
>>>>>> """
>>>>>> Connect
>>>>>>
>>>>>> Checks if verification is toggled; if not, just call
>>>>>> httplib.HTTPSConnection's connect
>>>>>> """
>>>>>> if not self.verify:
>>>>>> return httplib.HTTPSConnection.connect(self)
>>>>>>
>>>>>> # otherwise, create a connection and verify the hostname
>>>>>> # use socket.create_connection (in 2.6+) if possible
>>>>>> if getattr(socket, 'create_connection', None):
>>>>>> sock = socket.create_connection((self.host, self.port),
>>>>>> self.timeout)
>>>>>> else:
>>>>>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>>>>> sock.connect((self.host, self.port))
>>>>>>
>>>>>> # Support for HTTPS Proxy
>>>>>> if self._tunnel_host:
>>>>>> self.sock = sock
>>>>>> self._tunnel()
>>>>>>
>>>>>> self.sock = ssl.wrap_socket(sock,
>>>>>> self.key_file,
>>>>>> self.cert_file,
>>>>>> cert_reqs=ssl.CERT_REQUIRED,
>>>>>> ca_certs=self.ca_cert,
>>>>>> ssl_version=ssl.PROTOCOL_TLSv1)
>>>>>> cert = self.sock.getpeercert()
>>>>>> if not self._verify_hostname(self.host, cert):
>>>>>> raise ssl.SSLError('Failed to verify hostname')
>>>>>>
>>>>>>
>>>>>> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I try to use Libcloud to connect to EC2.
>>>>>>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy
>>> server that requires authentication (MS ISA Proxy Server).
>>>>>>> I tried with defining environment variables HTTP_PROXY and
>>> HTTPS_PROXY, but still doesn't work.
>>>>>>> Is there a way to set a proxy and proxy credentials directly using
>>> Libcloud ? Or should I have to patch the http_lib to implement UrlLib2
>>> ProxyHandler ?
>>>>>>>
>>>>>>> Thanks for your help,
>>>>>>> CL
>>>>>>
>>>>>> --
>>>>>> Scanned by iCritical.
>>>>>
>>>>> --
>>>>> Scanned by iCritical.
>>>>
>>>> --
>>>> Scanned by iCritical.
>>>
>>> --
>>> Scanned by iCritical.
>>>
>>
>>
--
Scanned by iCritical.
Re: [dev] Libcloud behind ISA Proxy
Posted by Tomaz Muraus <to...@apache.org>.
Another "just FYI" - I've pushed some more changes to trunk and now HTTP
proxy support in Libcloud also supports basic access authentication.
You can find some examples of how to use it in the documentation -
https://libcloud.readthedocs.org/en/latest/other/using-http-proxy.html
On Tue, Aug 19, 2014 at 7:17 PM, Tomaz Muraus <to...@apache.org> wrote:
> I just wanted to let you know that we also needed support for HTTP proxy
> for some things we are working on at DivvyCloud so I have pushed a
> preliminary support to trunk -
> https://github.com/apache/libcloud/commit/f3f600028384bc19c0f7ff051306a401d57afd35
>
> There are some know limitations (doesn't support with Python 2.6 and it
> doesn't work with proxies which require an authentication), but besides
> that, I have tested it with 2.7, PyPy, 3.1, 3.2, 3.3 and 3.4 and everything
> appears to be working fine.
>
> You can find some documentation on how to use it at
> https://libcloud.readthedocs.org/en/latest/other/using-http-proxy.html
>
>
> On Wed, Jul 16, 2014 at 2:39 PM, Cedric Lebrun <Ce...@asg.com>
> wrote:
>
>> I have also installed CNTLM to bypass proxy authentication.
>> Not the best solution, but a temporary trick to be able to progress on my
>> prototype :-)
>>
>> Cheers
>> Cedric
>>
>> -----Original Message-----
>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>> Sent: mercredi 16 juillet 2014 14:33
>> To: dev@libcloud.apache.org
>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>
>> Hi Cedric,
>>
>> That's great news.
>>
>> I thought you'd mentioned that the proxy you were connecting through also
>> needed authentication credentials. - Is that the case and if so what
>> additional changes did you need to make to the code?
>>
>> Cheers,
>> Phil
>>
>> On 16 Jul 2014, at 13:29, Cedric Lebrun <Ce...@asg.com> wrote:
>>
>> > With Python 3.4 (on Windows), I have finally succeeded using your patch.
>> >
>> > I have just had to replace
>> >
>> > self._set_hostport(proxy_host, None)
>> >
>> > by
>> >
>> > self.host = proxyHost
>> > self.port = proxyPort
>> >
>> >
>> > With the appropriate values for proxyHost and proxyPort.
>> >
>> >
>> > Cedric
>> >
>> > -----Original Message-----
>> > From: Cedric Lebrun [mailto:Cedric.Lebrun@asg.com]
>> > Sent: mercredi 16 juillet 2014 08:22
>> > To: dev@libcloud.apache.org
>> > Subject: RE: [dev] Libcloud behind ISA Proxy
>> >
>> > Hi Phil,
>> >
>> > Sorry for my late reply because of vacations :-) I've tried with Python
>> 2.7 on a CentOS environment as well as with Python 3.4 on a Windows 7
>> environment. But in both case, no success so far.
>> > Will continue investigating.
>> >
>> > Thank you.
>> > Cedric
>> >
>> > -----Original Message-----
>> > From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>> > Sent: mardi 8 juillet 2014 15:29
>> > To: dev@libcloud.apache.org
>> > Subject: Re: [dev] Libcloud behind ISA Proxy
>> >
>> > Hi Cedric,
>> >
>> > I'm using Python 2.7.2. There seems to be differences between the
>> two. I would adapt your code to do the job of _set_hostport and set host
>> and port separately. They are public attributes in any case.
>> >
>> > Cheers,
>> > Phil
>> >
>> > On 8 Jul 2014, at 13:50, Cedric Lebrun <Ce...@asg.com> wrote:
>> >
>> >> Hi Philip,
>> >>
>> >> I'm using Python 2.7 as well (2.7.7).
>> >> And Libcloud 0.15.1
>> >>
>> >> Cedric
>> >>
>> >> -----Original Message-----
>> >> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>> >> Sent: mardi 8 juillet 2014 12:03
>> >> To: dev@libcloud.apache.org
>> >> Subject: Re: [dev] Libcloud behind ISA Proxy
>> >>
>> >> Hi Cedric,
>> >>
>> >> What version of Python are you using?
>> >>
>> >> In my Python 2.7 installation, _set_hostport is a method of
>> httplib.HTTPConnection. It basically does some format checking and assigns
>> host and port number.
>> >>
>> >> Clearly some more work will be needed for a generic solution across
>> >> supported versions of Python :)
>> >>
>> >> Cheers,
>> >> Phil
>> >>
>> >>
>> >> def _set_hostport(self, host, port):
>> >> if port is None:
>> >> i = host.rfind(':')
>> >> j = host.rfind(']') # ipv6 addresses have [...]
>> >> if i > j:
>> >> try:
>> >> port = int(host[i+1:])
>> >> except ValueError:
>> >> raise InvalidURL("nonnumeric port: '%s'" %
>> host[i+1:])
>> >> host = host[:i]
>> >> else:
>> >> port = self.default_port
>> >> if host and host[0] == '[' and host[-1] == ']':
>> >> host = host[1:-1]
>> >> self.host = host
>> >> self.port = port
>> >>
>> >>
>> >> On 8 Jul 2014, at 10:45, Cedric Lebrun <Ce...@asg.com> wrote:
>> >>
>> >>> Hi Philip,
>> >>>
>> >>> Thanks for your help.
>> >>> But after patching httplib_ssl, I fall into now another error:
>> >>>
>> >>> AttributeError: LibcloudHTTPSConnection instance has no attribute
>> '_set_hostport'
>> >>>
>> >>>
>> >>> Cedric
>> >>>
>> >>> -----Original Message-----
>> >>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>> >>> Sent: mardi 8 juillet 2014 09:56
>> >>> To: dev@libcloud.apache.org
>> >>> Subject: Re: [dev] Libcloud behind ISA Proxy
>> >>>
>> >>> Hi Cedric,
>> >>>
>> >>> I ran into the same issue and patched libcloud.httplib_ssl. I've
>> included the code snippets below. The proxy I use doesn't require
>> authentication but hopefully you can adapt to make it do what you need. I
>> tested with Python 2.7 - may need tweaking for Python 3.
>> >>>
>> >>> Cheers,
>> >>> Phil
>> >>>
>> >>>
>> >>> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy
>> settings from the environment:
>> >>>
>> >>>
>> >>> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
>> >>> """
>> >>> LibcloudHTTPSConnection
>> >>>
>> >>> Subclass of HTTPSConnection which verifies certificate names if
>> >>> and only if CA certificates are available.
>> >>> """
>> >>> verify = True # verify by default
>> >>> ca_cert = None # no default CA Certificate
>> >>>
>> >>> def __init__(self, *args, **kwargs):
>> >>> """
>> >>> Constructor
>> >>> """
>> >>> self._setup_verify()
>> >>> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>> >>>
>> >>> # Support for HTTPS Proxy
>> >>> if 'https_proxy' in os.environ:
>> >>> from urlparse import urlparse
>> >>>
>> >>> self.set_tunnel(self.host, port=self.port)
>> >>>
>> >>> proxy_host = urlparse(os.environ['https_proxy']).netloc
>> >>> self._set_hostport(proxy_host, None)
>> >>>
>> >>> .
>> >>> .
>> >>> .
>> >>>
>> >>> And then modified the connect call to use the tunnel:
>> >>>
>> >>>
>> >>> def connect(self):
>> >>> """
>> >>> Connect
>> >>>
>> >>> Checks if verification is toggled; if not, just call
>> >>> httplib.HTTPSConnection's connect
>> >>> """
>> >>> if not self.verify:
>> >>> return httplib.HTTPSConnection.connect(self)
>> >>>
>> >>> # otherwise, create a connection and verify the hostname
>> >>> # use socket.create_connection (in 2.6+) if possible
>> >>> if getattr(socket, 'create_connection', None):
>> >>> sock = socket.create_connection((self.host, self.port),
>> >>> self.timeout)
>> >>> else:
>> >>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>> >>> sock.connect((self.host, self.port))
>> >>>
>> >>> # Support for HTTPS Proxy
>> >>> if self._tunnel_host:
>> >>> self.sock = sock
>> >>> self._tunnel()
>> >>>
>> >>> self.sock = ssl.wrap_socket(sock,
>> >>> self.key_file,
>> >>> self.cert_file,
>> >>> cert_reqs=ssl.CERT_REQUIRED,
>> >>> ca_certs=self.ca_cert,
>> >>> ssl_version=ssl.PROTOCOL_TLSv1)
>> >>> cert = self.sock.getpeercert()
>> >>> if not self._verify_hostname(self.host, cert):
>> >>> raise ssl.SSLError('Failed to verify hostname')
>> >>>
>> >>>
>> >>> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>> >>>
>> >>>> Hi All,
>> >>>>
>> >>>> I try to use Libcloud to connect to EC2.
>> >>>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy
>> server that requires authentication (MS ISA Proxy Server).
>> >>>> I tried with defining environment variables HTTP_PROXY and
>> HTTPS_PROXY, but still doesn't work.
>> >>>> Is there a way to set a proxy and proxy credentials directly using
>> Libcloud ? Or should I have to patch the http_lib to implement UrlLib2
>> ProxyHandler ?
>> >>>>
>> >>>> Thanks for your help,
>> >>>> CL
>> >>>
>> >>> --
>> >>> Scanned by iCritical.
>> >>
>> >> --
>> >> Scanned by iCritical.
>> >
>> > --
>> > Scanned by iCritical.
>>
>> --
>> Scanned by iCritical.
>>
>
>
Re: [dev] Libcloud behind ISA Proxy
Posted by Tomaz Muraus <to...@apache.org>.
I just wanted to let you know that we also needed support for HTTP proxy
for some things we are working on at DivvyCloud so I have pushed a
preliminary support to trunk -
https://github.com/apache/libcloud/commit/f3f600028384bc19c0f7ff051306a401d57afd35
There are some know limitations (doesn't support with Python 2.6 and it
doesn't work with proxies which require an authentication), but besides
that, I have tested it with 2.7, PyPy, 3.1, 3.2, 3.3 and 3.4 and everything
appears to be working fine.
You can find some documentation on how to use it at
https://libcloud.readthedocs.org/en/latest/other/using-http-proxy.html
On Wed, Jul 16, 2014 at 2:39 PM, Cedric Lebrun <Ce...@asg.com>
wrote:
> I have also installed CNTLM to bypass proxy authentication.
> Not the best solution, but a temporary trick to be able to progress on my
> prototype :-)
>
> Cheers
> Cedric
>
> -----Original Message-----
> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> Sent: mercredi 16 juillet 2014 14:33
> To: dev@libcloud.apache.org
> Subject: Re: [dev] Libcloud behind ISA Proxy
>
> Hi Cedric,
>
> That's great news.
>
> I thought you'd mentioned that the proxy you were connecting through also
> needed authentication credentials. - Is that the case and if so what
> additional changes did you need to make to the code?
>
> Cheers,
> Phil
>
> On 16 Jul 2014, at 13:29, Cedric Lebrun <Ce...@asg.com> wrote:
>
> > With Python 3.4 (on Windows), I have finally succeeded using your patch.
> >
> > I have just had to replace
> >
> > self._set_hostport(proxy_host, None)
> >
> > by
> >
> > self.host = proxyHost
> > self.port = proxyPort
> >
> >
> > With the appropriate values for proxyHost and proxyPort.
> >
> >
> > Cedric
> >
> > -----Original Message-----
> > From: Cedric Lebrun [mailto:Cedric.Lebrun@asg.com]
> > Sent: mercredi 16 juillet 2014 08:22
> > To: dev@libcloud.apache.org
> > Subject: RE: [dev] Libcloud behind ISA Proxy
> >
> > Hi Phil,
> >
> > Sorry for my late reply because of vacations :-) I've tried with Python
> 2.7 on a CentOS environment as well as with Python 3.4 on a Windows 7
> environment. But in both case, no success so far.
> > Will continue investigating.
> >
> > Thank you.
> > Cedric
> >
> > -----Original Message-----
> > From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> > Sent: mardi 8 juillet 2014 15:29
> > To: dev@libcloud.apache.org
> > Subject: Re: [dev] Libcloud behind ISA Proxy
> >
> > Hi Cedric,
> >
> > I'm using Python 2.7.2. There seems to be differences between the two.
> I would adapt your code to do the job of _set_hostport and set host and
> port separately. They are public attributes in any case.
> >
> > Cheers,
> > Phil
> >
> > On 8 Jul 2014, at 13:50, Cedric Lebrun <Ce...@asg.com> wrote:
> >
> >> Hi Philip,
> >>
> >> I'm using Python 2.7 as well (2.7.7).
> >> And Libcloud 0.15.1
> >>
> >> Cedric
> >>
> >> -----Original Message-----
> >> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> >> Sent: mardi 8 juillet 2014 12:03
> >> To: dev@libcloud.apache.org
> >> Subject: Re: [dev] Libcloud behind ISA Proxy
> >>
> >> Hi Cedric,
> >>
> >> What version of Python are you using?
> >>
> >> In my Python 2.7 installation, _set_hostport is a method of
> httplib.HTTPConnection. It basically does some format checking and assigns
> host and port number.
> >>
> >> Clearly some more work will be needed for a generic solution across
> >> supported versions of Python :)
> >>
> >> Cheers,
> >> Phil
> >>
> >>
> >> def _set_hostport(self, host, port):
> >> if port is None:
> >> i = host.rfind(':')
> >> j = host.rfind(']') # ipv6 addresses have [...]
> >> if i > j:
> >> try:
> >> port = int(host[i+1:])
> >> except ValueError:
> >> raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
> >> host = host[:i]
> >> else:
> >> port = self.default_port
> >> if host and host[0] == '[' and host[-1] == ']':
> >> host = host[1:-1]
> >> self.host = host
> >> self.port = port
> >>
> >>
> >> On 8 Jul 2014, at 10:45, Cedric Lebrun <Ce...@asg.com> wrote:
> >>
> >>> Hi Philip,
> >>>
> >>> Thanks for your help.
> >>> But after patching httplib_ssl, I fall into now another error:
> >>>
> >>> AttributeError: LibcloudHTTPSConnection instance has no attribute
> '_set_hostport'
> >>>
> >>>
> >>> Cedric
> >>>
> >>> -----Original Message-----
> >>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> >>> Sent: mardi 8 juillet 2014 09:56
> >>> To: dev@libcloud.apache.org
> >>> Subject: Re: [dev] Libcloud behind ISA Proxy
> >>>
> >>> Hi Cedric,
> >>>
> >>> I ran into the same issue and patched libcloud.httplib_ssl. I've
> included the code snippets below. The proxy I use doesn't require
> authentication but hopefully you can adapt to make it do what you need. I
> tested with Python 2.7 - may need tweaking for Python 3.
> >>>
> >>> Cheers,
> >>> Phil
> >>>
> >>>
> >>> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy
> settings from the environment:
> >>>
> >>>
> >>> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
> >>> """
> >>> LibcloudHTTPSConnection
> >>>
> >>> Subclass of HTTPSConnection which verifies certificate names if
> >>> and only if CA certificates are available.
> >>> """
> >>> verify = True # verify by default
> >>> ca_cert = None # no default CA Certificate
> >>>
> >>> def __init__(self, *args, **kwargs):
> >>> """
> >>> Constructor
> >>> """
> >>> self._setup_verify()
> >>> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
> >>>
> >>> # Support for HTTPS Proxy
> >>> if 'https_proxy' in os.environ:
> >>> from urlparse import urlparse
> >>>
> >>> self.set_tunnel(self.host, port=self.port)
> >>>
> >>> proxy_host = urlparse(os.environ['https_proxy']).netloc
> >>> self._set_hostport(proxy_host, None)
> >>>
> >>> .
> >>> .
> >>> .
> >>>
> >>> And then modified the connect call to use the tunnel:
> >>>
> >>>
> >>> def connect(self):
> >>> """
> >>> Connect
> >>>
> >>> Checks if verification is toggled; if not, just call
> >>> httplib.HTTPSConnection's connect
> >>> """
> >>> if not self.verify:
> >>> return httplib.HTTPSConnection.connect(self)
> >>>
> >>> # otherwise, create a connection and verify the hostname
> >>> # use socket.create_connection (in 2.6+) if possible
> >>> if getattr(socket, 'create_connection', None):
> >>> sock = socket.create_connection((self.host, self.port),
> >>> self.timeout)
> >>> else:
> >>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> >>> sock.connect((self.host, self.port))
> >>>
> >>> # Support for HTTPS Proxy
> >>> if self._tunnel_host:
> >>> self.sock = sock
> >>> self._tunnel()
> >>>
> >>> self.sock = ssl.wrap_socket(sock,
> >>> self.key_file,
> >>> self.cert_file,
> >>> cert_reqs=ssl.CERT_REQUIRED,
> >>> ca_certs=self.ca_cert,
> >>> ssl_version=ssl.PROTOCOL_TLSv1)
> >>> cert = self.sock.getpeercert()
> >>> if not self._verify_hostname(self.host, cert):
> >>> raise ssl.SSLError('Failed to verify hostname')
> >>>
> >>>
> >>> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
> >>>
> >>>> Hi All,
> >>>>
> >>>> I try to use Libcloud to connect to EC2.
> >>>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy
> server that requires authentication (MS ISA Proxy Server).
> >>>> I tried with defining environment variables HTTP_PROXY and
> HTTPS_PROXY, but still doesn't work.
> >>>> Is there a way to set a proxy and proxy credentials directly using
> Libcloud ? Or should I have to patch the http_lib to implement UrlLib2
> ProxyHandler ?
> >>>>
> >>>> Thanks for your help,
> >>>> CL
> >>>
> >>> --
> >>> Scanned by iCritical.
> >>
> >> --
> >> Scanned by iCritical.
> >
> > --
> > Scanned by iCritical.
>
> --
> Scanned by iCritical.
>
RE: [dev] Libcloud behind ISA Proxy
Posted by Cedric Lebrun <Ce...@asg.com>.
I have also installed CNTLM to bypass proxy authentication.
Not the best solution, but a temporary trick to be able to progress on my prototype :-)
Cheers
Cedric
-----Original Message-----
From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
Sent: mercredi 16 juillet 2014 14:33
To: dev@libcloud.apache.org
Subject: Re: [dev] Libcloud behind ISA Proxy
Hi Cedric,
That's great news.
I thought you'd mentioned that the proxy you were connecting through also needed authentication credentials. - Is that the case and if so what additional changes did you need to make to the code?
Cheers,
Phil
On 16 Jul 2014, at 13:29, Cedric Lebrun <Ce...@asg.com> wrote:
> With Python 3.4 (on Windows), I have finally succeeded using your patch.
>
> I have just had to replace
>
> self._set_hostport(proxy_host, None)
>
> by
>
> self.host = proxyHost
> self.port = proxyPort
>
>
> With the appropriate values for proxyHost and proxyPort.
>
>
> Cedric
>
> -----Original Message-----
> From: Cedric Lebrun [mailto:Cedric.Lebrun@asg.com]
> Sent: mercredi 16 juillet 2014 08:22
> To: dev@libcloud.apache.org
> Subject: RE: [dev] Libcloud behind ISA Proxy
>
> Hi Phil,
>
> Sorry for my late reply because of vacations :-) I've tried with Python 2.7 on a CentOS environment as well as with Python 3.4 on a Windows 7 environment. But in both case, no success so far.
> Will continue investigating.
>
> Thank you.
> Cedric
>
> -----Original Message-----
> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> Sent: mardi 8 juillet 2014 15:29
> To: dev@libcloud.apache.org
> Subject: Re: [dev] Libcloud behind ISA Proxy
>
> Hi Cedric,
>
> I'm using Python 2.7.2. There seems to be differences between the two. I would adapt your code to do the job of _set_hostport and set host and port separately. They are public attributes in any case.
>
> Cheers,
> Phil
>
> On 8 Jul 2014, at 13:50, Cedric Lebrun <Ce...@asg.com> wrote:
>
>> Hi Philip,
>>
>> I'm using Python 2.7 as well (2.7.7).
>> And Libcloud 0.15.1
>>
>> Cedric
>>
>> -----Original Message-----
>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>> Sent: mardi 8 juillet 2014 12:03
>> To: dev@libcloud.apache.org
>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>
>> Hi Cedric,
>>
>> What version of Python are you using?
>>
>> In my Python 2.7 installation, _set_hostport is a method of httplib.HTTPConnection. It basically does some format checking and assigns host and port number.
>>
>> Clearly some more work will be needed for a generic solution across
>> supported versions of Python :)
>>
>> Cheers,
>> Phil
>>
>>
>> def _set_hostport(self, host, port):
>> if port is None:
>> i = host.rfind(':')
>> j = host.rfind(']') # ipv6 addresses have [...]
>> if i > j:
>> try:
>> port = int(host[i+1:])
>> except ValueError:
>> raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
>> host = host[:i]
>> else:
>> port = self.default_port
>> if host and host[0] == '[' and host[-1] == ']':
>> host = host[1:-1]
>> self.host = host
>> self.port = port
>>
>>
>> On 8 Jul 2014, at 10:45, Cedric Lebrun <Ce...@asg.com> wrote:
>>
>>> Hi Philip,
>>>
>>> Thanks for your help.
>>> But after patching httplib_ssl, I fall into now another error:
>>>
>>> AttributeError: LibcloudHTTPSConnection instance has no attribute '_set_hostport'
>>>
>>>
>>> Cedric
>>>
>>> -----Original Message-----
>>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>>> Sent: mardi 8 juillet 2014 09:56
>>> To: dev@libcloud.apache.org
>>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>>
>>> Hi Cedric,
>>>
>>> I ran into the same issue and patched libcloud.httplib_ssl. I've included the code snippets below. The proxy I use doesn't require authentication but hopefully you can adapt to make it do what you need. I tested with Python 2.7 - may need tweaking for Python 3.
>>>
>>> Cheers,
>>> Phil
>>>
>>>
>>> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy settings from the environment:
>>>
>>>
>>> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
>>> """
>>> LibcloudHTTPSConnection
>>>
>>> Subclass of HTTPSConnection which verifies certificate names if
>>> and only if CA certificates are available.
>>> """
>>> verify = True # verify by default
>>> ca_cert = None # no default CA Certificate
>>>
>>> def __init__(self, *args, **kwargs):
>>> """
>>> Constructor
>>> """
>>> self._setup_verify()
>>> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>>>
>>> # Support for HTTPS Proxy
>>> if 'https_proxy' in os.environ:
>>> from urlparse import urlparse
>>>
>>> self.set_tunnel(self.host, port=self.port)
>>>
>>> proxy_host = urlparse(os.environ['https_proxy']).netloc
>>> self._set_hostport(proxy_host, None)
>>>
>>> .
>>> .
>>> .
>>>
>>> And then modified the connect call to use the tunnel:
>>>
>>>
>>> def connect(self):
>>> """
>>> Connect
>>>
>>> Checks if verification is toggled; if not, just call
>>> httplib.HTTPSConnection's connect
>>> """
>>> if not self.verify:
>>> return httplib.HTTPSConnection.connect(self)
>>>
>>> # otherwise, create a connection and verify the hostname
>>> # use socket.create_connection (in 2.6+) if possible
>>> if getattr(socket, 'create_connection', None):
>>> sock = socket.create_connection((self.host, self.port),
>>> self.timeout)
>>> else:
>>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> sock.connect((self.host, self.port))
>>>
>>> # Support for HTTPS Proxy
>>> if self._tunnel_host:
>>> self.sock = sock
>>> self._tunnel()
>>>
>>> self.sock = ssl.wrap_socket(sock,
>>> self.key_file,
>>> self.cert_file,
>>> cert_reqs=ssl.CERT_REQUIRED,
>>> ca_certs=self.ca_cert,
>>> ssl_version=ssl.PROTOCOL_TLSv1)
>>> cert = self.sock.getpeercert()
>>> if not self._verify_hostname(self.host, cert):
>>> raise ssl.SSLError('Failed to verify hostname')
>>>
>>>
>>> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I try to use Libcloud to connect to EC2.
>>>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server that requires authentication (MS ISA Proxy Server).
>>>> I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY, but still doesn't work.
>>>> Is there a way to set a proxy and proxy credentials directly using Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 ProxyHandler ?
>>>>
>>>> Thanks for your help,
>>>> CL
>>>
>>> --
>>> Scanned by iCritical.
>>
>> --
>> Scanned by iCritical.
>
> --
> Scanned by iCritical.
--
Scanned by iCritical.
Re: [dev] Libcloud behind ISA Proxy
Posted by ph...@stfc.ac.uk.
Hi Cedric,
That’s great news.
I thought you’d mentioned that the proxy you were connecting through also needed authentication credentials. - Is that the case and if so what additional changes did you need to make to the code?
Cheers,
Phil
On 16 Jul 2014, at 13:29, Cedric Lebrun <Ce...@asg.com> wrote:
> With Python 3.4 (on Windows), I have finally succeeded using your patch.
>
> I have just had to replace
>
> self._set_hostport(proxy_host, None)
>
> by
>
> self.host = proxyHost
> self.port = proxyPort
>
>
> With the appropriate values for proxyHost and proxyPort.
>
>
> Cedric
>
> -----Original Message-----
> From: Cedric Lebrun [mailto:Cedric.Lebrun@asg.com]
> Sent: mercredi 16 juillet 2014 08:22
> To: dev@libcloud.apache.org
> Subject: RE: [dev] Libcloud behind ISA Proxy
>
> Hi Phil,
>
> Sorry for my late reply because of vacations :-) I've tried with Python 2.7 on a CentOS environment as well as with Python 3.4 on a Windows 7 environment. But in both case, no success so far.
> Will continue investigating.
>
> Thank you.
> Cedric
>
> -----Original Message-----
> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> Sent: mardi 8 juillet 2014 15:29
> To: dev@libcloud.apache.org
> Subject: Re: [dev] Libcloud behind ISA Proxy
>
> Hi Cedric,
>
> I'm using Python 2.7.2. There seems to be differences between the two. I would adapt your code to do the job of _set_hostport and set host and port separately. They are public attributes in any case.
>
> Cheers,
> Phil
>
> On 8 Jul 2014, at 13:50, Cedric Lebrun <Ce...@asg.com> wrote:
>
>> Hi Philip,
>>
>> I'm using Python 2.7 as well (2.7.7).
>> And Libcloud 0.15.1
>>
>> Cedric
>>
>> -----Original Message-----
>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>> Sent: mardi 8 juillet 2014 12:03
>> To: dev@libcloud.apache.org
>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>
>> Hi Cedric,
>>
>> What version of Python are you using?
>>
>> In my Python 2.7 installation, _set_hostport is a method of httplib.HTTPConnection. It basically does some format checking and assigns host and port number.
>>
>> Clearly some more work will be needed for a generic solution across
>> supported versions of Python :)
>>
>> Cheers,
>> Phil
>>
>>
>> def _set_hostport(self, host, port):
>> if port is None:
>> i = host.rfind(':')
>> j = host.rfind(']') # ipv6 addresses have [...]
>> if i > j:
>> try:
>> port = int(host[i+1:])
>> except ValueError:
>> raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
>> host = host[:i]
>> else:
>> port = self.default_port
>> if host and host[0] == '[' and host[-1] == ']':
>> host = host[1:-1]
>> self.host = host
>> self.port = port
>>
>>
>> On 8 Jul 2014, at 10:45, Cedric Lebrun <Ce...@asg.com> wrote:
>>
>>> Hi Philip,
>>>
>>> Thanks for your help.
>>> But after patching httplib_ssl, I fall into now another error:
>>>
>>> AttributeError: LibcloudHTTPSConnection instance has no attribute '_set_hostport'
>>>
>>>
>>> Cedric
>>>
>>> -----Original Message-----
>>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>>> Sent: mardi 8 juillet 2014 09:56
>>> To: dev@libcloud.apache.org
>>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>>
>>> Hi Cedric,
>>>
>>> I ran into the same issue and patched libcloud.httplib_ssl. I've included the code snippets below. The proxy I use doesn't require authentication but hopefully you can adapt to make it do what you need. I tested with Python 2.7 - may need tweaking for Python 3.
>>>
>>> Cheers,
>>> Phil
>>>
>>>
>>> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy settings from the environment:
>>>
>>>
>>> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
>>> """
>>> LibcloudHTTPSConnection
>>>
>>> Subclass of HTTPSConnection which verifies certificate names
>>> if and only if CA certificates are available.
>>> """
>>> verify = True # verify by default
>>> ca_cert = None # no default CA Certificate
>>>
>>> def __init__(self, *args, **kwargs):
>>> """
>>> Constructor
>>> """
>>> self._setup_verify()
>>> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>>>
>>> # Support for HTTPS Proxy
>>> if 'https_proxy' in os.environ:
>>> from urlparse import urlparse
>>>
>>> self.set_tunnel(self.host, port=self.port)
>>>
>>> proxy_host = urlparse(os.environ['https_proxy']).netloc
>>> self._set_hostport(proxy_host, None)
>>>
>>> .
>>> .
>>> .
>>>
>>> And then modified the connect call to use the tunnel:
>>>
>>>
>>> def connect(self):
>>> """
>>> Connect
>>>
>>> Checks if verification is toggled; if not, just call
>>> httplib.HTTPSConnection's connect
>>> """
>>> if not self.verify:
>>> return httplib.HTTPSConnection.connect(self)
>>>
>>> # otherwise, create a connection and verify the hostname
>>> # use socket.create_connection (in 2.6+) if possible
>>> if getattr(socket, 'create_connection', None):
>>> sock = socket.create_connection((self.host, self.port),
>>> self.timeout)
>>> else:
>>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> sock.connect((self.host, self.port))
>>>
>>> # Support for HTTPS Proxy
>>> if self._tunnel_host:
>>> self.sock = sock
>>> self._tunnel()
>>>
>>> self.sock = ssl.wrap_socket(sock,
>>> self.key_file,
>>> self.cert_file,
>>> cert_reqs=ssl.CERT_REQUIRED,
>>> ca_certs=self.ca_cert,
>>> ssl_version=ssl.PROTOCOL_TLSv1)
>>> cert = self.sock.getpeercert()
>>> if not self._verify_hostname(self.host, cert):
>>> raise ssl.SSLError('Failed to verify hostname')
>>>
>>>
>>> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I try to use Libcloud to connect to EC2.
>>>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server that requires authentication (MS ISA Proxy Server).
>>>> I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY, but still doesn't work.
>>>> Is there a way to set a proxy and proxy credentials directly using Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 ProxyHandler ?
>>>>
>>>> Thanks for your help,
>>>> CL
>>>
>>> --
>>> Scanned by iCritical.
>>
>> --
>> Scanned by iCritical.
>
> --
> Scanned by iCritical.
--
Scanned by iCritical.
RE: [dev] Libcloud behind ISA Proxy
Posted by Cedric Lebrun <Ce...@asg.com>.
With Python 3.4 (on Windows), I have finally succeeded using your patch.
I have just had to replace
self._set_hostport(proxy_host, None)
by
self.host = proxyHost
self.port = proxyPort
With the appropriate values for proxyHost and proxyPort.
Cedric
-----Original Message-----
From: Cedric Lebrun [mailto:Cedric.Lebrun@asg.com]
Sent: mercredi 16 juillet 2014 08:22
To: dev@libcloud.apache.org
Subject: RE: [dev] Libcloud behind ISA Proxy
Hi Phil,
Sorry for my late reply because of vacations :-) I've tried with Python 2.7 on a CentOS environment as well as with Python 3.4 on a Windows 7 environment. But in both case, no success so far.
Will continue investigating.
Thank you.
Cedric
-----Original Message-----
From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
Sent: mardi 8 juillet 2014 15:29
To: dev@libcloud.apache.org
Subject: Re: [dev] Libcloud behind ISA Proxy
Hi Cedric,
I'm using Python 2.7.2. There seems to be differences between the two. I would adapt your code to do the job of _set_hostport and set host and port separately. They are public attributes in any case.
Cheers,
Phil
On 8 Jul 2014, at 13:50, Cedric Lebrun <Ce...@asg.com> wrote:
> Hi Philip,
>
> I'm using Python 2.7 as well (2.7.7).
> And Libcloud 0.15.1
>
> Cedric
>
> -----Original Message-----
> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> Sent: mardi 8 juillet 2014 12:03
> To: dev@libcloud.apache.org
> Subject: Re: [dev] Libcloud behind ISA Proxy
>
> Hi Cedric,
>
> What version of Python are you using?
>
> In my Python 2.7 installation, _set_hostport is a method of httplib.HTTPConnection. It basically does some format checking and assigns host and port number.
>
> Clearly some more work will be needed for a generic solution across
> supported versions of Python :)
>
> Cheers,
> Phil
>
>
> def _set_hostport(self, host, port):
> if port is None:
> i = host.rfind(':')
> j = host.rfind(']') # ipv6 addresses have [...]
> if i > j:
> try:
> port = int(host[i+1:])
> except ValueError:
> raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
> host = host[:i]
> else:
> port = self.default_port
> if host and host[0] == '[' and host[-1] == ']':
> host = host[1:-1]
> self.host = host
> self.port = port
>
>
> On 8 Jul 2014, at 10:45, Cedric Lebrun <Ce...@asg.com> wrote:
>
>> Hi Philip,
>>
>> Thanks for your help.
>> But after patching httplib_ssl, I fall into now another error:
>>
>> AttributeError: LibcloudHTTPSConnection instance has no attribute '_set_hostport'
>>
>>
>> Cedric
>>
>> -----Original Message-----
>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>> Sent: mardi 8 juillet 2014 09:56
>> To: dev@libcloud.apache.org
>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>
>> Hi Cedric,
>>
>> I ran into the same issue and patched libcloud.httplib_ssl. I've included the code snippets below. The proxy I use doesn't require authentication but hopefully you can adapt to make it do what you need. I tested with Python 2.7 - may need tweaking for Python 3.
>>
>> Cheers,
>> Phil
>>
>>
>> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy settings from the environment:
>>
>>
>> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
>> """
>> LibcloudHTTPSConnection
>>
>> Subclass of HTTPSConnection which verifies certificate names
>> if and only if CA certificates are available.
>> """
>> verify = True # verify by default
>> ca_cert = None # no default CA Certificate
>>
>> def __init__(self, *args, **kwargs):
>> """
>> Constructor
>> """
>> self._setup_verify()
>> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>>
>> # Support for HTTPS Proxy
>> if 'https_proxy' in os.environ:
>> from urlparse import urlparse
>>
>> self.set_tunnel(self.host, port=self.port)
>>
>> proxy_host = urlparse(os.environ['https_proxy']).netloc
>> self._set_hostport(proxy_host, None)
>>
>> .
>> .
>> .
>>
>> And then modified the connect call to use the tunnel:
>>
>>
>> def connect(self):
>> """
>> Connect
>>
>> Checks if verification is toggled; if not, just call
>> httplib.HTTPSConnection's connect
>> """
>> if not self.verify:
>> return httplib.HTTPSConnection.connect(self)
>>
>> # otherwise, create a connection and verify the hostname
>> # use socket.create_connection (in 2.6+) if possible
>> if getattr(socket, 'create_connection', None):
>> sock = socket.create_connection((self.host, self.port),
>> self.timeout)
>> else:
>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>> sock.connect((self.host, self.port))
>>
>> # Support for HTTPS Proxy
>> if self._tunnel_host:
>> self.sock = sock
>> self._tunnel()
>>
>> self.sock = ssl.wrap_socket(sock,
>> self.key_file,
>> self.cert_file,
>> cert_reqs=ssl.CERT_REQUIRED,
>> ca_certs=self.ca_cert,
>> ssl_version=ssl.PROTOCOL_TLSv1)
>> cert = self.sock.getpeercert()
>> if not self._verify_hostname(self.host, cert):
>> raise ssl.SSLError('Failed to verify hostname')
>>
>>
>> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>>
>>> Hi All,
>>>
>>> I try to use Libcloud to connect to EC2.
>>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server that requires authentication (MS ISA Proxy Server).
>>> I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY, but still doesn't work.
>>> Is there a way to set a proxy and proxy credentials directly using Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 ProxyHandler ?
>>>
>>> Thanks for your help,
>>> CL
>>
>> --
>> Scanned by iCritical.
>
> --
> Scanned by iCritical.
--
Scanned by iCritical.
RE: [dev] Libcloud behind ISA Proxy
Posted by Cedric Lebrun <Ce...@asg.com>.
Hi Phil,
Sorry for my late reply because of vacations :-)
I've tried with Python 2.7 on a CentOS environment as well as with Python 3.4 on a Windows 7 environment. But in both case, no success so far.
Will continue investigating.
Thank you.
Cedric
-----Original Message-----
From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
Sent: mardi 8 juillet 2014 15:29
To: dev@libcloud.apache.org
Subject: Re: [dev] Libcloud behind ISA Proxy
Hi Cedric,
I'm using Python 2.7.2. There seems to be differences between the two. I would adapt your code to do the job of _set_hostport and set host and port separately. They are public attributes in any case.
Cheers,
Phil
On 8 Jul 2014, at 13:50, Cedric Lebrun <Ce...@asg.com> wrote:
> Hi Philip,
>
> I'm using Python 2.7 as well (2.7.7).
> And Libcloud 0.15.1
>
> Cedric
>
> -----Original Message-----
> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> Sent: mardi 8 juillet 2014 12:03
> To: dev@libcloud.apache.org
> Subject: Re: [dev] Libcloud behind ISA Proxy
>
> Hi Cedric,
>
> What version of Python are you using?
>
> In my Python 2.7 installation, _set_hostport is a method of httplib.HTTPConnection. It basically does some format checking and assigns host and port number.
>
> Clearly some more work will be needed for a generic solution across supported versions of Python :)
>
> Cheers,
> Phil
>
>
> def _set_hostport(self, host, port):
> if port is None:
> i = host.rfind(':')
> j = host.rfind(']') # ipv6 addresses have [...]
> if i > j:
> try:
> port = int(host[i+1:])
> except ValueError:
> raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
> host = host[:i]
> else:
> port = self.default_port
> if host and host[0] == '[' and host[-1] == ']':
> host = host[1:-1]
> self.host = host
> self.port = port
>
>
> On 8 Jul 2014, at 10:45, Cedric Lebrun <Ce...@asg.com> wrote:
>
>> Hi Philip,
>>
>> Thanks for your help.
>> But after patching httplib_ssl, I fall into now another error:
>>
>> AttributeError: LibcloudHTTPSConnection instance has no attribute '_set_hostport'
>>
>>
>> Cedric
>>
>> -----Original Message-----
>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>> Sent: mardi 8 juillet 2014 09:56
>> To: dev@libcloud.apache.org
>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>
>> Hi Cedric,
>>
>> I ran into the same issue and patched libcloud.httplib_ssl. I've included the code snippets below. The proxy I use doesn't require authentication but hopefully you can adapt to make it do what you need. I tested with Python 2.7 - may need tweaking for Python 3.
>>
>> Cheers,
>> Phil
>>
>>
>> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy settings from the environment:
>>
>>
>> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
>> """
>> LibcloudHTTPSConnection
>>
>> Subclass of HTTPSConnection which verifies certificate names
>> if and only if CA certificates are available.
>> """
>> verify = True # verify by default
>> ca_cert = None # no default CA Certificate
>>
>> def __init__(self, *args, **kwargs):
>> """
>> Constructor
>> """
>> self._setup_verify()
>> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>>
>> # Support for HTTPS Proxy
>> if 'https_proxy' in os.environ:
>> from urlparse import urlparse
>>
>> self.set_tunnel(self.host, port=self.port)
>>
>> proxy_host = urlparse(os.environ['https_proxy']).netloc
>> self._set_hostport(proxy_host, None)
>>
>> .
>> .
>> .
>>
>> And then modified the connect call to use the tunnel:
>>
>>
>> def connect(self):
>> """
>> Connect
>>
>> Checks if verification is toggled; if not, just call
>> httplib.HTTPSConnection's connect
>> """
>> if not self.verify:
>> return httplib.HTTPSConnection.connect(self)
>>
>> # otherwise, create a connection and verify the hostname
>> # use socket.create_connection (in 2.6+) if possible
>> if getattr(socket, 'create_connection', None):
>> sock = socket.create_connection((self.host, self.port),
>> self.timeout)
>> else:
>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>> sock.connect((self.host, self.port))
>>
>> # Support for HTTPS Proxy
>> if self._tunnel_host:
>> self.sock = sock
>> self._tunnel()
>>
>> self.sock = ssl.wrap_socket(sock,
>> self.key_file,
>> self.cert_file,
>> cert_reqs=ssl.CERT_REQUIRED,
>> ca_certs=self.ca_cert,
>> ssl_version=ssl.PROTOCOL_TLSv1)
>> cert = self.sock.getpeercert()
>> if not self._verify_hostname(self.host, cert):
>> raise ssl.SSLError('Failed to verify hostname')
>>
>>
>> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>>
>>> Hi All,
>>>
>>> I try to use Libcloud to connect to EC2.
>>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server that requires authentication (MS ISA Proxy Server).
>>> I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY, but still doesn't work.
>>> Is there a way to set a proxy and proxy credentials directly using Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 ProxyHandler ?
>>>
>>> Thanks for your help,
>>> CL
>>
>> --
>> Scanned by iCritical.
>
> --
> Scanned by iCritical.
--
Scanned by iCritical.
Re: [dev] Libcloud behind ISA Proxy
Posted by ph...@stfc.ac.uk.
Hi Cedric,
I’m using Python 2.7.2. There seems to be differences between the two. I would adapt your code to do the job of _set_hostport and set host and port separately. They are public attributes in any case.
Cheers,
Phil
On 8 Jul 2014, at 13:50, Cedric Lebrun <Ce...@asg.com> wrote:
> Hi Philip,
>
> I'm using Python 2.7 as well (2.7.7).
> And Libcloud 0.15.1
>
> Cedric
>
> -----Original Message-----
> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> Sent: mardi 8 juillet 2014 12:03
> To: dev@libcloud.apache.org
> Subject: Re: [dev] Libcloud behind ISA Proxy
>
> Hi Cedric,
>
> What version of Python are you using?
>
> In my Python 2.7 installation, _set_hostport is a method of httplib.HTTPConnection. It basically does some format checking and assigns host and port number.
>
> Clearly some more work will be needed for a generic solution across supported versions of Python :)
>
> Cheers,
> Phil
>
>
> def _set_hostport(self, host, port):
> if port is None:
> i = host.rfind(':')
> j = host.rfind(']') # ipv6 addresses have [...]
> if i > j:
> try:
> port = int(host[i+1:])
> except ValueError:
> raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
> host = host[:i]
> else:
> port = self.default_port
> if host and host[0] == '[' and host[-1] == ']':
> host = host[1:-1]
> self.host = host
> self.port = port
>
>
> On 8 Jul 2014, at 10:45, Cedric Lebrun <Ce...@asg.com> wrote:
>
>> Hi Philip,
>>
>> Thanks for your help.
>> But after patching httplib_ssl, I fall into now another error:
>>
>> AttributeError: LibcloudHTTPSConnection instance has no attribute '_set_hostport'
>>
>>
>> Cedric
>>
>> -----Original Message-----
>> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
>> Sent: mardi 8 juillet 2014 09:56
>> To: dev@libcloud.apache.org
>> Subject: Re: [dev] Libcloud behind ISA Proxy
>>
>> Hi Cedric,
>>
>> I ran into the same issue and patched libcloud.httplib_ssl. I've included the code snippets below. The proxy I use doesn't require authentication but hopefully you can adapt to make it do what you need. I tested with Python 2.7 - may need tweaking for Python 3.
>>
>> Cheers,
>> Phil
>>
>>
>> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy settings from the environment:
>>
>>
>> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
>> """
>> LibcloudHTTPSConnection
>>
>> Subclass of HTTPSConnection which verifies certificate names
>> if and only if CA certificates are available.
>> """
>> verify = True # verify by default
>> ca_cert = None # no default CA Certificate
>>
>> def __init__(self, *args, **kwargs):
>> """
>> Constructor
>> """
>> self._setup_verify()
>> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>>
>> # Support for HTTPS Proxy
>> if 'https_proxy' in os.environ:
>> from urlparse import urlparse
>>
>> self.set_tunnel(self.host, port=self.port)
>>
>> proxy_host = urlparse(os.environ['https_proxy']).netloc
>> self._set_hostport(proxy_host, None)
>>
>> .
>> .
>> .
>>
>> And then modified the connect call to use the tunnel:
>>
>>
>> def connect(self):
>> """
>> Connect
>>
>> Checks if verification is toggled; if not, just call
>> httplib.HTTPSConnection's connect
>> """
>> if not self.verify:
>> return httplib.HTTPSConnection.connect(self)
>>
>> # otherwise, create a connection and verify the hostname
>> # use socket.create_connection (in 2.6+) if possible
>> if getattr(socket, 'create_connection', None):
>> sock = socket.create_connection((self.host, self.port),
>> self.timeout)
>> else:
>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>> sock.connect((self.host, self.port))
>>
>> # Support for HTTPS Proxy
>> if self._tunnel_host:
>> self.sock = sock
>> self._tunnel()
>>
>> self.sock = ssl.wrap_socket(sock,
>> self.key_file,
>> self.cert_file,
>> cert_reqs=ssl.CERT_REQUIRED,
>> ca_certs=self.ca_cert,
>> ssl_version=ssl.PROTOCOL_TLSv1)
>> cert = self.sock.getpeercert()
>> if not self._verify_hostname(self.host, cert):
>> raise ssl.SSLError('Failed to verify hostname')
>>
>>
>> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>>
>>> Hi All,
>>>
>>> I try to use Libcloud to connect to EC2.
>>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server that requires authentication (MS ISA Proxy Server).
>>> I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY, but still doesn't work.
>>> Is there a way to set a proxy and proxy credentials directly using Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 ProxyHandler ?
>>>
>>> Thanks for your help,
>>> CL
>>
>> --
>> Scanned by iCritical.
>
> --
> Scanned by iCritical.
--
Scanned by iCritical.
RE: [dev] Libcloud behind ISA Proxy
Posted by Cedric Lebrun <Ce...@asg.com>.
Hi Philip,
I'm using Python 2.7 as well (2.7.7).
And Libcloud 0.15.1
Cedric
-----Original Message-----
From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
Sent: mardi 8 juillet 2014 12:03
To: dev@libcloud.apache.org
Subject: Re: [dev] Libcloud behind ISA Proxy
Hi Cedric,
What version of Python are you using?
In my Python 2.7 installation, _set_hostport is a method of httplib.HTTPConnection. It basically does some format checking and assigns host and port number.
Clearly some more work will be needed for a generic solution across supported versions of Python :)
Cheers,
Phil
def _set_hostport(self, host, port):
if port is None:
i = host.rfind(':')
j = host.rfind(']') # ipv6 addresses have [...]
if i > j:
try:
port = int(host[i+1:])
except ValueError:
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
host = host[:i]
else:
port = self.default_port
if host and host[0] == '[' and host[-1] == ']':
host = host[1:-1]
self.host = host
self.port = port
On 8 Jul 2014, at 10:45, Cedric Lebrun <Ce...@asg.com> wrote:
> Hi Philip,
>
> Thanks for your help.
> But after patching httplib_ssl, I fall into now another error:
>
> AttributeError: LibcloudHTTPSConnection instance has no attribute '_set_hostport'
>
>
> Cedric
>
> -----Original Message-----
> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> Sent: mardi 8 juillet 2014 09:56
> To: dev@libcloud.apache.org
> Subject: Re: [dev] Libcloud behind ISA Proxy
>
> Hi Cedric,
>
> I ran into the same issue and patched libcloud.httplib_ssl. I've included the code snippets below. The proxy I use doesn't require authentication but hopefully you can adapt to make it do what you need. I tested with Python 2.7 - may need tweaking for Python 3.
>
> Cheers,
> Phil
>
>
> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy settings from the environment:
>
>
> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
> """
> LibcloudHTTPSConnection
>
> Subclass of HTTPSConnection which verifies certificate names
> if and only if CA certificates are available.
> """
> verify = True # verify by default
> ca_cert = None # no default CA Certificate
>
> def __init__(self, *args, **kwargs):
> """
> Constructor
> """
> self._setup_verify()
> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>
> # Support for HTTPS Proxy
> if 'https_proxy' in os.environ:
> from urlparse import urlparse
>
> self.set_tunnel(self.host, port=self.port)
>
> proxy_host = urlparse(os.environ['https_proxy']).netloc
> self._set_hostport(proxy_host, None)
>
> .
> .
> .
>
> And then modified the connect call to use the tunnel:
>
>
> def connect(self):
> """
> Connect
>
> Checks if verification is toggled; if not, just call
> httplib.HTTPSConnection's connect
> """
> if not self.verify:
> return httplib.HTTPSConnection.connect(self)
>
> # otherwise, create a connection and verify the hostname
> # use socket.create_connection (in 2.6+) if possible
> if getattr(socket, 'create_connection', None):
> sock = socket.create_connection((self.host, self.port),
> self.timeout)
> else:
> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> sock.connect((self.host, self.port))
>
> # Support for HTTPS Proxy
> if self._tunnel_host:
> self.sock = sock
> self._tunnel()
>
> self.sock = ssl.wrap_socket(sock,
> self.key_file,
> self.cert_file,
> cert_reqs=ssl.CERT_REQUIRED,
> ca_certs=self.ca_cert,
> ssl_version=ssl.PROTOCOL_TLSv1)
> cert = self.sock.getpeercert()
> if not self._verify_hostname(self.host, cert):
> raise ssl.SSLError('Failed to verify hostname')
>
>
> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>
>> Hi All,
>>
>> I try to use Libcloud to connect to EC2.
>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server that requires authentication (MS ISA Proxy Server).
>> I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY, but still doesn't work.
>> Is there a way to set a proxy and proxy credentials directly using Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 ProxyHandler ?
>>
>> Thanks for your help,
>> CL
>
> --
> Scanned by iCritical.
--
Scanned by iCritical.
Re: [dev] Libcloud behind ISA Proxy
Posted by ph...@stfc.ac.uk.
Hi Cedric,
What version of Python are you using?
In my Python 2.7 installation, _set_hostport is a method of httplib.HTTPConnection. It basically does some format checking and assigns host and port number.
Clearly some more work will be needed for a generic solution across supported versions of Python :)
Cheers,
Phil
def _set_hostport(self, host, port):
if port is None:
i = host.rfind(':')
j = host.rfind(']') # ipv6 addresses have [...]
if i > j:
try:
port = int(host[i+1:])
except ValueError:
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
host = host[:i]
else:
port = self.default_port
if host and host[0] == '[' and host[-1] == ']':
host = host[1:-1]
self.host = host
self.port = port
On 8 Jul 2014, at 10:45, Cedric Lebrun <Ce...@asg.com> wrote:
> Hi Philip,
>
> Thanks for your help.
> But after patching httplib_ssl, I fall into now another error:
>
> AttributeError: LibcloudHTTPSConnection instance has no attribute '_set_hostport'
>
>
> Cedric
>
> -----Original Message-----
> From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
> Sent: mardi 8 juillet 2014 09:56
> To: dev@libcloud.apache.org
> Subject: Re: [dev] Libcloud behind ISA Proxy
>
> Hi Cedric,
>
> I ran into the same issue and patched libcloud.httplib_ssl. I've included the code snippets below. The proxy I use doesn't require authentication but hopefully you can adapt to make it do what you need. I tested with Python 2.7 - may need tweaking for Python 3.
>
> Cheers,
> Phil
>
>
> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy settings from the environment:
>
>
> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
> """
> LibcloudHTTPSConnection
>
> Subclass of HTTPSConnection which verifies certificate names
> if and only if CA certificates are available.
> """
> verify = True # verify by default
> ca_cert = None # no default CA Certificate
>
> def __init__(self, *args, **kwargs):
> """
> Constructor
> """
> self._setup_verify()
> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>
> # Support for HTTPS Proxy
> if 'https_proxy' in os.environ:
> from urlparse import urlparse
>
> self.set_tunnel(self.host, port=self.port)
>
> proxy_host = urlparse(os.environ['https_proxy']).netloc
> self._set_hostport(proxy_host, None)
>
> .
> .
> .
>
> And then modified the connect call to use the tunnel:
>
>
> def connect(self):
> """
> Connect
>
> Checks if verification is toggled; if not, just call
> httplib.HTTPSConnection's connect
> """
> if not self.verify:
> return httplib.HTTPSConnection.connect(self)
>
> # otherwise, create a connection and verify the hostname
> # use socket.create_connection (in 2.6+) if possible
> if getattr(socket, 'create_connection', None):
> sock = socket.create_connection((self.host, self.port),
> self.timeout)
> else:
> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> sock.connect((self.host, self.port))
>
> # Support for HTTPS Proxy
> if self._tunnel_host:
> self.sock = sock
> self._tunnel()
>
> self.sock = ssl.wrap_socket(sock,
> self.key_file,
> self.cert_file,
> cert_reqs=ssl.CERT_REQUIRED,
> ca_certs=self.ca_cert,
> ssl_version=ssl.PROTOCOL_TLSv1)
> cert = self.sock.getpeercert()
> if not self._verify_hostname(self.host, cert):
> raise ssl.SSLError('Failed to verify hostname')
>
>
> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>
>> Hi All,
>>
>> I try to use Libcloud to connect to EC2.
>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server that requires authentication (MS ISA Proxy Server).
>> I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY, but still doesn't work.
>> Is there a way to set a proxy and proxy credentials directly using Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 ProxyHandler ?
>>
>> Thanks for your help,
>> CL
>
> --
> Scanned by iCritical.
--
Scanned by iCritical.
RE: [dev] Libcloud behind ISA Proxy
Posted by Cedric Lebrun <Ce...@asg.com>.
Hi Philip,
Thanks for your help.
But after patching httplib_ssl, I fall into now another error:
AttributeError: LibcloudHTTPSConnection instance has no attribute '_set_hostport'
Cedric
-----Original Message-----
From: philip.kershaw@stfc.ac.uk [mailto:philip.kershaw@stfc.ac.uk]
Sent: mardi 8 juillet 2014 09:56
To: dev@libcloud.apache.org
Subject: Re: [dev] Libcloud behind ISA Proxy
Hi Cedric,
I ran into the same issue and patched libcloud.httplib_ssl. I've included the code snippets below. The proxy I use doesn't require authentication but hopefully you can adapt to make it do what you need. I tested with Python 2.7 - may need tweaking for Python 3.
Cheers,
Phil
I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy settings from the environment:
class LibcloudHTTPSConnection(httplib.HTTPSConnection):
"""
LibcloudHTTPSConnection
Subclass of HTTPSConnection which verifies certificate names
if and only if CA certificates are available.
"""
verify = True # verify by default
ca_cert = None # no default CA Certificate
def __init__(self, *args, **kwargs):
"""
Constructor
"""
self._setup_verify()
httplib.HTTPSConnection.__init__(self, *args, **kwargs)
# Support for HTTPS Proxy
if 'https_proxy' in os.environ:
from urlparse import urlparse
self.set_tunnel(self.host, port=self.port)
proxy_host = urlparse(os.environ['https_proxy']).netloc
self._set_hostport(proxy_host, None)
.
.
.
And then modified the connect call to use the tunnel:
def connect(self):
"""
Connect
Checks if verification is toggled; if not, just call
httplib.HTTPSConnection's connect
"""
if not self.verify:
return httplib.HTTPSConnection.connect(self)
# otherwise, create a connection and verify the hostname
# use socket.create_connection (in 2.6+) if possible
if getattr(socket, 'create_connection', None):
sock = socket.create_connection((self.host, self.port),
self.timeout)
else:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((self.host, self.port))
# Support for HTTPS Proxy
if self._tunnel_host:
self.sock = sock
self._tunnel()
self.sock = ssl.wrap_socket(sock,
self.key_file,
self.cert_file,
cert_reqs=ssl.CERT_REQUIRED,
ca_certs=self.ca_cert,
ssl_version=ssl.PROTOCOL_TLSv1)
cert = self.sock.getpeercert()
if not self._verify_hostname(self.host, cert):
raise ssl.SSLError('Failed to verify hostname')
On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
> Hi All,
>
> I try to use Libcloud to connect to EC2.
> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server that requires authentication (MS ISA Proxy Server).
> I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY, but still doesn't work.
> Is there a way to set a proxy and proxy credentials directly using Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 ProxyHandler ?
>
> Thanks for your help,
> CL
--
Scanned by iCritical.
Re: [dev] Libcloud behind ISA Proxy
Posted by ph...@stfc.ac.uk.
Hi Tomaz,
I’d be happy to work with Cedric and anyone else to get a solution merged in.
Cheers,
Phil
On 8 Jul 2014, at 10:07, Tomaz Muraus <to...@apache.org> wrote:
> I remember seeing this question in the past.
>
> It would be great if we could get those changes working in all the
> supported Python versions, tested and merged upstream.
>
> On Tue, Jul 8, 2014 at 9:56 AM, <ph...@stfc.ac.uk> wrote:
>
>> Hi Cedric,
>>
>> I ran into the same issue and patched libcloud.httplib_ssl. I’ve included
>> the code snippets below. The proxy I use doesn’t require authentication
>> but hopefully you can adapt to make it do what you need. I tested with
>> Python 2.7 - may need tweaking for Python 3.
>>
>> Cheers,
>> Phil
>>
>>
>> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy
>> settings from the environment:
>>
>>
>> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
>> """
>> LibcloudHTTPSConnection
>>
>> Subclass of HTTPSConnection which verifies certificate names
>> if and only if CA certificates are available.
>> """
>> verify = True # verify by default
>> ca_cert = None # no default CA Certificate
>>
>> def __init__(self, *args, **kwargs):
>> """
>> Constructor
>> """
>> self._setup_verify()
>> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>>
>> # Support for HTTPS Proxy
>> if 'https_proxy' in os.environ:
>> from urlparse import urlparse
>>
>> self.set_tunnel(self.host, port=self.port)
>>
>> proxy_host = urlparse(os.environ['https_proxy']).netloc
>> self._set_hostport(proxy_host, None)
>>
>> .
>> .
>> .
>>
>> And then modified the connect call to use the tunnel:
>>
>>
>> def connect(self):
>> """
>> Connect
>>
>> Checks if verification is toggled; if not, just call
>> httplib.HTTPSConnection's connect
>> """
>> if not self.verify:
>> return httplib.HTTPSConnection.connect(self)
>>
>> # otherwise, create a connection and verify the hostname
>> # use socket.create_connection (in 2.6+) if possible
>> if getattr(socket, 'create_connection', None):
>> sock = socket.create_connection((self.host, self.port),
>> self.timeout)
>> else:
>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>> sock.connect((self.host, self.port))
>>
>> # Support for HTTPS Proxy
>> if self._tunnel_host:
>> self.sock = sock
>> self._tunnel()
>>
>> self.sock = ssl.wrap_socket(sock,
>> self.key_file,
>> self.cert_file,
>> cert_reqs=ssl.CERT_REQUIRED,
>> ca_certs=self.ca_cert,
>> ssl_version=ssl.PROTOCOL_TLSv1)
>> cert = self.sock.getpeercert()
>> if not self._verify_hostname(self.host, cert):
>> raise ssl.SSLError('Failed to verify hostname')
>>
>>
>> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>>
>>> Hi All,
>>>
>>> I try to use Libcloud to connect to EC2.
>>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server
>> that requires authentication (MS ISA Proxy Server).
>>> I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY,
>> but still doesn't work.
>>> Is there a way to set a proxy and proxy credentials directly using
>> Libcloud ? Or should I have to patch the http_lib to implement UrlLib2
>> ProxyHandler ?
>>>
>>> Thanks for your help,
>>> CL
>>
>> --
>> Scanned by iCritical.
>>
--
Scanned by iCritical.
Re: [dev] Libcloud behind ISA Proxy
Posted by Tomaz Muraus <to...@apache.org>.
I remember seeing this question in the past.
It would be great if we could get those changes working in all the
supported Python versions, tested and merged upstream.
On Tue, Jul 8, 2014 at 9:56 AM, <ph...@stfc.ac.uk> wrote:
> Hi Cedric,
>
> I ran into the same issue and patched libcloud.httplib_ssl. I’ve included
> the code snippets below. The proxy I use doesn’t require authentication
> but hopefully you can adapt to make it do what you need. I tested with
> Python 2.7 - may need tweaking for Python 3.
>
> Cheers,
> Phil
>
>
> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy
> settings from the environment:
>
>
> class LibcloudHTTPSConnection(httplib.HTTPSConnection):
> """
> LibcloudHTTPSConnection
>
> Subclass of HTTPSConnection which verifies certificate names
> if and only if CA certificates are available.
> """
> verify = True # verify by default
> ca_cert = None # no default CA Certificate
>
> def __init__(self, *args, **kwargs):
> """
> Constructor
> """
> self._setup_verify()
> httplib.HTTPSConnection.__init__(self, *args, **kwargs)
>
> # Support for HTTPS Proxy
> if 'https_proxy' in os.environ:
> from urlparse import urlparse
>
> self.set_tunnel(self.host, port=self.port)
>
> proxy_host = urlparse(os.environ['https_proxy']).netloc
> self._set_hostport(proxy_host, None)
>
> .
> .
> .
>
> And then modified the connect call to use the tunnel:
>
>
> def connect(self):
> """
> Connect
>
> Checks if verification is toggled; if not, just call
> httplib.HTTPSConnection's connect
> """
> if not self.verify:
> return httplib.HTTPSConnection.connect(self)
>
> # otherwise, create a connection and verify the hostname
> # use socket.create_connection (in 2.6+) if possible
> if getattr(socket, 'create_connection', None):
> sock = socket.create_connection((self.host, self.port),
> self.timeout)
> else:
> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> sock.connect((self.host, self.port))
>
> # Support for HTTPS Proxy
> if self._tunnel_host:
> self.sock = sock
> self._tunnel()
>
> self.sock = ssl.wrap_socket(sock,
> self.key_file,
> self.cert_file,
> cert_reqs=ssl.CERT_REQUIRED,
> ca_certs=self.ca_cert,
> ssl_version=ssl.PROTOCOL_TLSv1)
> cert = self.sock.getpeercert()
> if not self._verify_hostname(self.host, cert):
> raise ssl.SSLError('Failed to verify hostname')
>
>
> On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
>
> > Hi All,
> >
> > I try to use Libcloud to connect to EC2.
> > My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server
> that requires authentication (MS ISA Proxy Server).
> > I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY,
> but still doesn't work.
> > Is there a way to set a proxy and proxy credentials directly using
> Libcloud ? Or should I have to patch the http_lib to implement UrlLib2
> ProxyHandler ?
> >
> > Thanks for your help,
> > CL
>
> --
> Scanned by iCritical.
>
Re: [dev] Libcloud behind ISA Proxy
Posted by ph...@stfc.ac.uk.
Hi Cedric,
I ran into the same issue and patched libcloud.httplib_ssl. I’ve included the code snippets below. The proxy I use doesn’t require authentication but hopefully you can adapt to make it do what you need. I tested with Python 2.7 - may need tweaking for Python 3.
Cheers,
Phil
I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy settings from the environment:
class LibcloudHTTPSConnection(httplib.HTTPSConnection):
"""
LibcloudHTTPSConnection
Subclass of HTTPSConnection which verifies certificate names
if and only if CA certificates are available.
"""
verify = True # verify by default
ca_cert = None # no default CA Certificate
def __init__(self, *args, **kwargs):
"""
Constructor
"""
self._setup_verify()
httplib.HTTPSConnection.__init__(self, *args, **kwargs)
# Support for HTTPS Proxy
if 'https_proxy' in os.environ:
from urlparse import urlparse
self.set_tunnel(self.host, port=self.port)
proxy_host = urlparse(os.environ['https_proxy']).netloc
self._set_hostport(proxy_host, None)
.
.
.
And then modified the connect call to use the tunnel:
def connect(self):
"""
Connect
Checks if verification is toggled; if not, just call
httplib.HTTPSConnection's connect
"""
if not self.verify:
return httplib.HTTPSConnection.connect(self)
# otherwise, create a connection and verify the hostname
# use socket.create_connection (in 2.6+) if possible
if getattr(socket, 'create_connection', None):
sock = socket.create_connection((self.host, self.port),
self.timeout)
else:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((self.host, self.port))
# Support for HTTPS Proxy
if self._tunnel_host:
self.sock = sock
self._tunnel()
self.sock = ssl.wrap_socket(sock,
self.key_file,
self.cert_file,
cert_reqs=ssl.CERT_REQUIRED,
ca_certs=self.ca_cert,
ssl_version=ssl.PROTOCOL_TLSv1)
cert = self.sock.getpeercert()
if not self._verify_hostname(self.host, cert):
raise ssl.SSLError('Failed to verify hostname')
On 8 Jul 2014, at 07:23, Cedric Lebrun <Ce...@asg.com> wrote:
> Hi All,
>
> I try to use Libcloud to connect to EC2.
> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy server that requires authentication (MS ISA Proxy Server).
> I tried with defining environment variables HTTP_PROXY and HTTPS_PROXY, but still doesn't work.
> Is there a way to set a proxy and proxy credentials directly using Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 ProxyHandler ?
>
> Thanks for your help,
> CL
--
Scanned by iCritical.