You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@whirr.apache.org by John S <bu...@gmail.com> on 2014/04/01 14:03:28 UTC
Launch Hadoop CLuster Using Whirr within EC2 VPC
Hi All,
I was trying to launch hadoop cluster on Ec2 within a VPC subnet . Tried
with whir-0.8.2 ,checked options in jclouds, but am unable to start a
instance in VPC .
Please let me know if this is possible and any docs available on the same .
Regards
John
Re: Launch Hadoop CLuster Using Whirr within EC2 VPC
Posted by Samarth Gupta <sa...@gmail.com>.
not sure if this answers the question or is the right way but worked for me
.......
we had a use case to launch instance on private subnetid in our company.
I couldn't find any reference in whirr to do the same, so we made few code
changes in whirr core and added "subnetid" as a property in "cluster spec"
class file. Then able to launch instance in private subnet successfully.
Below is the api call we added in BootstrapTemplate.java
"template.getOptions().as(AWSEC2TemplateOptions.class)
.subnetId(spec.getAwsEc2SubnetID());"
On Tue, Apr 1, 2014 at 7:39 PM, John S <bu...@gmail.com> wrote:
> Hi Shane,
>
> Please confirm if the instance is brought under any vpc , when i launched
> instances are created in Public subnet and in some private subnet ,but
> unable to create within a VPC or i didnt find any reference to Subnetid
> using jclouds .....same concern Werner mentioned
>
> regards.
>
>
>
>
> On Tue, Apr 1, 2014 at 5:51 PM, Werner Buck <em...@wernerbuck.nl> wrote:
>
>> Are you sure VPC is supported? I can't find a reference to "subnet" in
>> all of the code in the whirr trunk.
>> http://jclouds.apache.org/guides/aws-ec2/
>>
>>
>> On Tue, Apr 1, 2014 at 2:16 PM, Shane O'Donnell <sh...@knownormal.com>wrote:
>>
>>> Are you having success doing this on a "public" subnet, but it's
>>> specifically failing on VPC, or you just happening to be using VPC when you
>>> notice the problems?
>>>
>>> I successfully brought up a cluster yesterday on EC2 "public". The
>>> biggest hurdle for me was the fact that the latest Java (1.7u51?) won't
>>> work because of a conflict in jclouds (specifically, in guava).
>>> Backleveling the java worked for me.
>>>
>>> Hope this helps -
>>>
>>> Shane O.
>>>
>>>
>>> On Tue, Apr 1, 2014 at 8:03 AM, John S <bu...@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I was trying to launch hadoop cluster on Ec2 within a VPC subnet .
>>>> Tried with whir-0.8.2 ,checked options in jclouds, but am unable to start a
>>>> instance in VPC .
>>>>
>>>> Please let me know if this is possible and any docs available on the
>>>> same .
>>>>
>>>> Regards
>>>> John
>>>>
>>>
>>>
>>>
>>> --
>>> Shane O.
>>> ========================
>>> Shane O'Donnell
>>> Know Normal, Inc.
>>> cell: +1.919.323.5380
>>> skype: shaneodonnell
>>> email: shaneo@knownormal.com
>>> ========================
>>> :wq!
>>>
>>
>>
>
Re: Launch Hadoop CLuster Using Whirr within EC2 VPC
Posted by John S <bu...@gmail.com>.
Hi Shane,
Please confirm if the instance is brought under any vpc , when i launched
instances are created in Public subnet and in some private subnet ,but
unable to create within a VPC or i didnt find any reference to Subnetid
using jclouds .....same concern Werner mentioned
regards.
On Tue, Apr 1, 2014 at 5:51 PM, Werner Buck <em...@wernerbuck.nl> wrote:
> Are you sure VPC is supported? I can't find a reference to "subnet" in all
> of the code in the whirr trunk.
> http://jclouds.apache.org/guides/aws-ec2/
>
>
> On Tue, Apr 1, 2014 at 2:16 PM, Shane O'Donnell <sh...@knownormal.com>wrote:
>
>> Are you having success doing this on a "public" subnet, but it's
>> specifically failing on VPC, or you just happening to be using VPC when you
>> notice the problems?
>>
>> I successfully brought up a cluster yesterday on EC2 "public". The
>> biggest hurdle for me was the fact that the latest Java (1.7u51?) won't
>> work because of a conflict in jclouds (specifically, in guava).
>> Backleveling the java worked for me.
>>
>> Hope this helps -
>>
>> Shane O.
>>
>>
>> On Tue, Apr 1, 2014 at 8:03 AM, John S <bu...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I was trying to launch hadoop cluster on Ec2 within a VPC subnet . Tried
>>> with whir-0.8.2 ,checked options in jclouds, but am unable to start a
>>> instance in VPC .
>>>
>>> Please let me know if this is possible and any docs available on the
>>> same .
>>>
>>> Regards
>>> John
>>>
>>
>>
>>
>> --
>> Shane O.
>> ========================
>> Shane O'Donnell
>> Know Normal, Inc.
>> cell: +1.919.323.5380
>> skype: shaneodonnell
>> email: shaneo@knownormal.com
>> ========================
>> :wq!
>>
>
>
Re: Launch Hadoop CLuster Using Whirr within EC2 VPC
Posted by Werner Buck <em...@wernerbuck.nl>.
Are you sure VPC is supported? I can't find a reference to "subnet" in all
of the code in the whirr trunk.
http://jclouds.apache.org/guides/aws-ec2/
On Tue, Apr 1, 2014 at 2:16 PM, Shane O'Donnell <sh...@knownormal.com>wrote:
> Are you having success doing this on a "public" subnet, but it's
> specifically failing on VPC, or you just happening to be using VPC when you
> notice the problems?
>
> I successfully brought up a cluster yesterday on EC2 "public". The
> biggest hurdle for me was the fact that the latest Java (1.7u51?) won't
> work because of a conflict in jclouds (specifically, in guava).
> Backleveling the java worked for me.
>
> Hope this helps -
>
> Shane O.
>
>
> On Tue, Apr 1, 2014 at 8:03 AM, John S <bu...@gmail.com> wrote:
>
>> Hi All,
>>
>> I was trying to launch hadoop cluster on Ec2 within a VPC subnet . Tried
>> with whir-0.8.2 ,checked options in jclouds, but am unable to start a
>> instance in VPC .
>>
>> Please let me know if this is possible and any docs available on the same
>> .
>>
>> Regards
>> John
>>
>
>
>
> --
> Shane O.
> ========================
> Shane O'Donnell
> Know Normal, Inc.
> cell: +1.919.323.5380
> skype: shaneodonnell
> email: shaneo@knownormal.com
> ========================
> :wq!
>
Re: Launch Hadoop CLuster Using Whirr within EC2 VPC
Posted by Shane O'Donnell <sh...@knownormal.com>.
Are you having success doing this on a "public" subnet, but it's
specifically failing on VPC, or you just happening to be using VPC when you
notice the problems?
I successfully brought up a cluster yesterday on EC2 "public". The biggest
hurdle for me was the fact that the latest Java (1.7u51?) won't work
because of a conflict in jclouds (specifically, in guava). Backleveling
the java worked for me.
Hope this helps -
Shane O.
On Tue, Apr 1, 2014 at 8:03 AM, John S <bu...@gmail.com> wrote:
> Hi All,
>
> I was trying to launch hadoop cluster on Ec2 within a VPC subnet . Tried
> with whir-0.8.2 ,checked options in jclouds, but am unable to start a
> instance in VPC .
>
> Please let me know if this is possible and any docs available on the same .
>
> Regards
> John
>
--
Shane O.
========================
Shane O'Donnell
Know Normal, Inc.
cell: +1.919.323.5380
skype: shaneodonnell
email: shaneo@knownormal.com
========================
:wq!