hosting recommandation for a small hadoop cluster?

[Thomas Koch <th...@koch.ro>]

Hi,

do you know and would maybe like to recommend hosting providers which have 
hadoop-friendly offerings? I'm new to hadoop, but what I've read is:

- virtualization makes no sense and only costs
- no raid, hdfs does the replication
- 16GB+, 2CPU+

My cluster should be around 10 machines. Most likely I'd take another 5 for 
development. I do very much prefer dedicated machines and Debian since we're 
not familiar with Amazon or other cloud providers.

I'm currently thinking about rackspace and hetzner.

Thanks for your insights,

Thomas Koch, http://www.koch.ro

Re: hosting recommandation for a small hadoop cluster?

[Patrick Angeles <pa...@gmail.com>]

Thomas,

Owning your machines and renting a 1/2 cabinet in a colo facility is the
cheapest way to go in the long run.

That said, you could also try www.softlayer.com. You'll get a good idea of
the pricing up front as they allow you to configure everything on the
website. You can also get the Cloudera Distro right on the machines which is
nice.

- P

On Tue, Feb 16, 2010 at 3:28 AM, Thomas Koch <th...@koch.ro> wrote:

> Hi,
>
> do you know and would maybe like to recommend hosting providers which have
> hadoop-friendly offerings? I'm new to hadoop, but what I've read is:
>
> - virtualization makes no sense and only costs
> - no raid, hdfs does the replication
> - 16GB+, 2CPU+
>
> My cluster should be around 10 machines. Most likely I'd take another 5 for
> development. I do very much prefer dedicated machines and Debian since
> we're
> not familiar with Amazon or other cloud providers.
>
> I'm currently thinking about rackspace and hetzner.
>
> Thanks for your insights,
>
> Thomas Koch, http://www.koch.ro
>

Re: hosting recommandation for a small hadoop cluster?

[Edward Capriolo <ed...@gmail.com>]

On Tue, Feb 16, 2010 at 9:53 AM, Eric Sammer <er...@lifeless.net> wrote:
> On 2/16/10 3:28 AM, Thomas Koch wrote:
>> Hi,
>>
>> do you know and would maybe like to recommend hosting providers which have
>> hadoop-friendly offerings? I'm new to hadoop, but what I've read is:
>>
>> - virtualization makes no sense and only costs
>> - no raid, hdfs does the replication
>> - 16GB+, 2CPU+
>>
>> My cluster should be around 10 machines. Most likely I'd take another 5 for
>> development. I do very much prefer dedicated machines and Debian since we're
>> not familiar with Amazon or other cloud providers.
>>
>> I'm currently thinking about rackspace and hetzner.
>
> Thomas:
>
> I don't have much experience with Rackspace or Hetzner as I've always
> just done hosting myself in carrier neutral facilities, but I wanted to
> give you another data point. When talking to these types of vendors, it
> may do you well try to and find out about physical machine placement. If
> you're getting 10 - 15 machines, they may be able to get you machines
> all on the same switch and within the same rack. Let them know what
> you're doing benefits from as few network devices between the machines
> as possible.
>
> Good luck!
> --
> Eric Sammer
> eric@lifeless.net
> http://esammer.blogspot.com
>

Rackspace is very managed hosting, I generally prefer more un-managed.
You probably want to specifically ask for a half/quarter cabinet and
not public colo space. I used to work at an ISP. it would be pretty
uncommon for a 10 + sever order to be placed on a public switch. you
are approaching an enterprise level,  private router + private switch+
public ip block. Public colo is normally for 1-5 servers. You would
not want a public switch for a number of reasons:

1) security (hadoop wants a trusted network)
2) performance (I am sure either people on the network are going to
affect you or more then likely your massive data jobs are going to
effect them)

You should ask for a dedicated switch and make sure it is a good/new
one. Sometimes, if you don't ask you might get the switch laying on
someones desk for a while.

Re: hosting recommandation for a small hadoop cluster?

[Eric Sammer <er...@lifeless.net>]

On 2/16/10 3:28 AM, Thomas Koch wrote:
> Hi,
> 
> do you know and would maybe like to recommend hosting providers which have 
> hadoop-friendly offerings? I'm new to hadoop, but what I've read is:
> 
> - virtualization makes no sense and only costs
> - no raid, hdfs does the replication
> - 16GB+, 2CPU+
> 
> My cluster should be around 10 machines. Most likely I'd take another 5 for 
> development. I do very much prefer dedicated machines and Debian since we're 
> not familiar with Amazon or other cloud providers.
> 
> I'm currently thinking about rackspace and hetzner.

Thomas:

I don't have much experience with Rackspace or Hetzner as I've always
just done hosting myself in carrier neutral facilities, but I wanted to
give you another data point. When talking to these types of vendors, it
may do you well try to and find out about physical machine placement. If
you're getting 10 - 15 machines, they may be able to get you machines
all on the same switch and within the same rack. Let them know what
you're doing benefits from as few network devices between the machines
as possible.

Good luck!
-- 
Eric Sammer
eric@lifeless.net
http://esammer.blogspot.com

Re: hosting recommandation for a small hadoop cluster?

[John Martyniak <jo...@beforedawnsolutions.com>]

The company that I work for offers hosting, and we are currently hosting our
hadoop cluster.  Before Dawn Solutions, Inc. the hosting information is at
http://www.beforedawnsolutions.com/hosting-services/denver-colocation-cat.

Reasonable prices in a top end hosting facility with a fast connection.

-John


On Tue, Feb 16, 2010 at 1:28 AM, Thomas Koch <th...@koch.ro> wrote:

> Hi,
>
> do you know and would maybe like to recommend hosting providers which have
> hadoop-friendly offerings? I'm new to hadoop, but what I've read is:
>
> - virtualization makes no sense and only costs
> - no raid, hdfs does the replication
> - 16GB+, 2CPU+
>
> My cluster should be around 10 machines. Most likely I'd take another 5 for
> development. I do very much prefer dedicated machines and Debian since
> we're
> not familiar with Amazon or other cloud providers.
>
> I'm currently thinking about rackspace and hetzner.
>
> Thanks for your insights,
>
> Thomas Koch, http://www.koch.ro
>



-- 
John Martyniak
President/CEO
Before Dawn Solutions, Inc.
9457 S. University Blvd #266
Highlands Ranch, CO 80126
o: 877-499-1562
c: 303-522-1756
e: john@beforedawnsoutions.com
w: http://www.beforedawnsolutions.com