You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jason Lotz <ja...@gmail.com> on 2010/11/03 14:21:03 UTC
Where do you get your hardware?
We are in the process of analyzing our options for the future purchases of
our Hadoop/HBase DN/RS servers. Currently, we purchase Dell PowerEdge
R710's which work well for us. However, we know that there are other
options that may give us more bang for our buck.
I'm not as interested in knowing the specs of the machines that people are
using. Rather, I'm curious to know where you buy them from or if you are
building them yourselves.
Any feedback on how you acquire server hardware in your environment would be
greatly appreciated.
Jason
Re: Where do you get your hardware?
Posted by tsuna <ts...@gmail.com>.
On Wed, Nov 3, 2010 at 11:45 AM, Jason Lotz <ja...@gmail.com> wrote:
> Thanks for the replies. My take away is that most organizations are buying
> from vendors (Dell, HP, SuperMicro, HP, etc.) While "build it yourself" is
> an approach, I'm not hearing a lot of companies that are doing it.
Maybe companies that are doing it aren't willing to talk too much about it?
I also like ASA Computers "Twin Servers" (2 nodes in 1U):
http://www.asaservers.com/showpages.asp?pid=1280
Good bang for the buck.
--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com
Re: Where do you get your hardware?
Posted by Andrew Purtell <ap...@apache.org>.
We have had ixSystems build hardware for us as well.
Best regards,
- Andy
----- Original Message ----
> From: Daniel Einspanjer <de...@mozilla.com>
> To: user@hbase.apache.org
> Cc: M. C. Srivas <mc...@gmail.com>
> Sent: Sun, November 7, 2010 1:27:58 AM
> Subject: Re: Where do you get your hardware?
>
> I set up a googledocs spreadsheet a while back and shared it on this
> list that specifically broke down the costs associated with doing
> SuperMicro 2 node 2U or 4 node 2U servers. The problem with the 2.5
> inch drives is that you can't get a large drive that is enterprise class
> (important for vibration in this case) without totally blowing the price
> curve out.
>
> We've built clusters using both HP and SuperMicro hardware from iX Systems.
>
> -Daniel
>
> On 11/6/10 5:44 PM, M. C. Srivas wrote:
> > On Sat, Nov 6, 2010 at 1:44 PM, Michael
>Segel<mi...@hotmail.com>wrote:
> >
> >>
> >> Nothing against Super Micro, but...
> >>
> >> You're paying a premium for the half board machines and you're also going
> >> to be paying a premium for 2.5" disks.
> >>
> >> Are you really that tight on rack space?
> >>
> > More tight on $ than rackspace :).
> >
> > I think the chassis costs about the same, if not cheaper, than Dell or HP.
> >
> > I just bought a 2.5" 5400 rpm 1TB drive from Fry's for $90 last Tuesday. So
> > yeah, its more $/TB/drive , since a 3.5" 1TB drive at the same rpm costs
> > about $70. But I got 2 more spindles per machine compared to a 4-drive
> > unit, so its 50% better I/O perf (both xfer rate and disk-ops) for about
> > $220 more ( = 2 x $90 + 4 x ($90 - $70)) more.
> >
> >
> >
> >
> >>
> >>> Date: Sat, 6 Nov 2010 07:25:04 -0700
> >>> Subject: Re: Where do you get your hardware?
> >>> From: mcsrivas@gmail.com
> >>> To: user@hbase.apache.org
> >>>
> >>> Supermicro also sells 4 nodes in a 2U, with 24 2.5" drives. You get 6
> >>> drives per node, which is 50% more disk per node. The prices seem to be
> >>> pretty good.
> >>>
> >>>
> >>> On Fri, Nov 5, 2010 at 10:17 PM, tsuna<ts...@gmail.com> wrote:
> >>>
> >>>> On Fri, Nov 5, 2010 at 8:59 PM, Patrick Angeles<pa...@cloudera.com>
> >>>> wrote:
> >>>>> Did you mean 2 nodes in 2U?
> >>>> No, I meant 2 nodes in 1U.
> >>>>
> >>>>> Dell, HP and SuperMicro all have models that fit the bill. If you
> >> really
> >>>> did
> >>>>> mean 2 nodes in 1U you're looking at either 2.5" drives or< 4
> >> spindles
> >>>> per
> >>>>> node, neither of which is ideal for Hadoop/HBase in terms of !/$
> >> (bang
> >>>> per
> >>>>> buck).
> >>>> Yeah those particular models have room for 4 x 3.5" disks (so 2 per
> >>>> node). Given how much they cost, they still give you a pretty good
> >>>> !/$. Plus, you can always mod them a little bit to fit more disks in
> >>>> the system (this, however, is left as an exercise for the reader).
> >>>>
> >>>> There are other vendors who sell cheap 2-in-1U servers, some may offer
> >>>> more 3.5" disk slots out of the box. Otherwise you can always do like
> >>>> Google and build your own. You just need a bunch of dudes who are
> >>>> passionate about that stuff :)
> >>>>
> >>>> --
> >>>> Benoit "tsuna" Sigoure
> >>>> Software Engineer @ www.StumbleUpon.com
> >>>>
> >>
>
Re: Where do you get your hardware?
Posted by Daniel Einspanjer <de...@mozilla.com>.
I set up a googledocs spreadsheet a while back and shared it on this
list that specifically broke down the costs associated with doing
SuperMicro 2 node 2U or 4 node 2U servers. The problem with the 2.5
inch drives is that you can't get a large drive that is enterprise class
(important for vibration in this case) without totally blowing the price
curve out.
We've built clusters using both HP and SuperMicro hardware from iX Systems.
-Daniel
On 11/6/10 5:44 PM, M. C. Srivas wrote:
> On Sat, Nov 6, 2010 at 1:44 PM, Michael Segel<mi...@hotmail.com>wrote:
>
>>
>> Nothing against Super Micro, but...
>>
>> You're paying a premium for the half board machines and you're also going
>> to be paying a premium for 2.5" disks.
>>
>> Are you really that tight on rack space?
>>
> More tight on $ than rackspace :).
>
> I think the chassis costs about the same, if not cheaper, than Dell or HP.
>
> I just bought a 2.5" 5400 rpm 1TB drive from Fry's for $90 last Tuesday. So
> yeah, its more $/TB/drive , since a 3.5" 1TB drive at the same rpm costs
> about $70. But I got 2 more spindles per machine compared to a 4-drive
> unit, so its 50% better I/O perf (both xfer rate and disk-ops) for about
> $220 more ( = 2 x $90 + 4 x ($90 - $70)) more.
>
>
>
>
>>
>>> Date: Sat, 6 Nov 2010 07:25:04 -0700
>>> Subject: Re: Where do you get your hardware?
>>> From: mcsrivas@gmail.com
>>> To: user@hbase.apache.org
>>>
>>> Supermicro also sells 4 nodes in a 2U, with 24 2.5" drives. You get 6
>>> drives per node, which is 50% more disk per node. The prices seem to be
>>> pretty good.
>>>
>>>
>>> On Fri, Nov 5, 2010 at 10:17 PM, tsuna<ts...@gmail.com> wrote:
>>>
>>>> On Fri, Nov 5, 2010 at 8:59 PM, Patrick Angeles<pa...@cloudera.com>
>>>> wrote:
>>>>> Did you mean 2 nodes in 2U?
>>>> No, I meant 2 nodes in 1U.
>>>>
>>>>> Dell, HP and SuperMicro all have models that fit the bill. If you
>> really
>>>> did
>>>>> mean 2 nodes in 1U you're looking at either 2.5" drives or< 4
>> spindles
>>>> per
>>>>> node, neither of which is ideal for Hadoop/HBase in terms of !/$
>> (bang
>>>> per
>>>>> buck).
>>>> Yeah those particular models have room for 4 x 3.5" disks (so 2 per
>>>> node). Given how much they cost, they still give you a pretty good
>>>> !/$. Plus, you can always mod them a little bit to fit more disks in
>>>> the system (this, however, is left as an exercise for the reader).
>>>>
>>>> There are other vendors who sell cheap 2-in-1U servers, some may offer
>>>> more 3.5" disk slots out of the box. Otherwise you can always do like
>>>> Google and build your own. You just need a bunch of dudes who are
>>>> passionate about that stuff :)
>>>>
>>>> --
>>>> Benoit "tsuna" Sigoure
>>>> Software Engineer @ www.StumbleUpon.com
>>>>
>>
Re: Where do you get your hardware?
Posted by "M. C. Srivas" <mc...@gmail.com>.
On Sat, Nov 6, 2010 at 1:44 PM, Michael Segel <mi...@hotmail.com>wrote:
>
>
> Nothing against Super Micro, but...
>
> You're paying a premium for the half board machines and you're also going
> to be paying a premium for 2.5" disks.
>
> Are you really that tight on rack space?
>
More tight on $ than rackspace :).
I think the chassis costs about the same, if not cheaper, than Dell or HP.
I just bought a 2.5" 5400 rpm 1TB drive from Fry's for $90 last Tuesday. So
yeah, its more $/TB/drive , since a 3.5" 1TB drive at the same rpm costs
about $70. But I got 2 more spindles per machine compared to a 4-drive
unit, so its 50% better I/O perf (both xfer rate and disk-ops) for about
$220 more ( = 2 x $90 + 4 x ($90 - $70)) more.
>
>
> > Date: Sat, 6 Nov 2010 07:25:04 -0700
> > Subject: Re: Where do you get your hardware?
> > From: mcsrivas@gmail.com
> > To: user@hbase.apache.org
> >
> > Supermicro also sells 4 nodes in a 2U, with 24 2.5" drives. You get 6
> > drives per node, which is 50% more disk per node. The prices seem to be
> > pretty good.
> >
> >
> > On Fri, Nov 5, 2010 at 10:17 PM, tsuna <ts...@gmail.com> wrote:
> >
> > > On Fri, Nov 5, 2010 at 8:59 PM, Patrick Angeles <pa...@cloudera.com>
> > > wrote:
> > > > Did you mean 2 nodes in 2U?
> > >
> > > No, I meant 2 nodes in 1U.
> > >
> > > > Dell, HP and SuperMicro all have models that fit the bill. If you
> really
> > > did
> > > > mean 2 nodes in 1U you're looking at either 2.5" drives or < 4
> spindles
> > > per
> > > > node, neither of which is ideal for Hadoop/HBase in terms of !/$
> (bang
> > > per
> > > > buck).
> > >
> > > Yeah those particular models have room for 4 x 3.5" disks (so 2 per
> > > node). Given how much they cost, they still give you a pretty good
> > > !/$. Plus, you can always mod them a little bit to fit more disks in
> > > the system (this, however, is left as an exercise for the reader).
> > >
> > > There are other vendors who sell cheap 2-in-1U servers, some may offer
> > > more 3.5" disk slots out of the box. Otherwise you can always do like
> > > Google and build your own. You just need a bunch of dudes who are
> > > passionate about that stuff :)
> > >
> > > --
> > > Benoit "tsuna" Sigoure
> > > Software Engineer @ www.StumbleUpon.com
> > >
>
>
RE: Where do you get your hardware?
Posted by Michael Segel <mi...@hotmail.com>.
Nothing against Super Micro, but...
You're paying a premium for the half board machines and you're also going to be paying a premium for 2.5" disks.
Are you really that tight on rack space?
> Date: Sat, 6 Nov 2010 07:25:04 -0700
> Subject: Re: Where do you get your hardware?
> From: mcsrivas@gmail.com
> To: user@hbase.apache.org
>
> Supermicro also sells 4 nodes in a 2U, with 24 2.5" drives. You get 6
> drives per node, which is 50% more disk per node. The prices seem to be
> pretty good.
>
>
> On Fri, Nov 5, 2010 at 10:17 PM, tsuna <ts...@gmail.com> wrote:
>
> > On Fri, Nov 5, 2010 at 8:59 PM, Patrick Angeles <pa...@cloudera.com>
> > wrote:
> > > Did you mean 2 nodes in 2U?
> >
> > No, I meant 2 nodes in 1U.
> >
> > > Dell, HP and SuperMicro all have models that fit the bill. If you really
> > did
> > > mean 2 nodes in 1U you're looking at either 2.5" drives or < 4 spindles
> > per
> > > node, neither of which is ideal for Hadoop/HBase in terms of !/$ (bang
> > per
> > > buck).
> >
> > Yeah those particular models have room for 4 x 3.5" disks (so 2 per
> > node). Given how much they cost, they still give you a pretty good
> > !/$. Plus, you can always mod them a little bit to fit more disks in
> > the system (this, however, is left as an exercise for the reader).
> >
> > There are other vendors who sell cheap 2-in-1U servers, some may offer
> > more 3.5" disk slots out of the box. Otherwise you can always do like
> > Google and build your own. You just need a bunch of dudes who are
> > passionate about that stuff :)
> >
> > --
> > Benoit "tsuna" Sigoure
> > Software Engineer @ www.StumbleUpon.com
> >
Re: Where do you get your hardware?
Posted by "M. C. Srivas" <mc...@gmail.com>.
Supermicro also sells 4 nodes in a 2U, with 24 2.5" drives. You get 6
drives per node, which is 50% more disk per node. The prices seem to be
pretty good.
On Fri, Nov 5, 2010 at 10:17 PM, tsuna <ts...@gmail.com> wrote:
> On Fri, Nov 5, 2010 at 8:59 PM, Patrick Angeles <pa...@cloudera.com>
> wrote:
> > Did you mean 2 nodes in 2U?
>
> No, I meant 2 nodes in 1U.
>
> > Dell, HP and SuperMicro all have models that fit the bill. If you really
> did
> > mean 2 nodes in 1U you're looking at either 2.5" drives or < 4 spindles
> per
> > node, neither of which is ideal for Hadoop/HBase in terms of !/$ (bang
> per
> > buck).
>
> Yeah those particular models have room for 4 x 3.5" disks (so 2 per
> node). Given how much they cost, they still give you a pretty good
> !/$. Plus, you can always mod them a little bit to fit more disks in
> the system (this, however, is left as an exercise for the reader).
>
> There are other vendors who sell cheap 2-in-1U servers, some may offer
> more 3.5" disk slots out of the box. Otherwise you can always do like
> Google and build your own. You just need a bunch of dudes who are
> passionate about that stuff :)
>
> --
> Benoit "tsuna" Sigoure
> Software Engineer @ www.StumbleUpon.com
>
Re: Where do you get your hardware?
Posted by tsuna <ts...@gmail.com>.
On Fri, Nov 5, 2010 at 8:59 PM, Patrick Angeles <pa...@cloudera.com> wrote:
> Did you mean 2 nodes in 2U?
No, I meant 2 nodes in 1U.
> Dell, HP and SuperMicro all have models that fit the bill. If you really did
> mean 2 nodes in 1U you're looking at either 2.5" drives or < 4 spindles per
> node, neither of which is ideal for Hadoop/HBase in terms of !/$ (bang per
> buck).
Yeah those particular models have room for 4 x 3.5" disks (so 2 per
node). Given how much they cost, they still give you a pretty good
!/$. Plus, you can always mod them a little bit to fit more disks in
the system (this, however, is left as an exercise for the reader).
There are other vendors who sell cheap 2-in-1U servers, some may offer
more 3.5" disk slots out of the box. Otherwise you can always do like
Google and build your own. You just need a bunch of dudes who are
passionate about that stuff :)
--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com
Re: Where do you get your hardware?
Posted by Patrick Angeles <pa...@cloudera.com>.
Did you mean 2 nodes in 2U?
Dell, HP and SuperMicro all have models that fit the bill. If you really did
mean 2 nodes in 1U you're looking at either 2.5" drives or < 4 spindles per
node, neither of which is ideal for Hadoop/HBase in terms of !/$ (bang per
buck).
On Fri, Nov 5, 2010 at 9:24 PM, Jason Lotz <ja...@gmail.com> wrote:
> Thanks for the continued input on this. Very interesting replies.
>
> I'm particularly interested in the 2 nodes in 1U servers. We'll look into
> it.
>
> Jason
>
> On Fri, Nov 5, 2010 at 6:46 PM, Jonathan Gray <jg...@facebook.com> wrote:
>
> > > Just avoid the dell hard drives, they are a super-rip off. Which btw
> > > means you'll have to avoid dells, because the _only_ way to get the
> > > dell disk trays which are required is to buy dell hard drives (3-4x
> > > markup btw).
> >
> > +1 on crazy markup. But there actually are some online retailers out
> there
> > that sell the trays. I think you can find them on ebay as well.
> >
> > I was forced to do this at one point because Dell wasn't stocking the
> > latest Intel SSDs about a year ago.
> >
>
Re: Where do you get your hardware?
Posted by Jason Lotz <ja...@gmail.com>.
Thanks for the continued input on this. Very interesting replies.
I'm particularly interested in the 2 nodes in 1U servers. We'll look into
it.
Jason
On Fri, Nov 5, 2010 at 6:46 PM, Jonathan Gray <jg...@facebook.com> wrote:
> > Just avoid the dell hard drives, they are a super-rip off. Which btw
> > means you'll have to avoid dells, because the _only_ way to get the
> > dell disk trays which are required is to buy dell hard drives (3-4x
> > markup btw).
>
> +1 on crazy markup. But there actually are some online retailers out there
> that sell the trays. I think you can find them on ebay as well.
>
> I was forced to do this at one point because Dell wasn't stocking the
> latest Intel SSDs about a year ago.
>
RE: Where do you get your hardware?
Posted by Jonathan Gray <jg...@facebook.com>.
> Just avoid the dell hard drives, they are a super-rip off. Which btw
> means you'll have to avoid dells, because the _only_ way to get the
> dell disk trays which are required is to buy dell hard drives (3-4x
> markup btw).
+1 on crazy markup. But there actually are some online retailers out there that sell the trays. I think you can find them on ebay as well.
I was forced to do this at one point because Dell wasn't stocking the latest Intel SSDs about a year ago.
Re: Where do you get your hardware?
Posted by Ryan Rawson <ry...@gmail.com>.
Most of the people building large clusters have good relationships
with dell and get a significant discount off the Dell list. This might
be because a reasonable cluster is $100k and on up.
Supermicro seems popular, they had a better choice of options about 18
months ago, but maybe dell has formed up behind them.
Just avoid the dell hard drives, they are a super-rip off. Which btw
means you'll have to avoid dells, because the _only_ way to get the
dell disk trays which are required is to buy dell hard drives (3-4x
markup btw).
-ryan
On Wed, Nov 3, 2010 at 2:36 PM, Jack Levin <ma...@gmail.com> wrote:
> We are doing it with system integrator called Racklogic in San Jose.
> We tell them what to build and they do it per our intructions. We are
> running 3 datacenters with 500 servers however, and 20 Gbps of traffic
> to the world... so, a lot of our stuff is custom made.
>
> -Jack
>
> On Wed, Nov 3, 2010 at 11:45 AM, Jason Lotz <ja...@gmail.com> wrote:
>> Thanks for the replies. My take away is that most organizations are buying
>> from vendors (Dell, HP, SuperMicro, HP, etc.) While "build it yourself" is
>> an approach, I'm not hearing a lot of companies that are doing it.
>>
>> Thanks again,
>> Jason
>>
>> On Wed, Nov 3, 2010 at 10:04 AM, Michael Segel <mi...@hotmail.com>wrote:
>>
>>>
>>> Well I usually go to Home Depot, even though there's an ACE a block away...
>>> :-)
>>> (Just kidding)
>>>
>>> If you're keen on Dell, I don't know if they are still making R410s.
>>>
>>> They're 1U so you can put in 4 Hot Swap drives giving you roughly 7TB per
>>> node.
>>> They have multiple 1GBe ports so you can bond them if you need to. Assuming
>>> you're using 'standard' SATA drives, then
>>> you will max out your drive i/o before you max out your networking
>>> bandwidth so if you do port bonding, you'll have enough head room.
>>>
>>> For your ToR switch, I'd recommend the new switch by Blade.
>>>
>>> http://www.bladenetwork.net/?pi_ad_id=6155346275&gclid=CISh1Y7khKUCFYPV5wod_X3kQA
>>> Note: IBM bought them out so prices may vary...
>>>
>>> They announced a new ToR Switch that had 42 (I think) 1 GBe ports w 4 10GBe
>>> uplink ports.
>>> Definitely something to consider because if you try to 'trunk' your switch
>>> over a 1GBe port you'll see the bottleneck between racks hit you hard.
>>>
>>>
>>> If you've got the budget you could go with 10GBe on the motherboard... or
>>> go with SolarFlare's nic cards:
>>> http://www.solarflare.com/index.php
>>>
>>> They have a sweet card that has 2 nic ports (SPIF) each capable of 10GBe
>>> bidirectional (so the card handles 40GBe).
>>> Definitely a good option if you're doing more things in memory, or you have
>>> 8 drive or more per node.
>>> Also gives you a future on your hardware.
>>> Note: 10GBe isn't 'cheap' and most people don't need it.
>>> 10K switch, 1K per nic card is a good budget price...
>>>
>>> If you want to get away from Dell, you can look at other hardware
>>> providers, or you could build your own white boxes for less money, provided
>>> you have people who know how to build, install and support your hardware/OS.
>>> Most corporations don't do this because its easier to pick up the phone and
>>> order a box already built and you get support.
>>>
>>> You may consider a hybrid approach. Go w Dell/IBM/Oracle/HP (weird saying
>>> Oracle and not Sun) for your 'master nodes' [NN,SN,ZKs] where you have
>>> raided drives (smaller) and more memory.
>>> Go white box for your DN (RS) where if you lose a box, you just bring up a
>>> new one in its place and re-balance.
>>>
>>> HTH
>>>
>>> -Mike
>>>
>>>
>>> > From: jasonlotz@gmail.com
>>> > Date: Wed, 3 Nov 2010 09:21:03 -0400
>>> > Subject: Where do you get your hardware?
>>> > To: user@hbase.apache.org
>>> >
>>> > We are in the process of analyzing our options for the future purchases
>>> of
>>> > our Hadoop/HBase DN/RS servers. Currently, we purchase Dell PowerEdge
>>> > R710's which work well for us. However, we know that there are other
>>> > options that may give us more bang for our buck.
>>> >
>>> > I'm not as interested in knowing the specs of the machines that people
>>> are
>>> > using. Rather, I'm curious to know where you buy them from or if you are
>>> > building them yourselves.
>>> >
>>> > Any feedback on how you acquire server hardware in your environment would
>>> be
>>> > greatly appreciated.
>>> >
>>> > Jason
>>>
>>>
>>
>
Re: Where do you get your hardware?
Posted by Jack Levin <ma...@gmail.com>.
We are doing it with system integrator called Racklogic in San Jose.
We tell them what to build and they do it per our intructions. We are
running 3 datacenters with 500 servers however, and 20 Gbps of traffic
to the world... so, a lot of our stuff is custom made.
-Jack
On Wed, Nov 3, 2010 at 11:45 AM, Jason Lotz <ja...@gmail.com> wrote:
> Thanks for the replies. My take away is that most organizations are buying
> from vendors (Dell, HP, SuperMicro, HP, etc.) While "build it yourself" is
> an approach, I'm not hearing a lot of companies that are doing it.
>
> Thanks again,
> Jason
>
> On Wed, Nov 3, 2010 at 10:04 AM, Michael Segel <mi...@hotmail.com>wrote:
>
>>
>> Well I usually go to Home Depot, even though there's an ACE a block away...
>> :-)
>> (Just kidding)
>>
>> If you're keen on Dell, I don't know if they are still making R410s.
>>
>> They're 1U so you can put in 4 Hot Swap drives giving you roughly 7TB per
>> node.
>> They have multiple 1GBe ports so you can bond them if you need to. Assuming
>> you're using 'standard' SATA drives, then
>> you will max out your drive i/o before you max out your networking
>> bandwidth so if you do port bonding, you'll have enough head room.
>>
>> For your ToR switch, I'd recommend the new switch by Blade.
>>
>> http://www.bladenetwork.net/?pi_ad_id=6155346275&gclid=CISh1Y7khKUCFYPV5wod_X3kQA
>> Note: IBM bought them out so prices may vary...
>>
>> They announced a new ToR Switch that had 42 (I think) 1 GBe ports w 4 10GBe
>> uplink ports.
>> Definitely something to consider because if you try to 'trunk' your switch
>> over a 1GBe port you'll see the bottleneck between racks hit you hard.
>>
>>
>> If you've got the budget you could go with 10GBe on the motherboard... or
>> go with SolarFlare's nic cards:
>> http://www.solarflare.com/index.php
>>
>> They have a sweet card that has 2 nic ports (SPIF) each capable of 10GBe
>> bidirectional (so the card handles 40GBe).
>> Definitely a good option if you're doing more things in memory, or you have
>> 8 drive or more per node.
>> Also gives you a future on your hardware.
>> Note: 10GBe isn't 'cheap' and most people don't need it.
>> 10K switch, 1K per nic card is a good budget price...
>>
>> If you want to get away from Dell, you can look at other hardware
>> providers, or you could build your own white boxes for less money, provided
>> you have people who know how to build, install and support your hardware/OS.
>> Most corporations don't do this because its easier to pick up the phone and
>> order a box already built and you get support.
>>
>> You may consider a hybrid approach. Go w Dell/IBM/Oracle/HP (weird saying
>> Oracle and not Sun) for your 'master nodes' [NN,SN,ZKs] where you have
>> raided drives (smaller) and more memory.
>> Go white box for your DN (RS) where if you lose a box, you just bring up a
>> new one in its place and re-balance.
>>
>> HTH
>>
>> -Mike
>>
>>
>> > From: jasonlotz@gmail.com
>> > Date: Wed, 3 Nov 2010 09:21:03 -0400
>> > Subject: Where do you get your hardware?
>> > To: user@hbase.apache.org
>> >
>> > We are in the process of analyzing our options for the future purchases
>> of
>> > our Hadoop/HBase DN/RS servers. Currently, we purchase Dell PowerEdge
>> > R710's which work well for us. However, we know that there are other
>> > options that may give us more bang for our buck.
>> >
>> > I'm not as interested in knowing the specs of the machines that people
>> are
>> > using. Rather, I'm curious to know where you buy them from or if you are
>> > building them yourselves.
>> >
>> > Any feedback on how you acquire server hardware in your environment would
>> be
>> > greatly appreciated.
>> >
>> > Jason
>>
>>
>
Re: Where do you get your hardware?
Posted by Jason Lotz <ja...@gmail.com>.
Thanks for the replies. My take away is that most organizations are buying
from vendors (Dell, HP, SuperMicro, HP, etc.) While "build it yourself" is
an approach, I'm not hearing a lot of companies that are doing it.
Thanks again,
Jason
On Wed, Nov 3, 2010 at 10:04 AM, Michael Segel <mi...@hotmail.com>wrote:
>
> Well I usually go to Home Depot, even though there's an ACE a block away...
> :-)
> (Just kidding)
>
> If you're keen on Dell, I don't know if they are still making R410s.
>
> They're 1U so you can put in 4 Hot Swap drives giving you roughly 7TB per
> node.
> They have multiple 1GBe ports so you can bond them if you need to. Assuming
> you're using 'standard' SATA drives, then
> you will max out your drive i/o before you max out your networking
> bandwidth so if you do port bonding, you'll have enough head room.
>
> For your ToR switch, I'd recommend the new switch by Blade.
>
> http://www.bladenetwork.net/?pi_ad_id=6155346275&gclid=CISh1Y7khKUCFYPV5wod_X3kQA
> Note: IBM bought them out so prices may vary...
>
> They announced a new ToR Switch that had 42 (I think) 1 GBe ports w 4 10GBe
> uplink ports.
> Definitely something to consider because if you try to 'trunk' your switch
> over a 1GBe port you'll see the bottleneck between racks hit you hard.
>
>
> If you've got the budget you could go with 10GBe on the motherboard... or
> go with SolarFlare's nic cards:
> http://www.solarflare.com/index.php
>
> They have a sweet card that has 2 nic ports (SPIF) each capable of 10GBe
> bidirectional (so the card handles 40GBe).
> Definitely a good option if you're doing more things in memory, or you have
> 8 drive or more per node.
> Also gives you a future on your hardware.
> Note: 10GBe isn't 'cheap' and most people don't need it.
> 10K switch, 1K per nic card is a good budget price...
>
> If you want to get away from Dell, you can look at other hardware
> providers, or you could build your own white boxes for less money, provided
> you have people who know how to build, install and support your hardware/OS.
> Most corporations don't do this because its easier to pick up the phone and
> order a box already built and you get support.
>
> You may consider a hybrid approach. Go w Dell/IBM/Oracle/HP (weird saying
> Oracle and not Sun) for your 'master nodes' [NN,SN,ZKs] where you have
> raided drives (smaller) and more memory.
> Go white box for your DN (RS) where if you lose a box, you just bring up a
> new one in its place and re-balance.
>
> HTH
>
> -Mike
>
>
> > From: jasonlotz@gmail.com
> > Date: Wed, 3 Nov 2010 09:21:03 -0400
> > Subject: Where do you get your hardware?
> > To: user@hbase.apache.org
> >
> > We are in the process of analyzing our options for the future purchases
> of
> > our Hadoop/HBase DN/RS servers. Currently, we purchase Dell PowerEdge
> > R710's which work well for us. However, we know that there are other
> > options that may give us more bang for our buck.
> >
> > I'm not as interested in knowing the specs of the machines that people
> are
> > using. Rather, I'm curious to know where you buy them from or if you are
> > building them yourselves.
> >
> > Any feedback on how you acquire server hardware in your environment would
> be
> > greatly appreciated.
> >
> > Jason
>
>
RE: Where do you get your hardware?
Posted by Michael Segel <mi...@hotmail.com>.
Well I usually go to Home Depot, even though there's an ACE a block away... :-)
(Just kidding)
If you're keen on Dell, I don't know if they are still making R410s.
They're 1U so you can put in 4 Hot Swap drives giving you roughly 7TB per node.
They have multiple 1GBe ports so you can bond them if you need to. Assuming you're using 'standard' SATA drives, then
you will max out your drive i/o before you max out your networking bandwidth so if you do port bonding, you'll have enough head room.
For your ToR switch, I'd recommend the new switch by Blade.
http://www.bladenetwork.net/?pi_ad_id=6155346275&gclid=CISh1Y7khKUCFYPV5wod_X3kQA
Note: IBM bought them out so prices may vary...
They announced a new ToR Switch that had 42 (I think) 1 GBe ports w 4 10GBe uplink ports.
Definitely something to consider because if you try to 'trunk' your switch over a 1GBe port you'll see the bottleneck between racks hit you hard.
If you've got the budget you could go with 10GBe on the motherboard... or go with SolarFlare's nic cards:
http://www.solarflare.com/index.php
They have a sweet card that has 2 nic ports (SPIF) each capable of 10GBe bidirectional (so the card handles 40GBe).
Definitely a good option if you're doing more things in memory, or you have 8 drive or more per node.
Also gives you a future on your hardware.
Note: 10GBe isn't 'cheap' and most people don't need it.
10K switch, 1K per nic card is a good budget price...
If you want to get away from Dell, you can look at other hardware providers, or you could build your own white boxes for less money, provided you have people who know how to build, install and support your hardware/OS. Most corporations don't do this because its easier to pick up the phone and order a box already built and you get support.
You may consider a hybrid approach. Go w Dell/IBM/Oracle/HP (weird saying Oracle and not Sun) for your 'master nodes' [NN,SN,ZKs] where you have raided drives (smaller) and more memory.
Go white box for your DN (RS) where if you lose a box, you just bring up a new one in its place and re-balance.
HTH
-Mike
> From: jasonlotz@gmail.com
> Date: Wed, 3 Nov 2010 09:21:03 -0400
> Subject: Where do you get your hardware?
> To: user@hbase.apache.org
>
> We are in the process of analyzing our options for the future purchases of
> our Hadoop/HBase DN/RS servers. Currently, we purchase Dell PowerEdge
> R710's which work well for us. However, we know that there are other
> options that may give us more bang for our buck.
>
> I'm not as interested in knowing the specs of the machines that people are
> using. Rather, I'm curious to know where you buy them from or if you are
> building them yourselves.
>
> Any feedback on how you acquire server hardware in your environment would be
> greatly appreciated.
>
> Jason
Re: Where do you get your hardware?
Posted by Tim Robertson <ti...@gmail.com>.
We just set up a cluster with Dells, and have a pretty fine
relationship with a local Dell supplier.
Tim
On Wed, Nov 3, 2010 at 2:21 PM, Jason Lotz <ja...@gmail.com> wrote:
> We are in the process of analyzing our options for the future purchases of
> our Hadoop/HBase DN/RS servers. Currently, we purchase Dell PowerEdge
> R710's which work well for us. However, we know that there are other
> options that may give us more bang for our buck.
>
> I'm not as interested in knowing the specs of the machines that people are
> using. Rather, I'm curious to know where you buy them from or if you are
> building them yourselves.
>
> Any feedback on how you acquire server hardware in your environment would be
> greatly appreciated.
>
> Jason
>
Re: Where do you get your hardware?
Posted by Jeremy Carroll <je...@networkedinsights.com>.
We have used the Dell Cloud Servers (C2100 to be exact). Turn around time
is a little slower, but it's worth it IMHO.
On 11/3/10 8:36 AM, "Patrick Angeles" <pa...@cloudera.com> wrote:
>Jason,
>
>Unless you're operating at Google scale, it doesn't make economic sense to
>build your own unless you're *really into that*. Most major vendors (HP,
>Dell, SuperMicro) will offer a configuration that is very suitable for
>Hadoop.
>
>Regards,
>
>- P
>
>
>On Wed, Nov 3, 2010 at 9:21 AM, Jason Lotz <ja...@gmail.com> wrote:
>
>> We are in the process of analyzing our options for the future purchases
>>of
>> our Hadoop/HBase DN/RS servers. Currently, we purchase Dell PowerEdge
>> R710's which work well for us. However, we know that there are other
>> options that may give us more bang for our buck.
>>
>> I'm not as interested in knowing the specs of the machines that people
>>are
>> using. Rather, I'm curious to know where you buy them from or if you
>>are
>> building them yourselves.
>>
>> Any feedback on how you acquire server hardware in your environment
>>would
>> be
>> greatly appreciated.
>>
>> Jason
>>
Re: Where do you get your hardware?
Posted by Patrick Angeles <pa...@cloudera.com>.
Jason,
Unless you're operating at Google scale, it doesn't make economic sense to
build your own unless you're *really into that*. Most major vendors (HP,
Dell, SuperMicro) will offer a configuration that is very suitable for
Hadoop.
Regards,
- P
On Wed, Nov 3, 2010 at 9:21 AM, Jason Lotz <ja...@gmail.com> wrote:
> We are in the process of analyzing our options for the future purchases of
> our Hadoop/HBase DN/RS servers. Currently, we purchase Dell PowerEdge
> R710's which work well for us. However, we know that there are other
> options that may give us more bang for our buck.
>
> I'm not as interested in knowing the specs of the machines that people are
> using. Rather, I'm curious to know where you buy them from or if you are
> building them yourselves.
>
> Any feedback on how you acquire server hardware in your environment would
> be
> greatly appreciated.
>
> Jason
>