You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Steve Baker <ba...@gmail.com> on 2015/12/27 16:39:28 UTC
Hadoop reference architecture
Dear All
Apology if this is not the right mailing list to ask this question or if
the request is too simple.
We are in process of designing a Hadoop cluster and we wonder if you could
share or direct us to a reference architecture design document or something
like that (Physical design). We have looked through Apache Hadoop document
but we were unable to find the required info. However, there are some
vendors specific documents on the web but we are after general document or
best practice to design a Hadoop cluster.
Many Thanks
Steve
Re: Hadoop reference architecture
Posted by Gautam Gopalakrishnan <gg...@cloudera.com>.
You could try these books for a start
Hadoop Application Architectures
http://shop.oreilly.com/product/0636920033196.do
Hadoop Operations
http://shop.oreilly.com/product/0636920025085.do
On Mon, Dec 28, 2015 at 2:12 PM, daemeon reiydelle <da...@gmail.com>
wrote:
> "Best" practices are either so generic as to be mostly useless, or
> dependent on very specific business processes, SLA's, and OLA's relevant to
> different data models, etc.
>
> If you can identify specific business use cases?
>
> Even basic attributes change the model: Containers on CoreOS? White boxes?
> SSD? Mixed HDD/SSD? AWS? OpenStack? VMWare?
>
> High volume ingest with long retentions? Fine grain, well structured data
> for which data mining is desired or log based analytics?
>
>
>
> *.......*
>
>
>
>
>
>
> *“Life should not be a journey to the grave with the intention of arriving
> safely in apretty and well preserved body, but rather to skid in broadside
> in a cloud of smoke,thoroughly used up, totally worn out, and loudly
> proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
> (+1) 415.501.0198 <%28%2B1%29%20415.501.0198>London (+44) (0) 20 8144 9872
> <%28%2B44%29%20%280%29%2020%208144%209872>*
>
> On Sun, Dec 27, 2015 at 7:39 AM, Steve Baker <ba...@gmail.com>
> wrote:
>
>> Dear All
>>
>> Apology if this is not the right mailing list to ask this question or if
>> the request is too simple.
>>
>> We are in process of designing a Hadoop cluster and we wonder if you
>> could share or direct us to a reference architecture design document or
>> something like that (Physical design). We have looked through Apache
>> Hadoop document but we were unable to find the required info. However,
>> there are some vendors specific documents on the web but we are after
>> general document or best practice to design a Hadoop cluster.
>>
>> Many Thanks
>>
>> Steve
>>
>
>
Re: Hadoop reference architecture
Posted by Gautam Gopalakrishnan <gg...@cloudera.com>.
You could try these books for a start
Hadoop Application Architectures
http://shop.oreilly.com/product/0636920033196.do
Hadoop Operations
http://shop.oreilly.com/product/0636920025085.do
On Mon, Dec 28, 2015 at 2:12 PM, daemeon reiydelle <da...@gmail.com>
wrote:
> "Best" practices are either so generic as to be mostly useless, or
> dependent on very specific business processes, SLA's, and OLA's relevant to
> different data models, etc.
>
> If you can identify specific business use cases?
>
> Even basic attributes change the model: Containers on CoreOS? White boxes?
> SSD? Mixed HDD/SSD? AWS? OpenStack? VMWare?
>
> High volume ingest with long retentions? Fine grain, well structured data
> for which data mining is desired or log based analytics?
>
>
>
> *.......*
>
>
>
>
>
>
> *“Life should not be a journey to the grave with the intention of arriving
> safely in apretty and well preserved body, but rather to skid in broadside
> in a cloud of smoke,thoroughly used up, totally worn out, and loudly
> proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
> (+1) 415.501.0198 <%28%2B1%29%20415.501.0198>London (+44) (0) 20 8144 9872
> <%28%2B44%29%20%280%29%2020%208144%209872>*
>
> On Sun, Dec 27, 2015 at 7:39 AM, Steve Baker <ba...@gmail.com>
> wrote:
>
>> Dear All
>>
>> Apology if this is not the right mailing list to ask this question or if
>> the request is too simple.
>>
>> We are in process of designing a Hadoop cluster and we wonder if you
>> could share or direct us to a reference architecture design document or
>> something like that (Physical design). We have looked through Apache
>> Hadoop document but we were unable to find the required info. However,
>> there are some vendors specific documents on the web but we are after
>> general document or best practice to design a Hadoop cluster.
>>
>> Many Thanks
>>
>> Steve
>>
>
>
Re: Hadoop reference architecture
Posted by Gautam Gopalakrishnan <gg...@cloudera.com>.
You could try these books for a start
Hadoop Application Architectures
http://shop.oreilly.com/product/0636920033196.do
Hadoop Operations
http://shop.oreilly.com/product/0636920025085.do
On Mon, Dec 28, 2015 at 2:12 PM, daemeon reiydelle <da...@gmail.com>
wrote:
> "Best" practices are either so generic as to be mostly useless, or
> dependent on very specific business processes, SLA's, and OLA's relevant to
> different data models, etc.
>
> If you can identify specific business use cases?
>
> Even basic attributes change the model: Containers on CoreOS? White boxes?
> SSD? Mixed HDD/SSD? AWS? OpenStack? VMWare?
>
> High volume ingest with long retentions? Fine grain, well structured data
> for which data mining is desired or log based analytics?
>
>
>
> *.......*
>
>
>
>
>
>
> *“Life should not be a journey to the grave with the intention of arriving
> safely in apretty and well preserved body, but rather to skid in broadside
> in a cloud of smoke,thoroughly used up, totally worn out, and loudly
> proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
> (+1) 415.501.0198 <%28%2B1%29%20415.501.0198>London (+44) (0) 20 8144 9872
> <%28%2B44%29%20%280%29%2020%208144%209872>*
>
> On Sun, Dec 27, 2015 at 7:39 AM, Steve Baker <ba...@gmail.com>
> wrote:
>
>> Dear All
>>
>> Apology if this is not the right mailing list to ask this question or if
>> the request is too simple.
>>
>> We are in process of designing a Hadoop cluster and we wonder if you
>> could share or direct us to a reference architecture design document or
>> something like that (Physical design). We have looked through Apache
>> Hadoop document but we were unable to find the required info. However,
>> there are some vendors specific documents on the web but we are after
>> general document or best practice to design a Hadoop cluster.
>>
>> Many Thanks
>>
>> Steve
>>
>
>
Re: Hadoop reference architecture
Posted by Gautam Gopalakrishnan <gg...@cloudera.com>.
You could try these books for a start
Hadoop Application Architectures
http://shop.oreilly.com/product/0636920033196.do
Hadoop Operations
http://shop.oreilly.com/product/0636920025085.do
On Mon, Dec 28, 2015 at 2:12 PM, daemeon reiydelle <da...@gmail.com>
wrote:
> "Best" practices are either so generic as to be mostly useless, or
> dependent on very specific business processes, SLA's, and OLA's relevant to
> different data models, etc.
>
> If you can identify specific business use cases?
>
> Even basic attributes change the model: Containers on CoreOS? White boxes?
> SSD? Mixed HDD/SSD? AWS? OpenStack? VMWare?
>
> High volume ingest with long retentions? Fine grain, well structured data
> for which data mining is desired or log based analytics?
>
>
>
> *.......*
>
>
>
>
>
>
> *“Life should not be a journey to the grave with the intention of arriving
> safely in apretty and well preserved body, but rather to skid in broadside
> in a cloud of smoke,thoroughly used up, totally worn out, and loudly
> proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
> (+1) 415.501.0198 <%28%2B1%29%20415.501.0198>London (+44) (0) 20 8144 9872
> <%28%2B44%29%20%280%29%2020%208144%209872>*
>
> On Sun, Dec 27, 2015 at 7:39 AM, Steve Baker <ba...@gmail.com>
> wrote:
>
>> Dear All
>>
>> Apology if this is not the right mailing list to ask this question or if
>> the request is too simple.
>>
>> We are in process of designing a Hadoop cluster and we wonder if you
>> could share or direct us to a reference architecture design document or
>> something like that (Physical design). We have looked through Apache
>> Hadoop document but we were unable to find the required info. However,
>> there are some vendors specific documents on the web but we are after
>> general document or best practice to design a Hadoop cluster.
>>
>> Many Thanks
>>
>> Steve
>>
>
>
Re: Hadoop reference architecture
Posted by daemeon reiydelle <da...@gmail.com>.
"Best" practices are either so generic as to be mostly useless, or
dependent on very specific business processes, SLA's, and OLA's relevant to
different data models, etc.
If you can identify specific business use cases?
Even basic attributes change the model: Containers on CoreOS? White boxes?
SSD? Mixed HDD/SSD? AWS? OpenStack? VMWare?
High volume ingest with long retentions? Fine grain, well structured data
for which data mining is desired or log based analytics?
*.......*
*“Life should not be a journey to the grave with the intention of arriving
safely in apretty and well preserved body, but rather to skid in broadside
in a cloud of smoke,thoroughly used up, totally worn out, and loudly
proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
(+1) 415.501.0198London (+44) (0) 20 8144 9872*
On Sun, Dec 27, 2015 at 7:39 AM, Steve Baker <ba...@gmail.com>
wrote:
> Dear All
>
> Apology if this is not the right mailing list to ask this question or if
> the request is too simple.
>
> We are in process of designing a Hadoop cluster and we wonder if you could
> share or direct us to a reference architecture design document or something
> like that (Physical design). We have looked through Apache Hadoop document
> but we were unable to find the required info. However, there are some
> vendors specific documents on the web but we are after general document or
> best practice to design a Hadoop cluster.
>
> Many Thanks
>
> Steve
>
Re: Hadoop reference architecture
Posted by daemeon reiydelle <da...@gmail.com>.
"Best" practices are either so generic as to be mostly useless, or
dependent on very specific business processes, SLA's, and OLA's relevant to
different data models, etc.
If you can identify specific business use cases?
Even basic attributes change the model: Containers on CoreOS? White boxes?
SSD? Mixed HDD/SSD? AWS? OpenStack? VMWare?
High volume ingest with long retentions? Fine grain, well structured data
for which data mining is desired or log based analytics?
*.......*
*“Life should not be a journey to the grave with the intention of arriving
safely in apretty and well preserved body, but rather to skid in broadside
in a cloud of smoke,thoroughly used up, totally worn out, and loudly
proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
(+1) 415.501.0198London (+44) (0) 20 8144 9872*
On Sun, Dec 27, 2015 at 7:39 AM, Steve Baker <ba...@gmail.com>
wrote:
> Dear All
>
> Apology if this is not the right mailing list to ask this question or if
> the request is too simple.
>
> We are in process of designing a Hadoop cluster and we wonder if you could
> share or direct us to a reference architecture design document or something
> like that (Physical design). We have looked through Apache Hadoop document
> but we were unable to find the required info. However, there are some
> vendors specific documents on the web but we are after general document or
> best practice to design a Hadoop cluster.
>
> Many Thanks
>
> Steve
>
Re: Hadoop reference architecture
Posted by daemeon reiydelle <da...@gmail.com>.
"Best" practices are either so generic as to be mostly useless, or
dependent on very specific business processes, SLA's, and OLA's relevant to
different data models, etc.
If you can identify specific business use cases?
Even basic attributes change the model: Containers on CoreOS? White boxes?
SSD? Mixed HDD/SSD? AWS? OpenStack? VMWare?
High volume ingest with long retentions? Fine grain, well structured data
for which data mining is desired or log based analytics?
*.......*
*“Life should not be a journey to the grave with the intention of arriving
safely in apretty and well preserved body, but rather to skid in broadside
in a cloud of smoke,thoroughly used up, totally worn out, and loudly
proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
(+1) 415.501.0198London (+44) (0) 20 8144 9872*
On Sun, Dec 27, 2015 at 7:39 AM, Steve Baker <ba...@gmail.com>
wrote:
> Dear All
>
> Apology if this is not the right mailing list to ask this question or if
> the request is too simple.
>
> We are in process of designing a Hadoop cluster and we wonder if you could
> share or direct us to a reference architecture design document or something
> like that (Physical design). We have looked through Apache Hadoop document
> but we were unable to find the required info. However, there are some
> vendors specific documents on the web but we are after general document or
> best practice to design a Hadoop cluster.
>
> Many Thanks
>
> Steve
>
Re: Hadoop reference architecture
Posted by daemeon reiydelle <da...@gmail.com>.
"Best" practices are either so generic as to be mostly useless, or
dependent on very specific business processes, SLA's, and OLA's relevant to
different data models, etc.
If you can identify specific business use cases?
Even basic attributes change the model: Containers on CoreOS? White boxes?
SSD? Mixed HDD/SSD? AWS? OpenStack? VMWare?
High volume ingest with long retentions? Fine grain, well structured data
for which data mining is desired or log based analytics?
*.......*
*“Life should not be a journey to the grave with the intention of arriving
safely in apretty and well preserved body, but rather to skid in broadside
in a cloud of smoke,thoroughly used up, totally worn out, and loudly
proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
(+1) 415.501.0198London (+44) (0) 20 8144 9872*
On Sun, Dec 27, 2015 at 7:39 AM, Steve Baker <ba...@gmail.com>
wrote:
> Dear All
>
> Apology if this is not the right mailing list to ask this question or if
> the request is too simple.
>
> We are in process of designing a Hadoop cluster and we wonder if you could
> share or direct us to a reference architecture design document or something
> like that (Physical design). We have looked through Apache Hadoop document
> but we were unable to find the required info. However, there are some
> vendors specific documents on the web but we are after general document or
> best practice to design a Hadoop cluster.
>
> Many Thanks
>
> Steve
>