You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by David Medinets <da...@gmail.com> on 2012/07/07 17:13:31 UTC
Using a really small computer for accumulo experimentation.
http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
reason why Hadoop & Accumulo would not work on this kind of computer.
Any opinions? Buying a few desktops for my home use will make my
office too hot to use.
Re: Using a really small computer for accumulo experimentation.
Posted by William Slacum <wi...@accumulo.net>.
Do it!
On Sat, Jul 7, 2012 at 3:47 PM, Mattmann, Chris A (388J) <
chris.a.mattmann@jpl.nasa.gov> wrote:
> Reminded me of Raspberry Pi:
> http://downloads.element14.com/raspberryPi2.html?SKU=83T1943
>
> Accumulo + Hadoop + Raspberry Pi! :)
>
> Cheers,
> Chris
>
> On Jul 7, 2012, at 8:13 AM, David Medinets wrote:
>
> > http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
> > reason why Hadoop & Accumulo would not work on this kind of computer.
> > Any opinions? Buying a few desktops for my home use will make my
> > office too hot to use.
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
Re: Using a really small computer for accumulo experimentation.
Posted by Ed Kohlwey <ek...@gmail.com>.
A raspberry pi cluster is an interesting idea. Other people have toyed with
the idea of lots of small, low wattage hosts attached to relatively slow,
cheap storage and making up the difference in parallelism.
http://www.cs.cmu.edu/~fawnproj/
On Mon, Jul 9, 2012 at 9:53 AM, Cardon, Tejay E <te...@lmco.com>wrote:
> Chris,
> Have you actually tried a Raspberry Pi Hadoop cluster? I've been
> considering it for a while, but I'd love to know the results if anyone has
> actually done it.
>
> Tejay
>
> -----Original Message-----
> From: Mattmann, Chris A (388J) [mailto:chris.a.mattmann@jpl.nasa.gov]
> Sent: Saturday, July 07, 2012 1:48 PM
> To: <de...@accumulo.apache.org>
> Subject: EXTERNAL: Re: Using a really small computer for accumulo
> experimentation.
>
> Reminded me of Raspberry Pi:
> http://downloads.element14.com/raspberryPi2.html?SKU=83T1943
>
> Accumulo + Hadoop + Raspberry Pi! :)
>
> Cheers,
> Chris
>
> On Jul 7, 2012, at 8:13 AM, David Medinets wrote:
>
> > http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
> > reason why Hadoop & Accumulo would not work on this kind of computer.
> > Any opinions? Buying a few desktops for my home use will make my
> > office too hot to use.
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department University of
> Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
Re: Using a really small computer for accumulo experimentation.
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hi Tejay,
Thanks, no I haven't tried, but would love to. I missed the first Raspberry Pi window
but will be checking each day when they open sales back up in July.
Thanks!
Cheers,
Chris
On Jul 9, 2012, at 6:53 AM, Cardon, Tejay E wrote:
> Chris,
> Have you actually tried a Raspberry Pi Hadoop cluster? I've been considering it for a while, but I'd love to know the results if anyone has actually done it.
>
> Tejay
>
> -----Original Message-----
> From: Mattmann, Chris A (388J) [mailto:chris.a.mattmann@jpl.nasa.gov]
> Sent: Saturday, July 07, 2012 1:48 PM
> To: <de...@accumulo.apache.org>
> Subject: EXTERNAL: Re: Using a really small computer for accumulo experimentation.
>
> Reminded me of Raspberry Pi: http://downloads.element14.com/raspberryPi2.html?SKU=83T1943
>
> Accumulo + Hadoop + Raspberry Pi! :)
>
> Cheers,
> Chris
>
> On Jul 7, 2012, at 8:13 AM, David Medinets wrote:
>
>> http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
>> reason why Hadoop & Accumulo would not work on this kind of computer.
>> Any opinions? Buying a few desktops for my home use will make my
>> office too hot to use.
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
RE: Using a really small computer for accumulo experimentation.
Posted by "Cardon, Tejay E" <te...@lmco.com>.
Chris,
Have you actually tried a Raspberry Pi Hadoop cluster? I've been considering it for a while, but I'd love to know the results if anyone has actually done it.
Tejay
-----Original Message-----
From: Mattmann, Chris A (388J) [mailto:chris.a.mattmann@jpl.nasa.gov]
Sent: Saturday, July 07, 2012 1:48 PM
To: <de...@accumulo.apache.org>
Subject: EXTERNAL: Re: Using a really small computer for accumulo experimentation.
Reminded me of Raspberry Pi: http://downloads.element14.com/raspberryPi2.html?SKU=83T1943
Accumulo + Hadoop + Raspberry Pi! :)
Cheers,
Chris
On Jul 7, 2012, at 8:13 AM, David Medinets wrote:
> http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
> reason why Hadoop & Accumulo would not work on this kind of computer.
> Any opinions? Buying a few desktops for my home use will make my
> office too hot to use.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Using a really small computer for accumulo experimentation.
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Reminded me of Raspberry Pi: http://downloads.element14.com/raspberryPi2.html?SKU=83T1943
Accumulo + Hadoop + Raspberry Pi! :)
Cheers,
Chris
On Jul 7, 2012, at 8:13 AM, David Medinets wrote:
> http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
> reason why Hadoop & Accumulo would not work on this kind of computer.
> Any opinions? Buying a few desktops for my home use will make my
> office too hot to use.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Using a really small computer for accumulo experimentation.
Posted by David Medinets <da...@gmail.com>.
I took my own advice to buy a Linode 512 system. It took less than 40
minutes to have accumulo v1.5 working on it - start to finish. I
should have done this a long time ago.
Re: Using a really small computer for accumulo experimentation.
Posted by David Medinets <da...@gmail.com>.
Part of the impulse is the novelty. But also, I am concerned about the
price of EC2; how pausing the instances would affect accumulo; or with
continuously loading data.
With something like the Fit PC I can leave them running all the time
in my office. Of course, simply getting a Linode 512 from
http://www.linode.com/ might be better.
On Sat, Jul 7, 2012 at 11:37 AM, Jim Klucar <kl...@gmail.com> wrote:
> Doesn't seem like enough RAM or disk to me. I guess it depends on what
> you were wanting to process though. Ive had a localhost setup of
> Hadoop/ Zookeeper/ Accumulo on my macbook with 4GB of RAM for testing
> but its brutal. Is there a reason to not just go with an EC2 setup on
> amazon?
>
> Sent from my iPhone
>
> On Jul 7, 2012, at 11:14 AM, David Medinets <da...@gmail.com> wrote:
>
>> http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
>> reason why Hadoop & Accumulo would not work on this kind of computer.
>> Any opinions? Buying a few desktops for my home use will make my
>> office too hot to use.
Re: Using a really small computer for accumulo experimentation.
Posted by Ed Kohlwey <ek...@gmail.com>.
You could probably do it, but only small amounts of data. You can cut down
on running so many processes by not running HDFS.
On Jul 7, 2012 11:38 AM, "Jim Klucar" <kl...@gmail.com> wrote:
> Doesn't seem like enough RAM or disk to me. I guess it depends on what
> you were wanting to process though. Ive had a localhost setup of
> Hadoop/ Zookeeper/ Accumulo on my macbook with 4GB of RAM for testing
> but its brutal. Is there a reason to not just go with an EC2 setup on
> amazon?
>
> Sent from my iPhone
>
> On Jul 7, 2012, at 11:14 AM, David Medinets <da...@gmail.com>
> wrote:
>
> > http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
> > reason why Hadoop & Accumulo would not work on this kind of computer.
> > Any opinions? Buying a few desktops for my home use will make my
> > office too hot to use.
>
Re: Using a really small computer for accumulo experimentation.
Posted by John Vines <jo...@ugov.gov>.
Hdfs is required as we don't support running over the local filesystem.
However, you can cut out MapReduce related tasks.
Sent from my phone, so pardon the typos and brevity.
On Jul 8, 2012 8:48 AM, "Ed Kohlwey" <ek...@gmail.com> wrote:
> You could probably do it, but only small amounts of data. You can cut down
> on running so many processes by not running HDFS.
> On Jul 7, 2012 11:38 AM, "Jim Klucar" <kl...@gmail.com> wrote:
>
> > Doesn't seem like enough RAM or disk to me. I guess it depends on what
> > you were wanting to process though. Ive had a localhost setup of
> > Hadoop/ Zookeeper/ Accumulo on my macbook with 4GB of RAM for testing
> > but its brutal. Is there a reason to not just go with an EC2 setup on
> > amazon?
> >
> > Sent from my iPhone
> >
> > On Jul 7, 2012, at 11:14 AM, David Medinets <da...@gmail.com>
> > wrote:
> >
> > > http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
> > > reason why Hadoop & Accumulo would not work on this kind of computer.
> > > Any opinions? Buying a few desktops for my home use will make my
> > > office too hot to use.
> >
>
Re: Using a really small computer for accumulo experimentation.
Posted by Keith Turner <ke...@deenlo.com>.
You can Accumulo on the local filesystem by setting instance.dfs.uri,
but its not well tested. I have tried it once.
On Sun, Jul 8, 2012 at 10:11 AM, John Vines <jv...@gmail.com> wrote:
> Hdfs is necessary for Accumulo, as we currently don't support running of
> the local filesystem. You can cut out MapReduce though.
>
> Sent from my phone, so pardon the typos and brevity.
> On Jul 8, 2012 8:48 AM, "Ed Kohlwey" <ek...@gmail.com> wrote:
>
>> You could probably do it, but only small amounts of data. You can cut down
>> on running so many processes by not running HDFS.
>> On Jul 7, 2012 11:38 AM, "Jim Klucar" <kl...@gmail.com> wrote:
>>
>> > Doesn't seem like enough RAM or disk to me. I guess it depends on what
>> > you were wanting to process though. Ive had a localhost setup of
>> > Hadoop/ Zookeeper/ Accumulo on my macbook with 4GB of RAM for testing
>> > but its brutal. Is there a reason to not just go with an EC2 setup on
>> > amazon?
>> >
>> > Sent from my iPhone
>> >
>> > On Jul 7, 2012, at 11:14 AM, David Medinets <da...@gmail.com>
>> > wrote:
>> >
>> > > http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
>> > > reason why Hadoop & Accumulo would not work on this kind of computer.
>> > > Any opinions? Buying a few desktops for my home use will make my
>> > > office too hot to use.
>> >
>>
Re: Using a really small computer for accumulo experimentation.
Posted by John Vines <jv...@gmail.com>.
Hdfs is necessary for Accumulo, as we currently don't support running of
the local filesystem. You can cut out MapReduce though.
Sent from my phone, so pardon the typos and brevity.
On Jul 8, 2012 8:48 AM, "Ed Kohlwey" <ek...@gmail.com> wrote:
> You could probably do it, but only small amounts of data. You can cut down
> on running so many processes by not running HDFS.
> On Jul 7, 2012 11:38 AM, "Jim Klucar" <kl...@gmail.com> wrote:
>
> > Doesn't seem like enough RAM or disk to me. I guess it depends on what
> > you were wanting to process though. Ive had a localhost setup of
> > Hadoop/ Zookeeper/ Accumulo on my macbook with 4GB of RAM for testing
> > but its brutal. Is there a reason to not just go with an EC2 setup on
> > amazon?
> >
> > Sent from my iPhone
> >
> > On Jul 7, 2012, at 11:14 AM, David Medinets <da...@gmail.com>
> > wrote:
> >
> > > http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
> > > reason why Hadoop & Accumulo would not work on this kind of computer.
> > > Any opinions? Buying a few desktops for my home use will make my
> > > office too hot to use.
> >
>
Re: Using a really small computer for accumulo experimentation.
Posted by Jim Klucar <kl...@gmail.com>.
Doesn't seem like enough RAM or disk to me. I guess it depends on what
you were wanting to process though. Ive had a localhost setup of
Hadoop/ Zookeeper/ Accumulo on my macbook with 4GB of RAM for testing
but its brutal. Is there a reason to not just go with an EC2 setup on
amazon?
Sent from my iPhone
On Jul 7, 2012, at 11:14 AM, David Medinets <da...@gmail.com> wrote:
> http://www.fit-pc.com/web/fit-pc2-models/ - I wonder if there is any
> reason why Hadoop & Accumulo would not work on this kind of computer.
> Any opinions? Buying a few desktops for my home use will make my
> office too hot to use.