You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Saurabh Agarwal <sr...@gmail.com> on 2010/05/12 13:58:26 UTC

HDFS on raw device

Hii

Can HDFS be give path of raw device, say can I set dfs.data.dir to /dev/sda3
or something like that



Thanks and regards
Saurabh Agarwal

Re: HDFS on raw device

Posted by Saurabh Agarwal <sr...@gmail.com>.
Ok Thanks !!

Saurabh Agarwal


On Wed, May 12, 2010 at 11:52 PM, Eric Sammer <es...@cloudera.com> wrote:

> You can, but you'll lose all of the features and functionality such as
> data locality, replication, and so on. Things get tricky if you don't
> use HDFS. It's worth reading some of the docs at
> http://hadoop.apache.org and the map reduce paper prior to deviating
> from the common case. The Hadoop - the Definitive Guide book (Tom
> White, O'Reilly) is an excellent reference as well.
>
> On Wed, May 12, 2010 at 1:34 PM, Saurabh Agarwal <sr...@gmail.com>
> wrote:
> > thanks eric
> > also can i use bypass hdfs as a datastore and instead use ext3
> > Saurabh Agarwal
> >
> >
> > On Wed, May 12, 2010 at 10:40 PM, Eric Sammer <es...@cloudera.com>
> wrote:
> >>
> >> No. HDFS always stores data on a OS file system (e.g. ext3 on Linux).
> >>
> >> On Wed, May 12, 2010 at 7:58 AM, Saurabh Agarwal <sr...@gmail.com>
> >> wrote:
> >> > Hii
> >> > Can HDFS be give path of raw device, say can I set dfs.data.dir to
> >> > /dev/sda3
> >> > or something like that
> >> >
> >> >
> >> > Thanks and regards
> >> > Saurabh Agarwal
> >> >
> >>
> >>
> >>
> >> --
> >> Eric Sammer
> >> phone: +1-917-287-2675
> >> twitter: esammer
> >> data: www.cloudera.com
> >
> >
>
>
>
> --
> Eric Sammer
> phone: +1-917-287-2675
> twitter: esammer
> data: www.cloudera.com
>

Re: HDFS on raw device

Posted by Eric Sammer <es...@cloudera.com>.
You can, but you'll lose all of the features and functionality such as
data locality, replication, and so on. Things get tricky if you don't
use HDFS. It's worth reading some of the docs at
http://hadoop.apache.org and the map reduce paper prior to deviating
from the common case. The Hadoop - the Definitive Guide book (Tom
White, O'Reilly) is an excellent reference as well.

On Wed, May 12, 2010 at 1:34 PM, Saurabh Agarwal <sr...@gmail.com> wrote:
> thanks eric
> also can i use bypass hdfs as a datastore and instead use ext3
> Saurabh Agarwal
>
>
> On Wed, May 12, 2010 at 10:40 PM, Eric Sammer <es...@cloudera.com> wrote:
>>
>> No. HDFS always stores data on a OS file system (e.g. ext3 on Linux).
>>
>> On Wed, May 12, 2010 at 7:58 AM, Saurabh Agarwal <sr...@gmail.com>
>> wrote:
>> > Hii
>> > Can HDFS be give path of raw device, say can I set dfs.data.dir to
>> > /dev/sda3
>> > or something like that
>> >
>> >
>> > Thanks and regards
>> > Saurabh Agarwal
>> >
>>
>>
>>
>> --
>> Eric Sammer
>> phone: +1-917-287-2675
>> twitter: esammer
>> data: www.cloudera.com
>
>



-- 
Eric Sammer
phone: +1-917-287-2675
twitter: esammer
data: www.cloudera.com

Re: HDFS on raw device

Posted by Saurabh Agarwal <sr...@gmail.com>.
thanks eric

also can i use bypass hdfs as a datastore and instead use ext3
Saurabh Agarwal


On Wed, May 12, 2010 at 10:40 PM, Eric Sammer <es...@cloudera.com> wrote:

> No. HDFS always stores data on a OS file system (e.g. ext3 on Linux).
>
> On Wed, May 12, 2010 at 7:58 AM, Saurabh Agarwal <sr...@gmail.com>
> wrote:
> > Hii
> > Can HDFS be give path of raw device, say can I set dfs.data.dir to
> /dev/sda3
> > or something like that
> >
> >
> > Thanks and regards
> > Saurabh Agarwal
> >
>
>
>
> --
> Eric Sammer
> phone: +1-917-287-2675
> twitter: esammer
> data: www.cloudera.com
>

Re: HDFS on raw device

Posted by Eric Sammer <es...@cloudera.com>.
No. HDFS always stores data on a OS file system (e.g. ext3 on Linux).

On Wed, May 12, 2010 at 7:58 AM, Saurabh Agarwal <sr...@gmail.com> wrote:
> Hii
> Can HDFS be give path of raw device, say can I set dfs.data.dir to /dev/sda3
> or something like that
>
>
> Thanks and regards
> Saurabh Agarwal
>



-- 
Eric Sammer
phone: +1-917-287-2675
twitter: esammer
data: www.cloudera.com