Hi,can hbase store picture and video such as blob field ? Any other simply question.

[Rocks <nu...@gmail.com>]

Hi,i want to konw whether hbase can store picture  and video as blob field?
Maybe I  should store the picture path in it ,and point to the pic or video
in file system?
but how to store the pic and video ,how to define the path pointed to the
file system in top of hadoop? Can anyone give me some suggestion?
And  i am still uncertain where is my hbase storage file in my HDFS ?In the
command "bin/hadoop dfs -ls" , i cannot see anything related to hbase file.
It will be greatly appreciated if you give me some hlep.

Re: Hi,can hbase store picture and video such as blob field ? Any other simply question.

[Rocks <nu...@gmail.com>]

OK,thanks, i find it with your help.
On Tue, Aug 11, 2009 at 10:05 AM, tim robertson
<ti...@gmail.com>wrote:

> Depends where you configured it to go.  For me it is in /hbase
>
> I think you might need a / at the end of your ls to see the root of
> hdfs and not the user account:
> bin/hadoop dfs -ls /
>
> Cheers
>
> Tim
>
>
>
> On Tue, Aug 11, 2009 at 4:02 AM, Rocks<nu...@gmail.com> wrote:
> > And i am still puzzled with the simple question, where is my hbase
> storage
> > file in my HDFS ?In the command "bin/hadoop dfs -ls" , i cannot see
> anything
> > related to hbase file.
> >
> > can anyone give me a answer?
> >
> > On Tue, Aug 11, 2009 at 9:20 AM, Rocks <nu...@gmail.com> wrote:
> >
> >> oh，thanks, can you give me a CDNS tools or tutorial to solve the
> metioned
> >> problem?
> >>
> >>
> >>
> >> On Tue, Aug 11, 2009 at 4:20 AM, Ryan Rawson <ry...@gmail.com>
> wrote:
> >>
> >>> HBase can help one overcome the 'small file' problem of HDFS.  But
> >>> HBase may not be the best methodology once files get beyond a certain
> >>> size (that size might be in the 10mb+ area).  However that is no
> >>> longer 'small files' and HDFS would be suitable again.
> >>>
> >>> Another aspect to consider is that access for a single row is mediated
> >>> by a single regionserver, which means if you have extremely high load
> >>> on only a few rows, you might not achieve scalability.  Caching layers
> >>> (reverse proxies, cdns) can help solve this. But if you are looking at
> >>> a high volume, high performance download server that can handle
> >>> millions of clients asking for the same object, you might need to
> >>> think a bit more.
> >>>
> >>> -ryan
> >>>
> >>> On Mon, Aug 10, 2009 at 1:13 PM, Tim Sell<tr...@gmail.com> wrote:
> >>> > Yes you can store binary data in hbase, all data in hbase is treated
> >>> > as just raw bytes.
> >>> > It would probably suit well things like thumbnails? I think youtube
> >>> > uses bigtable for the thumbnail images of their videos. And they
> store
> >>> > their actual videos in cdn's (and gfs for the longtail I guess).
> >>> >
> >>> > So if you have a lot of large binaries, you'll probably want just the
> >>> > links in hbase, with the files elsewhere. Maybe mogilefs is more
> >>> > suitable then hdfs though, because hdfs doesn't like lots of "small"
> >>> > files?
> >>> >
> >>> > ~Tim.
> >>> >
> >>> >
> >>> > 2009/8/10 Rocks <nu...@gmail.com>:
> >>> >> Hi,i want to konw whether hbase can store picture  and video as blob
> >>> field?
> >>> >> Maybe I  should store the picture path in it ,and point to the pic
> or
> >>> video
> >>> >> in file system?
> >>> >> but how to store the pic and video ,how to define the path pointed
> to
> >>> the
> >>> >> file system in top of hadoop? Can anyone give me some suggestion?
> >>> >> And  i am still uncertain where is my hbase storage file in my HDFS
> ?In
> >>> the
> >>> >> command "bin/hadoop dfs -ls" , i cannot see anything related to
> hbase
> >>> file.
> >>> >> It will be greatly appreciated if you give me some hlep.
> >>> >>
> >>> >
> >>>
> >>
> >>
> >
>

Re: Hi,can hbase store picture and video such as blob field ? Any other simply question.

[tim robertson <ti...@gmail.com>]

Depends where you configured it to go.  For me it is in /hbase

I think you might need a / at the end of your ls to see the root of
hdfs and not the user account:
bin/hadoop dfs -ls /

Cheers

Tim



On Tue, Aug 11, 2009 at 4:02 AM, Rocks<nu...@gmail.com> wrote:
> And i am still puzzled with the simple question, where is my hbase storage
> file in my HDFS ?In the command "bin/hadoop dfs -ls" , i cannot see anything
> related to hbase file.
>
> can anyone give me a answer?
>
> On Tue, Aug 11, 2009 at 9:20 AM, Rocks <nu...@gmail.com> wrote:
>
>> oh，thanks, can you give me a CDNS tools or tutorial to solve the metioned
>> problem?
>>
>>
>>
>> On Tue, Aug 11, 2009 at 4:20 AM, Ryan Rawson <ry...@gmail.com> wrote:
>>
>>> HBase can help one overcome the 'small file' problem of HDFS.  But
>>> HBase may not be the best methodology once files get beyond a certain
>>> size (that size might be in the 10mb+ area).  However that is no
>>> longer 'small files' and HDFS would be suitable again.
>>>
>>> Another aspect to consider is that access for a single row is mediated
>>> by a single regionserver, which means if you have extremely high load
>>> on only a few rows, you might not achieve scalability.  Caching layers
>>> (reverse proxies, cdns) can help solve this. But if you are looking at
>>> a high volume, high performance download server that can handle
>>> millions of clients asking for the same object, you might need to
>>> think a bit more.
>>>
>>> -ryan
>>>
>>> On Mon, Aug 10, 2009 at 1:13 PM, Tim Sell<tr...@gmail.com> wrote:
>>> > Yes you can store binary data in hbase, all data in hbase is treated
>>> > as just raw bytes.
>>> > It would probably suit well things like thumbnails? I think youtube
>>> > uses bigtable for the thumbnail images of their videos. And they store
>>> > their actual videos in cdn's (and gfs for the longtail I guess).
>>> >
>>> > So if you have a lot of large binaries, you'll probably want just the
>>> > links in hbase, with the files elsewhere. Maybe mogilefs is more
>>> > suitable then hdfs though, because hdfs doesn't like lots of "small"
>>> > files?
>>> >
>>> > ~Tim.
>>> >
>>> >
>>> > 2009/8/10 Rocks <nu...@gmail.com>:
>>> >> Hi,i want to konw whether hbase can store picture  and video as blob
>>> field?
>>> >> Maybe I  should store the picture path in it ,and point to the pic or
>>> video
>>> >> in file system?
>>> >> but how to store the pic and video ,how to define the path pointed to
>>> the
>>> >> file system in top of hadoop? Can anyone give me some suggestion?
>>> >> And  i am still uncertain where is my hbase storage file in my HDFS ?In
>>> the
>>> >> command "bin/hadoop dfs -ls" , i cannot see anything related to hbase
>>> file.
>>> >> It will be greatly appreciated if you give me some hlep.
>>> >>
>>> >
>>>
>>
>>
>

Re: Hi,can hbase store picture and video such as blob field ? Any other simply question.

[Rocks <nu...@gmail.com>]

And i am still puzzled with the simple question, where is my hbase storage
file in my HDFS ?In the command "bin/hadoop dfs -ls" , i cannot see anything
related to hbase file.

can anyone give me a answer?

On Tue, Aug 11, 2009 at 9:20 AM, Rocks <nu...@gmail.com> wrote:

> oh，thanks, can you give me a CDNS tools or tutorial to solve the metioned
> problem?
>
>
>
> On Tue, Aug 11, 2009 at 4:20 AM, Ryan Rawson <ry...@gmail.com> wrote:
>
>> HBase can help one overcome the 'small file' problem of HDFS.  But
>> HBase may not be the best methodology once files get beyond a certain
>> size (that size might be in the 10mb+ area).  However that is no
>> longer 'small files' and HDFS would be suitable again.
>>
>> Another aspect to consider is that access for a single row is mediated
>> by a single regionserver, which means if you have extremely high load
>> on only a few rows, you might not achieve scalability.  Caching layers
>> (reverse proxies, cdns) can help solve this. But if you are looking at
>> a high volume, high performance download server that can handle
>> millions of clients asking for the same object, you might need to
>> think a bit more.
>>
>> -ryan
>>
>> On Mon, Aug 10, 2009 at 1:13 PM, Tim Sell<tr...@gmail.com> wrote:
>> > Yes you can store binary data in hbase, all data in hbase is treated
>> > as just raw bytes.
>> > It would probably suit well things like thumbnails? I think youtube
>> > uses bigtable for the thumbnail images of their videos. And they store
>> > their actual videos in cdn's (and gfs for the longtail I guess).
>> >
>> > So if you have a lot of large binaries, you'll probably want just the
>> > links in hbase, with the files elsewhere. Maybe mogilefs is more
>> > suitable then hdfs though, because hdfs doesn't like lots of "small"
>> > files?
>> >
>> > ~Tim.
>> >
>> >
>> > 2009/8/10 Rocks <nu...@gmail.com>:
>> >> Hi,i want to konw whether hbase can store picture  and video as blob
>> field?
>> >> Maybe I  should store the picture path in it ,and point to the pic or
>> video
>> >> in file system?
>> >> but how to store the pic and video ,how to define the path pointed to
>> the
>> >> file system in top of hadoop? Can anyone give me some suggestion?
>> >> And  i am still uncertain where is my hbase storage file in my HDFS ?In
>> the
>> >> command "bin/hadoop dfs -ls" , i cannot see anything related to hbase
>> file.
>> >> It will be greatly appreciated if you give me some hlep.
>> >>
>> >
>>
>
>

Re: Hi,can hbase store picture and video such as blob field ? Any other simply question.

[Rocks <nu...@gmail.com>]

oh，thanks, can you give me a CDNS tools or tutorial to solve the metioned
problem?


On Tue, Aug 11, 2009 at 4:20 AM, Ryan Rawson <ry...@gmail.com> wrote:

> HBase can help one overcome the 'small file' problem of HDFS.  But
> HBase may not be the best methodology once files get beyond a certain
> size (that size might be in the 10mb+ area).  However that is no
> longer 'small files' and HDFS would be suitable again.
>
> Another aspect to consider is that access for a single row is mediated
> by a single regionserver, which means if you have extremely high load
> on only a few rows, you might not achieve scalability.  Caching layers
> (reverse proxies, cdns) can help solve this. But if you are looking at
> a high volume, high performance download server that can handle
> millions of clients asking for the same object, you might need to
> think a bit more.
>
> -ryan
>
> On Mon, Aug 10, 2009 at 1:13 PM, Tim Sell<tr...@gmail.com> wrote:
> > Yes you can store binary data in hbase, all data in hbase is treated
> > as just raw bytes.
> > It would probably suit well things like thumbnails? I think youtube
> > uses bigtable for the thumbnail images of their videos. And they store
> > their actual videos in cdn's (and gfs for the longtail I guess).
> >
> > So if you have a lot of large binaries, you'll probably want just the
> > links in hbase, with the files elsewhere. Maybe mogilefs is more
> > suitable then hdfs though, because hdfs doesn't like lots of "small"
> > files?
> >
> > ~Tim.
> >
> >
> > 2009/8/10 Rocks <nu...@gmail.com>:
> >> Hi,i want to konw whether hbase can store picture  and video as blob
> field?
> >> Maybe I  should store the picture path in it ,and point to the pic or
> video
> >> in file system?
> >> but how to store the pic and video ,how to define the path pointed to
> the
> >> file system in top of hadoop? Can anyone give me some suggestion?
> >> And  i am still uncertain where is my hbase storage file in my HDFS ?In
> the
> >> command "bin/hadoop dfs -ls" , i cannot see anything related to hbase
> file.
> >> It will be greatly appreciated if you give me some hlep.
> >>
> >
>

Re: Hi,can hbase store picture and video such as blob field ? Any other simply question.

[Ryan Rawson <ry...@gmail.com>]

HBase can help one overcome the 'small file' problem of HDFS.  But
HBase may not be the best methodology once files get beyond a certain
size (that size might be in the 10mb+ area).  However that is no
longer 'small files' and HDFS would be suitable again.

Another aspect to consider is that access for a single row is mediated
by a single regionserver, which means if you have extremely high load
on only a few rows, you might not achieve scalability.  Caching layers
(reverse proxies, cdns) can help solve this. But if you are looking at
a high volume, high performance download server that can handle
millions of clients asking for the same object, you might need to
think a bit more.

-ryan

On Mon, Aug 10, 2009 at 1:13 PM, Tim Sell<tr...@gmail.com> wrote:
> Yes you can store binary data in hbase, all data in hbase is treated
> as just raw bytes.
> It would probably suit well things like thumbnails? I think youtube
> uses bigtable for the thumbnail images of their videos. And they store
> their actual videos in cdn's (and gfs for the longtail I guess).
>
> So if you have a lot of large binaries, you'll probably want just the
> links in hbase, with the files elsewhere. Maybe mogilefs is more
> suitable then hdfs though, because hdfs doesn't like lots of "small"
> files?
>
> ~Tim.
>
>
> 2009/8/10 Rocks <nu...@gmail.com>:
>> Hi,i want to konw whether hbase can store picture  and video as blob field?
>> Maybe I  should store the picture path in it ,and point to the pic or video
>> in file system?
>> but how to store the pic and video ,how to define the path pointed to the
>> file system in top of hadoop? Can anyone give me some suggestion?
>> And  i am still uncertain where is my hbase storage file in my HDFS ?In the
>> command "bin/hadoop dfs -ls" , i cannot see anything related to hbase file.
>> It will be greatly appreciated if you give me some hlep.
>>
>

Re: Hi,can hbase store picture and video such as blob field ? Any other simply question.

[Tim Sell <tr...@gmail.com>]

Yes you can store binary data in hbase, all data in hbase is treated
as just raw bytes.
It would probably suit well things like thumbnails? I think youtube
uses bigtable for the thumbnail images of their videos. And they store
their actual videos in cdn's (and gfs for the longtail I guess).

So if you have a lot of large binaries, you'll probably want just the
links in hbase, with the files elsewhere. Maybe mogilefs is more
suitable then hdfs though, because hdfs doesn't like lots of "small"
files?

~Tim.


2009/8/10 Rocks <nu...@gmail.com>:
> Hi,i want to konw whether hbase can store picture  and video as blob field?
> Maybe I  should store the picture path in it ,and point to the pic or video
> in file system?
> but how to store the pic and video ,how to define the path pointed to the
> file system in top of hadoop? Can anyone give me some suggestion?
> And  i am still uncertain where is my hbase storage file in my HDFS ?In the
> command "bin/hadoop dfs -ls" , i cannot see anything related to hbase file.
> It will be greatly appreciated if you give me some hlep.
>