You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by Donald Miner <dm...@clearedgeit.com> on 2014/02/25 16:49:31 UTC
HDFS caching w/ Accumulo?
HDFS caching is part of the new Hadoop 2.3 release. From what I understand,
it allows you to mark specific files to be held in memory for faster reads.
Has anyone thought about how Accumulo could leverage this?
Re: HDFS caching w/ Accumulo?
Posted by John Vines <vi...@apache.org>.
It would provide better speed for failovers/migrations/cold starts. But
block cache is definitely something we would want to rely on more for
those, IMO
On Tue, Feb 25, 2014 at 11:00 AM, Mike Drob <ma...@cloudera.com> wrote:
> Second instinct is that since we have block caches, it might not matter.
>
>
> On Tue, Feb 25, 2014 at 10:53 AM, Mike Drob <ma...@cloudera.com> wrote:
>
>> First instinct is to use it for the root/metadata tablets.
>>
>>
>> On Tue, Feb 25, 2014 at 10:49 AM, Donald Miner <dm...@clearedgeit.com>wrote:
>>
>>> HDFS caching is part of the new Hadoop 2.3 release. From what I
>>> understand, it allows you to mark specific files to be held in memory for
>>> faster reads.
>>>
>>> Has anyone thought about how Accumulo could leverage this?
>>>
>>
>>
>
Re: HDFS caching w/ Accumulo?
Posted by Mike Drob <ma...@cloudera.com>.
Second instinct is that since we have block caches, it might not matter.
On Tue, Feb 25, 2014 at 10:53 AM, Mike Drob <ma...@cloudera.com> wrote:
> First instinct is to use it for the root/metadata tablets.
>
>
> On Tue, Feb 25, 2014 at 10:49 AM, Donald Miner <dm...@clearedgeit.com>wrote:
>
>> HDFS caching is part of the new Hadoop 2.3 release. From what I
>> understand, it allows you to mark specific files to be held in memory for
>> faster reads.
>>
>> Has anyone thought about how Accumulo could leverage this?
>>
>
>
Re: HDFS caching w/ Accumulo?
Posted by Mike Drob <ma...@cloudera.com>.
First instinct is to use it for the root/metadata tablets.
On Tue, Feb 25, 2014 at 10:49 AM, Donald Miner <dm...@clearedgeit.com>wrote:
> HDFS caching is part of the new Hadoop 2.3 release. From what I
> understand, it allows you to mark specific files to be held in memory for
> faster reads.
>
> Has anyone thought about how Accumulo could leverage this?
>
Re: HDFS caching w/ Accumulo?
Posted by John Vines <vi...@apache.org>.
I could see it being used for the metadata and root table files, at a
minimum. Also useful for other reused information in HDFS- tableId for
HdfsZooInstance, KeyEncryptionKey if that's being used. Maybe root/metadata
WALogs as well.
On Tue, Feb 25, 2014 at 10:49 AM, Donald Miner <dm...@clearedgeit.com>wrote:
> HDFS caching is part of the new Hadoop 2.3 release. From what I
> understand, it allows you to mark specific files to be held in memory for
> faster reads.
>
> Has anyone thought about how Accumulo could leverage this?
>
Re: HDFS caching w/ Accumulo?
Posted by Adam Fuchs <af...@apache.org>.
Maybe this could be used to speed up WAL recovery for use cases that demand
really high availability and low latency?
Adam
On Feb 25, 2014 10:50 AM, "Donald Miner" <dm...@clearedgeit.com> wrote:
> HDFS caching is part of the new Hadoop 2.3 release. From what I
> understand, it allows you to mark specific files to be held in memory for
> faster reads.
>
> Has anyone thought about how Accumulo could leverage this?
>