You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Damien Carol <dc...@blitzbs.com> on 2014/08/13 11:14:37 UTC
Test read caching
Hi,
I'm trying to test Read Cache in HDFS on version 2.4.0.
I followed this article without success :
http://blog.cloudera.com/blog/2014/08/new-in-cdh-5-1-hdfs-read-caching/
I made this cmds :
>
hadoop-2.4.0/bin/hdfs cacheadmin -addPool testPool
>
hadoop-2.4.0/bin/hdfs cacheadmin -addDirective -path /myfile -pool testPool
>
hadoop2.4.0/bin/hdfs cacheadmin -listPools -stats testPool
I can see the directive with "-listDirectives" :
[hduser@nc-h04 ~]$ hadoop-2.4.0/bin/hdfs cacheadmin -listDirectives
Found 1 entry
ID POOL REPL EXPIRY PATH
2 testPool 1 never /user/hive/warehouse/casino.db/magasin
BUT the file is never cached. Even if I use this file. I wait 1 day.
Any tips on that ?
Regards,
--
Damien CAROL
* tél : +33 (0)4 74 96 88 14
* fax : +33 (0)4 74 96 31 88
* email :dcarol@blitzbs.com <ma...@blitzbs.com>
BLITZ BUSINESS SERVICE
Re: Test read caching
Posted by Yanbo Liang <ya...@gmail.com>.
You can check the response of your command.
For example, you can execute "hdfs dfsadmin -report"
and you will get reply like following and can ensure the space of cache
used and remaining is reasonable.
Configured Cache Capacity: 64000 (62.50 KB)
Cache Used: 4096 (4 KB)
Cache Remaining: 59904 (58.50 KB)
Cache Used%: 6.40%
Cache Remaining%: 93.60%
2014-08-13 17:14 GMT+08:00 Damien Carol <dc...@blitzbs.com>:
> Hi,
>
> I'm trying to test Read Cache in HDFS on version 2.4.0.
>
> I followed this article without success :
> http://blog.cloudera.com/blog/2014/08/new-in-cdh-5-1-hdfs-read-caching/
>
> I made this cmds :
>
> >
> hadoop-2.4.0/bin/hdfs cacheadmin -addPool testPool
> >
> hadoop-2.4.0/bin/hdfs cacheadmin -addDirective -path /myfile -pool testPool
> >
> hadoop2.4.0/bin/hdfs cacheadmin -listPools -stats testPool
>
> I can see the directive with "-listDirectives" :
>
> [hduser@nc-h04 ~]$ hadoop-2.4.0/bin/hdfs cacheadmin -listDirectives
> Found 1 entry
> ID POOL REPL EXPIRY PATH
> 2 testPool 1 never /user/hive/warehouse/casino.db/magasin
>
> BUT the file is never cached. Even if I use this file. I wait 1 day.
>
> Any tips on that ?
>
> Regards,
> --
>
> Damien CAROL
>
> - tél : +33 (0)4 74 96 88 14
> - fax : +33 (0)4 74 96 31 88
> - email : dcarol@blitzbs.com
>
> BLITZ BUSINESS SERVICE
>
Re: Test read caching
Posted by Yanbo Liang <ya...@gmail.com>.
You can check the response of your command.
For example, you can execute "hdfs dfsadmin -report"
and you will get reply like following and can ensure the space of cache
used and remaining is reasonable.
Configured Cache Capacity: 64000 (62.50 KB)
Cache Used: 4096 (4 KB)
Cache Remaining: 59904 (58.50 KB)
Cache Used%: 6.40%
Cache Remaining%: 93.60%
2014-08-13 17:14 GMT+08:00 Damien Carol <dc...@blitzbs.com>:
> Hi,
>
> I'm trying to test Read Cache in HDFS on version 2.4.0.
>
> I followed this article without success :
> http://blog.cloudera.com/blog/2014/08/new-in-cdh-5-1-hdfs-read-caching/
>
> I made this cmds :
>
> >
> hadoop-2.4.0/bin/hdfs cacheadmin -addPool testPool
> >
> hadoop-2.4.0/bin/hdfs cacheadmin -addDirective -path /myfile -pool testPool
> >
> hadoop2.4.0/bin/hdfs cacheadmin -listPools -stats testPool
>
> I can see the directive with "-listDirectives" :
>
> [hduser@nc-h04 ~]$ hadoop-2.4.0/bin/hdfs cacheadmin -listDirectives
> Found 1 entry
> ID POOL REPL EXPIRY PATH
> 2 testPool 1 never /user/hive/warehouse/casino.db/magasin
>
> BUT the file is never cached. Even if I use this file. I wait 1 day.
>
> Any tips on that ?
>
> Regards,
> --
>
> Damien CAROL
>
> - tél : +33 (0)4 74 96 88 14
> - fax : +33 (0)4 74 96 31 88
> - email : dcarol@blitzbs.com
>
> BLITZ BUSINESS SERVICE
>
Re: Test read caching
Posted by Yanbo Liang <ya...@gmail.com>.
You can check the response of your command.
For example, you can execute "hdfs dfsadmin -report"
and you will get reply like following and can ensure the space of cache
used and remaining is reasonable.
Configured Cache Capacity: 64000 (62.50 KB)
Cache Used: 4096 (4 KB)
Cache Remaining: 59904 (58.50 KB)
Cache Used%: 6.40%
Cache Remaining%: 93.60%
2014-08-13 17:14 GMT+08:00 Damien Carol <dc...@blitzbs.com>:
> Hi,
>
> I'm trying to test Read Cache in HDFS on version 2.4.0.
>
> I followed this article without success :
> http://blog.cloudera.com/blog/2014/08/new-in-cdh-5-1-hdfs-read-caching/
>
> I made this cmds :
>
> >
> hadoop-2.4.0/bin/hdfs cacheadmin -addPool testPool
> >
> hadoop-2.4.0/bin/hdfs cacheadmin -addDirective -path /myfile -pool testPool
> >
> hadoop2.4.0/bin/hdfs cacheadmin -listPools -stats testPool
>
> I can see the directive with "-listDirectives" :
>
> [hduser@nc-h04 ~]$ hadoop-2.4.0/bin/hdfs cacheadmin -listDirectives
> Found 1 entry
> ID POOL REPL EXPIRY PATH
> 2 testPool 1 never /user/hive/warehouse/casino.db/magasin
>
> BUT the file is never cached. Even if I use this file. I wait 1 day.
>
> Any tips on that ?
>
> Regards,
> --
>
> Damien CAROL
>
> - tél : +33 (0)4 74 96 88 14
> - fax : +33 (0)4 74 96 31 88
> - email : dcarol@blitzbs.com
>
> BLITZ BUSINESS SERVICE
>
Re: Test read caching
Posted by Yanbo Liang <ya...@gmail.com>.
You can check the response of your command.
For example, you can execute "hdfs dfsadmin -report"
and you will get reply like following and can ensure the space of cache
used and remaining is reasonable.
Configured Cache Capacity: 64000 (62.50 KB)
Cache Used: 4096 (4 KB)
Cache Remaining: 59904 (58.50 KB)
Cache Used%: 6.40%
Cache Remaining%: 93.60%
2014-08-13 17:14 GMT+08:00 Damien Carol <dc...@blitzbs.com>:
> Hi,
>
> I'm trying to test Read Cache in HDFS on version 2.4.0.
>
> I followed this article without success :
> http://blog.cloudera.com/blog/2014/08/new-in-cdh-5-1-hdfs-read-caching/
>
> I made this cmds :
>
> >
> hadoop-2.4.0/bin/hdfs cacheadmin -addPool testPool
> >
> hadoop-2.4.0/bin/hdfs cacheadmin -addDirective -path /myfile -pool testPool
> >
> hadoop2.4.0/bin/hdfs cacheadmin -listPools -stats testPool
>
> I can see the directive with "-listDirectives" :
>
> [hduser@nc-h04 ~]$ hadoop-2.4.0/bin/hdfs cacheadmin -listDirectives
> Found 1 entry
> ID POOL REPL EXPIRY PATH
> 2 testPool 1 never /user/hive/warehouse/casino.db/magasin
>
> BUT the file is never cached. Even if I use this file. I wait 1 day.
>
> Any tips on that ?
>
> Regards,
> --
>
> Damien CAROL
>
> - tél : +33 (0)4 74 96 88 14
> - fax : +33 (0)4 74 96 31 88
> - email : dcarol@blitzbs.com
>
> BLITZ BUSINESS SERVICE
>