Create cache fails on first time

[Arpit Tak <ar...@sigmoidanalytics.com>]

I am loading some data(25GB) in shark from hdfs : spark,shark ( both- 0.9)
. Generally it happens that caching a table some time fails, for the very
first time we are   caching data. Second time it runs successfully ...

Anybody facing same issue ??..

*Shark Client Log:*
> create table sample_cached as access_cached * from access;
[Hive Error]: Query returned non-zero code: 9, cause: FAILED: Execution
Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
Time taken (including network latency): 68.016 seconds

*SharkServer Log:*
Moving data to: hdfs://
ec2-xyz.compute-1.amazonaws.com:9000/user/hive/warehouse/access_cached
Failed with exception Unable to rename: hdfs://
ec2-xyz.compute-1.amazonaws.com:9000/tmp/hive-root/hive_2014-04-16_10-52-10_487_3421016764043167178/-ext-10004to:
hdfs://
ec2-xyz.compute-1.amazonaws.com:9000/user/hive/warehouse/access_cached
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MoveTask


Regards,
Arpit Tak

Re: Create cache fails on first time

[Andre Bois-Crettez <an...@kelkoo.com>]

It could be GC issue, first time it triggers a full GC that takes too
much time ?
Make sure you have Xms and Xms at the same values, and try
-XX:+UseConcMarkSweepGC
And analyse GC logs.

André Bois-Crettez

On 2014-04-16 16:44, Arpit Tak wrote:
> I am loading some data(25GB) in shark from hdfs : spark,shark ( both-
> 0.9) . Generally it happens that caching a table some time fails, for
> the very first time we are   caching data. Second time it runs
> successfully ...
> [...]
> Regards,
> Arpit Tak

--
André Bois-Crettez

Software Architect
Big Data Developer
http://www.kelkoo.com/


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 8, rue du Sentier 75002 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.