You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Joel D <ga...@gmail.com> on 2018/01/10 18:00:16 UTC
No Tasks have reported metrics yet
Hi,
I've a job which takes a HiveQL joining 2 tables (2.5 TB, 45GB),
repartitions to 100 and then does some other transformations. This
executed fine earlier.
Job stages:
Stage 0: hive table 1 scan
Stage 1: Hive table 2 scan
Stage 2: Tungsten exchange for the join
Stage 3: Tungsten exchange for the reparation
Today the job is stuck in Stage 2. Out of 200 tasks which are supposed to
be executed none of them have started but 290 have failed due to preempted
executors.
Any inputs on how to resolve this issue? I'll try reducing the executor
memory to see if resource allocation is the issue.
Thanks.