You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@crail.apache.org by "Jonas Pfefferle (JIRA)" <ji...@apache.org> on 2019/06/20 11:10:00 UTC
[jira] [Commented] (CRAIL-98) Crail, used as temporary results
storage (type2 storage), misses close from TeraSort
[ https://issues.apache.org/jira/browse/CRAIL-98?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868450#comment-16868450 ]
Jonas Pfefferle commented on CRAIL-98:
--------------------------------------
Hi David,
I found the issue and have a fix. I'm now trying to make sure it doesn't break anything else. Once I'm done with the testing I will update you.
Regards,
Jonas
> Crail, used as temporary results storage (type2 storage), misses close from TeraSort
> ------------------------------------------------------------------------------------
>
> Key: CRAIL-98
> URL: https://issues.apache.org/jira/browse/CRAIL-98
> Project: Apache Crail
> Issue Type: Bug
> Affects Versions: 1.2
> Reporter: David Crespi
> Priority: Major
>
> Running the latest Crail: v1.1-7-ga6e622f
> I’m running Crail as the temporary backend storage for Terasort.
> After each section (TeraGen, TeraSort, TeraVerify)
> the program waits until a Cntl-C is given, then moves on to the next
> section.
>
> Here’s a small snippet of the output. Terasort waits where the
> bolded “Number of records” is listed, until the ^c is given. Each of the three programs does the same, but the program does finish without errors.
> 19/06/18 15:13:19 DEBUG TaskSchedulerImpl: parentName: , name:TaskSet_1.0, runningTasks: 1
> 19/06/18 15:13:19 INFO TaskSetManager: Finished task 1.0 in stage 1.0 (TID 3) in 142 ms on 192.168.3.10 (executor 4) (1/2)
> 19/06/18 15:13:19 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 192.168.3.12:34011 (size: 1825.0 B, free: 366.3 MB)
> 19/06/18 15:13:19 DEBUG TaskSchedulerImpl: parentName: , name: TaskSet_1.0, runningTasks: 0
> 19/06/18 15:13:19 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 2) in 977 ms on 192.168.3.12 (executor 3) (2/2)
> 19/06/18 15:13:19 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool
> 19/06/18 15:13:19 INFO DAGScheduler: ResultStage 1 (count at TeraGen.scala:94) finished in 0.995 s
> 19/06/18 15:13:19 DEBUG DAGScheduler: After removal of stage 1, remaining stages = 0
> 19/06/18 15:13:19 INFO DAGScheduler: Job 1 finished: count at TeraGen.scala:94, took 1.003537 s
> *Number of records written: 10000*
> {color:#FF0000}*^C*{color}19/06/18 15:13:36 INFO SparkContext: Invoking stop() from shutdown hook
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)