You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Vihang Karajgaonkar (JIRA)" <ji...@apache.org> on 2018/05/14 04:13:00 UTC
[jira] [Commented] (HIVE-19514) Successful test logs should be
copied
[ https://issues.apache.org/jira/browse/HIVE-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473752#comment-16473752 ]
Vihang Karajgaonkar commented on HIVE-19514:
--------------------------------------------
Ptest server now only copies successful logs. This inprovement shaves off 10 min from the total execution time.
{noformat}
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase took 64 minutes
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase.TotalRsyncElapsedTime took 8 minutes
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase took 5 minutes
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: ReportingPhase took 0 minutes
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: TestCheckPhase took 0 minutes
2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase took 0 minutes
{noformat}
> Successful test logs should be copied
> -------------------------------------
>
> Key: HIVE-19514
> URL: https://issues.apache.org/jira/browse/HIVE-19514
> Project: Hive
> Issue Type: Sub-task
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Major
>
> Ptest tries to copy ~20G worth of logs from the worker nodes to the server. The total aggregate time spent by all the Ptest server in copying these files is close to 15 min (although much of that is in parallel threads). Most of the times we don't check for the logs. We seldom check for logs of successful tests. We should ignore copying successful test logs by default to improve runtime.
> {noformat}
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:235 Executed 13568 tests
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase took 71 minutes
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase.TotalRsyncElapsedTime took 15 minutes
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase took 5 minutes
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: ReportingPhase took 0 minutes
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: TestCheckPhase took 0 minutes
> 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase took 0 minutes
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)