[jira] [Updated] (BEAM-8213) Run and report python tox tasks separately within Jenkins

["Chad Dombrova (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chad Dombrova updated BEAM-8213:
--------------------------------
    Description: 
As a python developer, the speed and comprehensibility of the jenkins PreCommit job could be greatly improved.

Here are some of the problems

- when a lint job fails, it's not reported in the test results summary, so even though the job is marked as failed, I see "Test Result (no failures)" which is quite confusing
- I have to wait for over an hour to discover the lint failed, which takes about a minute to run on its own
- The logs are a jumbled mess of all the different tasks running on top of each other
- The test results give no indication of which version of python they use.  I click on Test results, then the test module, then the test class, then I see 4 tests named the same thing.  I assume that the first is python 2.7, the second is 3.5 and so on.   It takes 5 clicks and then reading the log output to know which version a single error pertains to.  This makes it very difficult to discover problems, and deduce that they may have something to do with python version mismatches.

I believe the solution to this is to split up the single monolithic python PreCommit job into sub-jobs (possibly using a pipeline with steps).  This would give us the following benefits:

- sub job results should become available as they finish, so for example, lint results should be available very early on
- sub job results will be reported separately, and there will be a job for each py2, py35, py36 and so on, so it will be clear when an error is related to a particular python version
- sub jobs without reports, like docs and lint, will have their own failure status and logs, so when they fail it will be more obvious what went wrong.

I'm happy to help out once I get some feedback on the desired way forward.


  was:
As a python developer, the speed and comprehensibility of the jenkins PreCommit job could be greatly improved.

Here are some of the problems

- when a lint job fails, it's not reported in the Test results, so even though the job failed, I see "Test Result (no failures)"
- I have to wait for over an hour to discover the lint failed, which takes about a minute to run
- The logs are a jumbled mess of all the different tasks running on top of each other
- The test results give no indication of whether they come from python 27, 35, etc.  I click on Test results, then the test module, then the test class, then I see 4 tests named the same thing.  I assume that the first is python 2.7, the second is 3.5 and so on.   This makes it very difficult to discover problems, and deduce that they may have something to do with python version mismatches.

I believe the solution to this is to split up the single monolithic python PreCommit job into sub-jobs (possibly using a pipeline with steps).  This would give us the following benefits:

- sub-job results should become available as they finish, so lint results should be available very early on
- sub-job results will be reported separately, so it will be clear when an error is related to a particular python version
- sub-jobs without reports, like docs and lint, will have their own failure status and logs, so it will be more obvious when they fail what went wrong.

I'm happy to help out once I get some feedback on the desired way forward.



> Run and report python tox tasks separately within Jenkins
> ---------------------------------------------------------
>
>                 Key: BEAM-8213
>                 URL: https://issues.apache.org/jira/browse/BEAM-8213
>             Project: Beam
>          Issue Type: Improvement
>          Components: build-system
>            Reporter: Chad Dombrova
>            Priority: Major
>
> As a python developer, the speed and comprehensibility of the jenkins PreCommit job could be greatly improved.
> Here are some of the problems
> - when a lint job fails, it's not reported in the test results summary, so even though the job is marked as failed, I see "Test Result (no failures)" which is quite confusing
> - I have to wait for over an hour to discover the lint failed, which takes about a minute to run on its own
> - The logs are a jumbled mess of all the different tasks running on top of each other
> - The test results give no indication of which version of python they use.  I click on Test results, then the test module, then the test class, then I see 4 tests named the same thing.  I assume that the first is python 2.7, the second is 3.5 and so on.   It takes 5 clicks and then reading the log output to know which version a single error pertains to.  This makes it very difficult to discover problems, and deduce that they may have something to do with python version mismatches.
> I believe the solution to this is to split up the single monolithic python PreCommit job into sub-jobs (possibly using a pipeline with steps).  This would give us the following benefits:
> - sub job results should become available as they finish, so for example, lint results should be available very early on
> - sub job results will be reported separately, and there will be a job for each py2, py35, py36 and so on, so it will be clear when an error is related to a particular python version
> - sub jobs without reports, like docs and lint, will have their own failure status and logs, so when they fail it will be more obvious what went wrong.
> I'm happy to help out once I get some feedback on the desired way forward.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)