[jira] [Work logged] (BEAM-6527) Parallel tox (unit) tests run on Jenkins

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-6527?focusedWorklogId=209861&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-209861 ]

ASF GitHub Bot logged work on BEAM-6527:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 07/Mar/19 23:42
            Start Date: 07/Mar/19 23:42
    Worklog Time Spent: 10m 
      Work Description: markflyhigh commented on pull request #7675: [BEAM-6527] Use Gradle to parallel Python tox tests
URL: https://github.com/apache/beam/pull/7675#discussion_r263616689
 
 

 ##########
 File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
 ##########
 @@ -1611,18 +1611,48 @@ class BeamModulePlugin implements Plugin<Project> {
         outputs.dirs(project.ext.envdir)
       }
 
+      def pythonSdkDeps = project.files(
+              project.fileTree(dir: "${project.rootDir}", include: [
+                'model/**',
+                'sdks/python/apache_beam/**/*.py',
+                'sdks/python/apache_beam/**/*.pyx',
+                'sdks/python/apache_beam/**/*.pxd',
+                'sdks/python/apache_beam/testing/data/**',
+                'sdks/python/apache_beam/scripts/**',
+                'sdks/python/.pylintrc',
+                'sdks/python/MANIFEST.in',
+                'sdks/python/gen_protos.py',
+                'sdks/python/setup.cfg',
+                'sdks/python/setup.py',
+                'sdks/python/test_config.py',
+                'sdks/python/tox.ini',
+              ])
+              )
+
       project.configurations { distConfig }
 
       project.task('sdist', dependsOn: 'setupVirtualenv') {
         doLast {
+          // Copy sdk sources to isolate directory
+          def copiedSrcDir = "${project.buildDir}/srcs"
+          project.copy {
+            from pythonSdkDeps
+            into copiedSrcDir
+          }
+
+          // Build artifact
           project.exec {
             executable 'sh'
-            args '-c', ". ${project.ext.envdir}/bin/activate && cd ${pythonRootDir} && python setup.py sdist --keep-temp --formats zip,gztar --dist-dir ${project.buildDir}"
+            args '-c', ". ${project.ext.envdir}/bin/activate && cd ${copiedSrcDir}/sdks/python && python setup.py sdist --formats zip,gztar --dist-dir ${project.buildDir}"
 
 Review comment:
   Since we start to use copy of sdk sources to build artifact, there is no need to keep temp directory around after build. So remove `--keep-temp` flag here.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 209861)
    Time Spent: 1h 20m  (was: 1h 10m)

> Parallel tox (unit) tests run on Jenkins
> ----------------------------------------
>
>                 Key: BEAM-6527
>                 URL: https://issues.apache.org/jira/browse/BEAM-6527
>             Project: Beam
>          Issue Type: Sub-task
>          Components: testing
>            Reporter: Mark Liu
>            Assignee: Mark Liu
>            Priority: Major
>              Labels: triaged
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Existing tox unit test suite (basic, gcp and cython) will be enabled in multiple version of Python 3, which will significantly increase runtime of Pre/PostCommit build. A parallel is wanted in tox or Gradle invocation to control the time in a reasonable range (<30mins for PreCommit is desired).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)