Python postcommit and precommit

[Udi Meiri <eh...@google.com>]

Hi,

I noticed that Python precommit runs using this command:
  mvn clean install -pl sdks/python -am -amd
while postcommit invocation is simply a bash script:
  bash sdks/python/run_postcommit.sh

Both run unit tests via Tox, however since the runtime environment setup is
configured in different files (pom.xml vs shell script), they don't always
agree in their results (precommit is currently succeeded while postcommit
is failing).

So my naive question is: why does Python precommit run via Maven/Gradle?
Could we not just use a script like run_postcommit.sh?

(Side note: there's a lot of code/config duplication, such as: pypi package
versions, *.c, *.so, etc. cleanup)

Re: Python postcommit and precommit

[Alan Myrvold <am...@google.com>]

I think we should replace the shell script with a top level
pythonPostCommit gradle target, similar to the precomment.

On Mon, Apr 9, 2018 at 12:12 PM Lukasz Cwik <lc...@google.com> wrote:

> The shell scripts still exist instead of using Gradle. Migrating to Gradle
> as the build system hasn't addressed this (only change in the Gradle
> migration was an improvement where Gradle now creates a virtualenv
> automatically for building).
>
> Alan, any plans to integrate more closely with Gradle going forward
> instead of using shell scripts for task/input/output management?
>
> On Wed, Apr 4, 2018 at 2:11 PM Kenneth Knowles <kl...@google.com> wrote:
>
>> Was this resolved off list? I think it makes sense to have a
>> dependency-driven build tool as the entry point to these processes. So in
>> our case, Gradle. If setting it up in Gradle/Groovy is a pain, having it
>> shell out seems fine as an implementation detail, but you need to set up
>> inputs/outputs of the Gradle tasks properly.
>>
>> Kenn
>>
>> On Fri, Mar 30, 2018 at 3:30 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> Hi,
>>>
>>> I noticed that Python precommit runs using this command:
>>>   mvn clean install -pl sdks/python -am -amd
>>> while postcommit invocation is simply a bash script:
>>>   bash sdks/python/run_postcommit.sh
>>>
>>> Both run unit tests via Tox, however since the runtime environment setup
>>> is configured in different files (pom.xml vs shell script), they don't
>>> always agree in their results (precommit is currently succeeded while
>>> postcommit is failing).
>>>
>>> So my naive question is: why does Python precommit run via Maven/Gradle?
>>> Could we not just use a script like run_postcommit.sh?
>>>
>>> (Side note: there's a lot of code/config duplication, such as: pypi
>>> package versions, *.c, *.so, etc. cleanup)
>>>
>>

Re: Python postcommit and precommit

[Lukasz Cwik <lc...@google.com>]

The shell scripts still exist instead of using Gradle. Migrating to Gradle
as the build system hasn't addressed this (only change in the Gradle
migration was an improvement where Gradle now creates a virtualenv
automatically for building).

Alan, any plans to integrate more closely with Gradle going forward instead
of using shell scripts for task/input/output management?

On Wed, Apr 4, 2018 at 2:11 PM Kenneth Knowles <kl...@google.com> wrote:

> Was this resolved off list? I think it makes sense to have a
> dependency-driven build tool as the entry point to these processes. So in
> our case, Gradle. If setting it up in Gradle/Groovy is a pain, having it
> shell out seems fine as an implementation detail, but you need to set up
> inputs/outputs of the Gradle tasks properly.
>
> Kenn
>
> On Fri, Mar 30, 2018 at 3:30 PM Udi Meiri <eh...@google.com> wrote:
>
>> Hi,
>>
>> I noticed that Python precommit runs using this command:
>>   mvn clean install -pl sdks/python -am -amd
>> while postcommit invocation is simply a bash script:
>>   bash sdks/python/run_postcommit.sh
>>
>> Both run unit tests via Tox, however since the runtime environment setup
>> is configured in different files (pom.xml vs shell script), they don't
>> always agree in their results (precommit is currently succeeded while
>> postcommit is failing).
>>
>> So my naive question is: why does Python precommit run via Maven/Gradle?
>> Could we not just use a script like run_postcommit.sh?
>>
>> (Side note: there's a lot of code/config duplication, such as: pypi
>> package versions, *.c, *.so, etc. cleanup)
>>
>

Re: Python postcommit and precommit

[Kenneth Knowles <kl...@google.com>]

Was this resolved off list? I think it makes sense to have a
dependency-driven build tool as the entry point to these processes. So in
our case, Gradle. If setting it up in Gradle/Groovy is a pain, having it
shell out seems fine as an implementation detail, but you need to set up
inputs/outputs of the Gradle tasks properly.

Kenn

On Fri, Mar 30, 2018 at 3:30 PM Udi Meiri <eh...@google.com> wrote:

> Hi,
>
> I noticed that Python precommit runs using this command:
>   mvn clean install -pl sdks/python -am -amd
> while postcommit invocation is simply a bash script:
>   bash sdks/python/run_postcommit.sh
>
> Both run unit tests via Tox, however since the runtime environment setup
> is configured in different files (pom.xml vs shell script), they don't
> always agree in their results (precommit is currently succeeded while
> postcommit is failing).
>
> So my naive question is: why does Python precommit run via Maven/Gradle?
> Could we not just use a script like run_postcommit.sh?
>
> (Side note: there's a lot of code/config duplication, such as: pypi
> package versions, *.c, *.so, etc. cleanup)
>