You are viewing a plain text version of this content. The canonical link for it is here.
Posted to proton@qpid.apache.org by Gordon Sim <gs...@redhat.com> on 2015/06/09 17:57:25 UTC
something rotten in the state of... something or other
I've recently started seeing errors[1] when running tests due to left
over artefacts of previous builds. This happens even for a completely
clean build directory, as some of the offending artefacts seem to be
created in the source tree.
Jython seems to be trying and failing to load cproton. With a completely
clean source and build tree, everything passes, but it is kind of
annoying to have to rely on that. Is anyone else seeing anything
similar? Any ideas as to the cause (I've only seen it happening quite
recently) or possible cures?
[1]:
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running org.apache.qpid.proton.InteropTest
> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.119 sec
> Running org.apache.qpid.proton.JythonTest
> 2015-06-09 16:49:29.705 INFO About to call Jython test script: '/home/gordon/projects/proton-git/tests/python/proton-test' with '/home/gordon/projects/proton-git/tests/python' added to Jython path
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.207 sec <<< FAILURE!
> test(org.apache.qpid.proton.JythonTest) Time elapsed: 5.203 sec <<< FAILURE!
> java.lang.AssertionError: Caught PyException on invocation number 2: Traceback (most recent call last):
> File "/home/gordon/projects/proton-git/tests/python/proton-test", line 616, in <module>
> m = __import__(name, None, None, ["dummy"])
> File "/home/gordon/projects/proton-git/tests/python/proton_tests/__init__.py", line 20, in <module>
> import proton_tests.codec
> File "/home/gordon/projects/proton-git/tests/python/proton_tests/codec.py", line 20, in <module>
> import os, common, sys
> File "/home/gordon/projects/proton-git/tests/python/proton_tests/common.py", line 26, in <module>
> from proton import Connection, Transport, SASL, Endpoint, Delivery, SSL
> File "/home/gordon/projects/proton-git/tests/../proton-c/bindings/python/proton/__init__.py", line 33, in <module>
> from cproton import *
> File "/home/gordon/projects/proton-git/tests/../proton-c/bindings/python/cproton.py", line 29, in <module>
> import _cproton
> ImportError: No module named _cproton
> with message: null
> at org.junit.Assert.fail(Assert.java:93)
> at org.apache.qpid.proton.JythonTest.runTestOnce(JythonTest.java:120)
> at org.apache.qpid.proton.JythonTest.test(JythonTest.java:95)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
> at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
> at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
> at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
> at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
> at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
>
>
> Results :
>
> Failed tests: test(org.apache.qpid.proton.JythonTest): Caught PyException on invocation number 2: Traceback (most recent call last):(..)
>
> Tests run: 9, Failures: 1, Errors: 0, Skipped: 0
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] proton-project .................................... SUCCESS [1.207s]
> [INFO] proton-j .......................................... SUCCESS [2.310s]
> [INFO] proton-jms ........................................ SUCCESS [0.623s]
> [INFO] proton-hawtdispatch ............................... SUCCESS [1.558s]
> [INFO] proton-tests ...................................... FAILURE [5.737s]
> [INFO] proton-j-messenger-example ........................ SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 11.637s
> [INFO] Finished at: Tue Jun 09 16:49:34 BST 2015
> [INFO] Final Memory: 14M/163M
> [INFO] ------------------------------------------------------------------------
Re: something rotten in the state of... something or other
Posted by aconway <ac...@redhat.com>.
On Tue, 2015-06-09 at 17:38 +0100, Robbie Gemmell wrote:
> I'm not seeing that currently, but I have seen similar sort of things
> a couple of times in the past.
>
> As you mention, some files get created in the source tree (presumably
> by or due to use of Jython), outwith the normal build areas they
> would
> be (which would lead to them being cleaned up), and I think that is
> part of the problem sometimes. If the shim, binding or test files get
> updated in certain ways, some bits can become out of sync, leading to
> the type of issue you saw.
>
> CI doesnt see the issue as it blows away all unversioned files before
> each update. If I see it locally I've just used git-clean to tidy up
> my checkout. I'm not sure what we can do otherwise except put
> something togehter that targets all the extraneous files and removes
> them.
Better to fix dependencies so things get rebuilt properly than to
simply blow them away. Could it be broken swig dependencies leaving out
of date .py files around? I've noticed before that swig does not always
get re-run when it should be.
> Robbie
>
> On 9 June 2015 at 16:57, Gordon Sim <gs...@redhat.com> wrote:
> > I've recently started seeing errors[1] when running tests due to
> > left over
> > artefacts of previous builds. This happens even for a completely
> > clean build
> > directory, as some of the offending artefacts seem to be created in
> > the
> > source tree.
> >
> > Jython seems to be trying and failing to load cproton. With a
> > completely
> > clean source and build tree, everything passes, but it is kind of
> > annoying
> > to have to rely on that. Is anyone else seeing anything similar?
> > Any ideas
> > as to the cause (I've only seen it happening quite recently) or
> > possible
> > cures?
> >
> >
> > [1]:
> >
> > > -------------------------------------------------------
> > > T E S T S
> > > -------------------------------------------------------
> > > Running org.apache.qpid.proton.InteropTest
> > > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > > 0.119 sec
> > > Running org.apache.qpid.proton.JythonTest
> > > 2015-06-09 16:49:29.705 INFO About to call Jython test script:
> > > '/home/gordon/projects/proton-git/tests/python/proton-test' with
> > > '/home/gordon/projects/proton-git/tests/python' added to Jython
> > > path
> > > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> > > 5.207 sec
> > > <<< FAILURE!
> > > test(org.apache.qpid.proton.JythonTest) Time elapsed: 5.203 sec
> > > <<<
> > > FAILURE!
> > > java.lang.AssertionError: Caught PyException on invocation number
> > > 2:
> > > Traceback (most recent call last):
> > > File "/home/gordon/projects/proton-git/tests/python/proton
> > > -test", line
> > > 616, in <module>
> > > m = __import__(name, None, None, ["dummy"])
> > > File
> > > "/home/gordon/projects/proton
> > > -git/tests/python/proton_tests/__init__.py",
> > > line 20, in <module>
> > > import proton_tests.codec
> > > File
> > > "/home/gordon/projects/proton
> > > -git/tests/python/proton_tests/codec.py", line
> > > 20, in <module>
> > > import os, common, sys
> > > File
> > > "/home/gordon/projects/proton
> > > -git/tests/python/proton_tests/common.py", line
> > > 26, in <module>
> > > from proton import Connection, Transport, SASL, Endpoint,
> > > Delivery,
> > > SSL
> > > File
> > > "/home/gordon/projects/proton-git/tests/../proton
> > > -c/bindings/python/proton/__init__.py",
> > > line 33, in <module>
> > > from cproton import *
> > > File
> > > "/home/gordon/projects/proton-git/tests/../proton
> > > -c/bindings/python/cproton.py",
> > > line 29, in <module>
> > > import _cproton
> > > ImportError: No module named _cproton
> > > with message: null
> > > at org.junit.Assert.fail(Assert.java:93)
> > > at
> > > org.apache.qpid.proton.JythonTest.runTestOnce(JythonTest.java:120
> > > )
> > > at
> > > org.apache.qpid.proton.JythonTest.test(JythonTest.java:95)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > Method)
> > > at
> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorI
> > > mpl.java:57)
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodA
> > > ccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:606)
> > > at
> > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Frame
> > > workMethod.java:45)
> > > at
> > > org.junit.internal.runners.model.ReflectiveCallable.run(Reflectiv
> > > eCallable.java:15)
> > > at
> > > org.junit.runners.model.FrameworkMethod.invokeExplosively(Framewo
> > > rkMethod.java:42)
> > > at
> > > org.junit.internal.runners.statements.InvokeMethod.evaluate(Invok
> > > eMethod.java:20)
> > > at
> > > org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> > > at
> > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4Clas
> > > sRunner.java:68)
> > > at
> > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4Clas
> > > sRunner.java:47)
> > > at
> > > org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > > at
> > > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > > at
> > > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > > at
> > > org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > > at
> > > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > > at
> > > org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Pro
> > > vider.java:252)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JU
> > > nit4Provider.java:141)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Prov
> > > ider.java:112)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > Method)
> > > at
> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorI
> > > mpl.java:57)
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodA
> > > ccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:606)
> > > at
> > > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithAr
> > > ray(ReflectionUtils.java:189)
> > > at
> > > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.in
> > > voke(ProviderFactory.java:165)
> > > at
> > > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(P
> > > roviderFactory.java:85)
> > > at
> > > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(
> > > ForkedBooter.java:115)
> > > at
> > > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.j
> > > ava:75)
> > >
> > >
> > > Results :
> > >
> > > Failed tests: test(org.apache.qpid.proton.JythonTest): Caught
> > > PyException on invocation number 2: Traceback (most recent call
> > > last):(..)
> > >
> > > Tests run: 9, Failures: 1, Errors: 0, Skipped: 0
> > >
> > > [INFO]
> > > -----------------------------------------------------------------
> > > -------
> > > [INFO] Reactor Summary:
> > > [INFO]
> > > [INFO] proton-project ....................................
> > > SUCCESS
> > > [1.207s]
> > > [INFO] proton-j ..........................................
> > > SUCCESS
> > > [2.310s]
> > > [INFO] proton-jms ........................................
> > > SUCCESS
> > > [0.623s]
> > > [INFO] proton-hawtdispatch ...............................
> > > SUCCESS
> > > [1.558s]
> > > [INFO] proton-tests ......................................
> > > FAILURE
> > > [5.737s]
> > > [INFO] proton-j-messenger-example ........................
> > > SKIPPED
> > > [INFO]
> > > -----------------------------------------------------------------
> > > -------
> > > [INFO] BUILD FAILURE
> > > [INFO]
> > > -----------------------------------------------------------------
> > > -------
> > > [INFO] Total time: 11.637s
> > > [INFO] Finished at: Tue Jun 09 16:49:34 BST 2015
> > > [INFO] Final Memory: 14M/163M
> > > [INFO]
> > > -----------------------------------------------------------------
> > > -------
Re: something rotten in the state of... something or other
Posted by Robbie Gemmell <ro...@gmail.com>.
I'm not seeing that currently, but I have seen similar sort of things
a couple of times in the past.
As you mention, some files get created in the source tree (presumably
by or due to use of Jython), outwith the normal build areas they would
be (which would lead to them being cleaned up), and I think that is
part of the problem sometimes. If the shim, binding or test files get
updated in certain ways, some bits can become out of sync, leading to
the type of issue you saw.
CI doesnt see the issue as it blows away all unversioned files before
each update. If I see it locally I've just used git-clean to tidy up
my checkout. I'm not sure what we can do otherwise except put
something togehter that targets all the extraneous files and removes
them.
Robbie
On 9 June 2015 at 16:57, Gordon Sim <gs...@redhat.com> wrote:
> I've recently started seeing errors[1] when running tests due to left over
> artefacts of previous builds. This happens even for a completely clean build
> directory, as some of the offending artefacts seem to be created in the
> source tree.
>
> Jython seems to be trying and failing to load cproton. With a completely
> clean source and build tree, everything passes, but it is kind of annoying
> to have to rely on that. Is anyone else seeing anything similar? Any ideas
> as to the cause (I've only seen it happening quite recently) or possible
> cures?
>
>
> [1]:
>
>> -------------------------------------------------------
>> T E S T S
>> -------------------------------------------------------
>> Running org.apache.qpid.proton.InteropTest
>> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.119 sec
>> Running org.apache.qpid.proton.JythonTest
>> 2015-06-09 16:49:29.705 INFO About to call Jython test script:
>> '/home/gordon/projects/proton-git/tests/python/proton-test' with
>> '/home/gordon/projects/proton-git/tests/python' added to Jython path
>> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.207 sec
>> <<< FAILURE!
>> test(org.apache.qpid.proton.JythonTest) Time elapsed: 5.203 sec <<<
>> FAILURE!
>> java.lang.AssertionError: Caught PyException on invocation number 2:
>> Traceback (most recent call last):
>> File "/home/gordon/projects/proton-git/tests/python/proton-test", line
>> 616, in <module>
>> m = __import__(name, None, None, ["dummy"])
>> File
>> "/home/gordon/projects/proton-git/tests/python/proton_tests/__init__.py",
>> line 20, in <module>
>> import proton_tests.codec
>> File
>> "/home/gordon/projects/proton-git/tests/python/proton_tests/codec.py", line
>> 20, in <module>
>> import os, common, sys
>> File
>> "/home/gordon/projects/proton-git/tests/python/proton_tests/common.py", line
>> 26, in <module>
>> from proton import Connection, Transport, SASL, Endpoint, Delivery,
>> SSL
>> File
>> "/home/gordon/projects/proton-git/tests/../proton-c/bindings/python/proton/__init__.py",
>> line 33, in <module>
>> from cproton import *
>> File
>> "/home/gordon/projects/proton-git/tests/../proton-c/bindings/python/cproton.py",
>> line 29, in <module>
>> import _cproton
>> ImportError: No module named _cproton
>> with message: null
>> at org.junit.Assert.fail(Assert.java:93)
>> at
>> org.apache.qpid.proton.JythonTest.runTestOnce(JythonTest.java:120)
>> at org.apache.qpid.proton.JythonTest.test(JythonTest.java:95)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>> at
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>> at
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>> at
>> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>> at
>> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>> at
>> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
>> at
>> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
>> at
>> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
>> at
>> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
>> at
>> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
>> at
>> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
>> at
>> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
>>
>>
>> Results :
>>
>> Failed tests: test(org.apache.qpid.proton.JythonTest): Caught
>> PyException on invocation number 2: Traceback (most recent call last):(..)
>>
>> Tests run: 9, Failures: 1, Errors: 0, Skipped: 0
>>
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] proton-project .................................... SUCCESS
>> [1.207s]
>> [INFO] proton-j .......................................... SUCCESS
>> [2.310s]
>> [INFO] proton-jms ........................................ SUCCESS
>> [0.623s]
>> [INFO] proton-hawtdispatch ............................... SUCCESS
>> [1.558s]
>> [INFO] proton-tests ...................................... FAILURE
>> [5.737s]
>> [INFO] proton-j-messenger-example ........................ SKIPPED
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD FAILURE
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 11.637s
>> [INFO] Finished at: Tue Jun 09 16:49:34 BST 2015
>> [INFO] Final Memory: 14M/163M
>> [INFO]
>> ------------------------------------------------------------------------
Re: something rotten in the state of... something or other
Posted by Gordon Sim <gs...@redhat.com>.
On 06/11/2015 09:41 PM, Flavio Percoco wrote:
> On 11/06/15 16:08 +0100, Gordon Sim wrote:
>> On 06/11/2015 02:39 PM, Flavio Percoco wrote:
>>> On 11/06/15 09:33 -0400, Ken Giusti wrote:
>>>> Yeah, jython's PYTHONPATH points to that directory - it has to in
>>>> order to pick up the python sources.
>>>>
>>>> we really should clean that cproton.py up. In fact, we probably
>>>> shouldn't be writing generated files anywhere in the source tree.
>>>> Everything should go under the build directory where cmake is run.
>>>
>>> It's actually something that swig does itself when building the python
>>> extension, I don't think there's much we can do unless we move
>>> cproton.i somewhere else, which is not a crazy thing to do.
>>
>> The cmake based build generates it under the build tree. The swig
>> command seems to have an -outdir option, maybe we could use?
>
> Unfortunately, distutils doesn't have the right abstraction to change
> the output dir for swig. However, this should fix the current issue:
>
> https://reviews.apache.org/r/35369/
>
> I believe we should still find a better place to put those files but
> we can do that later.
That fixes it! I've committed your patch, thanks again Flavio!
Re: something rotten in the state of... something or other
Posted by Flavio Percoco <fl...@redhat.com>.
On 11/06/15 16:08 +0100, Gordon Sim wrote:
>On 06/11/2015 02:39 PM, Flavio Percoco wrote:
>>On 11/06/15 09:33 -0400, Ken Giusti wrote:
>>>Yeah, jython's PYTHONPATH points to that directory - it has to in
>>>order to pick up the python sources.
>>>
>>>we really should clean that cproton.py up. In fact, we probably
>>>shouldn't be writing generated files anywhere in the source tree.
>>>Everything should go under the build directory where cmake is run.
>>
>>It's actually something that swig does itself when building the python
>>extension, I don't think there's much we can do unless we move
>>cproton.i somewhere else, which is not a crazy thing to do.
>
>The cmake based build generates it under the build tree. The swig
>command seems to have an -outdir option, maybe we could use?
Unfortunately, distutils doesn't have the right abstraction to change
the output dir for swig. However, this should fix the current issue:
https://reviews.apache.org/r/35369/
I believe we should still find a better place to put those files but
we can do that later.
Cheers,
Flavio
>
>>I tried moving it under a different package but that ends up in the
>>build step copying the whole package into site-packages instead of
>>just copying the generated `cproton.py` module.
>>
>>That said, distutils has enough hooks to make something like this
>>work, I'll take a look.
>>
>>Cheers,
>>Flavio
>>
>>P.S: I was finally able to replicate this issue.
>
--
@flaper87
Flavio Percoco
Re: something rotten in the state of... something or other
Posted by Gordon Sim <gs...@redhat.com>.
On 06/11/2015 02:39 PM, Flavio Percoco wrote:
> On 11/06/15 09:33 -0400, Ken Giusti wrote:
>> Yeah, jython's PYTHONPATH points to that directory - it has to in
>> order to pick up the python sources.
>>
>> we really should clean that cproton.py up. In fact, we probably
>> shouldn't be writing generated files anywhere in the source tree.
>> Everything should go under the build directory where cmake is run.
>
> It's actually something that swig does itself when building the python
> extension, I don't think there's much we can do unless we move
> cproton.i somewhere else, which is not a crazy thing to do.
The cmake based build generates it under the build tree. The swig
command seems to have an -outdir option, maybe we could use?
> I tried moving it under a different package but that ends up in the
> build step copying the whole package into site-packages instead of
> just copying the generated `cproton.py` module.
>
> That said, distutils has enough hooks to make something like this
> work, I'll take a look.
>
> Cheers,
> Flavio
>
> P.S: I was finally able to replicate this issue.
Re: something rotten in the state of... something or other
Posted by Flavio Percoco <fl...@redhat.com>.
On 11/06/15 09:33 -0400, Ken Giusti wrote:
>Yeah, jython's PYTHONPATH points to that directory - it has to in order to pick up the python sources.
>
>we really should clean that cproton.py up. In fact, we probably shouldn't be writing generated files anywhere in the source tree. Everything should go under the build directory where cmake is run.
It's actually something that swig does itself when building the python
extension, I don't think there's much we can do unless we move
cproton.i somewhere else, which is not a crazy thing to do.
I tried moving it under a different package but that ends up in the
build step copying the whole package into site-packages instead of
just copying the generated `cproton.py` module.
That said, distutils has enough hooks to make something like this
work, I'll take a look.
Cheers,
Flavio
P.S: I was finally able to replicate this issue.
>
>
>----- Original Message -----
>> From: "Gordon Sim" <gs...@redhat.com>
>> To: proton@qpid.apache.org
>> Cc: "Flavio Percoco" <fl...@redhat.com>, flaper87@gmail.com
>> Sent: Wednesday, June 10, 2015 12:56:51 PM
>> Subject: Re: something rotten in the state of... something or other
>>
>> On 06/10/2015 03:24 PM, Flavio Percoco wrote:
>> > On 09/06/15 12:30 -0400, Ken Giusti wrote:
>> >> A betting man would wager it has something to do with the recent
>> >> changes to the python setup.py.
>> >>
>> >> I'll have a look into it.
>> >>
>> >> ----- Original Message -----
>> >>> From: "Gordon Sim" <gs...@redhat.com>
>> >>> To: proton@qpid.apache.org
>> >>> Sent: Tuesday, June 9, 2015 11:57:25 AM
>> >>> Subject: something rotten in the state of... something or other
>> >>>
>> >>> I've recently started seeing errors[1] when running tests due to left
>> >>> over artefacts of previous builds. This happens even for a completely
>> >>> clean build directory, as some of the offending artefacts seem to be
>> >>> created in the source tree.
>> >>>
>> >>> Jython seems to be trying and failing to load cproton. With a completely
>> >>> clean source and build tree, everything passes, but it is kind of
>> >>> annoying to have to rely on that. Is anyone else seeing anything
>> >>> similar? Any ideas as to the cause (I've only seen it happening quite
>> >>> recently) or possible cures?
>> >
>> > I haven't been able to replicate this but I'm afraid it might be my
>> > fault. Does it happen to you every time?
>>
>> Pretty much, yes. On more digging, it appears that the
>> proton-c/bindings/python/cproton.py file in the source tree is causing
>> the problem. It seems to get generated when running the
>> python-tox-tests, and once its there the proton-java tests fail with the
>> error pasted previously.
>>
>> If I delete that file and also delete the .pyc, .class and .pyo objects
>> in the source tree, then the java tests pass again.
>>
>
>--
>-K
--
@flaper87
Flavio Percoco
Re: something rotten in the state of... something or other
Posted by Ken Giusti <kg...@redhat.com>.
Yeah, jython's PYTHONPATH points to that directory - it has to in order to pick up the python sources.
we really should clean that cproton.py up. In fact, we probably shouldn't be writing generated files anywhere in the source tree. Everything should go under the build directory where cmake is run.
----- Original Message -----
> From: "Gordon Sim" <gs...@redhat.com>
> To: proton@qpid.apache.org
> Cc: "Flavio Percoco" <fl...@redhat.com>, flaper87@gmail.com
> Sent: Wednesday, June 10, 2015 12:56:51 PM
> Subject: Re: something rotten in the state of... something or other
>
> On 06/10/2015 03:24 PM, Flavio Percoco wrote:
> > On 09/06/15 12:30 -0400, Ken Giusti wrote:
> >> A betting man would wager it has something to do with the recent
> >> changes to the python setup.py.
> >>
> >> I'll have a look into it.
> >>
> >> ----- Original Message -----
> >>> From: "Gordon Sim" <gs...@redhat.com>
> >>> To: proton@qpid.apache.org
> >>> Sent: Tuesday, June 9, 2015 11:57:25 AM
> >>> Subject: something rotten in the state of... something or other
> >>>
> >>> I've recently started seeing errors[1] when running tests due to left
> >>> over artefacts of previous builds. This happens even for a completely
> >>> clean build directory, as some of the offending artefacts seem to be
> >>> created in the source tree.
> >>>
> >>> Jython seems to be trying and failing to load cproton. With a completely
> >>> clean source and build tree, everything passes, but it is kind of
> >>> annoying to have to rely on that. Is anyone else seeing anything
> >>> similar? Any ideas as to the cause (I've only seen it happening quite
> >>> recently) or possible cures?
> >
> > I haven't been able to replicate this but I'm afraid it might be my
> > fault. Does it happen to you every time?
>
> Pretty much, yes. On more digging, it appears that the
> proton-c/bindings/python/cproton.py file in the source tree is causing
> the problem. It seems to get generated when running the
> python-tox-tests, and once its there the proton-java tests fail with the
> error pasted previously.
>
> If I delete that file and also delete the .pyc, .class and .pyo objects
> in the source tree, then the java tests pass again.
>
--
-K
Re: something rotten in the state of... something or other
Posted by Gordon Sim <gs...@redhat.com>.
On 06/10/2015 03:24 PM, Flavio Percoco wrote:
> On 09/06/15 12:30 -0400, Ken Giusti wrote:
>> A betting man would wager it has something to do with the recent
>> changes to the python setup.py.
>>
>> I'll have a look into it.
>>
>> ----- Original Message -----
>>> From: "Gordon Sim" <gs...@redhat.com>
>>> To: proton@qpid.apache.org
>>> Sent: Tuesday, June 9, 2015 11:57:25 AM
>>> Subject: something rotten in the state of... something or other
>>>
>>> I've recently started seeing errors[1] when running tests due to left
>>> over artefacts of previous builds. This happens even for a completely
>>> clean build directory, as some of the offending artefacts seem to be
>>> created in the source tree.
>>>
>>> Jython seems to be trying and failing to load cproton. With a completely
>>> clean source and build tree, everything passes, but it is kind of
>>> annoying to have to rely on that. Is anyone else seeing anything
>>> similar? Any ideas as to the cause (I've only seen it happening quite
>>> recently) or possible cures?
>
> I haven't been able to replicate this but I'm afraid it might be my
> fault. Does it happen to you every time?
Pretty much, yes. On more digging, it appears that the
proton-c/bindings/python/cproton.py file in the source tree is causing
the problem. It seems to get generated when running the
python-tox-tests, and once its there the proton-java tests fail with the
error pasted previously.
If I delete that file and also delete the .pyc, .class and .pyo objects
in the source tree, then the java tests pass again.
Re: something rotten in the state of... something or other
Posted by Flavio Percoco <fl...@redhat.com>.
On 09/06/15 12:30 -0400, Ken Giusti wrote:
>A betting man would wager it has something to do with the recent changes to the python setup.py.
>
>I'll have a look into it.
>
>----- Original Message -----
>> From: "Gordon Sim" <gs...@redhat.com>
>> To: proton@qpid.apache.org
>> Sent: Tuesday, June 9, 2015 11:57:25 AM
>> Subject: something rotten in the state of... something or other
>>
>> I've recently started seeing errors[1] when running tests due to left
>> over artefacts of previous builds. This happens even for a completely
>> clean build directory, as some of the offending artefacts seem to be
>> created in the source tree.
>>
>> Jython seems to be trying and failing to load cproton. With a completely
>> clean source and build tree, everything passes, but it is kind of
>> annoying to have to rely on that. Is anyone else seeing anything
>> similar? Any ideas as to the cause (I've only seen it happening quite
>> recently) or possible cures?
I haven't been able to replicate this but I'm afraid it might be my
fault. Does it happen to you every time?
Thanks,
Flavio
>>
>>
>> [1]:
>>
>> > -------------------------------------------------------
>> > T E S T S
>> > -------------------------------------------------------
>> > Running org.apache.qpid.proton.InteropTest
>> > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.119 sec
>> > Running org.apache.qpid.proton.JythonTest
>> > 2015-06-09 16:49:29.705 INFO About to call Jython test script:
>> > '/home/gordon/projects/proton-git/tests/python/proton-test' with
>> > '/home/gordon/projects/proton-git/tests/python' added to Jython path
>> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.207 sec
>> > <<< FAILURE!
>> > test(org.apache.qpid.proton.JythonTest) Time elapsed: 5.203 sec <<<
>> > FAILURE!
>> > java.lang.AssertionError: Caught PyException on invocation number 2:
>> > Traceback (most recent call last):
>> > File "/home/gordon/projects/proton-git/tests/python/proton-test", line
>> > 616, in <module>
>> > m = __import__(name, None, None, ["dummy"])
>> > File
>> > "/home/gordon/projects/proton-git/tests/python/proton_tests/__init__.py",
>> > line 20, in <module>
>> > import proton_tests.codec
>> > File
>> > "/home/gordon/projects/proton-git/tests/python/proton_tests/codec.py",
>> > line 20, in <module>
>> > import os, common, sys
>> > File
>> > "/home/gordon/projects/proton-git/tests/python/proton_tests/common.py",
>> > line 26, in <module>
>> > from proton import Connection, Transport, SASL, Endpoint, Delivery, SSL
>> > File
>> > "/home/gordon/projects/proton-git/tests/../proton-c/bindings/python/proton/__init__.py",
>> > line 33, in <module>
>> > from cproton import *
>> > File
>> > "/home/gordon/projects/proton-git/tests/../proton-c/bindings/python/cproton.py",
>> > line 29, in <module>
>> > import _cproton
>> > ImportError: No module named _cproton
>> > with message: null
>> > at org.junit.Assert.fail(Assert.java:93)
>> > at org.apache.qpid.proton.JythonTest.runTestOnce(JythonTest.java:120)
>> > at org.apache.qpid.proton.JythonTest.test(JythonTest.java:95)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> > at
>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > at java.lang.reflect.Method.invoke(Method.java:606)
>> > at
>> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>> > at
>> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>> > at
>> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>> > at
>> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>> > at
>> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>> > at
>> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>> > at
>> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
>> > at
>> > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
>> > at
>> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> > at
>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > at java.lang.reflect.Method.invoke(Method.java:606)
>> > at
>> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
>> > at
>> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
>> > at
>> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
>> > at
>> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
>> > at
>> > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
>> >
>> >
>> > Results :
>> >
>> > Failed tests: test(org.apache.qpid.proton.JythonTest): Caught PyException
>> > on invocation number 2: Traceback (most recent call last):(..)
>> >
>> > Tests run: 9, Failures: 1, Errors: 0, Skipped: 0
>> >
>> > [INFO]
>> > ------------------------------------------------------------------------
>> > [INFO] Reactor Summary:
>> > [INFO]
>> > [INFO] proton-project .................................... SUCCESS [1.207s]
>> > [INFO] proton-j .......................................... SUCCESS [2.310s]
>> > [INFO] proton-jms ........................................ SUCCESS [0.623s]
>> > [INFO] proton-hawtdispatch ............................... SUCCESS [1.558s]
>> > [INFO] proton-tests ...................................... FAILURE [5.737s]
>> > [INFO] proton-j-messenger-example ........................ SKIPPED
>> > [INFO]
>> > ------------------------------------------------------------------------
>> > [INFO] BUILD FAILURE
>> > [INFO]
>> > ------------------------------------------------------------------------
>> > [INFO] Total time: 11.637s
>> > [INFO] Finished at: Tue Jun 09 16:49:34 BST 2015
>> > [INFO] Final Memory: 14M/163M
>> > [INFO]
>> > ------------------------------------------------------------------------
>>
>
>--
>-K
--
@flaper87
Flavio Percoco
Re: something rotten in the state of... something or other
Posted by Ken Giusti <kg...@redhat.com>.
A betting man would wager it has something to do with the recent changes to the python setup.py.
I'll have a look into it.
----- Original Message -----
> From: "Gordon Sim" <gs...@redhat.com>
> To: proton@qpid.apache.org
> Sent: Tuesday, June 9, 2015 11:57:25 AM
> Subject: something rotten in the state of... something or other
>
> I've recently started seeing errors[1] when running tests due to left
> over artefacts of previous builds. This happens even for a completely
> clean build directory, as some of the offending artefacts seem to be
> created in the source tree.
>
> Jython seems to be trying and failing to load cproton. With a completely
> clean source and build tree, everything passes, but it is kind of
> annoying to have to rely on that. Is anyone else seeing anything
> similar? Any ideas as to the cause (I've only seen it happening quite
> recently) or possible cures?
>
>
> [1]:
>
> > -------------------------------------------------------
> > T E S T S
> > -------------------------------------------------------
> > Running org.apache.qpid.proton.InteropTest
> > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.119 sec
> > Running org.apache.qpid.proton.JythonTest
> > 2015-06-09 16:49:29.705 INFO About to call Jython test script:
> > '/home/gordon/projects/proton-git/tests/python/proton-test' with
> > '/home/gordon/projects/proton-git/tests/python' added to Jython path
> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.207 sec
> > <<< FAILURE!
> > test(org.apache.qpid.proton.JythonTest) Time elapsed: 5.203 sec <<<
> > FAILURE!
> > java.lang.AssertionError: Caught PyException on invocation number 2:
> > Traceback (most recent call last):
> > File "/home/gordon/projects/proton-git/tests/python/proton-test", line
> > 616, in <module>
> > m = __import__(name, None, None, ["dummy"])
> > File
> > "/home/gordon/projects/proton-git/tests/python/proton_tests/__init__.py",
> > line 20, in <module>
> > import proton_tests.codec
> > File
> > "/home/gordon/projects/proton-git/tests/python/proton_tests/codec.py",
> > line 20, in <module>
> > import os, common, sys
> > File
> > "/home/gordon/projects/proton-git/tests/python/proton_tests/common.py",
> > line 26, in <module>
> > from proton import Connection, Transport, SASL, Endpoint, Delivery, SSL
> > File
> > "/home/gordon/projects/proton-git/tests/../proton-c/bindings/python/proton/__init__.py",
> > line 33, in <module>
> > from cproton import *
> > File
> > "/home/gordon/projects/proton-git/tests/../proton-c/bindings/python/cproton.py",
> > line 29, in <module>
> > import _cproton
> > ImportError: No module named _cproton
> > with message: null
> > at org.junit.Assert.fail(Assert.java:93)
> > at org.apache.qpid.proton.JythonTest.runTestOnce(JythonTest.java:120)
> > at org.apache.qpid.proton.JythonTest.test(JythonTest.java:95)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> > at
> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> > at
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> > at
> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
> > at
> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
> > at
> > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
> >
> >
> > Results :
> >
> > Failed tests: test(org.apache.qpid.proton.JythonTest): Caught PyException
> > on invocation number 2: Traceback (most recent call last):(..)
> >
> > Tests run: 9, Failures: 1, Errors: 0, Skipped: 0
> >
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Reactor Summary:
> > [INFO]
> > [INFO] proton-project .................................... SUCCESS [1.207s]
> > [INFO] proton-j .......................................... SUCCESS [2.310s]
> > [INFO] proton-jms ........................................ SUCCESS [0.623s]
> > [INFO] proton-hawtdispatch ............................... SUCCESS [1.558s]
> > [INFO] proton-tests ...................................... FAILURE [5.737s]
> > [INFO] proton-j-messenger-example ........................ SKIPPED
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] BUILD FAILURE
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Total time: 11.637s
> > [INFO] Finished at: Tue Jun 09 16:49:34 BST 2015
> > [INFO] Final Memory: 14M/163M
> > [INFO]
> > ------------------------------------------------------------------------
>
--
-K