You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Joshua Cohen <jc...@apache.org> on 2016/12/06 01:57:06 UTC
Re: Review Request 53836: Get pants using the same thrift binary as
gradle.
> On Nov. 17, 2016, 11:28 p.m., Joshua Cohen wrote:
> > I'm seeing intermittent failures running the e2e tests with this patch applied. That's not necessarily alarming, as we've had reports of this on master as well, however the behavior in the event of these failures is different with this patch applied. See this gist which contains the full output: https://gist.github.com/jcohen/ee69208fbca714d3f3437e4039961edf. Note the failure about 80% through, followed by some strange compile issues?
>
> John Sirois wrote:
> Can you click on a line number and paste that url? - not seeing it.
>
> Joshua Cohen wrote:
> I think it's just reviewboard counting the period as part of the url? try this: https://gist.github.com/jcohen/ee69208fbca714d3f3437e4039961edf
>
> John Sirois wrote:
> Ah - my miscommunication. URL worked fine, I thought clicking on a line number would ammend the url to an anchor that would skip me down to the error you saw. Moot since the gist is truncated and you have to view it raw to see the whole thing (as I now realize).
>
>
> So:
> ```
> :commons:generateThriftResources UP-TO-DATE
> :commons:processResources
> :commons:classes
> :commons:jar
> :compileJava
> /home/vagrant/aurora/src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java:82: error: cannot find symbol
> class WriteAheadStorage extends WriteAheadStorageForwarder implements
> ^
> symbol: class WriteAheadStorageForwarder
> error: Problem loading existing flags on compile time classpath: java.io.EOFException: Unexpected end of ZLIB input stream
> at java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:417)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
> at java.io.InputStreamReader.read(InputStreamReader.java:184)
> at com.google.common.io.MultiReader.read(MultiReader.java:55)
> at com.google.common.io.LineReader.readLine(LineReader.java:77)
> at com.google.common.io.CharStreams.readLines(CharStreams.java:146)
> at org.apache.aurora.common.args.apt.Configuration.load(Configuration.java:409)
> at org.apache.aurora.common.args.apt.Configuration.load(Configuration.java:325)
> at org.apache.aurora.common.args.apt.CmdLineProcessor$1.get(CmdLineProcessor.java:103)
> at org.apache.aurora.common.args.apt.CmdLineProcessor$1.get(CmdLineProcessor.java:100)
> at com.google.common.base.Suppliers$MemoizingSupplier.get(Suppliers.java:120)
> at org.apache.aurora.common.args.apt.CmdLineProcessor.process(CmdLineProcessor.java:174)
> at com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:794)
> at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:705)
> at com.sun.tools.javac.processing.JavacProcessingEnvironment.access$1800(JavacProcessingEnvironment.java:91)
> at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1035)
> at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1176)
> at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1170)
> at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:856)
> at com.sun.tools.javac.main.Main.compile(Main.java:523)
> ```
>
> Which has to be unrelated to these changes?
>
> John Sirois wrote:
> Basically the error above is an annotation processor trying to read classpath resources and encountering a jar on the classpath that is truncated.
>
> Joshua Cohen wrote:
> The odd thing is I'm able to reproduce about 50% of the time with a clean vagrant image (i.e. vagrant destroy before running the e2e tests). But it only seems to be failing in this particular way with this patch applied.
>
> Probably we need a broader ticket to investigate the recent e2e flakiness though, as regardless of this patch there are issues on master.
>
> John Sirois wrote:
> OK - Since I seem to be blocked on this e2e flakiness, I'll see if I can find some time in the next few days to file and take a look at the problem.
fwiw, in passing I traced the above e2e error to gradle daemon, when I get some time I'll investigate if there's a fix other than disabling the daemon. Until then, I don't think it should block this change.
- Joshua
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53836/#review156262
-----------------------------------------------------------
On Nov. 17, 2016, 4:46 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53836/
> -----------------------------------------------------------
>
> (Updated Nov. 17, 2016, 4:46 a.m.)
>
>
> Review request for Aurora, Joshua Cohen, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This required an upgrade to the latest pants dev release to correct
> an issue with the setup.py binary packager we use to generate sdists.
>
> This is for sanity sake, and, once the TODO in
> `build-support/thrift/prepare_binary.sh` is addressed, it will also
> allow pants-patch-free addition of new platforms (thinking ARM).
>
> api/src/main/thrift/org/apache/aurora/gen/BUILD | 3 +++
> api/src/main/thrift/org/apache/thermos/BUILD | 1 +
> build-support/thrift/.gitignore | 1 +
> build-support/thrift/AURORA-1727.compiler.cpp.Makefile.in.patch | 42 +++++++++++++------------------
> build-support/thrift/AURORA-1727.lib.py.setup.py.patch | 45 +++++++++++++++++++++++++++++++++
> {api/src/main/thrift/org/apache/thermos => build-support/thrift}/BUILD | 18 ++++---------
> build-support/thrift/Makefile | 2 +-
> build-support/thrift/prepare_binary.sh | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++
> build-support/thrift/thriftw | 12 +++++++--
> pants.ini | 7 +++++-
> 10 files changed, 158 insertions(+), 41 deletions(-)
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/BUILD f4004a3ca36078c6d24991db4beb68903a05652c
> api/src/main/thrift/org/apache/thermos/BUILD cb655a2fd35d22f7d7e80c4311742bad763d8614
> build-support/thrift/.gitignore 9a3adb69210ba3dbf3c1f408895561da37e8f4c3
> build-support/thrift/AURORA-1727.compiler.cpp.Makefile.in.patch b69e3fef137cd73c6f2b73201463a0705ef8082a
> build-support/thrift/AURORA-1727.lib.py.setup.py.patch PRE-CREATION
> build-support/thrift/BUILD PRE-CREATION
> build-support/thrift/Makefile 48b174ad622288d2738a5fa37bbb72385fcc3a27
> build-support/thrift/prepare_binary.sh PRE-CREATION
> build-support/thrift/thriftw 50d6dfdeb16ca8bf14aaff7aa826e3d69c5e13f0
> pants.ini cecdb277f327f77b2652f76a30fc8d4ffd9ff1db
>
> Diff: https://reviews.apache.org/r/53836/diff/
>
>
> Testing
> -------
>
> Locally green:
> ```
> rm -rf ~/.cache/pants .cache/
> git clean -fdx build-support/thrift
> ./pants clean-all
> ./build-support/jenkins/build.sh
>
> vagrant ssh --command "rm -rf ~/.cache/pants && cd aurora && rm -rf .cache/ && ./pants clean-all"
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> ./build-support/python/make-pycharm-virtualenv
> ```
>
>
> Thanks,
>
> John Sirois
>
>