You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Alex Amato <aj...@google.com> on 2019/01/23 19:05:36 UTC
compileJava broken on master see: BEAM-6495
https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
Any ideas, how this got through the precommit?
> Task :beam-sdks-java-extensions-sql:compileJava FAILED
/usr/local/google/home/ajamato/go/src/
github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
error: cannot find symbol
import
org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
^
symbol: class BeamSqlParserImpl
location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
1 error
Re: compileJava broken on master see: BEAM-6495
Posted by Ruoyun Huang <ru...@google.com>.
On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com> wrote:
> Okay, make sense perhaps we can somehow make it fail when it fails to
> generate the dep, rather than when compiling the java code later on
>
That would be a good improvement on error message. :-)
Or, does it make sense to manually check-in this particular error-prone
auto-gen code until this issue is resolved?
>
> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com> wrote:
>
>> ParserImpl is autogenerated by Calcite at build time. It seems that
>> there's a race condition there and it sometimes fails. Rerunning the build
>> works for me.
>>
>> Regards,
>> Anton
>>
>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com> wrote:
>>
>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>
>>> Any ideas, how this got through the precommit?
>>>
>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>
>>> /usr/local/google/home/ajamato/go/src/
>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>> error: cannot find symbol
>>>
>>> import
>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>
>>> ^
>>>
>>> symbol: class BeamSqlParserImpl
>>>
>>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>
>>> 1 error
>>>
>>>
--
================
Ruoyun Huang
Re: compileJava broken on master see: BEAM-6495
Posted by Michael Luckey <ad...@gmail.com>.
Unfortunately, I am not able to reproduce the issue. I tried very hard,
even downgrading the JDK. To no avail.
Looking at the logs you provided, everything seems fine. But obviously
there is something.
Did you ever try to switch of incremental compile (on that project)? No
idea, why this should cause the issue, but at least you might try to
exclude any issue here. Other point is the setup of the javacc output,
which is suspicious. But had not hat the time to look at it. Will do...
michel
On Wed, Feb 20, 2019 at 4:23 AM Ryan Williams <ry...@runsascoded.com> wrote:
> Just saw a similar problem for the first time with `
> :beam-sdks-java-io-clickhouse:compileJava`; I got the errors below toward
> the end of a full-project compile, and when I re-ran just that target,
> compilation succeeded.
>
> Intriguingly, extensions/sql and clickhouse are the only two modules that
> use the compileJavacc task to generate code that then gets compileJava'd.
> Something is up with it!
>
> I looked at the debug log from my previous email a bit, but haven't found
> anything.
>
> ```
> > Task :beam-sdks-java-io-clickhouse:compileJava
> …/sdks/java/io/clickhouse/src/main/java/org/apache/beam/sdk/io/clickhouse/TableSchema.java:228:
> error: cannot find symbol
> return new
> org.apache.beam.sdk.io.clickhouse.impl.parser.ColumnTypeParser(
> ^
> symbol: class ColumnTypeParser
> location: package org.apache.beam.sdk.io.clickhouse.impl.parser
> …/sdks/java/io/clickhouse/src/main/java/org/apache/beam/sdk/io/clickhouse/TableSchema.java:248:
> error: cannot find symbol
> new
> org.apache.beam.sdk.io.clickhouse.impl.parser.ColumnTypeParser(
> ^
> symbol: class ColumnTypeParser
> location: package org.apache.beam.sdk.io.clickhouse.impl.parser
> 2 errors
>
> > Task :beam-sdks-java-io-clickhouse:compileJava FAILED
> ```
>
> On Sun, Feb 17, 2019 at 4:00 PM Ryan Williams <ry...@runsascoded.com>
> wrote:
>
>> I just captured a debug log while hitting this issue, running: `./gradlew
>> -d :beam-sdks-java-extensions-sql:compileJava`
>>
>> The log is here
>> <https://gist.githubusercontent.com/ryan-williams/1c0178b7a0f33a25c62a5d5c07d690fa/raw/05f74ca200149bd04e7aa6426bf7de9a651b0a6d/gistfile1.txt>.
>> It's 18MB and doesn't really load in Chrome for me; I'd recommend wget'ing
>> it.
>>
>> I haven't had a chance to look at it too deeply yet.
>>
>> On Sun, Feb 17, 2019 at 1:49 PM Ryan Williams <ry...@runsascoded.com>
>> wrote:
>>
>>> Good to know, Reuven!
>>>
>>> I just hit it again with `--console=verbose`, and have more info:
>>> https://gist.github.com/ryan-williams/252597d9ecd6973378d5cc2e9dabceb1
>>>
>>> It seems that the generated files are correctly in place when the
>>> failing compileJava is invoked.
>>>
>>> Perhaps something is wrong (and nondeterministic) with the classpath of
>>> the `:beam-sdks-java-extensions-sql:compileJava` task?
>>>
>>> I'll see if I can run with more verbose/DEBUG logging and get any more
>>> info.
>>>
>>> On Sun, Feb 17, 2019 at 1:42 PM Reuven Lax <re...@google.com> wrote:
>>>
>>>> I also see this almost every other run.
>>>>
>>>> On Sun, Feb 17, 2019 at 8:24 AM Ryan Williams <ry...@runsascoded.com>
>>>> wrote:
>>>>
>>>>> I still see this about 20% of the time I run
>>>>> `./gradlew compileTestJava spotlessApply checkstyleMain checkstyleTest`
>>>>> (with 8-way parallelism), which I've taken to running as a kind of
>>>>> "pre-PreCommit" (for Java).
>>>>>
>>>>> In total I'd estimate I've seen this error 20 times (across multiple
>>>>> `./gradlew clean`s and spanning several weeks), but never twice in a row: I
>>>>> always immediately re-run, and the error goes away.
>>>>>
>>>>> That seems consistent with the idea that gradle is racing the codegen
>>>>> task against the compile task that requires the codegen's outputs.
>>>>>
>>>>> However, `./gradlew -p sdks/java/extensions/sql -m compileJava | grep
>>>>> compileJavacc` implies that Gradle knows that compilation depends on the
>>>>> codegen having run.
>>>>>
>>>>> Here are three successive invocations of `./gradlew compileTestJava
>>>>> spotlessApply checkstyleMain checkstyleTest`
>>>>> <https://gist.github.com/ryan-williams/6d2735b2484e9459be656abf379533c5>
>>>>> ;
>>>>>
>>>>> - The first one succeeds (and includes output showing that the
>>>>> codegen task, `:beam-sdks-java-extensions-sql:compileJavacc`, was run).
>>>>> - Then, I rebased on one new upstream commit and ran again, and
>>>>> saw the failure.
>>>>> - Immediately re-running saw the failure go away.
>>>>>
>>>>> I guess in that output, tasks that are run but produce no
>>>>> stdout/stderr are elided from the output, so we can't completely tell when
>>>>> the codegen and compile tasks in question are being run. I'll start running
>>>>> with --console=verbose
>>>>> <https://stackoverflow.com/questions/45883963/gradle-4-0-does-not-display-executed-tasks-in-command-line#comment92914258_45889966>,
>>>>> which I think will give a better sense of how this is happening.
>>>>>
>>>>>
>>>>> On Fri, Feb 8, 2019 at 12:41 AM Ryan Williams <ry...@runsascoded.com>
>>>>> wrote:
>>>>>
>>>>>> After your last message, Alex, I saw the issue on a branch with
>>>>>> minimal unrelated changes on top of b4b5495307
>>>>>> <https://github.com/apache/beam/commit/b4b549530730d9e0283ad0cdb203d384a107dfbf> (January
>>>>>> 28).
>>>>>>
>>>>>> I ran `./gradlew clean` for the first time in a while at that time,
>>>>>> and worked happily for 11 days, but just saw the issue again on a branch
>>>>>> rebased on top of 381ab55b59
>>>>>> <https://github.com/apache/beam/commit/381ab55b59> (today):
>>>>>>
>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>> …/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>> error: cannot find symbol
>>>>>> import
>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>> ^
>>>>>> symbol: class BeamSqlParserImpl
>>>>>> location: package
>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>> 1 error
>>>>>>
>>>>>> Every time I've seen it, I immediately re-run the compile, and it
>>>>>> succeeds.
>>>>>>
>>>>>> Perhaps something is still wrong with my environment, but otherwise
>>>>>> it would seem that something is still flaky here.
>>>>>>
>>>>>> I'm compiling on an 8-core macOS machine, fwiw, and usually running
>>>>>> `./gradlew compileTestJava` which compiles many projects concurrently.
>>>>>>
>>>>>> On Mon, Jan 28, 2019 at 4:12 PM Alex Amato <aj...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> If it continues to occur, maybe it is an environmental issue, be
>>>>>>> sure to try to clean as well.
>>>>>>> ./gradlew clean
>>>>>>>
>>>>>>> On Mon, Jan 28, 2019 at 11:12 AM Ryan Williams <ry...@runsascoded.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Yea I was rebased on top of a more recent master than your previous
>>>>>>>> message, when I saw it again. Perhaps I was mistaken. I'll ping here if I
>>>>>>>> see it again, thanks.
>>>>>>>>
>>>>>>>> On Mon, Jan 28, 2019 at 1:52 PM Alex Amato <aj...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> After I did a rebase, it went away for me. So I think that this
>>>>>>>>> should work. Are you saying that you did rebase ontop of master and it
>>>>>>>>> still occurred? Strange.
>>>>>>>>>
>>>>>>>>> On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <
>>>>>>>>> ryan@runsascoded.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hm, I just encountered this again on a branch that based on
>>>>>>>>>> 5b46b02b49 (top of trunk from this afternoon). Is it definitely supposed to
>>>>>>>>>> be fixed?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Please try rebasing from master, I believe this issue has been
>>>>>>>>>>> resolved.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <
>>>>>>>>>>> ryan@runsascoded.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I'm seeing every ≈third `./gradlew compileJava` fail locally
>>>>>>>>>>>> due to this; re-running the commit has always succeeded, so far.
>>>>>>>>>>>>
>>>>>>>>>>>> Sounds like there is not an immediate fix in the works / no one
>>>>>>>>>>>> assigned on the JIRA?
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <
>>>>>>>>>>>> kenn@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> This might connect to vendoring Calcite. It will be easiest,
>>>>>>>>>>>>> and have the best incremental build, if we separate the generated code into
>>>>>>>>>>>>> its own module that has relocation to match the vendored Calcite.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Kenn
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> We don't pre-generate the code as a separate step. Code gen
>>>>>>>>>>>>>> from the SQL parser syntax spec and its compilation happens both during the
>>>>>>>>>>>>>> Beam SQL build task. Splitting the code generation and compilation might
>>>>>>>>>>>>>> not be trivial. We definitely should look into fixing this though.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Anton
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <
>>>>>>>>>>>>>> ajamato@google.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Okay, make sense perhaps we can somehow make it fail when it
>>>>>>>>>>>>>>> fails to generate the dep, rather than when compiling the java code later on
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <
>>>>>>>>>>>>>>> kedin@google.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ParserImpl is autogenerated by Calcite at build time. It
>>>>>>>>>>>>>>>> seems that there's a race condition there and it sometimes fails. Rerunning
>>>>>>>>>>>>>>>> the build works for me.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Anton
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <
>>>>>>>>>>>>>>>> ajamato@google.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>>>>>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>>>>>>>>>>>> error: cannot find symbol
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> import
>>>>>>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ^
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> location: package
>>>>>>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1 error
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Ryan Williams <ry...@runsascoded.com>.
Just saw a similar problem for the first time with `
:beam-sdks-java-io-clickhouse:compileJava`; I got the errors below toward
the end of a full-project compile, and when I re-ran just that target,
compilation succeeded.
Intriguingly, extensions/sql and clickhouse are the only two modules that
use the compileJavacc task to generate code that then gets compileJava'd.
Something is up with it!
I looked at the debug log from my previous email a bit, but haven't found
anything.
```
> Task :beam-sdks-java-io-clickhouse:compileJava
…/sdks/java/io/clickhouse/src/main/java/org/apache/beam/sdk/io/clickhouse/TableSchema.java:228:
error: cannot find symbol
return new
org.apache.beam.sdk.io.clickhouse.impl.parser.ColumnTypeParser(
^
symbol: class ColumnTypeParser
location: package org.apache.beam.sdk.io.clickhouse.impl.parser
…/sdks/java/io/clickhouse/src/main/java/org/apache/beam/sdk/io/clickhouse/TableSchema.java:248:
error: cannot find symbol
new
org.apache.beam.sdk.io.clickhouse.impl.parser.ColumnTypeParser(
^
symbol: class ColumnTypeParser
location: package org.apache.beam.sdk.io.clickhouse.impl.parser
2 errors
> Task :beam-sdks-java-io-clickhouse:compileJava FAILED
```
On Sun, Feb 17, 2019 at 4:00 PM Ryan Williams <ry...@runsascoded.com> wrote:
> I just captured a debug log while hitting this issue, running: `./gradlew
> -d :beam-sdks-java-extensions-sql:compileJava`
>
> The log is here
> <https://gist.githubusercontent.com/ryan-williams/1c0178b7a0f33a25c62a5d5c07d690fa/raw/05f74ca200149bd04e7aa6426bf7de9a651b0a6d/gistfile1.txt>.
> It's 18MB and doesn't really load in Chrome for me; I'd recommend wget'ing
> it.
>
> I haven't had a chance to look at it too deeply yet.
>
> On Sun, Feb 17, 2019 at 1:49 PM Ryan Williams <ry...@runsascoded.com>
> wrote:
>
>> Good to know, Reuven!
>>
>> I just hit it again with `--console=verbose`, and have more info:
>> https://gist.github.com/ryan-williams/252597d9ecd6973378d5cc2e9dabceb1
>>
>> It seems that the generated files are correctly in place when the failing
>> compileJava is invoked.
>>
>> Perhaps something is wrong (and nondeterministic) with the classpath of
>> the `:beam-sdks-java-extensions-sql:compileJava` task?
>>
>> I'll see if I can run with more verbose/DEBUG logging and get any more
>> info.
>>
>> On Sun, Feb 17, 2019 at 1:42 PM Reuven Lax <re...@google.com> wrote:
>>
>>> I also see this almost every other run.
>>>
>>> On Sun, Feb 17, 2019 at 8:24 AM Ryan Williams <ry...@runsascoded.com>
>>> wrote:
>>>
>>>> I still see this about 20% of the time I run `./gradlew compileTestJava
>>>> spotlessApply checkstyleMain checkstyleTest` (with 8-way parallelism),
>>>> which I've taken to running as a kind of "pre-PreCommit" (for Java).
>>>>
>>>> In total I'd estimate I've seen this error 20 times (across multiple
>>>> `./gradlew clean`s and spanning several weeks), but never twice in a row: I
>>>> always immediately re-run, and the error goes away.
>>>>
>>>> That seems consistent with the idea that gradle is racing the codegen
>>>> task against the compile task that requires the codegen's outputs.
>>>>
>>>> However, `./gradlew -p sdks/java/extensions/sql -m compileJava | grep
>>>> compileJavacc` implies that Gradle knows that compilation depends on the
>>>> codegen having run.
>>>>
>>>> Here are three successive invocations of `./gradlew compileTestJava
>>>> spotlessApply checkstyleMain checkstyleTest`
>>>> <https://gist.github.com/ryan-williams/6d2735b2484e9459be656abf379533c5>
>>>> ;
>>>>
>>>> - The first one succeeds (and includes output showing that the
>>>> codegen task, `:beam-sdks-java-extensions-sql:compileJavacc`, was run).
>>>> - Then, I rebased on one new upstream commit and ran again, and saw
>>>> the failure.
>>>> - Immediately re-running saw the failure go away.
>>>>
>>>> I guess in that output, tasks that are run but produce no stdout/stderr
>>>> are elided from the output, so we can't completely tell when the codegen
>>>> and compile tasks in question are being run. I'll start running with
>>>> --console=verbose
>>>> <https://stackoverflow.com/questions/45883963/gradle-4-0-does-not-display-executed-tasks-in-command-line#comment92914258_45889966>,
>>>> which I think will give a better sense of how this is happening.
>>>>
>>>>
>>>> On Fri, Feb 8, 2019 at 12:41 AM Ryan Williams <ry...@runsascoded.com>
>>>> wrote:
>>>>
>>>>> After your last message, Alex, I saw the issue on a branch with
>>>>> minimal unrelated changes on top of b4b5495307
>>>>> <https://github.com/apache/beam/commit/b4b549530730d9e0283ad0cdb203d384a107dfbf> (January
>>>>> 28).
>>>>>
>>>>> I ran `./gradlew clean` for the first time in a while at that time,
>>>>> and worked happily for 11 days, but just saw the issue again on a branch
>>>>> rebased on top of 381ab55b59
>>>>> <https://github.com/apache/beam/commit/381ab55b59> (today):
>>>>>
>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>> …/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>> error: cannot find symbol
>>>>> import
>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>> ^
>>>>> symbol: class BeamSqlParserImpl
>>>>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>> 1 error
>>>>>
>>>>> Every time I've seen it, I immediately re-run the compile, and it
>>>>> succeeds.
>>>>>
>>>>> Perhaps something is still wrong with my environment, but otherwise it
>>>>> would seem that something is still flaky here.
>>>>>
>>>>> I'm compiling on an 8-core macOS machine, fwiw, and usually running
>>>>> `./gradlew compileTestJava` which compiles many projects concurrently.
>>>>>
>>>>> On Mon, Jan 28, 2019 at 4:12 PM Alex Amato <aj...@google.com> wrote:
>>>>>
>>>>>> If it continues to occur, maybe it is an environmental issue, be sure
>>>>>> to try to clean as well.
>>>>>> ./gradlew clean
>>>>>>
>>>>>> On Mon, Jan 28, 2019 at 11:12 AM Ryan Williams <ry...@runsascoded.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yea I was rebased on top of a more recent master than your previous
>>>>>>> message, when I saw it again. Perhaps I was mistaken. I'll ping here if I
>>>>>>> see it again, thanks.
>>>>>>>
>>>>>>> On Mon, Jan 28, 2019 at 1:52 PM Alex Amato <aj...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> After I did a rebase, it went away for me. So I think that this
>>>>>>>> should work. Are you saying that you did rebase ontop of master and it
>>>>>>>> still occurred? Strange.
>>>>>>>>
>>>>>>>> On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <ry...@runsascoded.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hm, I just encountered this again on a branch that based on
>>>>>>>>> 5b46b02b49 (top of trunk from this afternoon). Is it definitely supposed to
>>>>>>>>> be fixed?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Please try rebasing from master, I believe this issue has been
>>>>>>>>>> resolved.
>>>>>>>>>>
>>>>>>>>>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <
>>>>>>>>>> ryan@runsascoded.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I'm seeing every ≈third `./gradlew compileJava` fail locally due
>>>>>>>>>>> to this; re-running the commit has always succeeded, so far.
>>>>>>>>>>>
>>>>>>>>>>> Sounds like there is not an immediate fix in the works / no one
>>>>>>>>>>> assigned on the JIRA?
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> This might connect to vendoring Calcite. It will be easiest,
>>>>>>>>>>>> and have the best incremental build, if we separate the generated code into
>>>>>>>>>>>> its own module that has relocation to match the vendored Calcite.
>>>>>>>>>>>>
>>>>>>>>>>>> Kenn
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> We don't pre-generate the code as a separate step. Code gen
>>>>>>>>>>>>> from the SQL parser syntax spec and its compilation happens both during the
>>>>>>>>>>>>> Beam SQL build task. Splitting the code generation and compilation might
>>>>>>>>>>>>> not be trivial. We definitely should look into fixing this though.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Anton
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <
>>>>>>>>>>>>> ajamato@google.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Okay, make sense perhaps we can somehow make it fail when it
>>>>>>>>>>>>>> fails to generate the dep, rather than when compiling the java code later on
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <
>>>>>>>>>>>>>> kedin@google.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ParserImpl is autogenerated by Calcite at build time. It
>>>>>>>>>>>>>>> seems that there's a race condition there and it sometimes fails. Rerunning
>>>>>>>>>>>>>>> the build works for me.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Anton
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <
>>>>>>>>>>>>>>> ajamato@google.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>>>>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>>>>>>>>>>> error: cannot find symbol
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> import
>>>>>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ^
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> location: package
>>>>>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1 error
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Ryan Williams <ry...@runsascoded.com>.
I just captured a debug log while hitting this issue, running: `./gradlew
-d :beam-sdks-java-extensions-sql:compileJava`
The log is here
<https://gist.githubusercontent.com/ryan-williams/1c0178b7a0f33a25c62a5d5c07d690fa/raw/05f74ca200149bd04e7aa6426bf7de9a651b0a6d/gistfile1.txt>.
It's 18MB and doesn't really load in Chrome for me; I'd recommend wget'ing
it.
I haven't had a chance to look at it too deeply yet.
On Sun, Feb 17, 2019 at 1:49 PM Ryan Williams <ry...@runsascoded.com> wrote:
> Good to know, Reuven!
>
> I just hit it again with `--console=verbose`, and have more info:
> https://gist.github.com/ryan-williams/252597d9ecd6973378d5cc2e9dabceb1
>
> It seems that the generated files are correctly in place when the failing
> compileJava is invoked.
>
> Perhaps something is wrong (and nondeterministic) with the classpath of
> the `:beam-sdks-java-extensions-sql:compileJava` task?
>
> I'll see if I can run with more verbose/DEBUG logging and get any more
> info.
>
> On Sun, Feb 17, 2019 at 1:42 PM Reuven Lax <re...@google.com> wrote:
>
>> I also see this almost every other run.
>>
>> On Sun, Feb 17, 2019 at 8:24 AM Ryan Williams <ry...@runsascoded.com>
>> wrote:
>>
>>> I still see this about 20% of the time I run `./gradlew compileTestJava
>>> spotlessApply checkstyleMain checkstyleTest` (with 8-way parallelism),
>>> which I've taken to running as a kind of "pre-PreCommit" (for Java).
>>>
>>> In total I'd estimate I've seen this error 20 times (across multiple
>>> `./gradlew clean`s and spanning several weeks), but never twice in a row: I
>>> always immediately re-run, and the error goes away.
>>>
>>> That seems consistent with the idea that gradle is racing the codegen
>>> task against the compile task that requires the codegen's outputs.
>>>
>>> However, `./gradlew -p sdks/java/extensions/sql -m compileJava | grep
>>> compileJavacc` implies that Gradle knows that compilation depends on the
>>> codegen having run.
>>>
>>> Here are three successive invocations of `./gradlew compileTestJava
>>> spotlessApply checkstyleMain checkstyleTest`
>>> <https://gist.github.com/ryan-williams/6d2735b2484e9459be656abf379533c5>
>>> ;
>>>
>>> - The first one succeeds (and includes output showing that the
>>> codegen task, `:beam-sdks-java-extensions-sql:compileJavacc`, was run).
>>> - Then, I rebased on one new upstream commit and ran again, and saw
>>> the failure.
>>> - Immediately re-running saw the failure go away.
>>>
>>> I guess in that output, tasks that are run but produce no stdout/stderr
>>> are elided from the output, so we can't completely tell when the codegen
>>> and compile tasks in question are being run. I'll start running with
>>> --console=verbose
>>> <https://stackoverflow.com/questions/45883963/gradle-4-0-does-not-display-executed-tasks-in-command-line#comment92914258_45889966>,
>>> which I think will give a better sense of how this is happening.
>>>
>>>
>>> On Fri, Feb 8, 2019 at 12:41 AM Ryan Williams <ry...@runsascoded.com>
>>> wrote:
>>>
>>>> After your last message, Alex, I saw the issue on a branch with minimal
>>>> unrelated changes on top of b4b5495307
>>>> <https://github.com/apache/beam/commit/b4b549530730d9e0283ad0cdb203d384a107dfbf> (January
>>>> 28).
>>>>
>>>> I ran `./gradlew clean` for the first time in a while at that time, and
>>>> worked happily for 11 days, but just saw the issue again on a branch
>>>> rebased on top of 381ab55b59
>>>> <https://github.com/apache/beam/commit/381ab55b59> (today):
>>>>
>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>> …/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>> error: cannot find symbol
>>>> import
>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>> ^
>>>> symbol: class BeamSqlParserImpl
>>>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>> 1 error
>>>>
>>>> Every time I've seen it, I immediately re-run the compile, and it
>>>> succeeds.
>>>>
>>>> Perhaps something is still wrong with my environment, but otherwise it
>>>> would seem that something is still flaky here.
>>>>
>>>> I'm compiling on an 8-core macOS machine, fwiw, and usually running
>>>> `./gradlew compileTestJava` which compiles many projects concurrently.
>>>>
>>>> On Mon, Jan 28, 2019 at 4:12 PM Alex Amato <aj...@google.com> wrote:
>>>>
>>>>> If it continues to occur, maybe it is an environmental issue, be sure
>>>>> to try to clean as well.
>>>>> ./gradlew clean
>>>>>
>>>>> On Mon, Jan 28, 2019 at 11:12 AM Ryan Williams <ry...@runsascoded.com>
>>>>> wrote:
>>>>>
>>>>>> Yea I was rebased on top of a more recent master than your previous
>>>>>> message, when I saw it again. Perhaps I was mistaken. I'll ping here if I
>>>>>> see it again, thanks.
>>>>>>
>>>>>> On Mon, Jan 28, 2019 at 1:52 PM Alex Amato <aj...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> After I did a rebase, it went away for me. So I think that this
>>>>>>> should work. Are you saying that you did rebase ontop of master and it
>>>>>>> still occurred? Strange.
>>>>>>>
>>>>>>> On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <ry...@runsascoded.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hm, I just encountered this again on a branch that based on
>>>>>>>> 5b46b02b49 (top of trunk from this afternoon). Is it definitely supposed to
>>>>>>>> be fixed?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Please try rebasing from master, I believe this issue has been
>>>>>>>>> resolved.
>>>>>>>>>
>>>>>>>>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <
>>>>>>>>> ryan@runsascoded.com> wrote:
>>>>>>>>>
>>>>>>>>>> I'm seeing every ≈third `./gradlew compileJava` fail locally due
>>>>>>>>>> to this; re-running the commit has always succeeded, so far.
>>>>>>>>>>
>>>>>>>>>> Sounds like there is not an immediate fix in the works / no one
>>>>>>>>>> assigned on the JIRA?
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> This might connect to vendoring Calcite. It will be easiest, and
>>>>>>>>>>> have the best incremental build, if we separate the generated code into its
>>>>>>>>>>> own module that has relocation to match the vendored Calcite.
>>>>>>>>>>>
>>>>>>>>>>> Kenn
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> We don't pre-generate the code as a separate step. Code gen
>>>>>>>>>>>> from the SQL parser syntax spec and its compilation happens both during the
>>>>>>>>>>>> Beam SQL build task. Splitting the code generation and compilation might
>>>>>>>>>>>> not be trivial. We definitely should look into fixing this though.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Anton
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Okay, make sense perhaps we can somehow make it fail when it
>>>>>>>>>>>>> fails to generate the dep, rather than when compiling the java code later on
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> ParserImpl is autogenerated by Calcite at build time. It
>>>>>>>>>>>>>> seems that there's a race condition there and it sometimes fails. Rerunning
>>>>>>>>>>>>>> the build works for me.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Anton
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>>>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>>>>>>>>>> error: cannot find symbol
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> import
>>>>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ^
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> location: package
>>>>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1 error
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Ryan Williams <ry...@runsascoded.com>.
Good to know, Reuven!
I just hit it again with `--console=verbose`, and have more info:
https://gist.github.com/ryan-williams/252597d9ecd6973378d5cc2e9dabceb1
It seems that the generated files are correctly in place when the failing
compileJava is invoked.
Perhaps something is wrong (and nondeterministic) with the classpath of the
`:beam-sdks-java-extensions-sql:compileJava` task?
I'll see if I can run with more verbose/DEBUG logging and get any more info.
On Sun, Feb 17, 2019 at 1:42 PM Reuven Lax <re...@google.com> wrote:
> I also see this almost every other run.
>
> On Sun, Feb 17, 2019 at 8:24 AM Ryan Williams <ry...@runsascoded.com>
> wrote:
>
>> I still see this about 20% of the time I run `./gradlew compileTestJava
>> spotlessApply checkstyleMain checkstyleTest` (with 8-way parallelism),
>> which I've taken to running as a kind of "pre-PreCommit" (for Java).
>>
>> In total I'd estimate I've seen this error 20 times (across multiple
>> `./gradlew clean`s and spanning several weeks), but never twice in a row: I
>> always immediately re-run, and the error goes away.
>>
>> That seems consistent with the idea that gradle is racing the codegen
>> task against the compile task that requires the codegen's outputs.
>>
>> However, `./gradlew -p sdks/java/extensions/sql -m compileJava | grep
>> compileJavacc` implies that Gradle knows that compilation depends on the
>> codegen having run.
>>
>> Here are three successive invocations of `./gradlew compileTestJava
>> spotlessApply checkstyleMain checkstyleTest`
>> <https://gist.github.com/ryan-williams/6d2735b2484e9459be656abf379533c5>;
>>
>> - The first one succeeds (and includes output showing that the
>> codegen task, `:beam-sdks-java-extensions-sql:compileJavacc`, was run).
>> - Then, I rebased on one new upstream commit and ran again, and saw
>> the failure.
>> - Immediately re-running saw the failure go away.
>>
>> I guess in that output, tasks that are run but produce no stdout/stderr
>> are elided from the output, so we can't completely tell when the codegen
>> and compile tasks in question are being run. I'll start running with
>> --console=verbose
>> <https://stackoverflow.com/questions/45883963/gradle-4-0-does-not-display-executed-tasks-in-command-line#comment92914258_45889966>,
>> which I think will give a better sense of how this is happening.
>>
>>
>> On Fri, Feb 8, 2019 at 12:41 AM Ryan Williams <ry...@runsascoded.com>
>> wrote:
>>
>>> After your last message, Alex, I saw the issue on a branch with minimal
>>> unrelated changes on top of b4b5495307
>>> <https://github.com/apache/beam/commit/b4b549530730d9e0283ad0cdb203d384a107dfbf> (January
>>> 28).
>>>
>>> I ran `./gradlew clean` for the first time in a while at that time, and
>>> worked happily for 11 days, but just saw the issue again on a branch
>>> rebased on top of 381ab55b59
>>> <https://github.com/apache/beam/commit/381ab55b59> (today):
>>>
>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>> …/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>> error: cannot find symbol
>>> import
>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>> ^
>>> symbol: class BeamSqlParserImpl
>>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>> 1 error
>>>
>>> Every time I've seen it, I immediately re-run the compile, and it
>>> succeeds.
>>>
>>> Perhaps something is still wrong with my environment, but otherwise it
>>> would seem that something is still flaky here.
>>>
>>> I'm compiling on an 8-core macOS machine, fwiw, and usually running
>>> `./gradlew compileTestJava` which compiles many projects concurrently.
>>>
>>> On Mon, Jan 28, 2019 at 4:12 PM Alex Amato <aj...@google.com> wrote:
>>>
>>>> If it continues to occur, maybe it is an environmental issue, be sure
>>>> to try to clean as well.
>>>> ./gradlew clean
>>>>
>>>> On Mon, Jan 28, 2019 at 11:12 AM Ryan Williams <ry...@runsascoded.com>
>>>> wrote:
>>>>
>>>>> Yea I was rebased on top of a more recent master than your previous
>>>>> message, when I saw it again. Perhaps I was mistaken. I'll ping here if I
>>>>> see it again, thanks.
>>>>>
>>>>> On Mon, Jan 28, 2019 at 1:52 PM Alex Amato <aj...@google.com> wrote:
>>>>>
>>>>>> After I did a rebase, it went away for me. So I think that this
>>>>>> should work. Are you saying that you did rebase ontop of master and it
>>>>>> still occurred? Strange.
>>>>>>
>>>>>> On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <ry...@runsascoded.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hm, I just encountered this again on a branch that based on
>>>>>>> 5b46b02b49 (top of trunk from this afternoon). Is it definitely supposed to
>>>>>>> be fixed?
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Please try rebasing from master, I believe this issue has been
>>>>>>>> resolved.
>>>>>>>>
>>>>>>>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <ry...@runsascoded.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I'm seeing every ≈third `./gradlew compileJava` fail locally due
>>>>>>>>> to this; re-running the commit has always succeeded, so far.
>>>>>>>>>
>>>>>>>>> Sounds like there is not an immediate fix in the works / no one
>>>>>>>>> assigned on the JIRA?
>>>>>>>>>
>>>>>>>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> This might connect to vendoring Calcite. It will be easiest, and
>>>>>>>>>> have the best incremental build, if we separate the generated code into its
>>>>>>>>>> own module that has relocation to match the vendored Calcite.
>>>>>>>>>>
>>>>>>>>>> Kenn
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> We don't pre-generate the code as a separate step. Code gen from
>>>>>>>>>>> the SQL parser syntax spec and its compilation happens both during the Beam
>>>>>>>>>>> SQL build task. Splitting the code generation and compilation might not be
>>>>>>>>>>> trivial. We definitely should look into fixing this though.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Anton
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Okay, make sense perhaps we can somehow make it fail when it
>>>>>>>>>>>> fails to generate the dep, rather than when compiling the java code later on
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> ParserImpl is autogenerated by Calcite at build time. It seems
>>>>>>>>>>>>> that there's a race condition there and it sometimes fails. Rerunning the
>>>>>>>>>>>>> build works for me.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Anton
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>>>>>>>>> error: cannot find symbol
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> import
>>>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ^
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> location: package
>>>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1 error
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Reuven Lax <re...@google.com>.
I also see this almost every other run.
On Sun, Feb 17, 2019 at 8:24 AM Ryan Williams <ry...@runsascoded.com> wrote:
> I still see this about 20% of the time I run `./gradlew compileTestJava
> spotlessApply checkstyleMain checkstyleTest` (with 8-way parallelism),
> which I've taken to running as a kind of "pre-PreCommit" (for Java).
>
> In total I'd estimate I've seen this error 20 times (across multiple
> `./gradlew clean`s and spanning several weeks), but never twice in a row: I
> always immediately re-run, and the error goes away.
>
> That seems consistent with the idea that gradle is racing the codegen task
> against the compile task that requires the codegen's outputs.
>
> However, `./gradlew -p sdks/java/extensions/sql -m compileJava | grep
> compileJavacc` implies that Gradle knows that compilation depends on the
> codegen having run.
>
> Here are three successive invocations of `./gradlew compileTestJava
> spotlessApply checkstyleMain checkstyleTest`
> <https://gist.github.com/ryan-williams/6d2735b2484e9459be656abf379533c5>;
>
> - The first one succeeds (and includes output showing that the codegen
> task, `:beam-sdks-java-extensions-sql:compileJavacc`, was run).
> - Then, I rebased on one new upstream commit and ran again, and saw
> the failure.
> - Immediately re-running saw the failure go away.
>
> I guess in that output, tasks that are run but produce no stdout/stderr
> are elided from the output, so we can't completely tell when the codegen
> and compile tasks in question are being run. I'll start running with
> --console=verbose
> <https://stackoverflow.com/questions/45883963/gradle-4-0-does-not-display-executed-tasks-in-command-line#comment92914258_45889966>,
> which I think will give a better sense of how this is happening.
>
>
> On Fri, Feb 8, 2019 at 12:41 AM Ryan Williams <ry...@runsascoded.com>
> wrote:
>
>> After your last message, Alex, I saw the issue on a branch with minimal
>> unrelated changes on top of b4b5495307
>> <https://github.com/apache/beam/commit/b4b549530730d9e0283ad0cdb203d384a107dfbf> (January
>> 28).
>>
>> I ran `./gradlew clean` for the first time in a while at that time, and
>> worked happily for 11 days, but just saw the issue again on a branch
>> rebased on top of 381ab55b59
>> <https://github.com/apache/beam/commit/381ab55b59> (today):
>>
>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>> …/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>> error: cannot find symbol
>> import
>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>> ^
>> symbol: class BeamSqlParserImpl
>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>> 1 error
>>
>> Every time I've seen it, I immediately re-run the compile, and it
>> succeeds.
>>
>> Perhaps something is still wrong with my environment, but otherwise it
>> would seem that something is still flaky here.
>>
>> I'm compiling on an 8-core macOS machine, fwiw, and usually running
>> `./gradlew compileTestJava` which compiles many projects concurrently.
>>
>> On Mon, Jan 28, 2019 at 4:12 PM Alex Amato <aj...@google.com> wrote:
>>
>>> If it continues to occur, maybe it is an environmental issue, be sure to
>>> try to clean as well.
>>> ./gradlew clean
>>>
>>> On Mon, Jan 28, 2019 at 11:12 AM Ryan Williams <ry...@runsascoded.com>
>>> wrote:
>>>
>>>> Yea I was rebased on top of a more recent master than your previous
>>>> message, when I saw it again. Perhaps I was mistaken. I'll ping here if I
>>>> see it again, thanks.
>>>>
>>>> On Mon, Jan 28, 2019 at 1:52 PM Alex Amato <aj...@google.com> wrote:
>>>>
>>>>> After I did a rebase, it went away for me. So I think that this should
>>>>> work. Are you saying that you did rebase ontop of master and it still
>>>>> occurred? Strange.
>>>>>
>>>>> On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <ry...@runsascoded.com>
>>>>> wrote:
>>>>>
>>>>>> Hm, I just encountered this again on a branch that based on
>>>>>> 5b46b02b49 (top of trunk from this afternoon). Is it definitely supposed to
>>>>>> be fixed?
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Please try rebasing from master, I believe this issue has been
>>>>>>> resolved.
>>>>>>>
>>>>>>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <ry...@runsascoded.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I'm seeing every ≈third `./gradlew compileJava` fail locally due to
>>>>>>>> this; re-running the commit has always succeeded, so far.
>>>>>>>>
>>>>>>>> Sounds like there is not an immediate fix in the works / no one
>>>>>>>> assigned on the JIRA?
>>>>>>>>
>>>>>>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> This might connect to vendoring Calcite. It will be easiest, and
>>>>>>>>> have the best incremental build, if we separate the generated code into its
>>>>>>>>> own module that has relocation to match the vendored Calcite.
>>>>>>>>>
>>>>>>>>> Kenn
>>>>>>>>>
>>>>>>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> We don't pre-generate the code as a separate step. Code gen from
>>>>>>>>>> the SQL parser syntax spec and its compilation happens both during the Beam
>>>>>>>>>> SQL build task. Splitting the code generation and compilation might not be
>>>>>>>>>> trivial. We definitely should look into fixing this though.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Anton
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Okay, make sense perhaps we can somehow make it fail when it
>>>>>>>>>>> fails to generate the dep, rather than when compiling the java code later on
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> ParserImpl is autogenerated by Calcite at build time. It seems
>>>>>>>>>>>> that there's a race condition there and it sometimes fails. Rerunning the
>>>>>>>>>>>> build works for me.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Anton
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>>>>>>>
>>>>>>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>>>>>>>
>>>>>>>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>>>>>>>> error: cannot find symbol
>>>>>>>>>>>>>
>>>>>>>>>>>>> import
>>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>>>>>>>
>>>>>>>>>>>>> ^
>>>>>>>>>>>>>
>>>>>>>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>>>>>>>
>>>>>>>>>>>>> location: package
>>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1 error
>>>>>>>>>>>>>
>>>>>>>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Ryan Williams <ry...@runsascoded.com>.
I still see this about 20% of the time I run `./gradlew compileTestJava
spotlessApply checkstyleMain checkstyleTest` (with 8-way parallelism),
which I've taken to running as a kind of "pre-PreCommit" (for Java).
In total I'd estimate I've seen this error 20 times (across multiple
`./gradlew clean`s and spanning several weeks), but never twice in a row: I
always immediately re-run, and the error goes away.
That seems consistent with the idea that gradle is racing the codegen task
against the compile task that requires the codegen's outputs.
However, `./gradlew -p sdks/java/extensions/sql -m compileJava | grep
compileJavacc` implies that Gradle knows that compilation depends on the
codegen having run.
Here are three successive invocations of `./gradlew compileTestJava
spotlessApply checkstyleMain checkstyleTest`
<https://gist.github.com/ryan-williams/6d2735b2484e9459be656abf379533c5>;
- The first one succeeds (and includes output showing that the codegen
task, `:beam-sdks-java-extensions-sql:compileJavacc`, was run).
- Then, I rebased on one new upstream commit and ran again, and saw the
failure.
- Immediately re-running saw the failure go away.
I guess in that output, tasks that are run but produce no stdout/stderr are
elided from the output, so we can't completely tell when the codegen and
compile tasks in question are being run. I'll start running with
--console=verbose
<https://stackoverflow.com/questions/45883963/gradle-4-0-does-not-display-executed-tasks-in-command-line#comment92914258_45889966>,
which I think will give a better sense of how this is happening.
On Fri, Feb 8, 2019 at 12:41 AM Ryan Williams <ry...@runsascoded.com> wrote:
> After your last message, Alex, I saw the issue on a branch with minimal
> unrelated changes on top of b4b5495307
> <https://github.com/apache/beam/commit/b4b549530730d9e0283ad0cdb203d384a107dfbf> (January
> 28).
>
> I ran `./gradlew clean` for the first time in a while at that time, and
> worked happily for 11 days, but just saw the issue again on a branch
> rebased on top of 381ab55b59
> <https://github.com/apache/beam/commit/381ab55b59> (today):
>
> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
> …/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
> error: cannot find symbol
> import
> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
> ^
> symbol: class BeamSqlParserImpl
> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
> 1 error
>
> Every time I've seen it, I immediately re-run the compile, and it succeeds.
>
> Perhaps something is still wrong with my environment, but otherwise it
> would seem that something is still flaky here.
>
> I'm compiling on an 8-core macOS machine, fwiw, and usually running
> `./gradlew compileTestJava` which compiles many projects concurrently.
>
> On Mon, Jan 28, 2019 at 4:12 PM Alex Amato <aj...@google.com> wrote:
>
>> If it continues to occur, maybe it is an environmental issue, be sure to
>> try to clean as well.
>> ./gradlew clean
>>
>> On Mon, Jan 28, 2019 at 11:12 AM Ryan Williams <ry...@runsascoded.com>
>> wrote:
>>
>>> Yea I was rebased on top of a more recent master than your previous
>>> message, when I saw it again. Perhaps I was mistaken. I'll ping here if I
>>> see it again, thanks.
>>>
>>> On Mon, Jan 28, 2019 at 1:52 PM Alex Amato <aj...@google.com> wrote:
>>>
>>>> After I did a rebase, it went away for me. So I think that this should
>>>> work. Are you saying that you did rebase ontop of master and it still
>>>> occurred? Strange.
>>>>
>>>> On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <ry...@runsascoded.com>
>>>> wrote:
>>>>
>>>>> Hm, I just encountered this again on a branch that based on 5b46b02b49
>>>>> (top of trunk from this afternoon). Is it definitely supposed to be fixed?
>>>>>
>>>>>
>>>>> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com> wrote:
>>>>>
>>>>>> Please try rebasing from master, I believe this issue has been
>>>>>> resolved.
>>>>>>
>>>>>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <ry...@runsascoded.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I'm seeing every ≈third `./gradlew compileJava` fail locally due to
>>>>>>> this; re-running the commit has always succeeded, so far.
>>>>>>>
>>>>>>> Sounds like there is not an immediate fix in the works / no one
>>>>>>> assigned on the JIRA?
>>>>>>>
>>>>>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> This might connect to vendoring Calcite. It will be easiest, and
>>>>>>>> have the best incremental build, if we separate the generated code into its
>>>>>>>> own module that has relocation to match the vendored Calcite.
>>>>>>>>
>>>>>>>> Kenn
>>>>>>>>
>>>>>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> We don't pre-generate the code as a separate step. Code gen from
>>>>>>>>> the SQL parser syntax spec and its compilation happens both during the Beam
>>>>>>>>> SQL build task. Splitting the code generation and compilation might not be
>>>>>>>>> trivial. We definitely should look into fixing this though.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Anton
>>>>>>>>>
>>>>>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Okay, make sense perhaps we can somehow make it fail when it
>>>>>>>>>> fails to generate the dep, rather than when compiling the java code later on
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> ParserImpl is autogenerated by Calcite at build time. It seems
>>>>>>>>>>> that there's a race condition there and it sometimes fails. Rerunning the
>>>>>>>>>>> build works for me.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Anton
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>>>>>>
>>>>>>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>>>>>>
>>>>>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>>>>>>
>>>>>>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>>>>>>> error: cannot find symbol
>>>>>>>>>>>>
>>>>>>>>>>>> import
>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>>>>>>
>>>>>>>>>>>> ^
>>>>>>>>>>>>
>>>>>>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>>>>>>
>>>>>>>>>>>> location: package
>>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>>>>>>
>>>>>>>>>>>> 1 error
>>>>>>>>>>>>
>>>>>>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Ryan Williams <ry...@runsascoded.com>.
After your last message, Alex, I saw the issue on a branch with minimal
unrelated changes on top of b4b5495307
<https://github.com/apache/beam/commit/b4b549530730d9e0283ad0cdb203d384a107dfbf>
(January
28).
I ran `./gradlew clean` for the first time in a while at that time, and
worked happily for 11 days, but just saw the issue again on a branch
rebased on top of 381ab55b59
<https://github.com/apache/beam/commit/381ab55b59> (today):
> Task :beam-sdks-java-extensions-sql:compileJava FAILED
…/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
error: cannot find symbol
import
org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
^
symbol: class BeamSqlParserImpl
location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
1 error
Every time I've seen it, I immediately re-run the compile, and it succeeds.
Perhaps something is still wrong with my environment, but otherwise it
would seem that something is still flaky here.
I'm compiling on an 8-core macOS machine, fwiw, and usually running
`./gradlew compileTestJava` which compiles many projects concurrently.
On Mon, Jan 28, 2019 at 4:12 PM Alex Amato <aj...@google.com> wrote:
> If it continues to occur, maybe it is an environmental issue, be sure to
> try to clean as well.
> ./gradlew clean
>
> On Mon, Jan 28, 2019 at 11:12 AM Ryan Williams <ry...@runsascoded.com>
> wrote:
>
>> Yea I was rebased on top of a more recent master than your previous
>> message, when I saw it again. Perhaps I was mistaken. I'll ping here if I
>> see it again, thanks.
>>
>> On Mon, Jan 28, 2019 at 1:52 PM Alex Amato <aj...@google.com> wrote:
>>
>>> After I did a rebase, it went away for me. So I think that this should
>>> work. Are you saying that you did rebase ontop of master and it still
>>> occurred? Strange.
>>>
>>> On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <ry...@runsascoded.com>
>>> wrote:
>>>
>>>> Hm, I just encountered this again on a branch that based on 5b46b02b49
>>>> (top of trunk from this afternoon). Is it definitely supposed to be fixed?
>>>>
>>>>
>>>> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com> wrote:
>>>>
>>>>> Please try rebasing from master, I believe this issue has been
>>>>> resolved.
>>>>>
>>>>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <ry...@runsascoded.com>
>>>>> wrote:
>>>>>
>>>>>> I'm seeing every ≈third `./gradlew compileJava` fail locally due to
>>>>>> this; re-running the commit has always succeeded, so far.
>>>>>>
>>>>>> Sounds like there is not an immediate fix in the works / no one
>>>>>> assigned on the JIRA?
>>>>>>
>>>>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> This might connect to vendoring Calcite. It will be easiest, and
>>>>>>> have the best incremental build, if we separate the generated code into its
>>>>>>> own module that has relocation to match the vendored Calcite.
>>>>>>>
>>>>>>> Kenn
>>>>>>>
>>>>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> We don't pre-generate the code as a separate step. Code gen from
>>>>>>>> the SQL parser syntax spec and its compilation happens both during the Beam
>>>>>>>> SQL build task. Splitting the code generation and compilation might not be
>>>>>>>> trivial. We definitely should look into fixing this though.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Anton
>>>>>>>>
>>>>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Okay, make sense perhaps we can somehow make it fail when it fails
>>>>>>>>> to generate the dep, rather than when compiling the java code later on
>>>>>>>>>
>>>>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> ParserImpl is autogenerated by Calcite at build time. It seems
>>>>>>>>>> that there's a race condition there and it sometimes fails. Rerunning the
>>>>>>>>>> build works for me.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Anton
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>>>>>
>>>>>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>>>>>
>>>>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>>>>>
>>>>>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>>>>>> error: cannot find symbol
>>>>>>>>>>>
>>>>>>>>>>> import
>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>>>>>
>>>>>>>>>>> ^
>>>>>>>>>>>
>>>>>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>>>>>
>>>>>>>>>>> location: package
>>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>>>>>
>>>>>>>>>>> 1 error
>>>>>>>>>>>
>>>>>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Alex Amato <aj...@google.com>.
If it continues to occur, maybe it is an environmental issue, be sure to
try to clean as well.
./gradlew clean
On Mon, Jan 28, 2019 at 11:12 AM Ryan Williams <ry...@runsascoded.com> wrote:
> Yea I was rebased on top of a more recent master than your previous
> message, when I saw it again. Perhaps I was mistaken. I'll ping here if I
> see it again, thanks.
>
> On Mon, Jan 28, 2019 at 1:52 PM Alex Amato <aj...@google.com> wrote:
>
>> After I did a rebase, it went away for me. So I think that this should
>> work. Are you saying that you did rebase ontop of master and it still
>> occurred? Strange.
>>
>> On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <ry...@runsascoded.com>
>> wrote:
>>
>>> Hm, I just encountered this again on a branch that based on 5b46b02b49
>>> (top of trunk from this afternoon). Is it definitely supposed to be fixed?
>>>
>>>
>>> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com> wrote:
>>>
>>>> Please try rebasing from master, I believe this issue has been resolved.
>>>>
>>>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <ry...@runsascoded.com>
>>>> wrote:
>>>>
>>>>> I'm seeing every ≈third `./gradlew compileJava` fail locally due to
>>>>> this; re-running the commit has always succeeded, so far.
>>>>>
>>>>> Sounds like there is not an immediate fix in the works / no one
>>>>> assigned on the JIRA?
>>>>>
>>>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> This might connect to vendoring Calcite. It will be easiest, and have
>>>>>> the best incremental build, if we separate the generated code into its own
>>>>>> module that has relocation to match the vendored Calcite.
>>>>>>
>>>>>> Kenn
>>>>>>
>>>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> We don't pre-generate the code as a separate step. Code gen from the
>>>>>>> SQL parser syntax spec and its compilation happens both during the Beam SQL
>>>>>>> build task. Splitting the code generation and compilation might not be
>>>>>>> trivial. We definitely should look into fixing this though.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Anton
>>>>>>>
>>>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Okay, make sense perhaps we can somehow make it fail when it fails
>>>>>>>> to generate the dep, rather than when compiling the java code later on
>>>>>>>>
>>>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> ParserImpl is autogenerated by Calcite at build time. It seems
>>>>>>>>> that there's a race condition there and it sometimes fails. Rerunning the
>>>>>>>>> build works for me.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Anton
>>>>>>>>>
>>>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>>>>
>>>>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>>>>
>>>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>>>>
>>>>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>>>>> error: cannot find symbol
>>>>>>>>>>
>>>>>>>>>> import
>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>>>>
>>>>>>>>>> ^
>>>>>>>>>>
>>>>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>>>>
>>>>>>>>>> location: package
>>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>>>>
>>>>>>>>>> 1 error
>>>>>>>>>>
>>>>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Ryan Williams <ry...@runsascoded.com>.
Yea I was rebased on top of a more recent master than your previous
message, when I saw it again. Perhaps I was mistaken. I'll ping here if I
see it again, thanks.
On Mon, Jan 28, 2019 at 1:52 PM Alex Amato <aj...@google.com> wrote:
> After I did a rebase, it went away for me. So I think that this should
> work. Are you saying that you did rebase ontop of master and it still
> occurred? Strange.
>
> On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <ry...@runsascoded.com>
> wrote:
>
>> Hm, I just encountered this again on a branch that based on 5b46b02b49
>> (top of trunk from this afternoon). Is it definitely supposed to be fixed?
>>
>>
>> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com> wrote:
>>
>>> Please try rebasing from master, I believe this issue has been resolved.
>>>
>>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <ry...@runsascoded.com>
>>> wrote:
>>>
>>>> I'm seeing every ≈third `./gradlew compileJava` fail locally due to
>>>> this; re-running the commit has always succeeded, so far.
>>>>
>>>> Sounds like there is not an immediate fix in the works / no one
>>>> assigned on the JIRA?
>>>>
>>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>>
>>>>> This might connect to vendoring Calcite. It will be easiest, and have
>>>>> the best incremental build, if we separate the generated code into its own
>>>>> module that has relocation to match the vendored Calcite.
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com> wrote:
>>>>>
>>>>>> We don't pre-generate the code as a separate step. Code gen from the
>>>>>> SQL parser syntax spec and its compilation happens both during the Beam SQL
>>>>>> build task. Splitting the code generation and compilation might not be
>>>>>> trivial. We definitely should look into fixing this though.
>>>>>>
>>>>>> Regards,
>>>>>> Anton
>>>>>>
>>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Okay, make sense perhaps we can somehow make it fail when it fails
>>>>>>> to generate the dep, rather than when compiling the java code later on
>>>>>>>
>>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> ParserImpl is autogenerated by Calcite at build time. It seems that
>>>>>>>> there's a race condition there and it sometimes fails. Rerunning the build
>>>>>>>> works for me.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Anton
>>>>>>>>
>>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>>>
>>>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>>>
>>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>>>
>>>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>>>> error: cannot find symbol
>>>>>>>>>
>>>>>>>>> import
>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>>>
>>>>>>>>> ^
>>>>>>>>>
>>>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>>>
>>>>>>>>> location: package
>>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>>>
>>>>>>>>> 1 error
>>>>>>>>>
>>>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Alex Amato <aj...@google.com>.
After I did a rebase, it went away for me. So I think that this should
work. Are you saying that you did rebase ontop of master and it still
occurred? Strange.
On Sat, Jan 26, 2019 at 3:48 PM Ryan Williams <ry...@runsascoded.com> wrote:
> Hm, I just encountered this again on a branch that based on 5b46b02b49
> (top of trunk from this afternoon). Is it definitely supposed to be fixed?
>
>
> On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com> wrote:
>
>> Please try rebasing from master, I believe this issue has been resolved.
>>
>> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <ry...@runsascoded.com>
>> wrote:
>>
>>> I'm seeing every ≈third `./gradlew compileJava` fail locally due to
>>> this; re-running the commit has always succeeded, so far.
>>>
>>> Sounds like there is not an immediate fix in the works / no one assigned
>>> on the JIRA?
>>>
>>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>>> This might connect to vendoring Calcite. It will be easiest, and have
>>>> the best incremental build, if we separate the generated code into its own
>>>> module that has relocation to match the vendored Calcite.
>>>>
>>>> Kenn
>>>>
>>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com> wrote:
>>>>
>>>>> We don't pre-generate the code as a separate step. Code gen from the
>>>>> SQL parser syntax spec and its compilation happens both during the Beam SQL
>>>>> build task. Splitting the code generation and compilation might not be
>>>>> trivial. We definitely should look into fixing this though.
>>>>>
>>>>> Regards,
>>>>> Anton
>>>>>
>>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Okay, make sense perhaps we can somehow make it fail when it fails to
>>>>>> generate the dep, rather than when compiling the java code later on
>>>>>>
>>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> ParserImpl is autogenerated by Calcite at build time. It seems that
>>>>>>> there's a race condition there and it sometimes fails. Rerunning the build
>>>>>>> works for me.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Anton
>>>>>>>
>>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>>
>>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>>
>>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>>
>>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>>> error: cannot find symbol
>>>>>>>>
>>>>>>>> import
>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>>
>>>>>>>> ^
>>>>>>>>
>>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>>
>>>>>>>> location: package
>>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>>
>>>>>>>> 1 error
>>>>>>>>
>>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Ryan Williams <ry...@runsascoded.com>.
Hm, I just encountered this again on a branch that based on 5b46b02b49 (top
of trunk from this afternoon). Is it definitely supposed to be fixed?
On Thu, Jan 24, 2019 at 9:19 PM Alex Amato <aj...@google.com> wrote:
> Please try rebasing from master, I believe this issue has been resolved.
>
> On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <ry...@runsascoded.com>
> wrote:
>
>> I'm seeing every ≈third `./gradlew compileJava` fail locally due to this;
>> re-running the commit has always succeeded, so far.
>>
>> Sounds like there is not an immediate fix in the works / no one assigned
>> on the JIRA?
>>
>> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> This might connect to vendoring Calcite. It will be easiest, and have
>>> the best incremental build, if we separate the generated code into its own
>>> module that has relocation to match the vendored Calcite.
>>>
>>> Kenn
>>>
>>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com> wrote:
>>>
>>>> We don't pre-generate the code as a separate step. Code gen from the
>>>> SQL parser syntax spec and its compilation happens both during the Beam SQL
>>>> build task. Splitting the code generation and compilation might not be
>>>> trivial. We definitely should look into fixing this though.
>>>>
>>>> Regards,
>>>> Anton
>>>>
>>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com> wrote:
>>>>
>>>>> Okay, make sense perhaps we can somehow make it fail when it fails to
>>>>> generate the dep, rather than when compiling the java code later on
>>>>>
>>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com> wrote:
>>>>>
>>>>>> ParserImpl is autogenerated by Calcite at build time. It seems that
>>>>>> there's a race condition there and it sometimes fails. Rerunning the build
>>>>>> works for me.
>>>>>>
>>>>>> Regards,
>>>>>> Anton
>>>>>>
>>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com> wrote:
>>>>>>
>>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>>
>>>>>>> Any ideas, how this got through the precommit?
>>>>>>>
>>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>>
>>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>>> error: cannot find symbol
>>>>>>>
>>>>>>> import
>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>>
>>>>>>> ^
>>>>>>>
>>>>>>> symbol: class BeamSqlParserImpl
>>>>>>>
>>>>>>> location: package
>>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>>
>>>>>>> 1 error
>>>>>>>
>>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Alex Amato <aj...@google.com>.
Please try rebasing from master, I believe this issue has been resolved.
On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams <ry...@runsascoded.com> wrote:
> I'm seeing every ≈third `./gradlew compileJava` fail locally due to this;
> re-running the commit has always succeeded, so far.
>
> Sounds like there is not an immediate fix in the works / no one assigned
> on the JIRA?
>
> On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> This might connect to vendoring Calcite. It will be easiest, and have the
>> best incremental build, if we separate the generated code into its own
>> module that has relocation to match the vendored Calcite.
>>
>> Kenn
>>
>> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com> wrote:
>>
>>> We don't pre-generate the code as a separate step. Code gen from the SQL
>>> parser syntax spec and its compilation happens both during the Beam SQL
>>> build task. Splitting the code generation and compilation might not be
>>> trivial. We definitely should look into fixing this though.
>>>
>>> Regards,
>>> Anton
>>>
>>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com> wrote:
>>>
>>>> Okay, make sense perhaps we can somehow make it fail when it fails to
>>>> generate the dep, rather than when compiling the java code later on
>>>>
>>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com> wrote:
>>>>
>>>>> ParserImpl is autogenerated by Calcite at build time. It seems that
>>>>> there's a race condition there and it sometimes fails. Rerunning the build
>>>>> works for me.
>>>>>
>>>>> Regards,
>>>>> Anton
>>>>>
>>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com> wrote:
>>>>>
>>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>>
>>>>>> Any ideas, how this got through the precommit?
>>>>>>
>>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>>
>>>>>> /usr/local/google/home/ajamato/go/src/
>>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>>> error: cannot find symbol
>>>>>>
>>>>>> import
>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>>
>>>>>> ^
>>>>>>
>>>>>> symbol: class BeamSqlParserImpl
>>>>>>
>>>>>> location: package
>>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>>
>>>>>> 1 error
>>>>>>
>>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Ryan Williams <ry...@runsascoded.com>.
I'm seeing every ≈third `./gradlew compileJava` fail locally due to this;
re-running the commit has always succeeded, so far.
Sounds like there is not an immediate fix in the works / no one assigned on
the JIRA?
On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles <ke...@apache.org> wrote:
> This might connect to vendoring Calcite. It will be easiest, and have the
> best incremental build, if we separate the generated code into its own
> module that has relocation to match the vendored Calcite.
>
> Kenn
>
> On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com> wrote:
>
>> We don't pre-generate the code as a separate step. Code gen from the SQL
>> parser syntax spec and its compilation happens both during the Beam SQL
>> build task. Splitting the code generation and compilation might not be
>> trivial. We definitely should look into fixing this though.
>>
>> Regards,
>> Anton
>>
>> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com> wrote:
>>
>>> Okay, make sense perhaps we can somehow make it fail when it fails to
>>> generate the dep, rather than when compiling the java code later on
>>>
>>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com> wrote:
>>>
>>>> ParserImpl is autogenerated by Calcite at build time. It seems that
>>>> there's a race condition there and it sometimes fails. Rerunning the build
>>>> works for me.
>>>>
>>>> Regards,
>>>> Anton
>>>>
>>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com> wrote:
>>>>
>>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>>
>>>>> Any ideas, how this got through the precommit?
>>>>>
>>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>>
>>>>> /usr/local/google/home/ajamato/go/src/
>>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>>> error: cannot find symbol
>>>>>
>>>>> import
>>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>>
>>>>> ^
>>>>>
>>>>> symbol: class BeamSqlParserImpl
>>>>>
>>>>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>>
>>>>> 1 error
>>>>>
>>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Kenneth Knowles <ke...@apache.org>.
This might connect to vendoring Calcite. It will be easiest, and have the
best incremental build, if we separate the generated code into its own
module that has relocation to match the vendored Calcite.
Kenn
On Wed, Jan 23, 2019 at 11:29 AM Anton Kedin <ke...@google.com> wrote:
> We don't pre-generate the code as a separate step. Code gen from the SQL
> parser syntax spec and its compilation happens both during the Beam SQL
> build task. Splitting the code generation and compilation might not be
> trivial. We definitely should look into fixing this though.
>
> Regards,
> Anton
>
> On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com> wrote:
>
>> Okay, make sense perhaps we can somehow make it fail when it fails to
>> generate the dep, rather than when compiling the java code later on
>>
>> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com> wrote:
>>
>>> ParserImpl is autogenerated by Calcite at build time. It seems that
>>> there's a race condition there and it sometimes fails. Rerunning the build
>>> works for me.
>>>
>>> Regards,
>>> Anton
>>>
>>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com> wrote:
>>>
>>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>>
>>>> Any ideas, how this got through the precommit?
>>>>
>>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>>
>>>> /usr/local/google/home/ajamato/go/src/
>>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>>> error: cannot find symbol
>>>>
>>>> import
>>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>>
>>>> ^
>>>>
>>>> symbol: class BeamSqlParserImpl
>>>>
>>>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>>
>>>> 1 error
>>>>
>>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Anton Kedin <ke...@google.com>.
We don't pre-generate the code as a separate step. Code gen from the SQL
parser syntax spec and its compilation happens both during the Beam SQL
build task. Splitting the code generation and compilation might not be
trivial. We definitely should look into fixing this though.
Regards,
Anton
On Wed, Jan 23, 2019 at 11:13 AM Alex Amato <aj...@google.com> wrote:
> Okay, make sense perhaps we can somehow make it fail when it fails to
> generate the dep, rather than when compiling the java code later on
>
> On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com> wrote:
>
>> ParserImpl is autogenerated by Calcite at build time. It seems that
>> there's a race condition there and it sometimes fails. Rerunning the build
>> works for me.
>>
>> Regards,
>> Anton
>>
>> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com> wrote:
>>
>>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>>
>>> Any ideas, how this got through the precommit?
>>>
>>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>>
>>> /usr/local/google/home/ajamato/go/src/
>>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>>> error: cannot find symbol
>>>
>>> import
>>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>>
>>> ^
>>>
>>> symbol: class BeamSqlParserImpl
>>>
>>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>>
>>> 1 error
>>>
>>>
Re: compileJava broken on master see: BEAM-6495
Posted by Alex Amato <aj...@google.com>.
Okay, make sense perhaps we can somehow make it fail when it fails to
generate the dep, rather than when compiling the java code later on
On Wed, Jan 23, 2019 at 11:12 AM Anton Kedin <ke...@google.com> wrote:
> ParserImpl is autogenerated by Calcite at build time. It seems that
> there's a race condition there and it sometimes fails. Rerunning the build
> works for me.
>
> Regards,
> Anton
>
> On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com> wrote:
>
>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>
>> Any ideas, how this got through the precommit?
>>
>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>
>> /usr/local/google/home/ajamato/go/src/
>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>> error: cannot find symbol
>>
>> import
>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>
>> ^
>>
>> symbol: class BeamSqlParserImpl
>>
>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>
>> 1 error
>>
>>
Re: compileJava broken on master see: BEAM-6495
Posted by Anton Kedin <ke...@google.com>.
ParserImpl is autogenerated by Calcite at build time. It seems that there's
a race condition there and it sometimes fails. Rerunning the build works
for me.
Regards,
Anton
On Wed, Jan 23, 2019, 11:06 AM Alex Amato <aj...@google.com> wrote:
> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>
> Any ideas, how this got through the precommit?
>
> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>
> /usr/local/google/home/ajamato/go/src/
> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
> error: cannot find symbol
>
> import
> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>
> ^
>
> symbol: class BeamSqlParserImpl
>
> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>
> 1 error
>
>
Re: compileJava broken on master see: BEAM-6495
Posted by Rui Wang <ru...@google.com>.
https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/build.gradle#L95
After a glance on sql build code, seems like it does some file copy
operations and then compile. It might be the source of race condition.
-Rui
On Wed, Jan 23, 2019 at 11:17 AM Ruoyun Huang <ru...@google.com> wrote:
> I ran into the same issue. it is flaky, due to some depended package.
>
> short term solution is to rerun precommit. For me it was gone on second
> try.
>
> On Wed, Jan 23, 2019 at 11:06 AM Alex Amato <aj...@google.com> wrote:
>
>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>
>> Any ideas, how this got through the precommit?
>>
>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>
>> /usr/local/google/home/ajamato/go/src/
>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>> error: cannot find symbol
>>
>> import
>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>
>> ^
>>
>> symbol: class BeamSqlParserImpl
>>
>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>
>> 1 error
>>
>>
>
> --
> ================
> Ruoyun Huang
>
>
Re: compileJava broken on master see: BEAM-6495
Posted by Alex Amato <aj...@google.com>.
This kind of dependency can cause a flakey test? Seems like this kind of
thing would be a consistent failure, at least that's what I am seeing as
well
On Wed, Jan 23, 2019 at 11:11 AM Ruoyun Huang <ru...@google.com> wrote:
> I ran into the same issue. it is flaky, due to some depended package.
>
> short term solution is to rerun precommit. For me it was gone on second
> try.
>
> On Wed, Jan 23, 2019 at 11:06 AM Alex Amato <aj...@google.com> wrote:
>
>> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>>
>> Any ideas, how this got through the precommit?
>>
>> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>>
>> /usr/local/google/home/ajamato/go/src/
>> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
>> error: cannot find symbol
>>
>> import
>> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>>
>> ^
>>
>> symbol: class BeamSqlParserImpl
>>
>> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>>
>> 1 error
>>
>>
>
> --
> ================
> Ruoyun Huang
>
>
Re: compileJava broken on master see: BEAM-6495
Posted by Ruoyun Huang <ru...@google.com>.
I ran into the same issue. it is flaky, due to some depended package.
short term solution is to rerun precommit. For me it was gone on second
try.
On Wed, Jan 23, 2019 at 11:06 AM Alex Amato <aj...@google.com> wrote:
> https://jira.apache.org/jira/browse/BEAM-6495?filter=-2
>
> Any ideas, how this got through the precommit?
>
> > Task :beam-sdks-java-extensions-sql:compileJava FAILED
>
> /usr/local/google/home/ajamato/go/src/
> github.com/apache/beam/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JdbcFactory.java:29:
> error: cannot find symbol
>
> import
> org.apache.beam.sdk.extensions.sql.impl.parser.impl.BeamSqlParserImpl;
>
> ^
>
> symbol: class BeamSqlParserImpl
>
> location: package org.apache.beam.sdk.extensions.sql.impl.parser.impl
>
> 1 error
>
>
--
================
Ruoyun Huang