You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Elliotte Rusty Harold <el...@ibiblio.org> on 2019/11/21 11:10:07 UTC
goVet and clickHouse tests failing
I'm slowly working my way through getting the tests to run and pass.
We have a lot of work to do on the contributing docs to explain how to
setup and run the build. There's clearly a lot of knowledge in
developers' heads and workstations that hasn't yet made it into the
docs.
The latest is a problem finding "github.com/linkedin/goavro" when I
run goVet. I'm not a go person. Is this something that requires an
extra install? If so, how is it installed? Or is this some error in
the build.gradles? Or perhaps my go config is borked and gradle is
looking in the wrong directory?
> Task :sdks:go:examples:resolveBuildDependencies
Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go
.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2:
cannot find package "github.com/linkedin/goavro" in any of:
/home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro
(vendor tree)
/home/elharo/.gradle/go/binary/1.12/go/src/github.com/linkedin/goavro
(from $GOROOT)
/home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/linkedin/goavro
(from $GOPATH)
> Task :sdks:go:examples:goVet FAILED
I'm also seeing failures in ClickHouseIOTest:
> Task :sdks:java:io:clickhouse:test
org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
java.lang.IllegalStateException
org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
java.lang.NullPointerException
org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
java.lang.IllegalStateException
org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
java.lang.NullPointerException
--
Elliotte Rusty Harold
elharo@ibiblio.org
Re: goVet and clickHouse tests failing
Posted by Kenneth Knowles <ke...@apache.org>.
It would be nice if this failed gracefully, indicating what dependency is
not installed.
Kenn
On Thu, Nov 21, 2019 at 4:38 AM Elliotte Rusty Harold <el...@ibiblio.org>
wrote:
> That would explain it. I didn't have Docker installed. I do now, so
> let's see if it passes. Ultimately the root cause for a lot of this
> seems to be incomplete and conflicting docs. I hope we can invest some
> effort into improving this.
>
> On Thu, Nov 21, 2019 at 6:27 AM Gleb Kanterov <gl...@spotify.com> wrote:
> >
> > :sdks:java:io:clickhouse:test is using testcontainers. Testcontainers is
> a Java library that supports JUnit tests, providing lightweight, throwaway
> instances of common databases, Selenium web browsers, or anything else that
> can run in a Docker container. Therefore, ClickHouse tests require a local
> Docker environment. It might make sense to move tests using testcontainers
> into a separate gradle task if it's a common issue.
> >
> > On Thu, Nov 21, 2019 at 12:10 PM Elliotte Rusty Harold <
> elharo@ibiblio.org> wrote:
> >>
> >> I'm slowly working my way through getting the tests to run and pass.
> >> We have a lot of work to do on the contributing docs to explain how to
> >> setup and run the build. There's clearly a lot of knowledge in
> >> developers' heads and workstations that hasn't yet made it into the
> >> docs.
> >>
> >> The latest is a problem finding "github.com/linkedin/goavro" when I
> >> run goVet. I'm not a go person. Is this something that requires an
> >> extra install? If so, how is it installed? Or is this some error in
> >> the build.gradles? Or perhaps my go config is borked and gradle is
> >> looking in the wrong directory?
> >>
> >> > Task :sdks:go:examples:resolveBuildDependencies
> >> Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go
> >> .gogradle/project_gopath/src/
> github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2
> :
> >> cannot find package "github.com/linkedin/goavro" in any of:
> >> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
> github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro
> >> (vendor tree)
> >> /home/elharo/.gradle/go/binary/1.12/go/src/
> github.com/linkedin/goavro
> >> (from $GOROOT)
> >> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
> github.com/linkedin/goavro
> >> (from $GOPATH)
> >>
> >> > Task :sdks:go:examples:goVet FAILED
> >>
> >> I'm also seeing failures in ClickHouseIOTest:
> >>
> >> > Task :sdks:java:io:clickhouse:test
> >>
> >> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
> >> java.lang.IllegalStateException
> >>
> >> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
> >> java.lang.NullPointerException
> >>
> >> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
> >> java.lang.IllegalStateException
> >>
> >> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
> >> java.lang.NullPointerException
> >>
> >> --
> >> Elliotte Rusty Harold
> >> elharo@ibiblio.org
>
>
>
> --
> Elliotte Rusty Harold
> elharo@ibiblio.org
>
Re: goVet and clickHouse tests failing
Posted by Elliotte Rusty Harold <el...@ibiblio.org>.
That would explain it. I didn't have Docker installed. I do now, so
let's see if it passes. Ultimately the root cause for a lot of this
seems to be incomplete and conflicting docs. I hope we can invest some
effort into improving this.
On Thu, Nov 21, 2019 at 6:27 AM Gleb Kanterov <gl...@spotify.com> wrote:
>
> :sdks:java:io:clickhouse:test is using testcontainers. Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container. Therefore, ClickHouse tests require a local Docker environment. It might make sense to move tests using testcontainers into a separate gradle task if it's a common issue.
>
> On Thu, Nov 21, 2019 at 12:10 PM Elliotte Rusty Harold <el...@ibiblio.org> wrote:
>>
>> I'm slowly working my way through getting the tests to run and pass.
>> We have a lot of work to do on the contributing docs to explain how to
>> setup and run the build. There's clearly a lot of knowledge in
>> developers' heads and workstations that hasn't yet made it into the
>> docs.
>>
>> The latest is a problem finding "github.com/linkedin/goavro" when I
>> run goVet. I'm not a go person. Is this something that requires an
>> extra install? If so, how is it installed? Or is this some error in
>> the build.gradles? Or perhaps my go config is borked and gradle is
>> looking in the wrong directory?
>>
>> > Task :sdks:go:examples:resolveBuildDependencies
>> Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go
>> .gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2:
>> cannot find package "github.com/linkedin/goavro" in any of:
>> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro
>> (vendor tree)
>> /home/elharo/.gradle/go/binary/1.12/go/src/github.com/linkedin/goavro
>> (from $GOROOT)
>> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/linkedin/goavro
>> (from $GOPATH)
>>
>> > Task :sdks:go:examples:goVet FAILED
>>
>> I'm also seeing failures in ClickHouseIOTest:
>>
>> > Task :sdks:java:io:clickhouse:test
>>
>> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
>> java.lang.IllegalStateException
>>
>> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
>> java.lang.NullPointerException
>>
>> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
>> java.lang.IllegalStateException
>>
>> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
>> java.lang.NullPointerException
>>
>> --
>> Elliotte Rusty Harold
>> elharo@ibiblio.org
--
Elliotte Rusty Harold
elharo@ibiblio.org
Re: goVet and clickHouse tests failing
Posted by Gleb Kanterov <gl...@spotify.com>.
:sdks:java:io:clickhouse:test is using testcontainers. Testcontainers is a
Java library that supports JUnit tests, providing lightweight, throwaway
instances of common databases, Selenium web browsers, or anything else that
can run in a Docker container. Therefore, ClickHouse tests require a local
Docker environment. It might make sense to move tests using testcontainers
into a separate gradle task if it's a common issue.
On Thu, Nov 21, 2019 at 12:10 PM Elliotte Rusty Harold <el...@ibiblio.org>
wrote:
> I'm slowly working my way through getting the tests to run and pass.
> We have a lot of work to do on the contributing docs to explain how to
> setup and run the build. There's clearly a lot of knowledge in
> developers' heads and workstations that hasn't yet made it into the
> docs.
>
> The latest is a problem finding "github.com/linkedin/goavro" when I
> run goVet. I'm not a go person. Is this something that requires an
> extra install? If so, how is it installed? Or is this some error in
> the build.gradles? Or perhaps my go config is borked and gradle is
> looking in the wrong directory?
>
> > Task :sdks:go:examples:resolveBuildDependencies
> Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go
> .gogradle/project_gopath/src/
> github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2
> :
> cannot find package "github.com/linkedin/goavro" in any of:
> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
> github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro
> (vendor tree)
> /home/elharo/.gradle/go/binary/1.12/go/src/
> github.com/linkedin/goavro
> (from $GOROOT)
> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
> github.com/linkedin/goavro
> (from $GOPATH)
>
> > Task :sdks:go:examples:goVet FAILED
>
> I'm also seeing failures in ClickHouseIOTest:
>
> > Task :sdks:java:io:clickhouse:test
>
> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
> java.lang.IllegalStateException
>
> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
> java.lang.NullPointerException
>
> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
> java.lang.IllegalStateException
>
> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
> java.lang.NullPointerException
>
> --
> Elliotte Rusty Harold
> elharo@ibiblio.org
>
Re: goVet and clickHouse tests failing
Posted by Robert Burke <ro...@frantil.com>.
Hello!
A goal (that is up for anyone interested to work on) is to move the Beam Go
SDK to use Go Modules[2] rather than the GoGradle[3] plugin's lock file,
since this unifies the experience for both working on the Go SDK (as a Beam
dev) and with the Go SDK (as a Beam user). The plugin hasn't been updated
to deal with Go Modules, but it's possible to disable the locking and
.implicit vendoring in favour of the standard Go Module cache.
The gogradle lock file would need to be updated so that Gradle gets the
avro package into it's cache (if it's not listed already).
The SDK itself is compatible with Go 1.13 but that's outside of any
dependency module concerns.
Resolving this would let the SDK adopt new depencies more easily, and
better integrate with general Go tooling, rather than just with the
demi-integration we have with Gradle.
Robert B
[1] https://github.com/golang/go/wiki/Modules
[2] https://gogradle.github.io/gogradle/docs/getting-started.html
On Fri, 29 Nov 2019 at 08:18, Amogh Tiwari <am...@google.com> wrote:
> Adding +Sameer Abhyankar <sa...@google.com> for visibility.
>
> On Fri, Nov 29, 2019 at 4:10 PM Amogh Tiwari <am...@google.com> wrote:
>
>> Hi Elliotte,
>> Thanks for your reply.
>> Tried out the steps that you suggested, still not able to resolve this
>> issue.
>> Should I use a particular version of go for the build?
>> What I have noticed is that when I ./gradlew build, it automatically
>> downloads go 1.12 and sets the go env for it, basically ignoring whatever
>> version of go that I already have installed.
>> Please follow the last comment of this
>> https://issues.apache.org/jira/browse/BEAM-8564.
>> Regards
>> Amogh
>>
>>
>> On Wed, Nov 27, 2019 at 5:46 PM Elliotte Rusty Harold <el...@ibiblio.org>
>> wrote:
>>
>>> I did get through this one, and made the classic mistake of not
>>> immediately committing the steps I took to writing. I believe it
>>> involved some combination of setting go paths in environment
>>> variables. I seem to have added this to the end of my .profile:
>>>
>>> export GOROOT=/usr/local/go
>>> export GOPATH=$HOME/go/packages
>>> export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
>>>
>>> According to my history, I also ran
>>>
>>> go get github.com/linkedin/goavro
>>>
>>> I really need to dedicate some time to cleaning up the build
>>> documentation. See https://issues.apache.org/jira/browse/BEAM-8798
>>>
>>> Right now we have overlapping and sometimes contradictory docs in four
>>> different places:
>>>
>>> README.md
>>> CONTRIBUTING.md
>>> https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ
>>> https://beam.apache.org/contribute/
>>>
>>> We should probably pick one as the source of truth and rewrite the
>>> other three to simply point to it. I propose putting all checkout,
>>> build, test, commit, and push instructions in CONTRIBUTING.md in the
>>> repo. What do folks think?
>>>
>>> On Wed, Nov 27, 2019 at 3:51 AM Amogh Tiwari <am...@google.com> wrote:
>>> >
>>> > Hi Elliotte,
>>> > I am facing a similar goVet issue. It would be great if you can guide
>>> me through the solution. Please let me know the steps that you followed.
>>> > Regards,
>>> > Amogh
>>> >
>>> > On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold <
>>> elharo@ibiblio.org> wrote:
>>> >>
>>> >> Tentatively, the goVet issue does seem to have been an issue with my
>>> >> Go install I have now cleaned up. The clickhouse issue remains, as do
>>> >> several others I'm working through.
>>> >>
>>> >> I've filed https://issues.apache.org/jira/browse/BEAM-8798 to
>>> >> consolidate and update the instructions for getting to a working
>>> >> build.
>>> >>
>>> >> On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold
>>> >> <el...@ibiblio.org> wrote:
>>> >> >
>>> >> > I'm slowly working my way through getting the tests to run and pass.
>>> >> > We have a lot of work to do on the contributing docs to explain how
>>> to
>>> >> > setup and run the build. There's clearly a lot of knowledge in
>>> >> > developers' heads and workstations that hasn't yet made it into the
>>> >> > docs.
>>> >> >
>>> >> > The latest is a problem finding "github.com/linkedin/goavro" when I
>>> >> > run goVet. I'm not a go person. Is this something that requires an
>>> >> > extra install? If so, how is it installed? Or is this some error in
>>> >> > the build.gradles? Or perhaps my go config is borked and gradle is
>>> >> > looking in the wrong directory?
>>> >> >
>>> >> > > Task :sdks:go:examples:resolveBuildDependencies
>>> >> > Resolving ./
>>> github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go
>>> >> > .gogradle/project_gopath/src/
>>> github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2
>>> :
>>> >> > cannot find package "github.com/linkedin/goavro" in any of:
>>> >> >
>>> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
>>> github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro
>>> >> > (vendor tree)
>>> >> > /home/elharo/.gradle/go/binary/1.12/go/src/
>>> github.com/linkedin/goavro
>>> >> > (from $GOROOT)
>>> >> >
>>> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
>>> github.com/linkedin/goavro
>>> >> > (from $GOPATH)
>>> >> >
>>> >> > > Task :sdks:go:examples:goVet FAILED
>>> >> >
>>> >> > I'm also seeing failures in ClickHouseIOTest:
>>> >> >
>>> >> > > Task :sdks:java:io:clickhouse:test
>>> >> >
>>> >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod
>>> FAILED
>>> >> > java.lang.IllegalStateException
>>> >> >
>>> >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod
>>> FAILED
>>> >> > java.lang.NullPointerException
>>> >> >
>>> >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod
>>> FAILED
>>> >> > java.lang.IllegalStateException
>>> >> >
>>> >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod
>>> FAILED
>>> >> > java.lang.NullPointerException
>>> >> >
>>> >> > --
>>> >> > Elliotte Rusty Harold
>>> >> > elharo@ibiblio.org
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Elliotte Rusty Harold
>>> >> elharo@ibiblio.org
>>>
>>>
>>>
>>> --
>>> Elliotte Rusty Harold
>>> elharo@ibiblio.org
>>>
>>
Re: goVet and clickHouse tests failing
Posted by Amogh Tiwari <am...@google.com>.
Adding +Sameer Abhyankar <sa...@google.com> for visibility.
On Fri, Nov 29, 2019 at 4:10 PM Amogh Tiwari <am...@google.com> wrote:
> Hi Elliotte,
> Thanks for your reply.
> Tried out the steps that you suggested, still not able to resolve this
> issue.
> Should I use a particular version of go for the build?
> What I have noticed is that when I ./gradlew build, it automatically
> downloads go 1.12 and sets the go env for it, basically ignoring whatever
> version of go that I already have installed.
> Please follow the last comment of this
> https://issues.apache.org/jira/browse/BEAM-8564.
> Regards
> Amogh
>
>
> On Wed, Nov 27, 2019 at 5:46 PM Elliotte Rusty Harold <el...@ibiblio.org>
> wrote:
>
>> I did get through this one, and made the classic mistake of not
>> immediately committing the steps I took to writing. I believe it
>> involved some combination of setting go paths in environment
>> variables. I seem to have added this to the end of my .profile:
>>
>> export GOROOT=/usr/local/go
>> export GOPATH=$HOME/go/packages
>> export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
>>
>> According to my history, I also ran
>>
>> go get github.com/linkedin/goavro
>>
>> I really need to dedicate some time to cleaning up the build
>> documentation. See https://issues.apache.org/jira/browse/BEAM-8798
>>
>> Right now we have overlapping and sometimes contradictory docs in four
>> different places:
>>
>> README.md
>> CONTRIBUTING.md
>> https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ
>> https://beam.apache.org/contribute/
>>
>> We should probably pick one as the source of truth and rewrite the
>> other three to simply point to it. I propose putting all checkout,
>> build, test, commit, and push instructions in CONTRIBUTING.md in the
>> repo. What do folks think?
>>
>> On Wed, Nov 27, 2019 at 3:51 AM Amogh Tiwari <am...@google.com> wrote:
>> >
>> > Hi Elliotte,
>> > I am facing a similar goVet issue. It would be great if you can guide
>> me through the solution. Please let me know the steps that you followed.
>> > Regards,
>> > Amogh
>> >
>> > On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold <
>> elharo@ibiblio.org> wrote:
>> >>
>> >> Tentatively, the goVet issue does seem to have been an issue with my
>> >> Go install I have now cleaned up. The clickhouse issue remains, as do
>> >> several others I'm working through.
>> >>
>> >> I've filed https://issues.apache.org/jira/browse/BEAM-8798 to
>> >> consolidate and update the instructions for getting to a working
>> >> build.
>> >>
>> >> On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold
>> >> <el...@ibiblio.org> wrote:
>> >> >
>> >> > I'm slowly working my way through getting the tests to run and pass.
>> >> > We have a lot of work to do on the contributing docs to explain how
>> to
>> >> > setup and run the build. There's clearly a lot of knowledge in
>> >> > developers' heads and workstations that hasn't yet made it into the
>> >> > docs.
>> >> >
>> >> > The latest is a problem finding "github.com/linkedin/goavro" when I
>> >> > run goVet. I'm not a go person. Is this something that requires an
>> >> > extra install? If so, how is it installed? Or is this some error in
>> >> > the build.gradles? Or perhaps my go config is borked and gradle is
>> >> > looking in the wrong directory?
>> >> >
>> >> > > Task :sdks:go:examples:resolveBuildDependencies
>> >> > Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go
>> >> > .gogradle/project_gopath/src/
>> github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2
>> :
>> >> > cannot find package "github.com/linkedin/goavro" in any of:
>> >> >
>> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
>> github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro
>> >> > (vendor tree)
>> >> > /home/elharo/.gradle/go/binary/1.12/go/src/
>> github.com/linkedin/goavro
>> >> > (from $GOROOT)
>> >> >
>> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
>> github.com/linkedin/goavro
>> >> > (from $GOPATH)
>> >> >
>> >> > > Task :sdks:go:examples:goVet FAILED
>> >> >
>> >> > I'm also seeing failures in ClickHouseIOTest:
>> >> >
>> >> > > Task :sdks:java:io:clickhouse:test
>> >> >
>> >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod
>> FAILED
>> >> > java.lang.IllegalStateException
>> >> >
>> >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod
>> FAILED
>> >> > java.lang.NullPointerException
>> >> >
>> >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod
>> FAILED
>> >> > java.lang.IllegalStateException
>> >> >
>> >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod
>> FAILED
>> >> > java.lang.NullPointerException
>> >> >
>> >> > --
>> >> > Elliotte Rusty Harold
>> >> > elharo@ibiblio.org
>> >>
>> >>
>> >>
>> >> --
>> >> Elliotte Rusty Harold
>> >> elharo@ibiblio.org
>>
>>
>>
>> --
>> Elliotte Rusty Harold
>> elharo@ibiblio.org
>>
>
Re: goVet and clickHouse tests failing
Posted by Amogh Tiwari <am...@google.com>.
Hi Elliotte,
Thanks for your reply.
Tried out the steps that you suggested, still not able to resolve this
issue.
Should I use a particular version of go for the build?
What I have noticed is that when I ./gradlew build, it automatically
downloads go 1.12 and sets the go env for it, basically ignoring whatever
version of go that I already have installed.
Please follow the last comment of this
https://issues.apache.org/jira/browse/BEAM-8564.
Regards
Amogh
On Wed, Nov 27, 2019 at 5:46 PM Elliotte Rusty Harold <el...@ibiblio.org>
wrote:
> I did get through this one, and made the classic mistake of not
> immediately committing the steps I took to writing. I believe it
> involved some combination of setting go paths in environment
> variables. I seem to have added this to the end of my .profile:
>
> export GOROOT=/usr/local/go
> export GOPATH=$HOME/go/packages
> export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
>
> According to my history, I also ran
>
> go get github.com/linkedin/goavro
>
> I really need to dedicate some time to cleaning up the build
> documentation. See https://issues.apache.org/jira/browse/BEAM-8798
>
> Right now we have overlapping and sometimes contradictory docs in four
> different places:
>
> README.md
> CONTRIBUTING.md
> https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ
> https://beam.apache.org/contribute/
>
> We should probably pick one as the source of truth and rewrite the
> other three to simply point to it. I propose putting all checkout,
> build, test, commit, and push instructions in CONTRIBUTING.md in the
> repo. What do folks think?
>
> On Wed, Nov 27, 2019 at 3:51 AM Amogh Tiwari <am...@google.com> wrote:
> >
> > Hi Elliotte,
> > I am facing a similar goVet issue. It would be great if you can guide me
> through the solution. Please let me know the steps that you followed.
> > Regards,
> > Amogh
> >
> > On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold <
> elharo@ibiblio.org> wrote:
> >>
> >> Tentatively, the goVet issue does seem to have been an issue with my
> >> Go install I have now cleaned up. The clickhouse issue remains, as do
> >> several others I'm working through.
> >>
> >> I've filed https://issues.apache.org/jira/browse/BEAM-8798 to
> >> consolidate and update the instructions for getting to a working
> >> build.
> >>
> >> On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold
> >> <el...@ibiblio.org> wrote:
> >> >
> >> > I'm slowly working my way through getting the tests to run and pass.
> >> > We have a lot of work to do on the contributing docs to explain how to
> >> > setup and run the build. There's clearly a lot of knowledge in
> >> > developers' heads and workstations that hasn't yet made it into the
> >> > docs.
> >> >
> >> > The latest is a problem finding "github.com/linkedin/goavro" when I
> >> > run goVet. I'm not a go person. Is this something that requires an
> >> > extra install? If so, how is it installed? Or is this some error in
> >> > the build.gradles? Or perhaps my go config is borked and gradle is
> >> > looking in the wrong directory?
> >> >
> >> > > Task :sdks:go:examples:resolveBuildDependencies
> >> > Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go
> >> > .gogradle/project_gopath/src/
> github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2
> :
> >> > cannot find package "github.com/linkedin/goavro" in any of:
> >> >
> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
> github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro
> >> > (vendor tree)
> >> > /home/elharo/.gradle/go/binary/1.12/go/src/
> github.com/linkedin/goavro
> >> > (from $GOROOT)
> >> >
> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
> github.com/linkedin/goavro
> >> > (from $GOPATH)
> >> >
> >> > > Task :sdks:go:examples:goVet FAILED
> >> >
> >> > I'm also seeing failures in ClickHouseIOTest:
> >> >
> >> > > Task :sdks:java:io:clickhouse:test
> >> >
> >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod
> FAILED
> >> > java.lang.IllegalStateException
> >> >
> >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod
> FAILED
> >> > java.lang.NullPointerException
> >> >
> >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod
> FAILED
> >> > java.lang.IllegalStateException
> >> >
> >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod
> FAILED
> >> > java.lang.NullPointerException
> >> >
> >> > --
> >> > Elliotte Rusty Harold
> >> > elharo@ibiblio.org
> >>
> >>
> >>
> >> --
> >> Elliotte Rusty Harold
> >> elharo@ibiblio.org
>
>
>
> --
> Elliotte Rusty Harold
> elharo@ibiblio.org
>
Re: goVet and clickHouse tests failing
Posted by Elliotte Rusty Harold <el...@ibiblio.org>.
I did get through this one, and made the classic mistake of not
immediately committing the steps I took to writing. I believe it
involved some combination of setting go paths in environment
variables. I seem to have added this to the end of my .profile:
export GOROOT=/usr/local/go
export GOPATH=$HOME/go/packages
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
According to my history, I also ran
go get github.com/linkedin/goavro
I really need to dedicate some time to cleaning up the build
documentation. See https://issues.apache.org/jira/browse/BEAM-8798
Right now we have overlapping and sometimes contradictory docs in four
different places:
README.md
CONTRIBUTING.md
https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ
https://beam.apache.org/contribute/
We should probably pick one as the source of truth and rewrite the
other three to simply point to it. I propose putting all checkout,
build, test, commit, and push instructions in CONTRIBUTING.md in the
repo. What do folks think?
On Wed, Nov 27, 2019 at 3:51 AM Amogh Tiwari <am...@google.com> wrote:
>
> Hi Elliotte,
> I am facing a similar goVet issue. It would be great if you can guide me through the solution. Please let me know the steps that you followed.
> Regards,
> Amogh
>
> On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold <el...@ibiblio.org> wrote:
>>
>> Tentatively, the goVet issue does seem to have been an issue with my
>> Go install I have now cleaned up. The clickhouse issue remains, as do
>> several others I'm working through.
>>
>> I've filed https://issues.apache.org/jira/browse/BEAM-8798 to
>> consolidate and update the instructions for getting to a working
>> build.
>>
>> On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold
>> <el...@ibiblio.org> wrote:
>> >
>> > I'm slowly working my way through getting the tests to run and pass.
>> > We have a lot of work to do on the contributing docs to explain how to
>> > setup and run the build. There's clearly a lot of knowledge in
>> > developers' heads and workstations that hasn't yet made it into the
>> > docs.
>> >
>> > The latest is a problem finding "github.com/linkedin/goavro" when I
>> > run goVet. I'm not a go person. Is this something that requires an
>> > extra install? If so, how is it installed? Or is this some error in
>> > the build.gradles? Or perhaps my go config is borked and gradle is
>> > looking in the wrong directory?
>> >
>> > > Task :sdks:go:examples:resolveBuildDependencies
>> > Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go
>> > .gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2:
>> > cannot find package "github.com/linkedin/goavro" in any of:
>> > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro
>> > (vendor tree)
>> > /home/elharo/.gradle/go/binary/1.12/go/src/github.com/linkedin/goavro
>> > (from $GOROOT)
>> > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/linkedin/goavro
>> > (from $GOPATH)
>> >
>> > > Task :sdks:go:examples:goVet FAILED
>> >
>> > I'm also seeing failures in ClickHouseIOTest:
>> >
>> > > Task :sdks:java:io:clickhouse:test
>> >
>> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
>> > java.lang.IllegalStateException
>> >
>> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
>> > java.lang.NullPointerException
>> >
>> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
>> > java.lang.IllegalStateException
>> >
>> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
>> > java.lang.NullPointerException
>> >
>> > --
>> > Elliotte Rusty Harold
>> > elharo@ibiblio.org
>>
>>
>>
>> --
>> Elliotte Rusty Harold
>> elharo@ibiblio.org
--
Elliotte Rusty Harold
elharo@ibiblio.org
Re: goVet and clickHouse tests failing
Posted by Amogh Tiwari <am...@google.com>.
Hi Elliotte,
I am facing a similar goVet issue. It would be great if you can guide me
through the solution. Please let me know the steps that you followed.
Regards,
Amogh
On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold <el...@ibiblio.org>
wrote:
> Tentatively, the goVet issue does seem to have been an issue with my
> Go install I have now cleaned up. The clickhouse issue remains, as do
> several others I'm working through.
>
> I've filed https://issues.apache.org/jira/browse/BEAM-8798 to
> consolidate and update the instructions for getting to a working
> build.
>
> On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold
> <el...@ibiblio.org> wrote:
> >
> > I'm slowly working my way through getting the tests to run and pass.
> > We have a lot of work to do on the contributing docs to explain how to
> > setup and run the build. There's clearly a lot of knowledge in
> > developers' heads and workstations that hasn't yet made it into the
> > docs.
> >
> > The latest is a problem finding "github.com/linkedin/goavro" when I
> > run goVet. I'm not a go person. Is this something that requires an
> > extra install? If so, how is it installed? Or is this some error in
> > the build.gradles? Or perhaps my go config is borked and gradle is
> > looking in the wrong directory?
> >
> > > Task :sdks:go:examples:resolveBuildDependencies
> > Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go
> > .gogradle/project_gopath/src/
> github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2
> :
> > cannot find package "github.com/linkedin/goavro" in any of:
> > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
> github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro
> > (vendor tree)
> > /home/elharo/.gradle/go/binary/1.12/go/src/
> github.com/linkedin/goavro
> > (from $GOROOT)
> > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/
> github.com/linkedin/goavro
> > (from $GOPATH)
> >
> > > Task :sdks:go:examples:goVet FAILED
> >
> > I'm also seeing failures in ClickHouseIOTest:
> >
> > > Task :sdks:java:io:clickhouse:test
> >
> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
> > java.lang.IllegalStateException
> >
> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
> > java.lang.NullPointerException
> >
> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
> > java.lang.IllegalStateException
> >
> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
> > java.lang.NullPointerException
> >
> > --
> > Elliotte Rusty Harold
> > elharo@ibiblio.org
>
>
>
> --
> Elliotte Rusty Harold
> elharo@ibiblio.org
>
Re: goVet and clickHouse tests failing
Posted by Elliotte Rusty Harold <el...@ibiblio.org>.
Tentatively, the goVet issue does seem to have been an issue with my
Go install I have now cleaned up. The clickhouse issue remains, as do
several others I'm working through.
I've filed https://issues.apache.org/jira/browse/BEAM-8798 to
consolidate and update the instructions for getting to a working
build.
On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold
<el...@ibiblio.org> wrote:
>
> I'm slowly working my way through getting the tests to run and pass.
> We have a lot of work to do on the contributing docs to explain how to
> setup and run the build. There's clearly a lot of knowledge in
> developers' heads and workstations that hasn't yet made it into the
> docs.
>
> The latest is a problem finding "github.com/linkedin/goavro" when I
> run goVet. I'm not a go person. Is this something that requires an
> extra install? If so, how is it installed? Or is this some error in
> the build.gradles? Or perhaps my go config is borked and gradle is
> looking in the wrong directory?
>
> > Task :sdks:go:examples:resolveBuildDependencies
> Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go
> .gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2:
> cannot find package "github.com/linkedin/goavro" in any of:
> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro
> (vendor tree)
> /home/elharo/.gradle/go/binary/1.12/go/src/github.com/linkedin/goavro
> (from $GOROOT)
> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/linkedin/goavro
> (from $GOPATH)
>
> > Task :sdks:go:examples:goVet FAILED
>
> I'm also seeing failures in ClickHouseIOTest:
>
> > Task :sdks:java:io:clickhouse:test
>
> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
> java.lang.IllegalStateException
>
> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED
> java.lang.NullPointerException
>
> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
> java.lang.IllegalStateException
>
> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED
> java.lang.NullPointerException
>
> --
> Elliotte Rusty Harold
> elharo@ibiblio.org
--
Elliotte Rusty Harold
elharo@ibiblio.org