You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@fluo.apache.org by Kenneth McFarland <ke...@apache.org> on 2019/11/27 19:30:50 UTC

OracleIT and Rat check failures

Hello,

This is just a quick inquiry if anyone else running mvn verify is getting
rat errors that prevent maven from completing.

 I'm using -Drat.skip=true to bypass but is this normal?

Last thing is OracleIT is throwing errors again. I've checked out the new
code and am building on 16.10 Ubuntu x64.

If this is something others are experiencing maybe we can see if there is
an open ticket for OracleIT (thought we closed it with switch on lock
implementation in Curator). Rat check is new to me.

Thanks a bunch. This is informal, if I'm not insane I'll do a formal ticket
or something. Again, thanks!

Happy holidays also.

I

Re: OracleIT and Rat check failures

Posted by Christopher <ct...@apache.org>.

I wouldn't say "ok with it". I actually find it quite annoying... and
would love to see those expected stack traces suppressed in the
output... but haven't looked into it to figure out how to do it
myself.

(Re: Happy holidays: thanks! same to you!)

On Wed, Dec 18, 2019 at 2:30 PM Kenneth McFarland
<gl...@gmail.com> wrote:
>
> @Christopher I'll defer to your expertise. Since the test passes and
> everyone is ok with it, it's probably better to just leave it.
>
> Happy holidays
>
> On Tue, Dec 17, 2019, 10:48 PM Christopher <ct...@apache.org> wrote:
>
> > I'm not necessarily opposed, but I'm not sure it will help above and
> > beyond the fact that the test passes regardless of those messages. If
> > possible, it'd probably be better to modify the logging configuration
> > during the execution of that test to make it less spammy, rather than
> > add more debug messages.
> >
> > On Sat, Dec 14, 2019 at 12:51 PM Kenneth McFarland
> > <gl...@gmail.com> wrote:
> > >
> > > Does anyone have any issue with the idea of putting some debug message
> > that
> > > informs the user the OracleIT errors are expected?
> > >
> > > I'm in favor due to the limitations of my memory. I got confused. Maybe
> > > just wrap the output. Any objections? I feel it's good documentation.
> > >
> > > On Mon, Dec 2, 2019, 3:00 PM Christopher <ct...@apache.org> wrote:
> > >
> > > > I wouldn't worry about documenting it. It's more of a tooling (and
> > > > tooling familiarity) issue than anything pertaining to Fluo, so I
> > > > would consider it out of scope of Fluo's own documentation. Besides,
> > > > documenting everything that can go wrong with build tooling could
> > > > easily overwhelm the docs. It's better to just work through it in the
> > > > community forums like this email list, when somebody encounters an
> > > > issue.
> > > >
> > > > One reason this might not have been more obvious is that the
> > > > `.gitignore` file in Fluo is too overly-broad. It matches all files
> > > > and directories named `target` anywhere in the project (even those no
> > > > longer part of the project). So, `git status` wouldn't show them...
> > > > they'd be hidden instead. In Accumulo, we addressed this by making the
> > > > `.gitignore` file only match on `/target/` (this pattern ignores only
> > > > directories that are at the same depth as the `.gitignore` file
> > > > itself), and having a separate `.gitignore` file for each module with
> > > > the same contents. That way, if we removed a directory, no
> > > > `.gitignore` pattern would match on it any more. We could do the same
> > > > in Fluo.
> > > >
> > > > Another option would be that somebody could submit a patch for the
> > > > apache-rat-plugin that fixes its default excludes pattern matching to
> > > > use the same logic as that of the `.gitignore` pattern matching. It
> > > > would fix at least this plugin... but other plugins could still behave
> > > > badly when the workspace is "dirty" in this way.
> > > >
> > > > On Thu, Nov 28, 2019 at 3:26 PM Kenneth McFarland
> > > > <gl...@gmail.com> wrote:
> > > > >
> > > > > Thank you Christopher, that worked perfectly. Issue resolved, and I
> > > > learned
> > > > > something. Thanks again!
> > > > >
> > > > > I'd be happy to add this to some documentation but not sure where it
> > > > would
> > > > > belong. Might be a niche thing.
> > > > >
> > > > > Kenny
> > > > >
> > > > > On Thu, Nov 28, 2019, 11:30 AM Christopher <ct...@apache.org>
> > wrote:
> > > > >
> > > > > > Okay, so it looks like the failures you're seeing are the result
> > of an
> > > > old
> > > > > > modules/integration directory that was removed or renamed. Try
> > > > resetting
> > > > > > your clone with `git clean -fdx`
> > > > > >
> > > > > > On Thu, Nov 28, 2019, 00:39 Kenneth McFarland <
> > > > > > glorifiedcalculator@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > @Christopher : Good, I thought that race condition was fixed in a
> > > > > > previous
> > > > > > > issue so I was concerned and should have looked closer. Thank you
> > > > for the
> > > > > > > catch sir!
> > > > > > >
> > > > > > > I'm still confused as to why the RAT check is throwing errors. I
> > am
> > > > > > > working off a clean master with the same hash as the one Joe
> > > > mentioned
> > > > > > > (commit 549d645addb330f4ae2e074447428cb86b5a9a3f).
> > > > > > >
> > > > > > >  Here is some environment info:
> > > > > > >
> > > > > > > *MAVEN*
> > > > > > > Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
> > > > > > > 2017-10-18T00:58:13-07:00)
> > > > > > > Maven home: /opt/apache-maven-3.5.2
> > > > > > > Java version: 1.8.0_161, vendor: Oracle Corporation
> > > > > > > Java home: /usr/lib/jvm/jdk1.8.0_161/jre
> > > > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > > > OS name: "linux", version: "4.15.0-70-generic", arch: "amd64",
> > > > family:
> > > > > > > "unix"
> > > > > > >
> > > > > > > *JAVA*
> > > > > > > openjdk 11.0.5 2019-10-15
> > > > > > > OpenJDK Runtime Environment (build
> > > > 11.0.5+10-post-Ubuntu-2ubuntu116.04)
> > > > > > > OpenJDK 64-Bit Server VM (build
> > 11.0.5+10-post-Ubuntu-2ubuntu116.04,
> > > > > > mixed
> > > > > > > mode, sharing)
> > > > > > >
> > > > > > > I also have a few other java versions installed like the Oracle
> > 1.8.0
> > > > > > that
> > > > > > > maven reports above.
> > > > > > >
> > > > > > > *Operating System*
> > > > > > > Linux ubuntu 4.15.0-70-generic #79~16.04.1-Ubuntu SMP Tue Nov 12
> > > > 14:01:10
> > > > > > > UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> > > > > > >
> > > > > > > I will try this on my MacI notice the apache-rat
> > setDefaultExcludes
> > > > > > > variable is set to true. I'm currently baffled. The errors are
> > the
> > > > same
> > > > > > if
> > > > > > > I run 'mvn apache-rat:check' or if I run 'mvn verify' or 'mvn
> > clean
> > > > > > > verify''. The logs are kinda large so I simply piped it into a
> > text
> > > > file,
> > > > > > > as well as included the rat file.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Nov 27, 2019 at 7:06 PM Christopher <ctubbsii@apache.org
> > >
> > > > wrote:
> > > > > > >
> > > > > > >> I think the stack traces in the logs/test output are intended
> > for
> > > > that
> > > > > > >> test, because it's specifically checking error handling for a
> > > > running
> > > > > > >> server (if I remember correctly).
> > > > > > >>
> > > > > > >> On Wed, Nov 27, 2019, 18:42 Joseph Koshakow <ko...@gmail.com>
> > > > wrote:
> > > > > > >>
> > > > > > >> > I'm running `mvn clean verify` on the master branch
> > > > > > >> > (549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 1.8 on
> > > > Ubuntu
> > > > > > >> 18.04
> > > > > > >> > x86_64. I don't see any issues related to Rat. OracleIT passes
> > > > and the
> > > > > > >> > entire build is successful, but OracleIT does print the
> > following
> > > > > > >> errors in
> > > > > > >> > the logs
> > > > > > >> >
> > > > > > >> > Running org.apache.fluo.integration.impl.OracleIT
> > > > > > >> > 2019-11-27 18:18:24,087 [thrift.ProcessFunction] ERROR:
> > Internal
> > > > error
> > > > > > >> > processing getTimestamps
> > > > > > >> > java.lang.IllegalStateException: Received timestamp request
> > but
> > > > Oracle
> > > > > > >> is
> > > > > > >> > not leader
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.oracle.OracleServer.getTimestampsImpl(OracleServer.java:244)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.oracle.OracleServer.getTimestamps(OracleServer.java:224)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:279)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:257)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.shaded.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.shaded.thrift.server.Invocation.run(Invocation.java:18)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > >> >   at java.lang.Thread.run(Thread.java:748)
> > > > > > >> > 2019-11-27 18:18:24,091 [oracle.OracleClient] ERROR:
> > TException
> > > > > > >> occurred in
> > > > > > >> > doWork() method
> > > > > > >> > org.apache.fluo.core.shaded.thrift.TApplicationException:
> > Internal
> > > > > > error
> > > > > > >> > processing getTimestamps
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.shaded.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.thrift.OracleService$Client.recv_getTimestamps(OracleService.java:86)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.thrift.OracleService$Client.getTimestamps(OracleService.java:73)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.doWork(OracleClient.java:182)
> > > > > > >> >   at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.run(OracleClient.java:116)
> > > > > > >> >   at java.lang.Thread.run(Thread.java:748)
> > > > > > >> > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time
> > elapsed:
> > > > 22.911
> > > > > > >> sec
> > > > > > >> > - in org.apache.fluo.integration.impl.OracleIT
> > > > > > >> >
> > > > > > >> > -Joe
> > > > > > >> >
> > > > > > >> > On Wed, Nov 27, 2019 at 5:18 PM Christopher <
> > ctubbsii@apache.org>
> > > > > > >> wrote:
> > > > > > >> >
> > > > > > >> > > I have not seen any errors with `mvn clean verify` locally,
> > but
> > > > > > >> > > occasionally, there seems to be errors on Travis.
> > > > > > >> > > I'm building the master branch (currently
> > > > > > >> > > 549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on
> > > > Fedora
> > > > > > 31
> > > > > > >> > > x86_64 with no problems.
> > > > > > >> > >
> > > > > > >> > > On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
> > > > > > >> > > <ke...@apache.org> wrote:
> > > > > > >> > > >
> > > > > > >> > > > Hello,
> > > > > > >> > > >
> > > > > > >> > > > This is just a quick inquiry if anyone else running mvn
> > > > verify is
> > > > > > >> > getting
> > > > > > >> > > > rat errors that prevent maven from completing.
> > > > > > >> > > >
> > > > > > >> > > >  I'm using -Drat.skip=true to bypass but is this normal?
> > > > > > >> > > >
> > > > > > >> > > > Last thing is OracleIT is throwing errors again. I've
> > checked
> > > > out
> > > > > > >> the
> > > > > > >> > new
> > > > > > >> > > > code and am building on 16.10 Ubuntu x64.
> > > > > > >> > > >
> > > > > > >> > > > If this is something others are experiencing maybe we can
> > see
> > > > if
> > > > > > >> there
> > > > > > >> > is
> > > > > > >> > > > an open ticket for OracleIT (thought we closed it with
> > switch
> > > > on
> > > > > > >> lock
> > > > > > >> > > > implementation in Curator). Rat check is new to me.
> > > > > > >> > > >
> > > > > > >> > > > Thanks a bunch. This is informal, if I'm not insane I'll
> > do a
> > > > > > formal
> > > > > > >> > > ticket
> > > > > > >> > > > or something. Again, thanks!
> > > > > > >> > > >
> > > > > > >> > > > Happy holidays also.
> > > > > > >> > > >
> > > > > > >> > > > I
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > >
> > > >
> >

Re: OracleIT and Rat check failures

Posted by Kenneth McFarland <gl...@gmail.com>.

@Christopher I'll defer to your expertise. Since the test passes and
everyone is ok with it, it's probably better to just leave it.

Happy holidays

On Tue, Dec 17, 2019, 10:48 PM Christopher <ct...@apache.org> wrote:

> I'm not necessarily opposed, but I'm not sure it will help above and
> beyond the fact that the test passes regardless of those messages. If
> possible, it'd probably be better to modify the logging configuration
> during the execution of that test to make it less spammy, rather than
> add more debug messages.
>
> On Sat, Dec 14, 2019 at 12:51 PM Kenneth McFarland
> <gl...@gmail.com> wrote:
> >
> > Does anyone have any issue with the idea of putting some debug message
> that
> > informs the user the OracleIT errors are expected?
> >
> > I'm in favor due to the limitations of my memory. I got confused. Maybe
> > just wrap the output. Any objections? I feel it's good documentation.
> >
> > On Mon, Dec 2, 2019, 3:00 PM Christopher <ct...@apache.org> wrote:
> >
> > > I wouldn't worry about documenting it. It's more of a tooling (and
> > > tooling familiarity) issue than anything pertaining to Fluo, so I
> > > would consider it out of scope of Fluo's own documentation. Besides,
> > > documenting everything that can go wrong with build tooling could
> > > easily overwhelm the docs. It's better to just work through it in the
> > > community forums like this email list, when somebody encounters an
> > > issue.
> > >
> > > One reason this might not have been more obvious is that the
> > > `.gitignore` file in Fluo is too overly-broad. It matches all files
> > > and directories named `target` anywhere in the project (even those no
> > > longer part of the project). So, `git status` wouldn't show them...
> > > they'd be hidden instead. In Accumulo, we addressed this by making the
> > > `.gitignore` file only match on `/target/` (this pattern ignores only
> > > directories that are at the same depth as the `.gitignore` file
> > > itself), and having a separate `.gitignore` file for each module with
> > > the same contents. That way, if we removed a directory, no
> > > `.gitignore` pattern would match on it any more. We could do the same
> > > in Fluo.
> > >
> > > Another option would be that somebody could submit a patch for the
> > > apache-rat-plugin that fixes its default excludes pattern matching to
> > > use the same logic as that of the `.gitignore` pattern matching. It
> > > would fix at least this plugin... but other plugins could still behave
> > > badly when the workspace is "dirty" in this way.
> > >
> > > On Thu, Nov 28, 2019 at 3:26 PM Kenneth McFarland
> > > <gl...@gmail.com> wrote:
> > > >
> > > > Thank you Christopher, that worked perfectly. Issue resolved, and I
> > > learned
> > > > something. Thanks again!
> > > >
> > > > I'd be happy to add this to some documentation but not sure where it
> > > would
> > > > belong. Might be a niche thing.
> > > >
> > > > Kenny
> > > >
> > > > On Thu, Nov 28, 2019, 11:30 AM Christopher <ct...@apache.org>
> wrote:
> > > >
> > > > > Okay, so it looks like the failures you're seeing are the result
> of an
> > > old
> > > > > modules/integration directory that was removed or renamed. Try
> > > resetting
> > > > > your clone with `git clean -fdx`
> > > > >
> > > > > On Thu, Nov 28, 2019, 00:39 Kenneth McFarland <
> > > > > glorifiedcalculator@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > @Christopher : Good, I thought that race condition was fixed in a
> > > > > previous
> > > > > > issue so I was concerned and should have looked closer. Thank you
> > > for the
> > > > > > catch sir!
> > > > > >
> > > > > > I'm still confused as to why the RAT check is throwing errors. I
> am
> > > > > > working off a clean master with the same hash as the one Joe
> > > mentioned
> > > > > > (commit 549d645addb330f4ae2e074447428cb86b5a9a3f).
> > > > > >
> > > > > >  Here is some environment info:
> > > > > >
> > > > > > *MAVEN*
> > > > > > Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
> > > > > > 2017-10-18T00:58:13-07:00)
> > > > > > Maven home: /opt/apache-maven-3.5.2
> > > > > > Java version: 1.8.0_161, vendor: Oracle Corporation
> > > > > > Java home: /usr/lib/jvm/jdk1.8.0_161/jre
> > > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > > OS name: "linux", version: "4.15.0-70-generic", arch: "amd64",
> > > family:
> > > > > > "unix"
> > > > > >
> > > > > > *JAVA*
> > > > > > openjdk 11.0.5 2019-10-15
> > > > > > OpenJDK Runtime Environment (build
> > > 11.0.5+10-post-Ubuntu-2ubuntu116.04)
> > > > > > OpenJDK 64-Bit Server VM (build
> 11.0.5+10-post-Ubuntu-2ubuntu116.04,
> > > > > mixed
> > > > > > mode, sharing)
> > > > > >
> > > > > > I also have a few other java versions installed like the Oracle
> 1.8.0
> > > > > that
> > > > > > maven reports above.
> > > > > >
> > > > > > *Operating System*
> > > > > > Linux ubuntu 4.15.0-70-generic #79~16.04.1-Ubuntu SMP Tue Nov 12
> > > 14:01:10
> > > > > > UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> > > > > >
> > > > > > I will try this on my MacI notice the apache-rat
> setDefaultExcludes
> > > > > > variable is set to true. I'm currently baffled. The errors are
> the
> > > same
> > > > > if
> > > > > > I run 'mvn apache-rat:check' or if I run 'mvn verify' or 'mvn
> clean
> > > > > > verify''. The logs are kinda large so I simply piped it into a
> text
> > > file,
> > > > > > as well as included the rat file.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Nov 27, 2019 at 7:06 PM Christopher <ctubbsii@apache.org
> >
> > > wrote:
> > > > > >
> > > > > >> I think the stack traces in the logs/test output are intended
> for
> > > that
> > > > > >> test, because it's specifically checking error handling for a
> > > running
> > > > > >> server (if I remember correctly).
> > > > > >>
> > > > > >> On Wed, Nov 27, 2019, 18:42 Joseph Koshakow <ko...@gmail.com>
> > > wrote:
> > > > > >>
> > > > > >> > I'm running `mvn clean verify` on the master branch
> > > > > >> > (549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 1.8 on
> > > Ubuntu
> > > > > >> 18.04
> > > > > >> > x86_64. I don't see any issues related to Rat. OracleIT passes
> > > and the
> > > > > >> > entire build is successful, but OracleIT does print the
> following
> > > > > >> errors in
> > > > > >> > the logs
> > > > > >> >
> > > > > >> > Running org.apache.fluo.integration.impl.OracleIT
> > > > > >> > 2019-11-27 18:18:24,087 [thrift.ProcessFunction] ERROR:
> Internal
> > > error
> > > > > >> > processing getTimestamps
> > > > > >> > java.lang.IllegalStateException: Received timestamp request
> but
> > > Oracle
> > > > > >> is
> > > > > >> > not leader
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.oracle.OracleServer.getTimestampsImpl(OracleServer.java:244)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.oracle.OracleServer.getTimestamps(OracleServer.java:224)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:279)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:257)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.shaded.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.shaded.thrift.server.Invocation.run(Invocation.java:18)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > >> >   at java.lang.Thread.run(Thread.java:748)
> > > > > >> > 2019-11-27 18:18:24,091 [oracle.OracleClient] ERROR:
> TException
> > > > > >> occurred in
> > > > > >> > doWork() method
> > > > > >> > org.apache.fluo.core.shaded.thrift.TApplicationException:
> Internal
> > > > > error
> > > > > >> > processing getTimestamps
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.shaded.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.thrift.OracleService$Client.recv_getTimestamps(OracleService.java:86)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.thrift.OracleService$Client.getTimestamps(OracleService.java:73)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.doWork(OracleClient.java:182)
> > > > > >> >   at
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.run(OracleClient.java:116)
> > > > > >> >   at java.lang.Thread.run(Thread.java:748)
> > > > > >> > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time
> elapsed:
> > > 22.911
> > > > > >> sec
> > > > > >> > - in org.apache.fluo.integration.impl.OracleIT
> > > > > >> >
> > > > > >> > -Joe
> > > > > >> >
> > > > > >> > On Wed, Nov 27, 2019 at 5:18 PM Christopher <
> ctubbsii@apache.org>
> > > > > >> wrote:
> > > > > >> >
> > > > > >> > > I have not seen any errors with `mvn clean verify` locally,
> but
> > > > > >> > > occasionally, there seems to be errors on Travis.
> > > > > >> > > I'm building the master branch (currently
> > > > > >> > > 549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on
> > > Fedora
> > > > > 31
> > > > > >> > > x86_64 with no problems.
> > > > > >> > >
> > > > > >> > > On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
> > > > > >> > > <ke...@apache.org> wrote:
> > > > > >> > > >
> > > > > >> > > > Hello,
> > > > > >> > > >
> > > > > >> > > > This is just a quick inquiry if anyone else running mvn
> > > verify is
> > > > > >> > getting
> > > > > >> > > > rat errors that prevent maven from completing.
> > > > > >> > > >
> > > > > >> > > >  I'm using -Drat.skip=true to bypass but is this normal?
> > > > > >> > > >
> > > > > >> > > > Last thing is OracleIT is throwing errors again. I've
> checked
> > > out
> > > > > >> the
> > > > > >> > new
> > > > > >> > > > code and am building on 16.10 Ubuntu x64.
> > > > > >> > > >
> > > > > >> > > > If this is something others are experiencing maybe we can
> see
> > > if
> > > > > >> there
> > > > > >> > is
> > > > > >> > > > an open ticket for OracleIT (thought we closed it with
> switch
> > > on
> > > > > >> lock
> > > > > >> > > > implementation in Curator). Rat check is new to me.
> > > > > >> > > >
> > > > > >> > > > Thanks a bunch. This is informal, if I'm not insane I'll
> do a
> > > > > formal
> > > > > >> > > ticket
> > > > > >> > > > or something. Again, thanks!
> > > > > >> > > >
> > > > > >> > > > Happy holidays also.
> > > > > >> > > >
> > > > > >> > > > I
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > >
> > >
>

Re: OracleIT and Rat check failures

Posted by Christopher <ct...@apache.org>.

I'm not necessarily opposed, but I'm not sure it will help above and
beyond the fact that the test passes regardless of those messages. If
possible, it'd probably be better to modify the logging configuration
during the execution of that test to make it less spammy, rather than
add more debug messages.

On Sat, Dec 14, 2019 at 12:51 PM Kenneth McFarland
<gl...@gmail.com> wrote:
>
> Does anyone have any issue with the idea of putting some debug message that
> informs the user the OracleIT errors are expected?
>
> I'm in favor due to the limitations of my memory. I got confused. Maybe
> just wrap the output. Any objections? I feel it's good documentation.
>
> On Mon, Dec 2, 2019, 3:00 PM Christopher <ct...@apache.org> wrote:
>
> > I wouldn't worry about documenting it. It's more of a tooling (and
> > tooling familiarity) issue than anything pertaining to Fluo, so I
> > would consider it out of scope of Fluo's own documentation. Besides,
> > documenting everything that can go wrong with build tooling could
> > easily overwhelm the docs. It's better to just work through it in the
> > community forums like this email list, when somebody encounters an
> > issue.
> >
> > One reason this might not have been more obvious is that the
> > `.gitignore` file in Fluo is too overly-broad. It matches all files
> > and directories named `target` anywhere in the project (even those no
> > longer part of the project). So, `git status` wouldn't show them...
> > they'd be hidden instead. In Accumulo, we addressed this by making the
> > `.gitignore` file only match on `/target/` (this pattern ignores only
> > directories that are at the same depth as the `.gitignore` file
> > itself), and having a separate `.gitignore` file for each module with
> > the same contents. That way, if we removed a directory, no
> > `.gitignore` pattern would match on it any more. We could do the same
> > in Fluo.
> >
> > Another option would be that somebody could submit a patch for the
> > apache-rat-plugin that fixes its default excludes pattern matching to
> > use the same logic as that of the `.gitignore` pattern matching. It
> > would fix at least this plugin... but other plugins could still behave
> > badly when the workspace is "dirty" in this way.
> >
> > On Thu, Nov 28, 2019 at 3:26 PM Kenneth McFarland
> > <gl...@gmail.com> wrote:
> > >
> > > Thank you Christopher, that worked perfectly. Issue resolved, and I
> > learned
> > > something. Thanks again!
> > >
> > > I'd be happy to add this to some documentation but not sure where it
> > would
> > > belong. Might be a niche thing.
> > >
> > > Kenny
> > >
> > > On Thu, Nov 28, 2019, 11:30 AM Christopher <ct...@apache.org> wrote:
> > >
> > > > Okay, so it looks like the failures you're seeing are the result of an
> > old
> > > > modules/integration directory that was removed or renamed. Try
> > resetting
> > > > your clone with `git clean -fdx`
> > > >
> > > > On Thu, Nov 28, 2019, 00:39 Kenneth McFarland <
> > > > glorifiedcalculator@gmail.com>
> > > > wrote:
> > > >
> > > > > @Christopher : Good, I thought that race condition was fixed in a
> > > > previous
> > > > > issue so I was concerned and should have looked closer. Thank you
> > for the
> > > > > catch sir!
> > > > >
> > > > > I'm still confused as to why the RAT check is throwing errors. I am
> > > > > working off a clean master with the same hash as the one Joe
> > mentioned
> > > > > (commit 549d645addb330f4ae2e074447428cb86b5a9a3f).
> > > > >
> > > > >  Here is some environment info:
> > > > >
> > > > > *MAVEN*
> > > > > Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
> > > > > 2017-10-18T00:58:13-07:00)
> > > > > Maven home: /opt/apache-maven-3.5.2
> > > > > Java version: 1.8.0_161, vendor: Oracle Corporation
> > > > > Java home: /usr/lib/jvm/jdk1.8.0_161/jre
> > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > OS name: "linux", version: "4.15.0-70-generic", arch: "amd64",
> > family:
> > > > > "unix"
> > > > >
> > > > > *JAVA*
> > > > > openjdk 11.0.5 2019-10-15
> > > > > OpenJDK Runtime Environment (build
> > 11.0.5+10-post-Ubuntu-2ubuntu116.04)
> > > > > OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Ubuntu-2ubuntu116.04,
> > > > mixed
> > > > > mode, sharing)
> > > > >
> > > > > I also have a few other java versions installed like the Oracle 1.8.0
> > > > that
> > > > > maven reports above.
> > > > >
> > > > > *Operating System*
> > > > > Linux ubuntu 4.15.0-70-generic #79~16.04.1-Ubuntu SMP Tue Nov 12
> > 14:01:10
> > > > > UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> > > > >
> > > > > I will try this on my MacI notice the apache-rat setDefaultExcludes
> > > > > variable is set to true. I'm currently baffled. The errors are the
> > same
> > > > if
> > > > > I run 'mvn apache-rat:check' or if I run 'mvn verify' or 'mvn clean
> > > > > verify''. The logs are kinda large so I simply piped it into a text
> > file,
> > > > > as well as included the rat file.
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Nov 27, 2019 at 7:06 PM Christopher <ct...@apache.org>
> > wrote:
> > > > >
> > > > >> I think the stack traces in the logs/test output are intended for
> > that
> > > > >> test, because it's specifically checking error handling for a
> > running
> > > > >> server (if I remember correctly).
> > > > >>
> > > > >> On Wed, Nov 27, 2019, 18:42 Joseph Koshakow <ko...@gmail.com>
> > wrote:
> > > > >>
> > > > >> > I'm running `mvn clean verify` on the master branch
> > > > >> > (549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 1.8 on
> > Ubuntu
> > > > >> 18.04
> > > > >> > x86_64. I don't see any issues related to Rat. OracleIT passes
> > and the
> > > > >> > entire build is successful, but OracleIT does print the following
> > > > >> errors in
> > > > >> > the logs
> > > > >> >
> > > > >> > Running org.apache.fluo.integration.impl.OracleIT
> > > > >> > 2019-11-27 18:18:24,087 [thrift.ProcessFunction] ERROR: Internal
> > error
> > > > >> > processing getTimestamps
> > > > >> > java.lang.IllegalStateException: Received timestamp request but
> > Oracle
> > > > >> is
> > > > >> > not leader
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.oracle.OracleServer.getTimestampsImpl(OracleServer.java:244)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.oracle.OracleServer.getTimestamps(OracleServer.java:224)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:279)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:257)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.shaded.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.shaded.thrift.server.Invocation.run(Invocation.java:18)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > >> >   at java.lang.Thread.run(Thread.java:748)
> > > > >> > 2019-11-27 18:18:24,091 [oracle.OracleClient] ERROR: TException
> > > > >> occurred in
> > > > >> > doWork() method
> > > > >> > org.apache.fluo.core.shaded.thrift.TApplicationException: Internal
> > > > error
> > > > >> > processing getTimestamps
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.shaded.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.thrift.OracleService$Client.recv_getTimestamps(OracleService.java:86)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.thrift.OracleService$Client.getTimestamps(OracleService.java:73)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.doWork(OracleClient.java:182)
> > > > >> >   at
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.run(OracleClient.java:116)
> > > > >> >   at java.lang.Thread.run(Thread.java:748)
> > > > >> > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 22.911
> > > > >> sec
> > > > >> > - in org.apache.fluo.integration.impl.OracleIT
> > > > >> >
> > > > >> > -Joe
> > > > >> >
> > > > >> > On Wed, Nov 27, 2019 at 5:18 PM Christopher <ct...@apache.org>
> > > > >> wrote:
> > > > >> >
> > > > >> > > I have not seen any errors with `mvn clean verify` locally, but
> > > > >> > > occasionally, there seems to be errors on Travis.
> > > > >> > > I'm building the master branch (currently
> > > > >> > > 549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on
> > Fedora
> > > > 31
> > > > >> > > x86_64 with no problems.
> > > > >> > >
> > > > >> > > On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
> > > > >> > > <ke...@apache.org> wrote:
> > > > >> > > >
> > > > >> > > > Hello,
> > > > >> > > >
> > > > >> > > > This is just a quick inquiry if anyone else running mvn
> > verify is
> > > > >> > getting
> > > > >> > > > rat errors that prevent maven from completing.
> > > > >> > > >
> > > > >> > > >  I'm using -Drat.skip=true to bypass but is this normal?
> > > > >> > > >
> > > > >> > > > Last thing is OracleIT is throwing errors again. I've checked
> > out
> > > > >> the
> > > > >> > new
> > > > >> > > > code and am building on 16.10 Ubuntu x64.
> > > > >> > > >
> > > > >> > > > If this is something others are experiencing maybe we can see
> > if
> > > > >> there
> > > > >> > is
> > > > >> > > > an open ticket for OracleIT (thought we closed it with switch
> > on
> > > > >> lock
> > > > >> > > > implementation in Curator). Rat check is new to me.
> > > > >> > > >
> > > > >> > > > Thanks a bunch. This is informal, if I'm not insane I'll do a
> > > > formal
> > > > >> > > ticket
> > > > >> > > > or something. Again, thanks!
> > > > >> > > >
> > > > >> > > > Happy holidays also.
> > > > >> > > >
> > > > >> > > > I
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > >
> >

Re: OracleIT and Rat check failures

Posted by Kenneth McFarland <gl...@gmail.com>.

Does anyone have any issue with the idea of putting some debug message that
informs the user the OracleIT errors are expected?

I'm in favor due to the limitations of my memory. I got confused. Maybe
just wrap the output. Any objections? I feel it's good documentation.

On Mon, Dec 2, 2019, 3:00 PM Christopher <ct...@apache.org> wrote:

> I wouldn't worry about documenting it. It's more of a tooling (and
> tooling familiarity) issue than anything pertaining to Fluo, so I
> would consider it out of scope of Fluo's own documentation. Besides,
> documenting everything that can go wrong with build tooling could
> easily overwhelm the docs. It's better to just work through it in the
> community forums like this email list, when somebody encounters an
> issue.
>
> One reason this might not have been more obvious is that the
> `.gitignore` file in Fluo is too overly-broad. It matches all files
> and directories named `target` anywhere in the project (even those no
> longer part of the project). So, `git status` wouldn't show them...
> they'd be hidden instead. In Accumulo, we addressed this by making the
> `.gitignore` file only match on `/target/` (this pattern ignores only
> directories that are at the same depth as the `.gitignore` file
> itself), and having a separate `.gitignore` file for each module with
> the same contents. That way, if we removed a directory, no
> `.gitignore` pattern would match on it any more. We could do the same
> in Fluo.
>
> Another option would be that somebody could submit a patch for the
> apache-rat-plugin that fixes its default excludes pattern matching to
> use the same logic as that of the `.gitignore` pattern matching. It
> would fix at least this plugin... but other plugins could still behave
> badly when the workspace is "dirty" in this way.
>
> On Thu, Nov 28, 2019 at 3:26 PM Kenneth McFarland
> <gl...@gmail.com> wrote:
> >
> > Thank you Christopher, that worked perfectly. Issue resolved, and I
> learned
> > something. Thanks again!
> >
> > I'd be happy to add this to some documentation but not sure where it
> would
> > belong. Might be a niche thing.
> >
> > Kenny
> >
> > On Thu, Nov 28, 2019, 11:30 AM Christopher <ct...@apache.org> wrote:
> >
> > > Okay, so it looks like the failures you're seeing are the result of an
> old
> > > modules/integration directory that was removed or renamed. Try
> resetting
> > > your clone with `git clean -fdx`
> > >
> > > On Thu, Nov 28, 2019, 00:39 Kenneth McFarland <
> > > glorifiedcalculator@gmail.com>
> > > wrote:
> > >
> > > > @Christopher : Good, I thought that race condition was fixed in a
> > > previous
> > > > issue so I was concerned and should have looked closer. Thank you
> for the
> > > > catch sir!
> > > >
> > > > I'm still confused as to why the RAT check is throwing errors. I am
> > > > working off a clean master with the same hash as the one Joe
> mentioned
> > > > (commit 549d645addb330f4ae2e074447428cb86b5a9a3f).
> > > >
> > > >  Here is some environment info:
> > > >
> > > > *MAVEN*
> > > > Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
> > > > 2017-10-18T00:58:13-07:00)
> > > > Maven home: /opt/apache-maven-3.5.2
> > > > Java version: 1.8.0_161, vendor: Oracle Corporation
> > > > Java home: /usr/lib/jvm/jdk1.8.0_161/jre
> > > > Default locale: en_US, platform encoding: UTF-8
> > > > OS name: "linux", version: "4.15.0-70-generic", arch: "amd64",
> family:
> > > > "unix"
> > > >
> > > > *JAVA*
> > > > openjdk 11.0.5 2019-10-15
> > > > OpenJDK Runtime Environment (build
> 11.0.5+10-post-Ubuntu-2ubuntu116.04)
> > > > OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Ubuntu-2ubuntu116.04,
> > > mixed
> > > > mode, sharing)
> > > >
> > > > I also have a few other java versions installed like the Oracle 1.8.0
> > > that
> > > > maven reports above.
> > > >
> > > > *Operating System*
> > > > Linux ubuntu 4.15.0-70-generic #79~16.04.1-Ubuntu SMP Tue Nov 12
> 14:01:10
> > > > UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> > > >
> > > > I will try this on my MacI notice the apache-rat setDefaultExcludes
> > > > variable is set to true. I'm currently baffled. The errors are the
> same
> > > if
> > > > I run 'mvn apache-rat:check' or if I run 'mvn verify' or 'mvn clean
> > > > verify''. The logs are kinda large so I simply piped it into a text
> file,
> > > > as well as included the rat file.
> > > >
> > > >
> > > >
> > > > On Wed, Nov 27, 2019 at 7:06 PM Christopher <ct...@apache.org>
> wrote:
> > > >
> > > >> I think the stack traces in the logs/test output are intended for
> that
> > > >> test, because it's specifically checking error handling for a
> running
> > > >> server (if I remember correctly).
> > > >>
> > > >> On Wed, Nov 27, 2019, 18:42 Joseph Koshakow <ko...@gmail.com>
> wrote:
> > > >>
> > > >> > I'm running `mvn clean verify` on the master branch
> > > >> > (549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 1.8 on
> Ubuntu
> > > >> 18.04
> > > >> > x86_64. I don't see any issues related to Rat. OracleIT passes
> and the
> > > >> > entire build is successful, but OracleIT does print the following
> > > >> errors in
> > > >> > the logs
> > > >> >
> > > >> > Running org.apache.fluo.integration.impl.OracleIT
> > > >> > 2019-11-27 18:18:24,087 [thrift.ProcessFunction] ERROR: Internal
> error
> > > >> > processing getTimestamps
> > > >> > java.lang.IllegalStateException: Received timestamp request but
> Oracle
> > > >> is
> > > >> > not leader
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.oracle.OracleServer.getTimestampsImpl(OracleServer.java:244)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.oracle.OracleServer.getTimestamps(OracleServer.java:224)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:279)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:257)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.shaded.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.shaded.thrift.server.Invocation.run(Invocation.java:18)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > >> >   at java.lang.Thread.run(Thread.java:748)
> > > >> > 2019-11-27 18:18:24,091 [oracle.OracleClient] ERROR: TException
> > > >> occurred in
> > > >> > doWork() method
> > > >> > org.apache.fluo.core.shaded.thrift.TApplicationException: Internal
> > > error
> > > >> > processing getTimestamps
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.shaded.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.thrift.OracleService$Client.recv_getTimestamps(OracleService.java:86)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.thrift.OracleService$Client.getTimestamps(OracleService.java:73)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.doWork(OracleClient.java:182)
> > > >> >   at
> > > >> >
> > > >> >
> > > >>
> > >
> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.run(OracleClient.java:116)
> > > >> >   at java.lang.Thread.run(Thread.java:748)
> > > >> > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 22.911
> > > >> sec
> > > >> > - in org.apache.fluo.integration.impl.OracleIT
> > > >> >
> > > >> > -Joe
> > > >> >
> > > >> > On Wed, Nov 27, 2019 at 5:18 PM Christopher <ct...@apache.org>
> > > >> wrote:
> > > >> >
> > > >> > > I have not seen any errors with `mvn clean verify` locally, but
> > > >> > > occasionally, there seems to be errors on Travis.
> > > >> > > I'm building the master branch (currently
> > > >> > > 549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on
> Fedora
> > > 31
> > > >> > > x86_64 with no problems.
> > > >> > >
> > > >> > > On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
> > > >> > > <ke...@apache.org> wrote:
> > > >> > > >
> > > >> > > > Hello,
> > > >> > > >
> > > >> > > > This is just a quick inquiry if anyone else running mvn
> verify is
> > > >> > getting
> > > >> > > > rat errors that prevent maven from completing.
> > > >> > > >
> > > >> > > >  I'm using -Drat.skip=true to bypass but is this normal?
> > > >> > > >
> > > >> > > > Last thing is OracleIT is throwing errors again. I've checked
> out
> > > >> the
> > > >> > new
> > > >> > > > code and am building on 16.10 Ubuntu x64.
> > > >> > > >
> > > >> > > > If this is something others are experiencing maybe we can see
> if
> > > >> there
> > > >> > is
> > > >> > > > an open ticket for OracleIT (thought we closed it with switch
> on
> > > >> lock
> > > >> > > > implementation in Curator). Rat check is new to me.
> > > >> > > >
> > > >> > > > Thanks a bunch. This is informal, if I'm not insane I'll do a
> > > formal
> > > >> > > ticket
> > > >> > > > or something. Again, thanks!
> > > >> > > >
> > > >> > > > Happy holidays also.
> > > >> > > >
> > > >> > > > I
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
>

Re: OracleIT and Rat check failures

Posted by Christopher <ct...@apache.org>.

I wouldn't worry about documenting it. It's more of a tooling (and
tooling familiarity) issue than anything pertaining to Fluo, so I
would consider it out of scope of Fluo's own documentation. Besides,
documenting everything that can go wrong with build tooling could
easily overwhelm the docs. It's better to just work through it in the
community forums like this email list, when somebody encounters an
issue.

One reason this might not have been more obvious is that the
`.gitignore` file in Fluo is too overly-broad. It matches all files
and directories named `target` anywhere in the project (even those no
longer part of the project). So, `git status` wouldn't show them...
they'd be hidden instead. In Accumulo, we addressed this by making the
`.gitignore` file only match on `/target/` (this pattern ignores only
directories that are at the same depth as the `.gitignore` file
itself), and having a separate `.gitignore` file for each module with
the same contents. That way, if we removed a directory, no
`.gitignore` pattern would match on it any more. We could do the same
in Fluo.

Another option would be that somebody could submit a patch for the
apache-rat-plugin that fixes its default excludes pattern matching to
use the same logic as that of the `.gitignore` pattern matching. It
would fix at least this plugin... but other plugins could still behave
badly when the workspace is "dirty" in this way.

On Thu, Nov 28, 2019 at 3:26 PM Kenneth McFarland
<gl...@gmail.com> wrote:
>
> Thank you Christopher, that worked perfectly. Issue resolved, and I learned
> something. Thanks again!
>
> I'd be happy to add this to some documentation but not sure where it would
> belong. Might be a niche thing.
>
> Kenny
>
> On Thu, Nov 28, 2019, 11:30 AM Christopher <ct...@apache.org> wrote:
>
> > Okay, so it looks like the failures you're seeing are the result of an old
> > modules/integration directory that was removed or renamed. Try resetting
> > your clone with `git clean -fdx`
> >
> > On Thu, Nov 28, 2019, 00:39 Kenneth McFarland <
> > glorifiedcalculator@gmail.com>
> > wrote:
> >
> > > @Christopher : Good, I thought that race condition was fixed in a
> > previous
> > > issue so I was concerned and should have looked closer. Thank you for the
> > > catch sir!
> > >
> > > I'm still confused as to why the RAT check is throwing errors. I am
> > > working off a clean master with the same hash as the one Joe mentioned
> > > (commit 549d645addb330f4ae2e074447428cb86b5a9a3f).
> > >
> > >  Here is some environment info:
> > >
> > > *MAVEN*
> > > Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
> > > 2017-10-18T00:58:13-07:00)
> > > Maven home: /opt/apache-maven-3.5.2
> > > Java version: 1.8.0_161, vendor: Oracle Corporation
> > > Java home: /usr/lib/jvm/jdk1.8.0_161/jre
> > > Default locale: en_US, platform encoding: UTF-8
> > > OS name: "linux", version: "4.15.0-70-generic", arch: "amd64", family:
> > > "unix"
> > >
> > > *JAVA*
> > > openjdk 11.0.5 2019-10-15
> > > OpenJDK Runtime Environment (build 11.0.5+10-post-Ubuntu-2ubuntu116.04)
> > > OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Ubuntu-2ubuntu116.04,
> > mixed
> > > mode, sharing)
> > >
> > > I also have a few other java versions installed like the Oracle 1.8.0
> > that
> > > maven reports above.
> > >
> > > *Operating System*
> > > Linux ubuntu 4.15.0-70-generic #79~16.04.1-Ubuntu SMP Tue Nov 12 14:01:10
> > > UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> > >
> > > I will try this on my MacI notice the apache-rat setDefaultExcludes
> > > variable is set to true. I'm currently baffled. The errors are the same
> > if
> > > I run 'mvn apache-rat:check' or if I run 'mvn verify' or 'mvn clean
> > > verify''. The logs are kinda large so I simply piped it into a text file,
> > > as well as included the rat file.
> > >
> > >
> > >
> > > On Wed, Nov 27, 2019 at 7:06 PM Christopher <ct...@apache.org> wrote:
> > >
> > >> I think the stack traces in the logs/test output are intended for that
> > >> test, because it's specifically checking error handling for a running
> > >> server (if I remember correctly).
> > >>
> > >> On Wed, Nov 27, 2019, 18:42 Joseph Koshakow <ko...@gmail.com> wrote:
> > >>
> > >> > I'm running `mvn clean verify` on the master branch
> > >> > (549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 1.8 on Ubuntu
> > >> 18.04
> > >> > x86_64. I don't see any issues related to Rat. OracleIT passes and the
> > >> > entire build is successful, but OracleIT does print the following
> > >> errors in
> > >> > the logs
> > >> >
> > >> > Running org.apache.fluo.integration.impl.OracleIT
> > >> > 2019-11-27 18:18:24,087 [thrift.ProcessFunction] ERROR: Internal error
> > >> > processing getTimestamps
> > >> > java.lang.IllegalStateException: Received timestamp request but Oracle
> > >> is
> > >> > not leader
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.oracle.OracleServer.getTimestampsImpl(OracleServer.java:244)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.oracle.OracleServer.getTimestamps(OracleServer.java:224)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:279)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:257)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.shaded.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.shaded.thrift.server.Invocation.run(Invocation.java:18)
> > >> >   at
> > >> >
> > >> >
> > >>
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >> >   at
> > >> >
> > >> >
> > >>
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > >> >   at java.lang.Thread.run(Thread.java:748)
> > >> > 2019-11-27 18:18:24,091 [oracle.OracleClient] ERROR: TException
> > >> occurred in
> > >> > doWork() method
> > >> > org.apache.fluo.core.shaded.thrift.TApplicationException: Internal
> > error
> > >> > processing getTimestamps
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.shaded.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.thrift.OracleService$Client.recv_getTimestamps(OracleService.java:86)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.thrift.OracleService$Client.getTimestamps(OracleService.java:73)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.doWork(OracleClient.java:182)
> > >> >   at
> > >> >
> > >> >
> > >>
> > org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.run(OracleClient.java:116)
> > >> >   at java.lang.Thread.run(Thread.java:748)
> > >> > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.911
> > >> sec
> > >> > - in org.apache.fluo.integration.impl.OracleIT
> > >> >
> > >> > -Joe
> > >> >
> > >> > On Wed, Nov 27, 2019 at 5:18 PM Christopher <ct...@apache.org>
> > >> wrote:
> > >> >
> > >> > > I have not seen any errors with `mvn clean verify` locally, but
> > >> > > occasionally, there seems to be errors on Travis.
> > >> > > I'm building the master branch (currently
> > >> > > 549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on Fedora
> > 31
> > >> > > x86_64 with no problems.
> > >> > >
> > >> > > On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
> > >> > > <ke...@apache.org> wrote:
> > >> > > >
> > >> > > > Hello,
> > >> > > >
> > >> > > > This is just a quick inquiry if anyone else running mvn verify is
> > >> > getting
> > >> > > > rat errors that prevent maven from completing.
> > >> > > >
> > >> > > >  I'm using -Drat.skip=true to bypass but is this normal?
> > >> > > >
> > >> > > > Last thing is OracleIT is throwing errors again. I've checked out
> > >> the
> > >> > new
> > >> > > > code and am building on 16.10 Ubuntu x64.
> > >> > > >
> > >> > > > If this is something others are experiencing maybe we can see if
> > >> there
> > >> > is
> > >> > > > an open ticket for OracleIT (thought we closed it with switch on
> > >> lock
> > >> > > > implementation in Curator). Rat check is new to me.
> > >> > > >
> > >> > > > Thanks a bunch. This is informal, if I'm not insane I'll do a
> > formal
> > >> > > ticket
> > >> > > > or something. Again, thanks!
> > >> > > >
> > >> > > > Happy holidays also.
> > >> > > >
> > >> > > > I
> > >> > >
> > >> >
> > >>
> > >
> >

Re: OracleIT and Rat check failures

Posted by Kenneth McFarland <gl...@gmail.com>.

Thank you Christopher, that worked perfectly. Issue resolved, and I learned
something. Thanks again!

I'd be happy to add this to some documentation but not sure where it would
belong. Might be a niche thing.

Kenny

On Thu, Nov 28, 2019, 11:30 AM Christopher <ct...@apache.org> wrote:

> Okay, so it looks like the failures you're seeing are the result of an old
> modules/integration directory that was removed or renamed. Try resetting
> your clone with `git clean -fdx`
>
> On Thu, Nov 28, 2019, 00:39 Kenneth McFarland <
> glorifiedcalculator@gmail.com>
> wrote:
>
> > @Christopher : Good, I thought that race condition was fixed in a
> previous
> > issue so I was concerned and should have looked closer. Thank you for the
> > catch sir!
> >
> > I'm still confused as to why the RAT check is throwing errors. I am
> > working off a clean master with the same hash as the one Joe mentioned
> > (commit 549d645addb330f4ae2e074447428cb86b5a9a3f).
> >
> >  Here is some environment info:
> >
> > *MAVEN*
> > Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
> > 2017-10-18T00:58:13-07:00)
> > Maven home: /opt/apache-maven-3.5.2
> > Java version: 1.8.0_161, vendor: Oracle Corporation
> > Java home: /usr/lib/jvm/jdk1.8.0_161/jre
> > Default locale: en_US, platform encoding: UTF-8
> > OS name: "linux", version: "4.15.0-70-generic", arch: "amd64", family:
> > "unix"
> >
> > *JAVA*
> > openjdk 11.0.5 2019-10-15
> > OpenJDK Runtime Environment (build 11.0.5+10-post-Ubuntu-2ubuntu116.04)
> > OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Ubuntu-2ubuntu116.04,
> mixed
> > mode, sharing)
> >
> > I also have a few other java versions installed like the Oracle 1.8.0
> that
> > maven reports above.
> >
> > *Operating System*
> > Linux ubuntu 4.15.0-70-generic #79~16.04.1-Ubuntu SMP Tue Nov 12 14:01:10
> > UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> >
> > I will try this on my MacI notice the apache-rat setDefaultExcludes
> > variable is set to true. I'm currently baffled. The errors are the same
> if
> > I run 'mvn apache-rat:check' or if I run 'mvn verify' or 'mvn clean
> > verify''. The logs are kinda large so I simply piped it into a text file,
> > as well as included the rat file.
> >
> >
> >
> > On Wed, Nov 27, 2019 at 7:06 PM Christopher <ct...@apache.org> wrote:
> >
> >> I think the stack traces in the logs/test output are intended for that
> >> test, because it's specifically checking error handling for a running
> >> server (if I remember correctly).
> >>
> >> On Wed, Nov 27, 2019, 18:42 Joseph Koshakow <ko...@gmail.com> wrote:
> >>
> >> > I'm running `mvn clean verify` on the master branch
> >> > (549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 1.8 on Ubuntu
> >> 18.04
> >> > x86_64. I don't see any issues related to Rat. OracleIT passes and the
> >> > entire build is successful, but OracleIT does print the following
> >> errors in
> >> > the logs
> >> >
> >> > Running org.apache.fluo.integration.impl.OracleIT
> >> > 2019-11-27 18:18:24,087 [thrift.ProcessFunction] ERROR: Internal error
> >> > processing getTimestamps
> >> > java.lang.IllegalStateException: Received timestamp request but Oracle
> >> is
> >> > not leader
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.oracle.OracleServer.getTimestampsImpl(OracleServer.java:244)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.oracle.OracleServer.getTimestamps(OracleServer.java:224)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:279)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:257)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.shaded.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.shaded.thrift.server.Invocation.run(Invocation.java:18)
> >> >   at
> >> >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >> >   at
> >> >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >> >   at java.lang.Thread.run(Thread.java:748)
> >> > 2019-11-27 18:18:24,091 [oracle.OracleClient] ERROR: TException
> >> occurred in
> >> > doWork() method
> >> > org.apache.fluo.core.shaded.thrift.TApplicationException: Internal
> error
> >> > processing getTimestamps
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.shaded.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.thrift.OracleService$Client.recv_getTimestamps(OracleService.java:86)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.thrift.OracleService$Client.getTimestamps(OracleService.java:73)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.doWork(OracleClient.java:182)
> >> >   at
> >> >
> >> >
> >>
> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.run(OracleClient.java:116)
> >> >   at java.lang.Thread.run(Thread.java:748)
> >> > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.911
> >> sec
> >> > - in org.apache.fluo.integration.impl.OracleIT
> >> >
> >> > -Joe
> >> >
> >> > On Wed, Nov 27, 2019 at 5:18 PM Christopher <ct...@apache.org>
> >> wrote:
> >> >
> >> > > I have not seen any errors with `mvn clean verify` locally, but
> >> > > occasionally, there seems to be errors on Travis.
> >> > > I'm building the master branch (currently
> >> > > 549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on Fedora
> 31
> >> > > x86_64 with no problems.
> >> > >
> >> > > On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
> >> > > <ke...@apache.org> wrote:
> >> > > >
> >> > > > Hello,
> >> > > >
> >> > > > This is just a quick inquiry if anyone else running mvn verify is
> >> > getting
> >> > > > rat errors that prevent maven from completing.
> >> > > >
> >> > > >  I'm using -Drat.skip=true to bypass but is this normal?
> >> > > >
> >> > > > Last thing is OracleIT is throwing errors again. I've checked out
> >> the
> >> > new
> >> > > > code and am building on 16.10 Ubuntu x64.
> >> > > >
> >> > > > If this is something others are experiencing maybe we can see if
> >> there
> >> > is
> >> > > > an open ticket for OracleIT (thought we closed it with switch on
> >> lock
> >> > > > implementation in Curator). Rat check is new to me.
> >> > > >
> >> > > > Thanks a bunch. This is informal, if I'm not insane I'll do a
> formal
> >> > > ticket
> >> > > > or something. Again, thanks!
> >> > > >
> >> > > > Happy holidays also.
> >> > > >
> >> > > > I
> >> > >
> >> >
> >>
> >
>

Re: OracleIT and Rat check failures

Posted by Christopher <ct...@apache.org>.

Okay, so it looks like the failures you're seeing are the result of an old
modules/integration directory that was removed or renamed. Try resetting
your clone with `git clean -fdx`

On Thu, Nov 28, 2019, 00:39 Kenneth McFarland <gl...@gmail.com>
wrote:

> @Christopher : Good, I thought that race condition was fixed in a previous
> issue so I was concerned and should have looked closer. Thank you for the
> catch sir!
>
> I'm still confused as to why the RAT check is throwing errors. I am
> working off a clean master with the same hash as the one Joe mentioned
> (commit 549d645addb330f4ae2e074447428cb86b5a9a3f).
>
>  Here is some environment info:
>
> *MAVEN*
> Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
> 2017-10-18T00:58:13-07:00)
> Maven home: /opt/apache-maven-3.5.2
> Java version: 1.8.0_161, vendor: Oracle Corporation
> Java home: /usr/lib/jvm/jdk1.8.0_161/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "4.15.0-70-generic", arch: "amd64", family:
> "unix"
>
> *JAVA*
> openjdk 11.0.5 2019-10-15
> OpenJDK Runtime Environment (build 11.0.5+10-post-Ubuntu-2ubuntu116.04)
> OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Ubuntu-2ubuntu116.04, mixed
> mode, sharing)
>
> I also have a few other java versions installed like the Oracle 1.8.0 that
> maven reports above.
>
> *Operating System*
> Linux ubuntu 4.15.0-70-generic #79~16.04.1-Ubuntu SMP Tue Nov 12 14:01:10
> UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
>
> I will try this on my MacI notice the apache-rat setDefaultExcludes
> variable is set to true. I'm currently baffled. The errors are the same if
> I run 'mvn apache-rat:check' or if I run 'mvn verify' or 'mvn clean
> verify''. The logs are kinda large so I simply piped it into a text file,
> as well as included the rat file.
>
>
>
> On Wed, Nov 27, 2019 at 7:06 PM Christopher <ct...@apache.org> wrote:
>
>> I think the stack traces in the logs/test output are intended for that
>> test, because it's specifically checking error handling for a running
>> server (if I remember correctly).
>>
>> On Wed, Nov 27, 2019, 18:42 Joseph Koshakow <ko...@gmail.com> wrote:
>>
>> > I'm running `mvn clean verify` on the master branch
>> > (549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 1.8 on Ubuntu
>> 18.04
>> > x86_64. I don't see any issues related to Rat. OracleIT passes and the
>> > entire build is successful, but OracleIT does print the following
>> errors in
>> > the logs
>> >
>> > Running org.apache.fluo.integration.impl.OracleIT
>> > 2019-11-27 18:18:24,087 [thrift.ProcessFunction] ERROR: Internal error
>> > processing getTimestamps
>> > java.lang.IllegalStateException: Received timestamp request but Oracle
>> is
>> > not leader
>> >   at
>> >
>> >
>> org.apache.fluo.core.oracle.OracleServer.getTimestampsImpl(OracleServer.java:244)
>> >   at
>> >
>> >
>> org.apache.fluo.core.oracle.OracleServer.getTimestamps(OracleServer.java:224)
>> >   at
>> >
>> >
>> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:279)
>> >   at
>> >
>> >
>> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:257)
>> >   at
>> >
>> >
>> org.apache.fluo.core.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
>> >   at
>> >
>> >
>> org.apache.fluo.core.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>> >   at
>> >
>> >
>> org.apache.fluo.core.shaded.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
>> >   at
>> >
>> >
>> org.apache.fluo.core.shaded.thrift.server.Invocation.run(Invocation.java:18)
>> >   at
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >   at
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >   at java.lang.Thread.run(Thread.java:748)
>> > 2019-11-27 18:18:24,091 [oracle.OracleClient] ERROR: TException
>> occurred in
>> > doWork() method
>> > org.apache.fluo.core.shaded.thrift.TApplicationException: Internal error
>> > processing getTimestamps
>> >   at
>> >
>> >
>> org.apache.fluo.core.shaded.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
>> >   at
>> >
>> >
>> org.apache.fluo.core.thrift.OracleService$Client.recv_getTimestamps(OracleService.java:86)
>> >   at
>> >
>> >
>> org.apache.fluo.core.thrift.OracleService$Client.getTimestamps(OracleService.java:73)
>> >   at
>> >
>> >
>> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.doWork(OracleClient.java:182)
>> >   at
>> >
>> >
>> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.run(OracleClient.java:116)
>> >   at java.lang.Thread.run(Thread.java:748)
>> > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.911
>> sec
>> > - in org.apache.fluo.integration.impl.OracleIT
>> >
>> > -Joe
>> >
>> > On Wed, Nov 27, 2019 at 5:18 PM Christopher <ct...@apache.org>
>> wrote:
>> >
>> > > I have not seen any errors with `mvn clean verify` locally, but
>> > > occasionally, there seems to be errors on Travis.
>> > > I'm building the master branch (currently
>> > > 549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on Fedora 31
>> > > x86_64 with no problems.
>> > >
>> > > On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
>> > > <ke...@apache.org> wrote:
>> > > >
>> > > > Hello,
>> > > >
>> > > > This is just a quick inquiry if anyone else running mvn verify is
>> > getting
>> > > > rat errors that prevent maven from completing.
>> > > >
>> > > >  I'm using -Drat.skip=true to bypass but is this normal?
>> > > >
>> > > > Last thing is OracleIT is throwing errors again. I've checked out
>> the
>> > new
>> > > > code and am building on 16.10 Ubuntu x64.
>> > > >
>> > > > If this is something others are experiencing maybe we can see if
>> there
>> > is
>> > > > an open ticket for OracleIT (thought we closed it with switch on
>> lock
>> > > > implementation in Curator). Rat check is new to me.
>> > > >
>> > > > Thanks a bunch. This is informal, if I'm not insane I'll do a formal
>> > > ticket
>> > > > or something. Again, thanks!
>> > > >
>> > > > Happy holidays also.
>> > > >
>> > > > I
>> > >
>> >
>>
>

Re: OracleIT and Rat check failures

Posted by Kenneth McFarland <gl...@gmail.com>.

@Christopher : Good, I thought that race condition was fixed in a previous
issue so I was concerned and should have looked closer. Thank you for the
catch sir!

I'm still confused as to why the RAT check is throwing errors. I am working
off a clean master with the same hash as the one Joe mentioned (commit
549d645addb330f4ae2e074447428cb86b5a9a3f).

 Here is some environment info:

*MAVEN*
Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
2017-10-18T00:58:13-07:00)
Maven home: /opt/apache-maven-3.5.2
Java version: 1.8.0_161, vendor: Oracle Corporation
Java home: /usr/lib/jvm/jdk1.8.0_161/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.15.0-70-generic", arch: "amd64", family:
"unix"

*JAVA*
openjdk 11.0.5 2019-10-15
OpenJDK Runtime Environment (build 11.0.5+10-post-Ubuntu-2ubuntu116.04)
OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Ubuntu-2ubuntu116.04, mixed
mode, sharing)

I also have a few other java versions installed like the Oracle 1.8.0 that
maven reports above.

*Operating System*
Linux ubuntu 4.15.0-70-generic #79~16.04.1-Ubuntu SMP Tue Nov 12 14:01:10
UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

I will try this on my MacI notice the apache-rat setDefaultExcludes
variable is set to true. I'm currently baffled. The errors are the same if
I run 'mvn apache-rat:check' or if I run 'mvn verify' or 'mvn clean
verify''. The logs are kinda large so I simply piped it into a text file,
as well as included the rat file.



On Wed, Nov 27, 2019 at 7:06 PM Christopher <ct...@apache.org> wrote:

> I think the stack traces in the logs/test output are intended for that
> test, because it's specifically checking error handling for a running
> server (if I remember correctly).
>
> On Wed, Nov 27, 2019, 18:42 Joseph Koshakow <ko...@gmail.com> wrote:
>
> > I'm running `mvn clean verify` on the master branch
> > (549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 1.8 on Ubuntu
> 18.04
> > x86_64. I don't see any issues related to Rat. OracleIT passes and the
> > entire build is successful, but OracleIT does print the following errors
> in
> > the logs
> >
> > Running org.apache.fluo.integration.impl.OracleIT
> > 2019-11-27 18:18:24,087 [thrift.ProcessFunction] ERROR: Internal error
> > processing getTimestamps
> > java.lang.IllegalStateException: Received timestamp request but Oracle is
> > not leader
> >   at
> >
> >
> org.apache.fluo.core.oracle.OracleServer.getTimestampsImpl(OracleServer.java:244)
> >   at
> >
> >
> org.apache.fluo.core.oracle.OracleServer.getTimestamps(OracleServer.java:224)
> >   at
> >
> >
> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:279)
> >   at
> >
> >
> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:257)
> >   at
> >
> >
> org.apache.fluo.core.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
> >   at
> >
> >
> org.apache.fluo.core.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> >   at
> >
> >
> org.apache.fluo.core.shaded.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
> >   at
> >
> >
> org.apache.fluo.core.shaded.thrift.server.Invocation.run(Invocation.java:18)
> >   at
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >   at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >   at java.lang.Thread.run(Thread.java:748)
> > 2019-11-27 18:18:24,091 [oracle.OracleClient] ERROR: TException occurred
> in
> > doWork() method
> > org.apache.fluo.core.shaded.thrift.TApplicationException: Internal error
> > processing getTimestamps
> >   at
> >
> >
> org.apache.fluo.core.shaded.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> >   at
> >
> >
> org.apache.fluo.core.thrift.OracleService$Client.recv_getTimestamps(OracleService.java:86)
> >   at
> >
> >
> org.apache.fluo.core.thrift.OracleService$Client.getTimestamps(OracleService.java:73)
> >   at
> >
> >
> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.doWork(OracleClient.java:182)
> >   at
> >
> >
> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.run(OracleClient.java:116)
> >   at java.lang.Thread.run(Thread.java:748)
> > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.911
> sec
> > - in org.apache.fluo.integration.impl.OracleIT
> >
> > -Joe
> >
> > On Wed, Nov 27, 2019 at 5:18 PM Christopher <ct...@apache.org> wrote:
> >
> > > I have not seen any errors with `mvn clean verify` locally, but
> > > occasionally, there seems to be errors on Travis.
> > > I'm building the master branch (currently
> > > 549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on Fedora 31
> > > x86_64 with no problems.
> > >
> > > On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
> > > <ke...@apache.org> wrote:
> > > >
> > > > Hello,
> > > >
> > > > This is just a quick inquiry if anyone else running mvn verify is
> > getting
> > > > rat errors that prevent maven from completing.
> > > >
> > > >  I'm using -Drat.skip=true to bypass but is this normal?
> > > >
> > > > Last thing is OracleIT is throwing errors again. I've checked out the
> > new
> > > > code and am building on 16.10 Ubuntu x64.
> > > >
> > > > If this is something others are experiencing maybe we can see if
> there
> > is
> > > > an open ticket for OracleIT (thought we closed it with switch on lock
> > > > implementation in Curator). Rat check is new to me.
> > > >
> > > > Thanks a bunch. This is informal, if I'm not insane I'll do a formal
> > > ticket
> > > > or something. Again, thanks!
> > > >
> > > > Happy holidays also.
> > > >
> > > > I
> > >
> >
>

Re: OracleIT and Rat check failures

Posted by Christopher <ct...@apache.org>.

I think the stack traces in the logs/test output are intended for that
test, because it's specifically checking error handling for a running
server (if I remember correctly).

On Wed, Nov 27, 2019, 18:42 Joseph Koshakow <ko...@gmail.com> wrote:

> I'm running `mvn clean verify` on the master branch
> (549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 1.8 on Ubuntu 18.04
> x86_64. I don't see any issues related to Rat. OracleIT passes and the
> entire build is successful, but OracleIT does print the following errors in
> the logs
>
> Running org.apache.fluo.integration.impl.OracleIT
> 2019-11-27 18:18:24,087 [thrift.ProcessFunction] ERROR: Internal error
> processing getTimestamps
> java.lang.IllegalStateException: Received timestamp request but Oracle is
> not leader
>   at
>
> org.apache.fluo.core.oracle.OracleServer.getTimestampsImpl(OracleServer.java:244)
>   at
>
> org.apache.fluo.core.oracle.OracleServer.getTimestamps(OracleServer.java:224)
>   at
>
> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:279)
>   at
>
> org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:257)
>   at
>
> org.apache.fluo.core.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
>   at
>
> org.apache.fluo.core.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at
>
> org.apache.fluo.core.shaded.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
>   at
>
> org.apache.fluo.core.shaded.thrift.server.Invocation.run(Invocation.java:18)
>   at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> 2019-11-27 18:18:24,091 [oracle.OracleClient] ERROR: TException occurred in
> doWork() method
> org.apache.fluo.core.shaded.thrift.TApplicationException: Internal error
> processing getTimestamps
>   at
>
> org.apache.fluo.core.shaded.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
>   at
>
> org.apache.fluo.core.thrift.OracleService$Client.recv_getTimestamps(OracleService.java:86)
>   at
>
> org.apache.fluo.core.thrift.OracleService$Client.getTimestamps(OracleService.java:73)
>   at
>
> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.doWork(OracleClient.java:182)
>   at
>
> org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.run(OracleClient.java:116)
>   at java.lang.Thread.run(Thread.java:748)
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.911 sec
> - in org.apache.fluo.integration.impl.OracleIT
>
> -Joe
>
> On Wed, Nov 27, 2019 at 5:18 PM Christopher <ct...@apache.org> wrote:
>
> > I have not seen any errors with `mvn clean verify` locally, but
> > occasionally, there seems to be errors on Travis.
> > I'm building the master branch (currently
> > 549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on Fedora 31
> > x86_64 with no problems.
> >
> > On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
> > <ke...@apache.org> wrote:
> > >
> > > Hello,
> > >
> > > This is just a quick inquiry if anyone else running mvn verify is
> getting
> > > rat errors that prevent maven from completing.
> > >
> > >  I'm using -Drat.skip=true to bypass but is this normal?
> > >
> > > Last thing is OracleIT is throwing errors again. I've checked out the
> new
> > > code and am building on 16.10 Ubuntu x64.
> > >
> > > If this is something others are experiencing maybe we can see if there
> is
> > > an open ticket for OracleIT (thought we closed it with switch on lock
> > > implementation in Curator). Rat check is new to me.
> > >
> > > Thanks a bunch. This is informal, if I'm not insane I'll do a formal
> > ticket
> > > or something. Again, thanks!
> > >
> > > Happy holidays also.
> > >
> > > I
> >
>

Re: OracleIT and Rat check failures

Posted by Joseph Koshakow <ko...@gmail.com>.

I'm running `mvn clean verify` on the master branch
(549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 1.8 on Ubuntu 18.04
x86_64. I don't see any issues related to Rat. OracleIT passes and the
entire build is successful, but OracleIT does print the following errors in
the logs

Running org.apache.fluo.integration.impl.OracleIT
2019-11-27 18:18:24,087 [thrift.ProcessFunction] ERROR: Internal error
processing getTimestamps
java.lang.IllegalStateException: Received timestamp request but Oracle is
not leader
  at
org.apache.fluo.core.oracle.OracleServer.getTimestampsImpl(OracleServer.java:244)
  at
org.apache.fluo.core.oracle.OracleServer.getTimestamps(OracleServer.java:224)
  at
org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:279)
  at
org.apache.fluo.core.thrift.OracleService$Processor$getTimestamps.getResult(OracleService.java:257)
  at
org.apache.fluo.core.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38)
  at
org.apache.fluo.core.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
  at
org.apache.fluo.core.shaded.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
  at
org.apache.fluo.core.shaded.thrift.server.Invocation.run(Invocation.java:18)
  at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:748)
2019-11-27 18:18:24,091 [oracle.OracleClient] ERROR: TException occurred in
doWork() method
org.apache.fluo.core.shaded.thrift.TApplicationException: Internal error
processing getTimestamps
  at
org.apache.fluo.core.shaded.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
  at
org.apache.fluo.core.thrift.OracleService$Client.recv_getTimestamps(OracleService.java:86)
  at
org.apache.fluo.core.thrift.OracleService$Client.getTimestamps(OracleService.java:73)
  at
org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.doWork(OracleClient.java:182)
  at
org.apache.fluo.core.oracle.OracleClient$TimestampRetriever.run(OracleClient.java:116)
  at java.lang.Thread.run(Thread.java:748)
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.911 sec
- in org.apache.fluo.integration.impl.OracleIT

-Joe

On Wed, Nov 27, 2019 at 5:18 PM Christopher <ct...@apache.org> wrote:

> I have not seen any errors with `mvn clean verify` locally, but
> occasionally, there seems to be errors on Travis.
> I'm building the master branch (currently
> 549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on Fedora 31
> x86_64 with no problems.
>
> On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
> <ke...@apache.org> wrote:
> >
> > Hello,
> >
> > This is just a quick inquiry if anyone else running mvn verify is getting
> > rat errors that prevent maven from completing.
> >
> >  I'm using -Drat.skip=true to bypass but is this normal?
> >
> > Last thing is OracleIT is throwing errors again. I've checked out the new
> > code and am building on 16.10 Ubuntu x64.
> >
> > If this is something others are experiencing maybe we can see if there is
> > an open ticket for OracleIT (thought we closed it with switch on lock
> > implementation in Curator). Rat check is new to me.
> >
> > Thanks a bunch. This is informal, if I'm not insane I'll do a formal
> ticket
> > or something. Again, thanks!
> >
> > Happy holidays also.
> >
> > I
>

Re: OracleIT and Rat check failures

Posted by Christopher <ct...@apache.org>.

I have not seen any errors with `mvn clean verify` locally, but
occasionally, there seems to be errors on Travis.
I'm building the master branch (currently
549d645addb330f4ae2e074447428cb86b5a9a3f) with OpenJDK 11 on Fedora 31
x86_64 with no problems.

On Wed, Nov 27, 2019 at 2:31 PM Kenneth McFarland
<ke...@apache.org> wrote:
>
> Hello,
>
> This is just a quick inquiry if anyone else running mvn verify is getting
> rat errors that prevent maven from completing.
>
>  I'm using -Drat.skip=true to bypass but is this normal?
>
> Last thing is OracleIT is throwing errors again. I've checked out the new
> code and am building on 16.10 Ubuntu x64.
>
> If this is something others are experiencing maybe we can see if there is
> an open ticket for OracleIT (thought we closed it with switch on lock
> implementation in Curator). Rat check is new to me.
>
> Thanks a bunch. This is informal, if I'm not insane I'll do a formal ticket
> or something. Again, thanks!
>
> Happy holidays also.
>
> I