Re: excessive java precommit logging

[Udi Meiri <eh...@google.com>]

To follow up, I did some research yesterday on removing --info and my
findings are:
- Gradle Test tasks generate HTML and Junit XML reports. Both contain a
stacktrace, STDOUT, and STDERR of the failed test (example
<https://builds.apache.org/job/beam_PreCommit_Java_Phrase/515/testReport/junit/org.apache.beam.runners.samza.runtime/SamzaStoreStateInternalsTest/testSetStateIterator/>
).
So even though --info wasn't specified the output are not lost.
- Python SDK tests don't use Test tasks (they exec tox), and thus are not
affected by --info. Python tests aren't excessively verbose however.
- Go tests should also generate reports (via gogradle), but I haven't found
any and I can't seem to run ./gradlew :beam-sdks-go:test on my workstation.

Suggestion:
- Remove --info (https://github.com/apache/beam/pull/7409)
- If we find Gradle tasks that aren't somehow reporting or logging to
console on failure, that's a bug and the task should be fixed.

On Thu, Dec 20, 2018 at 6:09 AM Kenneth Knowles <ke...@apache.org> wrote:

> I support lowering the log level. The default is `lifecycle`.
>
> For reference, here's where it was increased to `info`:
> https://github.com/apache/beam/pull/4644
>
> The reason was to see more details about Gradle's dependency management.
> We were seeing dependency download flakes on things that should not require
> re-downloading. No longer an issue.
>
> To easily tweak  it on a one-off basis without having to change a Jenkins
> job, you can edit gradle.properties in a commit on your PR:
>
>     org.gradle.logging.level=(quiet,warn,lifecycle,info,debug)
>     org.gradle.warning.mode=(all,none,summary)
>     org.gradle.console=(auto,plain,rich,verbose)
>     org.gradle.caching.debug=(true,false)
>
> Kenn
>
> On Thu, Dec 20, 2018 at 6:49 AM Robert Bradshaw <ro...@google.com>
> wrote:
>
>> Interestingly, I was thinking exactly the same thing the other day.
>>
>> If we could drop the info logs for passing tests, that would be ideal.
>> Regardless, tests should fail (when possible) with actionable
>> messages. I think the rare case of not being able to reproduce the
>> error locally if info logs are needed makes it OK to go and add
>> logging to jenkins as a one-off. (If it's about jenkins build errors,
>> perhaps we could build with higher verbosity before testing with a
>> lower one.)
>> On Thu, Dec 20, 2018 at 11:24 AM Maximilian Michels <mx...@apache.org>
>> wrote:
>> >
>> > Thanks Udi for bringing this up. I'm also for dropping INFO. It's just
>> > incredible verbose. More importantly, from my experience the INFO level
>> doesn't
>> > help debugging problems, but it makes finding errors messages or
>> warnings harder.
>> >
>> > That said, here's what I do to search through the log:
>> >
>> > 1) curl <build_url>/consoleText | less
>> >
>> > This is when I just want to quickly look for something.
>> >
>> > 2) curl <build_url>/consoleText > log.txt
>> >     less log.txt
>> >
>> > Here we store the log to a file first, then use 'less' or 'grep' to
>> search it.
>> >
>> > When in 'less', I use '/' to grep through the lines. Pressing 'n' or
>> 'N' gets
>> > you forward and back in the search results.
>> >
>> > That works pretty well, but I think we would do us a favor by dropping
>> the log
>> > level. Shall we try it out?
>> >
>> > -Max
>> >
>> > On 19.12.18 23:27, Udi Meiri wrote:
>> > > The gradle scan doesn't pinpoint the error message, and it doesn't
>> contain all
>> > > the lines: https://scans.gradle.com/s/ckhjrjdexpuzm/console-log
>> > >
>> > > The logs might be useful, but usually not from passing tests. Doesn't
>> gradle log
>> > > output from failed tests by default?
>> > >
>> > > On Wed, Dec 19, 2018 at 1:22 PM Thomas Weise <thw@apache.org
>> > > <ma...@apache.org>> wrote:
>> > >
>> > >     I usually follow the download procedure outlined by Scott to look
>> at the logs.
>> > >
>> > >     These logs are big, but when there is a problem it is sometimes
>> essential to
>> > >     have the extra output, especially for less frequent flakes.
>> > >
>> > >     Reducing logs would then require the author to add extra logging
>> to the PR
>> > >     (and attempt to reproduce), which is also not nice.
>> > >
>> > >     Thomas
>> > >
>> > >
>> > >     On Wed, Dec 19, 2018 at 11:47 AM Scott Wegner <scott@apache.org
>> > >     <ma...@apache.org>> wrote:
>> > >
>> > >         I'm not sure what we lose by dropping the --info flag, but I
>> generally
>> > >         worry about reducing log output since logs are the main
>> resource for
>> > >         diagnosing Jenkins build errors.
>> > >
>> > >         It seems the issue is that Chrome doesn't scale well to large
>> log files.
>> > >         A few alternative solutions:
>> > >
>> > >         1. Use the produced Build Scan (example: [1]) instead of the
>> raw console
>> > >         log. The build scan is quite useful at pointing to what
>> actually failed,
>> > >         and filtering log output for only that task.
>> > >         2. Instead of consoleFull, use consoleText ("View as plain
>> text" link in
>> > >         Jenkins), which seems to be much easier on Chrome
>> > >         3. Download the consoleText output locally and use your
>> favorite log
>> > >         viewer that can scale to large files.
>> > >
>> > >         [1] https://gradle.com/s/ckhjrjdexpuzm
>> > >
>> > >         On Wed, Dec 19, 2018 at 10:42 AM Udi Meiri <ehudm@google.com
>> > >         <ma...@google.com>> wrote:
>> > >
>> > >             Hi all,
>> > >             I'd like to reduce precommit log sizes on Jenkins. For
>> example:
>> > >
>> https://builds.apache.org/job/beam_PreCommit_Java_Commit/3181/consoleFull
>> > >             is 79M, which makes Chrome sluggish to use on it (tab is
>> constantly
>> > >             using a whole cpu core).
>> > >
>> > >             I know this might be controversial, but I'd like to
>> propose to
>> > >             remove the --info flag from the gradlew command line.
>> > >
>> > >
>> > >
>> > >         --
>> > >
>> > >
>> > >
>> > >
>> > >         Got feedback? tinyurl.com/swegner-feedback
>> > >         <https://tinyurl.com/swegner-feedback>
>> > >
>>
>