Re: Make the TeamCity console quiet.

[Dmitriy Pavlov <dp...@apache.org>]

Hi Igniters,

For older versions, where the needed configuration is absent you may found
the following behavior of tests

 SPI [ tests 1 ] JDK8
 IgniteSpiTestSuite: TestSuite$1.warning

 Cache (Restarts) 1 [ tests 1 ] JDK8
 IgniteCacheRestartTestSuite: TestSuite$1.warning

If you found such issue consider unchecking `Ignite Quiet` mode for tests
run for an older version.

From the configuration point of view, you may set build parameter
reverse.dep.*.IGNITE_LOGGING_OPTS
to value:
-DIGNITE_QUIET=false

Sincerely,
Dmitriy Pavlov

пт, 19 апр. 2019 г. в 12:59, Maksim Stepachev <ma...@gmail.com>:

> Run params example:
>
> https://issues.apache.org/jira/secure/attachment/12966470/quiet-console-checkbox.png
>
> пт, 19 апр. 2019 г. в 12:55, Dmitriy Pavlov <dp...@apache.org>:
>
> > Hi Maxim,
> >
> > Apache Lists removes images from emails. Could you please add this image
> > somewhere and share a link?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 19 апр. 2019 г. в 12:52, Maksim Stepachev <
> maksim.stepachev@gmail.com
> > >:
> >
> > > Hi,
> > > The checkbox was added into the "Run Custom Build" window (Quiet
> console
> > > output *). Quiet is active by default.
> > >
> > > [image: quite-console-checkbox.png]
> > > https://issues.apache.org/jira/browse/IGNITE-11736
> > >
> > >
> > > вт, 9 апр. 2019 г. в 11:06, Павлухин Иван <vo...@gmail.com>:
> > >
> > >> Maksim,
> > >>
> > >> I like such approach. If nobody objects let's go ahead and try it out.
> > >>
> > >> вт, 9 апр. 2019 г. в 10:38, Maksim Stepachev <
> > maksim.stepachev@gmail.com
> > >> >:
> > >> >
> > >> > Ivan,
> > >> >
> > >> > 1. It'll some selector which changes ENV parameter for build run
> with
> > >> quiet
> > >> > by default.
> > >> > 2. Yes, the additional log config will be added for TC.
> > >> >
> > >> >
> > >> > пн, 8 апр. 2019 г. в 17:08, Павлухин Иван <vo...@gmail.com>:
> > >> >
> > >> > > Hi Maxim,
> > >> > >
> > >> > > Thank you for continuing with that initiative. Could you please
> > >> > > elaborate on following:
> > >> > > 1. What changes are required on TC?
> > >> > > 2. Will everything work without changes on a developer's local
> > >> machine?
> > >> > >
> > >> > > пн, 8 апр. 2019 г. в 13:13, Maksim Stepachev <
> > >> maksim.stepachev@gmail.com>:
> > >> > > >
> > >> > > > Hi, everyone!
> > >> > > >
> > >> > > > I want to push the topic up and discuss a final solution about
> the
> > >> quiet
> > >> > > > console.
> > >> > > >
> > >> > > > The suggestion above:
> > >> > > > 1. Rollover will be locked.
> > >> > > > Pros: Only one big file in an archive.
> > >> > > > Cons: Max size of the file isn't limited.
> > >> > > > 2. Run all will contain a parameter for switch off the quiet
> mode.
> > >> > > > 3. New config: log4j-tc-test.xml for TeamCity environment.
> > >> > > >
> > >> > > > чт, 14 февр. 2019 г. в 15:51, Павлухин Иван <
> vololo100@gmail.com
> > >:
> > >> > > >
> > >> > > > > Maksim,
> > >> > > > >
> > >> > > > > I tried configuration with quite console and unlimited log
> file
> > >> on my
> > >> > > > > machine. Naively configured in log4j-test.xml it works not so
> > >> fine in
> > >> > > > > development environment because log file is appended by each
> > test
> > >> > > > > execution. Console redirection works better in development
> > >> > > > > environment. Should we have separate log config for TC?
> > >> > > > >
> > >> > > > > By the way log redirection does not work well for every test
> in
> > >> > > > > development environment. Sometimes it seems that it could be
> > >> easier to
> > >> > > > > work with file. Sometimes IDEA shrinks console logs. Igniters,
> > do
> > >> you
> > >> > > > > have ideas how to work with logs produced by tests more
> > >> convenient?
> > >> > > > >
> > >> > > > > пн, 11 февр. 2019 г. в 17:29, Eduard Shangareev <
> > >> > > > > eduard.shangareev@gmail.com>:
> > >> > > > > >
> > >> > > > > > Maksim,
> > >> > > > > >
> > >> > > > > > sounds good.
> > >> > > > > >
> > >> > > > > > 1) Can we just turn off log rollover?
> > >> > > > > > 2) I believe, that we need the ability to override this by
> > >> setting
> > >> > > > > > parameter for Run All.
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Mon, Feb 11, 2019 at 4:12 PM Maksim Stepachev <
> > >> > > > > maksim.stepachev@gmail.com>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Ivan,
> > >> > > > > > >
> > >> > > > > > > Yes. It happens because we use the RollingFileAppender
> for a
> > >> file
> > >> > > > > logging.
> > >> > > > > > > This appender has the next properties:
> > >> > > > > > >
> > >> > > > > > > <param name="MaxFileSize" value="10MB"/>
> > >> > > > > > > <param name="MaxBackupIndex" value="10"/>
> > >> > > > > > >
> > >> > > > > > > Can I increase MaxFileSize by 200MB or more? I suppose
> these
> > >> limits
> > >> > > > > were
> > >> > > > > > > added for cases when the test writes a log. It makes the
> > >> guarantee
> > >> > > that
> > >> > > > > > > logs less than 10*10 = 100MBs. But it's wrong if it writes
> > >> into the
> > >> > > > > console
> > >> > > > > > > too.
> > >> > > > > > >
> > >> > > > > > > пн, 11 февр. 2019 г. в 13:14, Павлухин Иван <
> > >> vololo100@gmail.com>:
> > >> > > > > > >
> > >> > > > > > > > Maksim,
> > >> > > > > > > >
> > >> > > > > > > > Generally I like the idea. But there is one thing which
> > >> bothers
> > >> > > me a
> > >> > > > > > > > little bit. Usually I use "Download full build log" link
> > to
> > >> > > download
> > >> > > > > > > > log and then examine it as a single file. AFAIK artifact
> > >> with
> > >> > > logs
> > >> > > > > > > > contains several files. Could you suggest a way how can
> I
> > >> > > > > conveniently
> > >> > > > > > > > explore files archive as a single file?
> > >> > > > > > > >
> > >> > > > > > > > пн, 11 февр. 2019 г. в 13:04, Ilya Kasnacheev <
> > >> > > > > ilya.kasnacheev@gmail.com
> > >> > > > > > > >:
> > >> > > > > > > > >
> > >> > > > > > > > > Hello!
> > >> > > > > > > > >
> > >> > > > > > > > > Can we do IGNITE_QUIET=true for runAlls triggered by
> bot
> > >> (along
> > >> > > > > with
> > >> > > > > > > > > SCALE_FACTOR) and false for manually triggered builds
> > (or
> > >> > > > > retriggered
> > >> > > > > > > > > failures)?
> > >> > > > > > > > >
> > >> > > > > > > > > Regards,
> > >> > > > > > > > > --
> > >> > > > > > > > > Ilya Kasnacheev
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > пн, 11 февр. 2019 г. в 13:03, Dmitriy Pavlov <
> > >> > > dpavlov@apache.org>:
> > >> > > > > > > > >
> > >> > > > > > > > > > ++1 from my side. It is very interesting how overall
> > >> RunAll
> > >> > > run
> > >> > > > > time
> > >> > > > > > > > will
> > >> > > > > > > > > > be decreased.
> > >> > > > > > > > > >
> > >> > > > > > > > > > My previous experience with TeamCity shows that
> > removing
> > >> > > pressure
> > >> > > > > > > from
> > >> > > > > > > > the
> > >> > > > > > > > > > console and using adequate logging instead may bring
> > >> x1.5 -
> > >> > > x2
> > >> > > > > > > > performance
> > >> > > > > > > > > > boost for tests.
> > >> > > > > > > > > >
> > >> > > > > > > > > > One more benefit of moving from synced streams to
> > async
> > >> > > logging
> > >> > > > > will
> > >> > > > > > > > show
> > >> > > > > > > > > > us that bugs, which we can't see right now.
> > >> > > > > > > > > >
> > >> > > > > > > > > > So I absolutely agree to move logging data to a
> > logger.
> > >> > > > > > > > > >
> > >> > > > > > > > > > пн, 11 февр. 2019 г. в 11:14, Maksim Stepachev <
> > >> > > > > > > > maksim.stepachev@gmail.com
> > >> > > > > > > > > > >:
> > >> > > > > > > > > >
> > >> > > > > > > > > > > Igniters,
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > When I was working with flaky tests, I was
> surprised
> > >> that
> > >> > > one
> > >> > > > > of
> > >> > > > > > > the
> > >> > > > > > > > > > > reasons for failure was a log appender blocking
> the
> > >> > > console.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > I suppose it happened because of a TeamCity agent
> > >> > > communicating
> > >> > > > > > > with
> > >> > > > > > > > java
> > >> > > > > > > > > > > out through Linux pipe. This is a problem for
> tests
> > >> with
> > >> > > 1GB
> > >> > > > > log
> > >> > > > > > > > history.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > Our test by default writes logs into 2 sources,
> such
> > >> as the
> > >> > > > > console
> > >> > > > > > > > and
> > >> > > > > > > > > > the
> > >> > > > > > > > > > > file. I'm going to change mode for the console at
> > >> > > > > IGNITE_QUIET=true
> > >> > > > > > > > and
> > >> > > > > > > > > > > write only warn and error logs in it. Also, I’ll
> > >> include a
> > >> > > > > > > > diagnostic log
> > >> > > > > > > > > > > into it.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > You will be able to read the previous log from the
> > >> > > Artifacts
> > >> > > > > tab in
> > >> > > > > > > > your
> > >> > > > > > > > > > > build.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > This is a solution which has advantages:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > 1. First of all, we’ll cut down the space usage
> > >> pre-test
> > >> > > run by
> > >> > > > > > > ~6-7
> > >> > > > > > > > > > times.
> > >> > > > > > > > > > > For example - it's 1GB of saved space for 1 suit
> > like
> > >> > > "Cache
> > >> > > > > > > > (Restarts)
> > >> > > > > > > > > > 1".
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > 2. We’ll be able to use a new space for long
> history
> > >> > > storage
> > >> > > > > for
> > >> > > > > > > > builds.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > 3. The TeamCity bot won't be lagging when it
> parses
> > a
> > >> large
> > >> > > > > file.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > 4. Possibly some of tests won't be flaky.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > 5. And finally, the time of running my suite will
> be
> > >> cut
> > >> > > down
> > >> > > > > by
> > >> > > > > > > 5-7
> > >> > > > > > > > > > > minutes.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > Also, I'm going to make a workaround for previous
> > >> > > behavior. For
> > >> > > > > > > > example,
> > >> > > > > > > > > > > this flag will be added to build params. But I
> > should
> > >> > > > > investigate
> > >> > > > > > > it.
> > >> > > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > --
> > >> > > > > > > > Best regards,
> > >> > > > > > > > Ivan Pavlukhin
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > Best regards,
> > >> > > > > Ivan Pavlukhin
> > >> > > > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Best regards,
> > >> > > Ivan Pavlukhin
> > >> > >
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >> Ivan Pavlukhin
> > >>
> > >
> >
>