You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafodion.apache.org by Eric Owhadi <er...@esgyn.com> on 2015/08/04 16:01:47 UTC
Where are the logs for the first set of regression tests?
That may be an obvious question, but I can’t find the logs and diffs for
the first set of test showing up in the result of the runallsb:
I can see all the logs and diffs for categories like core, compGeneral etc,
but the first set of test labelled:
TEST001
TEST002
TEST003
TEST004
TEST005
TEST006
TEST015
TEST017
TEST018
TEST020
are not associated with a category, and they don’t have a subdirectory with
categoryname where I can find the other test results.
Thanks in advance for the help.
As a side note, if a test fails because of the compare, but the delta is
legit, what should one do? Update known diffs? Or set a new compare file?
Eric
RE: Where are the logs for the first set of regression tests?
Posted by Anoop Sharma <an...@esgyn.com>.
no, there is no tool.
Once you get your DIFF file and it is different than the existing known diff
file, then you should validate what the diff of diffs are.
If the diff of diffs is only due to line numbers, then check in the new
diff file as the known diff file.
if diffs are not just line num diffs, then you need to validate that this
new
set of diffs is expected due to your changes. If yes, then check in the diff
as
the known diff.
anoop
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:54 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
"regenerate a known diff file": is there a tool for that? Or is that a
manual process?
Eric
-----Original Message-----
From: Anoop Sharma [mailto:anoop.sharma@esgyn.com]
Sent: Tuesday, August 4, 2015 5:42 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
if a known diff file already exists for a test, then you cannot copy the log
file as the expected file. Doing that will mask the current known diffs.
You will need to manually update the expected file and regenerate a new
known diff file as diff line numbers may have changed.
anoop
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:37 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
OK clear,
Thanks again
Eric
On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are
> satisfied with your new results you could simply copy the LOGaaa file
> as the new EXPECTEDaaa file. There are filters in the test suite that
> will hide things like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes
> > that translate into false failure, should I modify the comparison
> > file, or the file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the
> > wiki only for the individual "pull request" automated jobs. I will
> > add them for the official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> > 2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> > number 48
> >
> > 2015-08-04 08:30:13 originally caused by:
> >
> > 2015-08-04 08:30:13 Started by timer
> >
> > 2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> > 2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> > (redhat_ahw2.2) in workspace
> > /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> > 2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> > /tmp/hudson2056470402644332566.sh
> >
> > 2015-08-04 08:30:14 Detailed logs:
> > *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/*
> > <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/
> > >
> >
> > == > Navigate to sql-regress-log
> >
> > You will see a directory for the component test
> > suite –
> eg
> > “executor” and a full log – “Regress.log”.
> >
> > If you navigate to the component directory eg
> > executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off
> > and get familiar with the official results, that'll give me some
> > learning opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run?
> > So that it is easy to compare with the failure one get running the
> > test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both
> > are
> being
> > investigated. Otherwise we have been having good clean runs. So any
> > other failures you see need to be investigated. If you can give me
> > VNC access
> to
> > your machine, I can take a quick look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checkin
> g_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all
> > on master branch? I am getting some failure, and I wonder if they
> > are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's
> > questions on the wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> > I can add a page to the wiki for this. We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the
> > first time.
> >
> > Also Roberta had a comprehensive document describing the whole
> > regress environment back in MX days , I think. Not sure if anyone
> > has that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time
> > saving.
> >
> > Also how to write and/or modify when tests are complaining, but we
> > know that this is just because regression is based on delta, and
> > cannot recognize legitimate expected delta (due to know feature we
> > just implemented), and real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to
> > run the regression tests and how to interpret and deal with the
> > results? If
> someone
> > would like to volunteer to write such a page, it would be very
> > useful to newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <su...@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This
> > is an artifact of how runallsb prints out results. At the end of
> > every suite a summary of that suite is printed out (with no header
> > what suite it refers to since we have been running that suite for
> > the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time
> > with a header for each suite. So if you kick off regressions now and
> > come back several hours later there will be a screen with a summary
> > for all suites and then before the summary for core there will be
> > list of results with no suite name. This is the same as the last
> > suite in the summary (the hive test suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir.
> > > For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one
> > > for
> >
> > > each category in $scriptsdir. Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi
> > > <er...@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and
> > > > diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a
> > > > new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>
RE: Where are the logs for the first set of regression tests?
Posted by Eric Owhadi <er...@esgyn.com>.
"regenerate a known diff file": is there a tool for that? Or is that a
manual process?
Eric
-----Original Message-----
From: Anoop Sharma [mailto:anoop.sharma@esgyn.com]
Sent: Tuesday, August 4, 2015 5:42 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
if a known diff file already exists for a test, then you cannot copy the log
file as the expected file. Doing that will mask the current known diffs.
You will need to manually update the expected file and regenerate a new
known diff file as diff line numbers may have changed.
anoop
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:37 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
OK clear,
Thanks again
Eric
On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are
> satisfied with your new results you could simply copy the LOGaaa file
> as the new EXPECTEDaaa file. There are filters in the test suite that
> will hide things like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes
> > that translate into false failure, should I modify the comparison
> > file, or the file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the
> > wiki only for the individual "pull request" automated jobs. I will
> > add them for the official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> > 2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> > number 48
> >
> > 2015-08-04 08:30:13 originally caused by:
> >
> > 2015-08-04 08:30:13 Started by timer
> >
> > 2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> > 2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> > (redhat_ahw2.2) in workspace
> > /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> > 2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> > /tmp/hudson2056470402644332566.sh
> >
> > 2015-08-04 08:30:14 Detailed logs:
> > *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/*
> > <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/
> > >
> >
> > == > Navigate to sql-regress-log
> >
> > You will see a directory for the component test
> > suite –
> eg
> > “executor” and a full log – “Regress.log”.
> >
> > If you navigate to the component directory eg
> > executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off
> > and get familiar with the official results, that'll give me some
> > learning opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run?
> > So that it is easy to compare with the failure one get running the
> > test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both
> > are
> being
> > investigated. Otherwise we have been having good clean runs. So any
> > other failures you see need to be investigated. If you can give me
> > VNC access
> to
> > your machine, I can take a quick look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checkin
> g_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all
> > on master branch? I am getting some failure, and I wonder if they
> > are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's
> > questions on the wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> > I can add a page to the wiki for this. We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the
> > first time.
> >
> > Also Roberta had a comprehensive document describing the whole
> > regress environment back in MX days , I think. Not sure if anyone
> > has that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time
> > saving.
> >
> > Also how to write and/or modify when tests are complaining, but we
> > know that this is just because regression is based on delta, and
> > cannot recognize legitimate expected delta (due to know feature we
> > just implemented), and real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to
> > run the regression tests and how to interpret and deal with the
> > results? If
> someone
> > would like to volunteer to write such a page, it would be very
> > useful to newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <su...@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This
> > is an artifact of how runallsb prints out results. At the end of
> > every suite a summary of that suite is printed out (with no header
> > what suite it refers to since we have been running that suite for
> > the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time
> > with a header for each suite. So if you kick off regressions now and
> > come back several hours later there will be a screen with a summary
> > for all suites and then before the summary for core there will be
> > list of results with no suite name. This is the same as the last
> > suite in the summary (the hive test suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir.
> > > For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one
> > > for
> >
> > > each category in $scriptsdir. Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi
> > > <er...@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and
> > > > diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a
> > > > new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>
RE: Where are the logs for the first set of regression tests?
Posted by Sandhya Sundaresan <sa...@esgyn.com>.
Thanks Eric. Will add these points.
About runallsb, you do not need to set rundir and scriptsdir manually. The
script does it by itself via the helper script "setuplnxenv".
As I have described, it needs to be manually set only when running
individual tests.
Thanks for your feedback !
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Monday, August 10, 2015 7:31 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hello Sandhya,
Thanks, that's very helpful indeed. You added the most important missing
info.
For improvement:
I would add something about the "known diff", the impact of messing with it
if you just replace the expected by log due to line number changes.
Also some test have :
Log*.SB, log*.SB.flt, log*.SB.srt, log*.SB.tflt Would be nice to know what
these are?
What is the process to deal with "known diff".
How to manage CQD used by the regression test.
What file to touch if one want to add a new test instead on appending an
existing one.
Oh and the setting of rundir and runscript is missing for the runallsb to
run all.
Thanks,
Eric
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
Sent: Wednesday, August 5, 2015 8:56 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
I have added the details to the wiki at :
https://wiki.trafodion.org/wiki/index.php/Test_Suites#Core
Eric, please take a look and see if this is helpful and if anything else
should be added.
Thanks
Sandhya
-----Original Message-----
From: Anoop Sharma [mailto:anoop.sharma@esgyn.com]
Sent: Tuesday, August 4, 2015 3:42 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
if a known diff file already exists for a test, then you cannot copy the log
file as the expected file. Doing that will mask the current known diffs.
You will need to manually update the expected file and regenerate a new
known diff file as diff line numbers may have changed.
anoop
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:37 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
OK clear,
Thanks again
Eric
On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are
> satisfied with your new results you could simply copy the LOGaaa file
> as the new EXPECTEDaaa file. There are filters in the test suite that
> will hide things like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes
> > that translate into false failure, should I modify the comparison
> > file, or the file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the
> > wiki only for the individual "pull request" automated jobs. I will
> > add them for the official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> > 2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> > number 48
> >
> > 2015-08-04 08:30:13 originally caused by:
> >
> > 2015-08-04 08:30:13 Started by timer
> >
> > 2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> > 2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> > (redhat_ahw2.2) in workspace
> > /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> > 2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> > /tmp/hudson2056470402644332566.sh
> >
> > 2015-08-04 08:30:14 Detailed logs:
> > *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/*
> > <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/
> > >
> >
> > == > Navigate to sql-regress-log
> >
> > You will see a directory for the component test
> > suite –
> eg
> > “executor” and a full log – “Regress.log”.
> >
> > If you navigate to the component directory eg
> > executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off
> > and get familiar with the official results, that'll give me some
> > learning opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run?
> > So that it is easy to compare with the failure one get running the
> > test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both
> > are
> being
> > investigated. Otherwise we have been having good clean runs. So any
> > other failures you see need to be investigated. If you can give me
> > VNC access
> to
> > your machine, I can take a quick look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checkin
> g_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all
> > on master branch? I am getting some failure, and I wonder if they
> > are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's
> > questions on the wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> > I can add a page to the wiki for this. We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the
> > first time.
> >
> > Also Roberta had a comprehensive document describing the whole
> > regress environment back in MX days , I think. Not sure if anyone
> > has that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time
> > saving.
> >
> > Also how to write and/or modify when tests are complaining, but we
> > know that this is just because regression is based on delta, and
> > cannot recognize legitimate expected delta (due to know feature we
> > just implemented), and real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to
> > run the regression tests and how to interpret and deal with the
> > results? If
> someone
> > would like to volunteer to write such a page, it would be very
> > useful to newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <su...@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This
> > is an artifact of how runallsb prints out results. At the end of
> > every suite a summary of that suite is printed out (with no header
> > what suite it refers to since we have been running that suite for
> > the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time
> > with a header for each suite. So if you kick off regressions now and
> > come back several hours later there will be a screen with a summary
> > for all suites and then before the summary for core there will be
> > list of results with no suite name. This is the same as the last
> > suite in the summary (the hive test suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir.
> > > For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one
> > > for
> >
> > > each category in $scriptsdir. Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi
> > > <er...@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and
> > > > diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a
> > > > new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>
RE: Where are the logs for the first set of regression tests?
Posted by Eric Owhadi <er...@esgyn.com>.
Hello Sandhya,
Thanks, that's very helpful indeed. You added the most important missing
info.
For improvement:
I would add something about the "known diff", the impact of messing with it
if you just replace the expected by log due to line number changes.
Also some test have :
Log*.SB, log*.SB.flt, log*.SB.srt, log*.SB.tflt
Would be nice to know what these are?
What is the process to deal with "known diff".
How to manage CQD used by the regression test.
What file to touch if one want to add a new test instead on appending an
existing one.
Oh and the setting of rundir and runscript is missing for the runallsb to
run all.
Thanks,
Eric
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
Sent: Wednesday, August 5, 2015 8:56 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
I have added the details to the wiki at :
https://wiki.trafodion.org/wiki/index.php/Test_Suites#Core
Eric, please take a look and see if this is helpful and if anything else
should be added.
Thanks
Sandhya
-----Original Message-----
From: Anoop Sharma [mailto:anoop.sharma@esgyn.com]
Sent: Tuesday, August 4, 2015 3:42 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
if a known diff file already exists for a test, then you cannot copy the log
file as the expected file. Doing that will mask the current known diffs.
You will need to manually update the expected file and regenerate a new
known diff file as diff line numbers may have changed.
anoop
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:37 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
OK clear,
Thanks again
Eric
On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are
> satisfied with your new results you could simply copy the LOGaaa file
> as the new EXPECTEDaaa file. There are filters in the test suite that
> will hide things like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes
> > that translate into false failure, should I modify the comparison
> > file, or the file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the
> > wiki only for the individual "pull request" automated jobs. I will
> > add them for the official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> > 2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> > number 48
> >
> > 2015-08-04 08:30:13 originally caused by:
> >
> > 2015-08-04 08:30:13 Started by timer
> >
> > 2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> > 2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> > (redhat_ahw2.2) in workspace
> > /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> > 2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> > /tmp/hudson2056470402644332566.sh
> >
> > 2015-08-04 08:30:14 Detailed logs:
> > *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/*
> > <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/
> > >
> >
> > == > Navigate to sql-regress-log
> >
> > You will see a directory for the component test
> > suite –
> eg
> > “executor” and a full log – “Regress.log”.
> >
> > If you navigate to the component directory eg
> > executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off
> > and get familiar with the official results, that'll give me some
> > learning opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run?
> > So that it is easy to compare with the failure one get running the
> > test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both
> > are
> being
> > investigated. Otherwise we have been having good clean runs. So any
> > other failures you see need to be investigated. If you can give me
> > VNC access
> to
> > your machine, I can take a quick look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checkin
> g_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all
> > on master branch? I am getting some failure, and I wonder if they
> > are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's
> > questions on the wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> > I can add a page to the wiki for this. We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the
> > first time.
> >
> > Also Roberta had a comprehensive document describing the whole
> > regress environment back in MX days , I think. Not sure if anyone
> > has that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time
> > saving.
> >
> > Also how to write and/or modify when tests are complaining, but we
> > know that this is just because regression is based on delta, and
> > cannot recognize legitimate expected delta (due to know feature we
> > just implemented), and real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to
> > run the regression tests and how to interpret and deal with the
> > results? If
> someone
> > would like to volunteer to write such a page, it would be very
> > useful to newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <su...@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This
> > is an artifact of how runallsb prints out results. At the end of
> > every suite a summary of that suite is printed out (with no header
> > what suite it refers to since we have been running that suite for
> > the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time
> > with a header for each suite. So if you kick off regressions now and
> > come back several hours later there will be a screen with a summary
> > for all suites and then before the summary for core there will be
> > list of results with no suite name. This is the same as the last
> > suite in the summary (the hive test suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir.
> > > For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one
> > > for
> >
> > > each category in $scriptsdir. Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi
> > > <er...@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and
> > > > diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a
> > > > new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>
RE: Where are the logs for the first set of regression tests?
Posted by Eric Owhadi <er...@esgyn.com>.
Sure I will as soon as I resume dealing with regression. I am completing the
feature first, as I see now that regression is not small task :-).
This is great to see all this stuff being checked automatically, but it
comes at a price.
Eric
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
Sent: Wednesday, August 5, 2015 8:56 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
I have added the details to the wiki at :
https://wiki.trafodion.org/wiki/index.php/Test_Suites#Core
Eric, please take a look and see if this is helpful and if anything else
should be added.
Thanks
Sandhya
-----Original Message-----
From: Anoop Sharma [mailto:anoop.sharma@esgyn.com]
Sent: Tuesday, August 4, 2015 3:42 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
if a known diff file already exists for a test, then you cannot copy the log
file as the expected file. Doing that will mask the current known diffs.
You will need to manually update the expected file and regenerate a new
known diff file as diff line numbers may have changed.
anoop
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:37 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
OK clear,
Thanks again
Eric
On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are
> satisfied with your new results you could simply copy the LOGaaa file
> as the new EXPECTEDaaa file. There are filters in the test suite that
> will hide things like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes
> > that translate into false failure, should I modify the comparison
> > file, or the file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the
> > wiki only for the individual "pull request" automated jobs. I will
> > add them for the official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> > 2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> > number 48
> >
> > 2015-08-04 08:30:13 originally caused by:
> >
> > 2015-08-04 08:30:13 Started by timer
> >
> > 2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> > 2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> > (redhat_ahw2.2) in workspace
> > /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> > 2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> > /tmp/hudson2056470402644332566.sh
> >
> > 2015-08-04 08:30:14 Detailed logs:
> > *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/*
> > <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/
> > >
> >
> > == > Navigate to sql-regress-log
> >
> > You will see a directory for the component test
> > suite –
> eg
> > “executor” and a full log – “Regress.log”.
> >
> > If you navigate to the component directory eg
> > executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off
> > and get familiar with the official results, that'll give me some
> > learning opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run?
> > So that it is easy to compare with the failure one get running the
> > test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both
> > are
> being
> > investigated. Otherwise we have been having good clean runs. So any
> > other failures you see need to be investigated. If you can give me
> > VNC access
> to
> > your machine, I can take a quick look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checkin
> g_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all
> > on master branch? I am getting some failure, and I wonder if they
> > are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's
> > questions on the wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> > I can add a page to the wiki for this. We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the
> > first time.
> >
> > Also Roberta had a comprehensive document describing the whole
> > regress environment back in MX days , I think. Not sure if anyone
> > has that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time
> > saving.
> >
> > Also how to write and/or modify when tests are complaining, but we
> > know that this is just because regression is based on delta, and
> > cannot recognize legitimate expected delta (due to know feature we
> > just implemented), and real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to
> > run the regression tests and how to interpret and deal with the
> > results? If
> someone
> > would like to volunteer to write such a page, it would be very
> > useful to newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <su...@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This
> > is an artifact of how runallsb prints out results. At the end of
> > every suite a summary of that suite is printed out (with no header
> > what suite it refers to since we have been running that suite for
> > the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time
> > with a header for each suite. So if you kick off regressions now and
> > come back several hours later there will be a screen with a summary
> > for all suites and then before the summary for core there will be
> > list of results with no suite name. This is the same as the last
> > suite in the summary (the hive test suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir.
> > > For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one
> > > for
> >
> > > each category in $scriptsdir. Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi
> > > <er...@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and
> > > > diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a
> > > > new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>
RE: Where are the logs for the first set of regression tests?
Posted by Sandhya Sundaresan <sa...@esgyn.com>.
I have added the details to the wiki at :
https://wiki.trafodion.org/wiki/index.php/Test_Suites#Core
Eric, please take a look and see if this is helpful and if anything else
should be added.
Thanks
Sandhya
-----Original Message-----
From: Anoop Sharma [mailto:anoop.sharma@esgyn.com]
Sent: Tuesday, August 4, 2015 3:42 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
if a known diff file already exists for a test, then you cannot copy the log
file as the expected file. Doing that will mask the current known diffs.
You will need to manually update the expected file and regenerate a new
known diff file as diff line numbers may have changed.
anoop
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:37 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
OK clear,
Thanks again
Eric
On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are
> satisfied with your new results you could simply copy the LOGaaa file
> as the new EXPECTEDaaa file. There are filters in the test suite that
> will hide things like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes
> > that translate into false failure, should I modify the comparison
> > file, or the file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the
> > wiki only for the individual "pull request" automated jobs. I will
> > add them for the official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> > 2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> > number 48
> >
> > 2015-08-04 08:30:13 originally caused by:
> >
> > 2015-08-04 08:30:13 Started by timer
> >
> > 2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> > 2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> > (redhat_ahw2.2) in workspace
> > /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> > 2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> > /tmp/hudson2056470402644332566.sh
> >
> > 2015-08-04 08:30:14 Detailed logs:
> > *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/*
> > <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/
> > >
> >
> > == > Navigate to sql-regress-log
> >
> > You will see a directory for the component test
> > suite –
> eg
> > “executor” and a full log – “Regress.log”.
> >
> > If you navigate to the component directory eg
> > executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off
> > and get familiar with the official results, that'll give me some
> > learning opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run?
> > So that it is easy to compare with the failure one get running the
> > test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both
> > are
> being
> > investigated. Otherwise we have been having good clean runs. So any
> > other failures you see need to be investigated. If you can give me
> > VNC access
> to
> > your machine, I can take a quick look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checkin
> g_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all
> > on master branch? I am getting some failure, and I wonder if they
> > are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's
> > questions on the wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> > I can add a page to the wiki for this. We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the
> > first time.
> >
> > Also Roberta had a comprehensive document describing the whole
> > regress environment back in MX days , I think. Not sure if anyone
> > has that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time
> > saving.
> >
> > Also how to write and/or modify when tests are complaining, but we
> > know that this is just because regression is based on delta, and
> > cannot recognize legitimate expected delta (due to know feature we
> > just implemented), and real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to
> > run the regression tests and how to interpret and deal with the
> > results? If
> someone
> > would like to volunteer to write such a page, it would be very
> > useful to newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <su...@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This
> > is an artifact of how runallsb prints out results. At the end of
> > every suite a summary of that suite is printed out (with no header
> > what suite it refers to since we have been running that suite for
> > the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time
> > with a header for each suite. So if you kick off regressions now and
> > come back several hours later there will be a screen with a summary
> > for all suites and then before the summary for core there will be
> > list of results with no suite name. This is the same as the last
> > suite in the summary (the hive test suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir.
> > > For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one
> > > for
> >
> > > each category in $scriptsdir. Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi
> > > <er...@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and
> > > > diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a
> > > > new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>
RE: Where are the logs for the first set of regression tests?
Posted by Anoop Sharma <an...@esgyn.com>.
if a known diff file already exists for a test, then you
cannot copy the log file as the expected file. Doing that
will mask the current known diffs.
You will need to manually update the expected file
and regenerate a new known diff file as diff line
numbers may have changed.
anoop
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:37 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
OK clear,
Thanks again
Eric
On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are
> satisfied with your new results you could simply copy the LOGaaa file
> as the new EXPECTEDaaa file. There are filters in the test suite that
> will hide things like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes
> > that translate into false failure, should I modify the comparison
> > file, or the file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the
> > wiki only for the individual "pull request" automated jobs. I will
> > add them for the official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> > 2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> > number 48
> >
> > 2015-08-04 08:30:13 originally caused by:
> >
> > 2015-08-04 08:30:13 Started by timer
> >
> > 2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> > 2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> > (redhat_ahw2.2) in workspace
> > /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> > 2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> > /tmp/hudson2056470402644332566.sh
> >
> > 2015-08-04 08:30:14 Detailed logs:
> > *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/*
> > <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/
> > >
> >
> > == > Navigate to sql-regress-log
> >
> > You will see a directory for the component test
> > suite –
> eg
> > “executor” and a full log – “Regress.log”.
> >
> > If you navigate to the component directory eg
> > executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off
> > and get familiar with the official results, that'll give me some
> > learning opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run?
> > So that it is easy to compare with the failure one get running the
> > test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both
> > are
> being
> > investigated. Otherwise we have been having good clean runs. So any
> > other failures you see need to be investigated. If you can give me
> > VNC access
> to
> > your machine, I can take a quick look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checkin
> g_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all
> > on master branch? I am getting some failure, and I wonder if they
> > are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's
> > questions on the wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> > I can add a page to the wiki for this. We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the
> > first time.
> >
> > Also Roberta had a comprehensive document describing the whole
> > regress environment back in MX days , I think. Not sure if anyone has
> > that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time
> > saving.
> >
> > Also how to write and/or modify when tests are complaining, but we
> > know that this is just because regression is based on delta, and
> > cannot recognize legitimate expected delta (due to know feature we
> > just implemented), and real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to
> > run the regression tests and how to interpret and deal with the
> > results? If
> someone
> > would like to volunteer to write such a page, it would be very
> > useful to newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <su...@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This
> > is an artifact of how runallsb prints out results. At the end of
> > every suite a summary of that suite is printed out (with no header
> > what suite it refers to since we have been running that suite for
> > the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time
> > with a header for each suite. So if you kick off regressions now and
> > come back several hours later there will be a screen with a summary
> > for all suites and then before the summary for core there will be
> > list of results with no suite name. This is the same as the last
> > suite in the summary (the hive test suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir.
> > > For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one
> > > for
> >
> > > each category in $scriptsdir. Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi
> > > <er...@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and
> > > > diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a
> > > > new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>
Re: Where are the logs for the first set of regression tests?
Posted by Eric Owhadi <er...@esgyn.com>.
OK clear,
Thanks again
Eric
On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <su...@gmail.com>
wrote:
> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are satisfied
> with your new results you could simply copy the LOGaaa file as the new
> EXPECTEDaaa file. There are filters in the test suite that will hide things
> like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes that
> > translate into false failure, should I modify the comparison file, or the
> > file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the wiki
> > only
> > for the individual "pull request" automated jobs. I will add them for the
> > official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> > 2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> > number 48
> >
> > 2015-08-04 08:30:13 originally caused by:
> >
> > 2015-08-04 08:30:13 Started by timer
> >
> > 2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> > 2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> > (redhat_ahw2.2) in workspace
> > /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> > 2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> > /tmp/hudson2056470402644332566.sh
> >
> > 2015-08-04 08:30:14 Detailed logs:
> > *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/*
> > <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/
> > >
> >
> > == > Navigate to sql-regress-log
> >
> > You will see a directory for the component test suite –
> eg
> > “executor” and a full log – “Regress.log”.
> >
> > If you navigate to the component directory eg executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off and get
> > familiar with the official results, that'll give me some learning
> > opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run? So
> > that
> > it is easy to compare with the failure one get running the test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both are
> being
> > investigated. Otherwise we have been having good clean runs. So any other
> > failures you see need to be investigated. If you can give me VNC access
> to
> > your machine, I can take a quick look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checking_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all on
> > master
> > branch? I am getting some failure, and I wonder if they are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's questions on
> > the
> > wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sa...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> > I can add a page to the wiki for this. We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the first
> > time.
> >
> > Also Roberta had a comprehensive document describing the whole regress
> > environment back in MX days , I think. Not sure if anyone has that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time saving.
> >
> > Also how to write and/or modify when tests are complaining, but we know
> > that
> > this is just because regression is based on delta, and cannot recognize
> > legitimate expected delta (due to know feature we just implemented), and
> > real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <da...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to run
> > the
> > regression tests and how to interpret and deal with the results? If
> someone
> > would like to volunteer to write such a page, it would be very useful to
> > newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <su...@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This is an
> > artifact of how runallsb prints out results. At the end of every suite a
> > summary of that suite is printed out (with no header what suite it refers
> > to
> > since we have been running that suite for the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time with a
> > header for each suite. So if you kick off regressions now and come back
> > several hours later there will be a screen with a summary for all suites
> > and
> > then before the summary for core there will be list of results with no
> > suite
> > name. This is the same as the last suite in the summary (the hive test
> > suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir. For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one for
> >
> > > each category in $scriptsdir. Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>
Re: Where are the logs for the first set of regression tests?
Posted by Suresh Subbiah <su...@gmail.com>.
Hi Eric,
Standard procedure is to modify the EXPECTEDaaa file. If you are satisfied
with your new results you could simply copy the LOGaaa file as the new
EXPECTEDaaa file. There are filters in the test suite that will hide things
like timestamp from the final comparison.
DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
while a fix is being worked on for a particular failure.
Thanks
Suresh
On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <er...@esgyn.com> wrote:
> Thanks Sandhya, very useful!
> BTW, if I want to change a test to take into account valid changes that
> translate into false failure, should I modify the comparison file, or the
> file that logs known legitimate differences?
> Thanks again for the help :-),
> Eric
>
> -----Original Message-----
> From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> Sent: Tuesday, August 4, 2015 1:19 PM
> To: dev@trafodion.incubator.apache.org
> Subject: RE: Where are the logs for the first set of regression tests?
>
> RE: Where are the logs for the first set of regression tests?
>
> There are logs and diffs. I notice I added the instructions on the wiki
> only
> for the individual "pull request" automated jobs. I will add them for the
> official test results as well.
>
> Here they are :
>
> Click on the link in the daily build email :
>
> *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> <https://jenkins02.trafodion.org/job/Check-Daily/48/>
>
> ==> Click on "Console Output" for any test suite you want to check
>
> ==> Around the 5th or 6th line you will see a link to detailed logs . Click
> that.
>
>
>
> 2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> number 48
>
> 2015-08-04 08:30:13 originally caused by:
>
> 2015-08-04 08:30:13 Started by timer
>
> 2015-08-04 08:30:13 [EnvInject] - Loading node environment variables.
>
> 2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> (redhat_ahw2.2) in workspace
> /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
>
> 2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> /tmp/hudson2056470402644332566.sh
>
> 2015-08-04 08:30:14 Detailed logs:
> *
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/*
> <
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/
> >
>
> == > Navigate to sql-regress-log
>
> You will see a directory for the component test suite – eg
> “executor” and a full log – “Regress.log”.
>
> If you navigate to the component directory eg executor, you
> will see all the DIFFs and LOG files .
>
> Sandhya
>
> -----Original Message-----
> From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
> Sent: Tuesday, August 4, 2015 11:06 AM
> To: dev@trafodion.incubator.apache.org
> Subject: RE: Where are the logs for the first set of regression tests?
>
> Hi Sandhya,
>
> Thanks for the offer, I'll first do a re-run with the feature off and get
> familiar with the official results, that'll give me some learning
> opportunity :-).
>
> There is no "diff" file on the official results automatic daily run? So
> that
> it is easy to compare with the failure one get running the test manually?
>
> Eric
>
> -----Original Message-----
>
> From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> <sa...@esgyn.com>]
>
> Sent: Tuesday, August 4, 2015 12:50 PM
>
> To: dev@trafodion.incubator.apache.org
>
> Subject: RE: Where are the logs for the first set of regression tests?
>
> Hi Eric,
>
> They are run on a nightly basis. Attached are the results form last night
> and previous night. As you can see 2 failures last night and both are being
> investigated. Otherwise we have been having good clean runs. So any other
> failures you see need to be investigated. If you can give me VNC access to
> your machine, I can take a quick look.
>
> I have documented how to read the official results on the wiki :
>
>
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checking_Test_Results
>
> Sandhya
>
> -----Original Message-----
>
> From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
>
> Sent: Tuesday, August 4, 2015 10:40 AM
>
> To: dev@trafodion.incubator.apache.org
>
> Subject: RE: Where are the logs for the first set of regression tests?
>
> Out of curiosity, anyone have recently run the regression test all on
> master
> branch? I am getting some failure, and I wonder if they are linked with my
> changes or if the current regression suit is having known temporary issues?
>
> Eric
>
> -----Original Message-----
>
> From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> <da...@esgyn.com>]
>
> Sent: Tuesday, August 4, 2015 11:26 AM
>
> To: dev@trafodion.incubator.apache.org
>
> Subject: RE: Where are the logs for the first set of regression tests?
>
> Right. Start with the simple stuff I think. Eric's e-mail trail is valuable
> as it is fresh newbie experience; if we could answer Eric's questions on
> the
> wiki I think that would be a great start.
>
> -----Original Message-----
>
> From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> <sa...@esgyn.com>]
>
> Sent: Tuesday, August 4, 2015 9:24 AM
>
> To: dev@trafodion.incubator.apache.org
>
> Subject: RE: Where are the logs for the first set of regression tests?
>
> Hi Eric, Dave,
>
> I can add a page to the wiki for this. We did have the steps documented
> in our old internal wiki for those starting on Trafodion for the first
> time.
>
> Also Roberta had a comprehensive document describing the whole regress
> environment back in MX days , I think. Not sure if anyone has that saved.
>
> Either way, for the wiki, perhaps a smaller page would be sufficient.
>
> Thanks
>
> Sandhya
>
> -----Original Message-----
>
> From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
>
> Sent: Tuesday, August 4, 2015 9:03 AM
>
> To: dev@trafodion.incubator.apache.org
>
> Subject: RE: Where are the logs for the first set of regression tests?
>
> I didn't want to ask, but indeed that would be fantastic and time saving.
>
> Also how to write and/or modify when tests are complaining, but we know
> that
> this is just because regression is based on delta, and cannot recognize
> legitimate expected delta (due to know feature we just implemented), and
> real regression.
>
> Eric
>
> -----Original Message-----
>
> From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> <da...@esgyn.com>]
>
> Sent: Tuesday, August 4, 2015 10:21 AM
>
> To: dev@trafodion.incubator.apache.org
>
> Subject: RE: Where are the logs for the first set of regression tests?
>
> Do we have a page on our Apache Trafodion wiki that describes how to run
> the
> regression tests and how to interpret and deal with the results? If someone
> would like to volunteer to write such a page, it would be very useful to
> newbies, I think.
>
> -----Original Message-----
>
> From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
>
> Sent: Tuesday, August 4, 2015 7:45 AM
>
> To: dev@trafodion.incubator.apache.org
>
> Subject: RE: Where are the logs for the first set of regression tests?
>
> Thanks Suresh, that's it indeed :-). Was driving me crazy.
>
> Eric
>
> -----Original Message-----
>
> From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> <su...@gmail.com>]
>
> Sent: Tuesday, August 4, 2015 9:41 AM
>
> To: dev@trafodion.incubator.apache.org
>
> Subject: Re: Where are the logs for the first set of regression tests?
>
> Hi Eric,
>
> I think the "first" set of tests is the same as the last set. This is an
> artifact of how runallsb prints out results. At the end of every suite a
> summary of that suite is printed out (with no header what suite it refers
> to
> since we have been running that suite for the last X minutes). Then at the
> end of a run it concatenates the summary of every suite, this time with a
> header for each suite. So if you kick off regressions now and come back
> several hours later there will be a screen with a summary for all suites
> and
> then before the summary for core there will be list of results with no
> suite
> name. This is the same as the last suite in the summary (the hive test
> suite).
>
> Thanks
>
> Suresh
>
> On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
>
> > Hi Eric,
>
> >
>
> > The results should be in one of the directories under $rundir. For
>
> > example, under $rundir/core, the result for TEST001 (under
>
> > $scriptsdir/core) is LOG001.
>
> >
>
> > In my ~/.bashrc file, I have these two variables defined.
>
> >
>
> > export rundir=/mnt/qfc/rundir
>
> >
>
> > export scriptsdir=$trafodionRoot/sql/regress
>
> >
>
> >
>
> > After mkdir $rundir, make sure $rundir contains directories, one for
>
> > each category in $scriptsdir. Once it is setup, you can run
>
> > regressions as many times as you want :-)
>
> >
>
> > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com>
> wrote:
>
> >
>
> > > That may be an obvious question, but I can’t find the logs and diffs
>
> > > for the first set of test showing up in the result of the runallsb:
>
> > >
>
> > > I can see all the logs and diffs for categories like core,
>
> > > compGeneral
>
> > etc,
>
> > > but the first set of test labelled:
>
> > >
>
> > > TEST001
>
> > >
>
> > > TEST002
>
> > >
>
> > > TEST003
>
> > >
>
> > > TEST004
>
> > >
>
> > > TEST005
>
> > >
>
> > > TEST006
>
> > >
>
> > > TEST015
>
> > >
>
> > > TEST017
>
> > >
>
> > > TEST018
>
> > >
>
> > > TEST020
>
> > >
>
> > >
>
> > >
>
> > > are not associated with a category, and they don’t have a
>
> > > subdirectory
>
> > with
>
> > > categoryname where I can find the other test results.
>
> > >
>
> > > Thanks in advance for the help.
>
> > >
>
> > >
>
> > >
>
> > > As a side note, if a test fails because of the compare, but the
>
> > > delta is legit, what should one do? Update known diffs? Or set a new
>
> > > compare file?
>
> > >
>
> > >
>
> > >
>
> > > Eric
>
> > >
>
> >
>
> >
>
> >
>
> > --
>
> > Regards, --Qifan
>
> >
>
RE: Where are the logs for the first set of regression tests?
Posted by Eric Owhadi <er...@esgyn.com>.
Thanks Sandhya, very useful!
BTW, if I want to change a test to take into account valid changes that
translate into false failure, should I modify the comparison file, or the
file that logs known legitimate differences?
Thanks again for the help :-),
Eric
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
Sent: Tuesday, August 4, 2015 1:19 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
RE: Where are the logs for the first set of regression tests?
There are logs and diffs. I notice I added the instructions on the wiki only
for the individual "pull request" automated jobs. I will add them for the
official test results as well.
Here they are :
Click on the link in the daily build email :
*https://jenkins02.trafodion.org/job/Check-Daily/48/*
<https://jenkins02.trafodion.org/job/Check-Daily/48/>
==> Click on "Console Output" for any test suite you want to check
==> Around the 5th or 6th line you will see a link to detailed logs . Click
that.
2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
number 48
2015-08-04 08:30:13 originally caused by:
2015-08-04 08:30:13 Started by timer
2015-08-04 08:30:13 [EnvInject] - Loading node environment variables.
2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
(redhat_ahw2.2) in workspace
/home/jenkins/workspace/core-regress-fullstack2-ahw2.2
2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
/tmp/hudson2056470402644332566.sh
2015-08-04 08:30:14 Detailed logs:
*http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/*
<http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/>
== > Navigate to sql-regress-log
You will see a directory for the component test suite – eg
“executor” and a full log – “Regress.log”.
If you navigate to the component directory eg executor, you
will see all the DIFFs and LOG files .
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
Sent: Tuesday, August 4, 2015 11:06 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Sandhya,
Thanks for the offer, I'll first do a re-run with the feature off and get
familiar with the official results, that'll give me some learning
opportunity :-).
There is no "diff" file on the official results automatic daily run? So that
it is easy to compare with the failure one get running the test manually?
Eric
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
<sa...@esgyn.com>]
Sent: Tuesday, August 4, 2015 12:50 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Eric,
They are run on a nightly basis. Attached are the results form last night
and previous night. As you can see 2 failures last night and both are being
investigated. Otherwise we have been having good clean runs. So any other
failures you see need to be investigated. If you can give me VNC access to
your machine, I can take a quick look.
I have documented how to read the official results on the wiki :
https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checking_Test_Results
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
Sent: Tuesday, August 4, 2015 10:40 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Out of curiosity, anyone have recently run the regression test all on master
branch? I am getting some failure, and I wonder if they are linked with my
changes or if the current regression suit is having known temporary issues?
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
<da...@esgyn.com>]
Sent: Tuesday, August 4, 2015 11:26 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Right. Start with the simple stuff I think. Eric's e-mail trail is valuable
as it is fresh newbie experience; if we could answer Eric's questions on the
wiki I think that would be a great start.
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
<sa...@esgyn.com>]
Sent: Tuesday, August 4, 2015 9:24 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Eric, Dave,
I can add a page to the wiki for this. We did have the steps documented
in our old internal wiki for those starting on Trafodion for the first time.
Also Roberta had a comprehensive document describing the whole regress
environment back in MX days , I think. Not sure if anyone has that saved.
Either way, for the wiki, perhaps a smaller page would be sufficient.
Thanks
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
Sent: Tuesday, August 4, 2015 9:03 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
I didn't want to ask, but indeed that would be fantastic and time saving.
Also how to write and/or modify when tests are complaining, but we know that
this is just because regression is based on delta, and cannot recognize
legitimate expected delta (due to know feature we just implemented), and
real regression.
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
<da...@esgyn.com>]
Sent: Tuesday, August 4, 2015 10:21 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Do we have a page on our Apache Trafodion wiki that describes how to run the
regression tests and how to interpret and deal with the results? If someone
would like to volunteer to write such a page, it would be very useful to
newbies, I think.
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
Sent: Tuesday, August 4, 2015 7:45 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Thanks Suresh, that's it indeed :-). Was driving me crazy.
Eric
-----Original Message-----
From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
<su...@gmail.com>]
Sent: Tuesday, August 4, 2015 9:41 AM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers to
since we have been running that suite for the last X minutes). Then at the
end of a run it concatenates the summary of every suite, this time with a
header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites and
then before the summary for core there will be list of results with no suite
name. This is the same as the last suite in the summary (the hive test
suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for
> each category in $scriptsdir. Once it is setup, you can run
> regressions as many times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs
> > for the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core,
> > compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a
> > subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the
> > delta is legit, what should one do? Update known diffs? Or set a new
> > compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
RE: Where are the logs for the first set of regression tests?
Posted by Sandhya Sundaresan <sa...@esgyn.com>.
RE: Where are the logs for the first set of regression tests?
There are logs and diffs. I notice I added the instructions on the wiki only
for the individual "pull request" automated jobs. I will add them for the
official test results as well.
Here they are :
Click on the link in the daily build email :
*https://jenkins02.trafodion.org/job/Check-Daily/48/*
<https://jenkins02.trafodion.org/job/Check-Daily/48/>
==> Click on "Console Output" for any test suite you want to check
==> Around the 5th or 6th line you will see a link to detailed logs . Click
that.
2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
number 48
2015-08-04 08:30:13 originally caused by:
2015-08-04 08:30:13 Started by timer
2015-08-04 08:30:13 [EnvInject] - Loading node environment variables.
2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
(redhat_ahw2.2) in workspace
/home/jenkins/workspace/core-regress-fullstack2-ahw2.2
2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
/tmp/hudson2056470402644332566.sh
2015-08-04 08:30:14 Detailed logs:
*http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/*
<http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/>
== > Navigate to sql-regress-log
You will see a directory for the component test suite – eg
“executor” and a full log – “Regress.log”.
If you navigate to the component directory eg executor, you
will see all the DIFFs and LOG files .
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
Sent: Tuesday, August 4, 2015 11:06 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Sandhya,
Thanks for the offer, I'll first do a re-run with the feature off and get
familiar with the official results, that'll give me some learning
opportunity :-).
There is no "diff" file on the official results automatic daily run? So
that it is easy to compare with the failure one get running the test
manually?
Eric
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
<sa...@esgyn.com>]
Sent: Tuesday, August 4, 2015 12:50 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Eric,
They are run on a nightly basis. Attached are the results form last night
and previous night. As you can see 2 failures last night and both are being
investigated. Otherwise we have been having good clean runs. So any other
failures you see need to be investigated. If you can give me VNC access to
your machine, I can take a quick look.
I have documented how to read the official results on the wiki :
https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checking_Test_Results
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
Sent: Tuesday, August 4, 2015 10:40 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Out of curiosity, anyone have recently run the regression test all on
master branch? I am getting some failure, and I wonder if they are linked
with my changes or if the current regression suit is having known temporary
issues?
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
<da...@esgyn.com>]
Sent: Tuesday, August 4, 2015 11:26 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Right. Start with the simple stuff I think. Eric's e-mail trail is valuable
as it is fresh newbie experience; if we could answer Eric's questions on
the wiki I think that would be a great start.
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
<sa...@esgyn.com>]
Sent: Tuesday, August 4, 2015 9:24 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Eric, Dave,
I can add a page to the wiki for this. We did have the steps documented
in our old internal wiki for those starting on Trafodion for the first time.
Also Roberta had a comprehensive document describing the whole regress
environment back in MX days , I think. Not sure if anyone has that saved.
Either way, for the wiki, perhaps a smaller page would be sufficient.
Thanks
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
Sent: Tuesday, August 4, 2015 9:03 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
I didn't want to ask, but indeed that would be fantastic and time saving.
Also how to write and/or modify when tests are complaining, but we know
that this is just because regression is based on delta, and cannot
recognize legitimate expected delta (due to know feature we just
implemented), and real regression.
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
<da...@esgyn.com>]
Sent: Tuesday, August 4, 2015 10:21 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Do we have a page on our Apache Trafodion wiki that describes how to run
the regression tests and how to interpret and deal with the results? If
someone would like to volunteer to write such a page, it would be very
useful to newbies, I think.
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <er...@esgyn.com>]
Sent: Tuesday, August 4, 2015 7:45 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Thanks Suresh, that's it indeed :-). Was driving me crazy.
Eric
-----Original Message-----
From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
<su...@gmail.com>]
Sent: Tuesday, August 4, 2015 9:41 AM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers
to since we have been running that suite for the last X minutes). Then at
the end of a run it concatenates the summary of every suite, this time with
a header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites
and then before the summary for core there will be list of results with no
suite name. This is the same as the last suite in the summary (the hive
test suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for
> each category in $scriptsdir. Once it is setup, you can run
> regressions as many times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs
> > for the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core,
> > compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a
> > subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the
> > delta is legit, what should one do? Update known diffs? Or set a new
> > compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
RE: Where are the logs for the first set of regression tests?
Posted by Eric Owhadi <er...@esgyn.com>.
Hi Sandhya,
Thanks for the offer, I'll first do a re-run with the feature off and get
familiar with the official results, that'll give me some learning
opportunity :-).
There is no "diff" file on the official results automatic daily run? So that
it is easy to compare with the failure one get running the test manually?
Eric
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
Sent: Tuesday, August 4, 2015 12:50 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Eric,
They are run on a nightly basis. Attached are the results form last night
and previous night. As you can see 2 failures last night and both are being
investigated. Otherwise we have been having good clean runs. So any other
failures you see need to be investigated. If you can give me VNC access to
your machine, I can take a quick look.
I have documented how to read the official results on the wiki :
https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checking_Test_Results
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 10:40 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Out of curiosity, anyone have recently run the regression test all on master
branch? I am getting some failure, and I wonder if they are linked with my
changes or if the current regression suit is having known temporary issues?
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com]
Sent: Tuesday, August 4, 2015 11:26 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Right. Start with the simple stuff I think. Eric's e-mail trail is valuable
as it is fresh newbie experience; if we could answer Eric's questions on the
wiki I think that would be a great start.
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
Sent: Tuesday, August 4, 2015 9:24 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Eric, Dave,
I can add a page to the wiki for this. We did have the steps documented
in our old internal wiki for those starting on Trafodion for the first time.
Also Roberta had a comprehensive document describing the whole regress
environment back in MX days , I think. Not sure if anyone has that saved.
Either way, for the wiki, perhaps a smaller page would be sufficient.
Thanks
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 9:03 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
I didn't want to ask, but indeed that would be fantastic and time saving.
Also how to write and/or modify when tests are complaining, but we know that
this is just because regression is based on delta, and cannot recognize
legitimate expected delta (due to know feature we just implemented), and
real regression.
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com]
Sent: Tuesday, August 4, 2015 10:21 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Do we have a page on our Apache Trafodion wiki that describes how to run the
regression tests and how to interpret and deal with the results? If someone
would like to volunteer to write such a page, it would be very useful to
newbies, I think.
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 7:45 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Thanks Suresh, that's it indeed :-). Was driving me crazy.
Eric
-----Original Message-----
From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com]
Sent: Tuesday, August 4, 2015 9:41 AM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers to
since we have been running that suite for the last X minutes). Then at the
end of a run it concatenates the summary of every suite, this time with a
header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites and
then before the summary for core there will be list of results with no suite
name. This is the same as the last suite in the summary (the hive test
suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for
> each category in $scriptsdir. Once it is setup, you can run
> regressions as many times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs
> > for the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core,
> > compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a
> > subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the
> > delta is legit, what should one do? Update known diffs? Or set a new
> > compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
RE: Where are the logs for the first set of regression tests?
Posted by Sandhya Sundaresan <sa...@esgyn.com>.
Hi Eric,
They are run on a nightly basis. Attached are the results form last night
and previous night. As you can see 2 failures last night and both are being
investigated. Otherwise we have been having good clean runs. So any other
failures you see need to be investigated. If you can give me VNC access to
your machine, I can take a quick look.
I have documented how to read the official results on the wiki :
https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checking_Test_Results
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 10:40 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Out of curiosity, anyone have recently run the regression test all on master
branch? I am getting some failure, and I wonder if they are linked with my
changes or if the current regression suit is having known temporary issues?
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com]
Sent: Tuesday, August 4, 2015 11:26 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Right. Start with the simple stuff I think. Eric's e-mail trail is valuable
as it is fresh newbie experience; if we could answer Eric's questions on the
wiki I think that would be a great start.
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
Sent: Tuesday, August 4, 2015 9:24 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Eric, Dave,
I can add a page to the wiki for this. We did have the steps documented
in our old internal wiki for those starting on Trafodion for the first time.
Also Roberta had a comprehensive document describing the whole regress
environment back in MX days , I think. Not sure if anyone has that saved.
Either way, for the wiki, perhaps a smaller page would be sufficient.
Thanks
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 9:03 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
I didn't want to ask, but indeed that would be fantastic and time saving.
Also how to write and/or modify when tests are complaining, but we know that
this is just because regression is based on delta, and cannot recognize
legitimate expected delta (due to know feature we just implemented), and
real regression.
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com]
Sent: Tuesday, August 4, 2015 10:21 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Do we have a page on our Apache Trafodion wiki that describes how to run the
regression tests and how to interpret and deal with the results? If someone
would like to volunteer to write such a page, it would be very useful to
newbies, I think.
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 7:45 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Thanks Suresh, that's it indeed :-). Was driving me crazy.
Eric
-----Original Message-----
From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com]
Sent: Tuesday, August 4, 2015 9:41 AM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers to
since we have been running that suite for the last X minutes). Then at the
end of a run it concatenates the summary of every suite, this time with a
header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites and
then before the summary for core there will be list of results with no suite
name. This is the same as the last suite in the summary (the hive test
suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for
> each category in $scriptsdir. Once it is setup, you can run
> regressions as many times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs
> > for the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core,
> > compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a
> > subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the
> > delta is legit, what should one do? Update known diffs? Or set a new
> > compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
RE: Where are the logs for the first set of regression tests?
Posted by Eric Owhadi <er...@esgyn.com>.
Out of curiosity, anyone have recently run the regression test all on master
branch? I am getting some failure, and I wonder if they are linked with my
changes or if the current regression suit is having known temporary issues?
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com]
Sent: Tuesday, August 4, 2015 11:26 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Right. Start with the simple stuff I think. Eric's e-mail trail is valuable
as it is fresh newbie experience; if we could answer Eric's questions on the
wiki I think that would be a great start.
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
Sent: Tuesday, August 4, 2015 9:24 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Eric, Dave,
I can add a page to the wiki for this. We did have the steps documented
in our old internal wiki for those starting on Trafodion for the first time.
Also Roberta had a comprehensive document describing the whole regress
environment back in MX days , I think. Not sure if anyone has that saved.
Either way, for the wiki, perhaps a smaller page would be sufficient.
Thanks
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 9:03 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
I didn't want to ask, but indeed that would be fantastic and time saving.
Also how to write and/or modify when tests are complaining, but we know that
this is just because regression is based on delta, and cannot recognize
legitimate expected delta (due to know feature we just implemented), and
real regression.
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com]
Sent: Tuesday, August 4, 2015 10:21 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Do we have a page on our Apache Trafodion wiki that describes how to run the
regression tests and how to interpret and deal with the results? If someone
would like to volunteer to write such a page, it would be very useful to
newbies, I think.
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 7:45 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Thanks Suresh, that's it indeed :-). Was driving me crazy.
Eric
-----Original Message-----
From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com]
Sent: Tuesday, August 4, 2015 9:41 AM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers to
since we have been running that suite for the last X minutes). Then at the
end of a run it concatenates the summary of every suite, this time with a
header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites and
then before the summary for core there will be list of results with no suite
name. This is the same as the last suite in the summary (the hive test
suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for
> each category in $scriptsdir. Once it is setup, you can run
> regressions as many times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs
> > for the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core,
> > compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a
> > subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the
> > delta is legit, what should one do? Update known diffs? Or set a new
> > compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
RE: Where are the logs for the first set of regression tests?
Posted by Dave Birdsall <da...@esgyn.com>.
Right. Start with the simple stuff I think. Eric's e-mail trail is valuable
as it is fresh newbie experience; if we could answer Eric's questions on the
wiki I think that would be a great start.
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
Sent: Tuesday, August 4, 2015 9:24 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Hi Eric, Dave,
I can add a page to the wiki for this. We did have the steps documented
in our old internal wiki for those starting on Trafodion for the first time.
Also Roberta had a comprehensive document describing the whole regress
environment back in MX days , I think. Not sure if anyone has that saved.
Either way, for the wiki, perhaps a smaller page would be sufficient.
Thanks
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 9:03 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
I didn't want to ask, but indeed that would be fantastic and time saving.
Also how to write and/or modify when tests are complaining, but we know that
this is just because regression is based on delta, and cannot recognize
legitimate expected delta (due to know feature we just implemented), and
real regression.
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com]
Sent: Tuesday, August 4, 2015 10:21 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Do we have a page on our Apache Trafodion wiki that describes how to run the
regression tests and how to interpret and deal with the results? If someone
would like to volunteer to write such a page, it would be very useful to
newbies, I think.
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 7:45 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Thanks Suresh, that's it indeed :-). Was driving me crazy.
Eric
-----Original Message-----
From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com]
Sent: Tuesday, August 4, 2015 9:41 AM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers to
since we have been running that suite for the last X minutes). Then at the
end of a run it concatenates the summary of every suite, this time with a
header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites and
then before the summary for core there will be list of results with no suite
name. This is the same as the last suite in the summary (the hive test
suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for
> each category in $scriptsdir. Once it is setup, you can run
> regressions as many times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs
> > for the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core,
> > compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a
> > subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the
> > delta is legit, what should one do? Update known diffs? Or set a new
> > compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
RE: Where are the logs for the first set of regression tests?
Posted by Sandhya Sundaresan <sa...@esgyn.com>.
Hi Eric, Dave,
I can add a page to the wiki for this. We did have the steps documented
in our old internal wiki for those starting on Trafodion for the first time.
Also Roberta had a comprehensive document describing the whole regress
environment back in MX days , I think. Not sure if anyone has that saved.
Either way, for the wiki, perhaps a smaller page would be sufficient.
Thanks
Sandhya
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 9:03 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
I didn't want to ask, but indeed that would be fantastic and time saving.
Also how to write and/or modify when tests are complaining, but we know that
this is just because regression is based on delta, and cannot recognize
legitimate expected delta (due to know feature we just implemented), and
real regression.
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com]
Sent: Tuesday, August 4, 2015 10:21 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Do we have a page on our Apache Trafodion wiki that describes how to run the
regression tests and how to interpret and deal with the results? If someone
would like to volunteer to write such a page, it would be very useful to
newbies, I think.
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 7:45 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Thanks Suresh, that's it indeed :-). Was driving me crazy.
Eric
-----Original Message-----
From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com]
Sent: Tuesday, August 4, 2015 9:41 AM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers to
since we have been running that suite for the last X minutes). Then at the
end of a run it concatenates the summary of every suite, this time with a
header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites and
then before the summary for core there will be list of results with no suite
name. This is the same as the last suite in the summary (the hive test
suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for
> each category in $scriptsdir. Once it is setup, you can run
> regressions as many times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs
> > for the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core,
> > compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a
> > subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the
> > delta is legit, what should one do? Update known diffs? Or set a new
> > compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
RE: Where are the logs for the first set of regression tests?
Posted by Eric Owhadi <er...@esgyn.com>.
I didn't want to ask, but indeed that would be fantastic and time saving.
Also how to write and/or modify when tests are complaining, but we know that
this is just because regression is based on delta, and cannot recognize
legitimate expected delta (due to know feature we just implemented), and
real regression.
Eric
-----Original Message-----
From: Dave Birdsall [mailto:dave.birdsall@esgyn.com]
Sent: Tuesday, August 4, 2015 10:21 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Do we have a page on our Apache Trafodion wiki that describes how to run the
regression tests and how to interpret and deal with the results? If someone
would like to volunteer to write such a page, it would be very useful to
newbies, I think.
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 7:45 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Thanks Suresh, that's it indeed :-). Was driving me crazy.
Eric
-----Original Message-----
From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com]
Sent: Tuesday, August 4, 2015 9:41 AM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers to
since we have been running that suite for the last X minutes). Then at the
end of a run it concatenates the summary of every suite, this time with a
header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites and
then before the summary for core there will be list of results with no suite
name. This is the same as the last suite in the summary (the hive test
suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for
> each category in $scriptsdir. Once it is setup, you can run
> regressions as many times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs
> > for the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core,
> > compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a
> > subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the
> > delta is legit, what should one do? Update known diffs? Or set a new
> > compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
RE: Where are the logs for the first set of regression tests?
Posted by Dave Birdsall <da...@esgyn.com>.
Do we have a page on our Apache Trafodion wiki that describes how to run the
regression tests and how to interpret and deal with the results? If someone
would like to volunteer to write such a page, it would be very useful to
newbies, I think.
-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 7:45 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?
Thanks Suresh, that's it indeed :-). Was driving me crazy.
Eric
-----Original Message-----
From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com]
Sent: Tuesday, August 4, 2015 9:41 AM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers to
since we have been running that suite for the last X minutes). Then at the
end of a run it concatenates the summary of every suite, this time with a
header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites and
then before the summary for core there will be list of results with no suite
name. This is the same as the last suite in the summary (the hive test
suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for
> each category in $scriptsdir. Once it is setup, you can run
> regressions as many times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs
> > for the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core,
> > compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a
> > subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the
> > delta is legit, what should one do? Update known diffs? Or set a new
> > compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
RE: Where are the logs for the first set of regression tests?
Posted by Eric Owhadi <er...@esgyn.com>.
Thanks Suresh, that's it indeed :-). Was driving me crazy.
Eric
-----Original Message-----
From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com]
Sent: Tuesday, August 4, 2015 9:41 AM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers to
since we have been running that suite for the last X minutes). Then at the
end of a run it concatenates the summary of every suite, this time with a
header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites and
then before the summary for core there will be list of results with no suite
name. This is the same as the last suite in the summary (the hive test
suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for
> each category in $scriptsdir. Once it is setup, you can run
> regressions as many times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs
> > for the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core,
> > compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a
> > subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the
> > delta is legit, what should one do? Update known diffs? Or set a new
> > compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
Re: Where are the logs for the first set of regression tests?
Posted by Suresh Subbiah <su...@gmail.com>.
Hi Eric,
I think the "first" set of tests is the same as the last set. This is an
artifact of how runallsb prints out results. At the end of every suite a
summary of that suite is printed out (with no header what suite it refers
to since we have been running that suite for the last X minutes). Then at
the end of a run it concatenates the summary of every suite, this time with
a header for each suite. So if you kick off regressions now and come back
several hours later there will be a screen with a summary for all suites
and then before the summary for core there will be list of results with no
suite name. This is the same as the last suite in the summary (the hive
test suite).
Thanks
Suresh
On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qi...@esgyn.com> wrote:
> Hi Eric,
>
> The results should be in one of the directories under $rundir. For
> example, under $rundir/core, the result for TEST001 (under
> $scriptsdir/core) is LOG001.
>
> In my ~/.bashrc file, I have these two variables defined.
>
> export rundir=/mnt/qfc/rundir
>
> export scriptsdir=$trafodionRoot/sql/regress
>
>
> After mkdir $rundir, make sure $rundir contains directories, one for each
> category in $scriptsdir. Once it is setup, you can run regressions as many
> times as you want :-)
>
> On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
>
> > That may be an obvious question, but I can’t find the logs and diffs for
> > the first set of test showing up in the result of the runallsb:
> >
> > I can see all the logs and diffs for categories like core, compGeneral
> etc,
> > but the first set of test labelled:
> >
> > TEST001
> >
> > TEST002
> >
> > TEST003
> >
> > TEST004
> >
> > TEST005
> >
> > TEST006
> >
> > TEST015
> >
> > TEST017
> >
> > TEST018
> >
> > TEST020
> >
> >
> >
> > are not associated with a category, and they don’t have a subdirectory
> with
> > categoryname where I can find the other test results.
> >
> > Thanks in advance for the help.
> >
> >
> >
> > As a side note, if a test fails because of the compare, but the delta is
> > legit, what should one do? Update known diffs? Or set a new compare file?
> >
> >
> >
> > Eric
> >
>
>
>
> --
> Regards, --Qifan
>
RE: Where are the logs for the first set of regression tests?
Posted by Eric Owhadi <er...@esgyn.com>.
Hi Qifan,
I understood the one that belong to a group like core, compGeneral,
executor and all the other categories and was able to locate the logs and
diffs for these.
The problem is for the first set of results that don't belong to a category.
There is no subdirectory with the category name and the logs and diffs in
it.
And I happen to have lots of fails in this set (9 out of 10). Only test 17
pass. 1,2,3,4,5,6,15,18, 20 fail.
Eric
-----Original Message-----
From: Qifan Chen [mailto:qifan.chen@esgyn.com]
Sent: Tuesday, August 4, 2015 9:20 AM
To: dev
Subject: Re: Where are the logs for the first set of regression tests?
Hi Eric,
The results should be in one of the directories under $rundir. For example,
under $rundir/core, the result for TEST001 (under
$scriptsdir/core) is LOG001.
In my ~/.bashrc file, I have these two variables defined.
export rundir=/mnt/qfc/rundir
export scriptsdir=$trafodionRoot/sql/regress
After mkdir $rundir, make sure $rundir contains directories, one for each
category in $scriptsdir. Once it is setup, you can run regressions as many
times as you want :-)
On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
> That may be an obvious question, but I can’t find the logs and diffs
> for the first set of test showing up in the result of the runallsb:
>
> I can see all the logs and diffs for categories like core, compGeneral
> etc, but the first set of test labelled:
>
> TEST001
>
> TEST002
>
> TEST003
>
> TEST004
>
> TEST005
>
> TEST006
>
> TEST015
>
> TEST017
>
> TEST018
>
> TEST020
>
>
>
> are not associated with a category, and they don’t have a subdirectory
> with categoryname where I can find the other test results.
>
> Thanks in advance for the help.
>
>
>
> As a side note, if a test fails because of the compare, but the delta
> is legit, what should one do? Update known diffs? Or set a new compare
> file?
>
>
>
> Eric
>
--
Regards, --Qifan
Re: Where are the logs for the first set of regression tests?
Posted by Qifan Chen <qi...@esgyn.com>.
Hi Eric,
The results should be in one of the directories under $rundir. For
example, under $rundir/core, the result for TEST001 (under
$scriptsdir/core) is LOG001.
In my ~/.bashrc file, I have these two variables defined.
export rundir=/mnt/qfc/rundir
export scriptsdir=$trafodionRoot/sql/regress
After mkdir $rundir, make sure $rundir contains directories, one for each
category in $scriptsdir. Once it is setup, you can run regressions as many
times as you want :-)
On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <er...@esgyn.com> wrote:
> That may be an obvious question, but I can’t find the logs and diffs for
> the first set of test showing up in the result of the runallsb:
>
> I can see all the logs and diffs for categories like core, compGeneral etc,
> but the first set of test labelled:
>
> TEST001
>
> TEST002
>
> TEST003
>
> TEST004
>
> TEST005
>
> TEST006
>
> TEST015
>
> TEST017
>
> TEST018
>
> TEST020
>
>
>
> are not associated with a category, and they don’t have a subdirectory with
> categoryname where I can find the other test results.
>
> Thanks in advance for the help.
>
>
>
> As a side note, if a test fails because of the compare, but the delta is
> legit, what should one do? Update known diffs? Or set a new compare file?
>
>
>
> Eric
>
--
Regards, --Qifan