You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafodion.apache.org by Gunnar Tapper <ta...@gmail.com> on 2015/11/23 06:50:21 UTC
Tests
Hi,
Can someone explain how a contributor goes about creating a new test and a
new test library?
SQL Core
- Test library: For example, sql/regress/catman1
- Test: For example: cidefs
DCS
It seems that the main test is AppTest. dcs/src/test doesn't contain a main
README file.
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Tests
Posted by Gunnar Tapper <ta...@gmail.com>.
BTW, this version contains fixes to the issues that Trina reported the
other day.
On Sun, Nov 22, 2015 at 11:49 PM, Gunnar Tapper <ta...@gmail.com>
wrote:
> Hi Sandhya,
>
> I just finished porting this page including the note. You can review it by:
>
> 1. Downloading and unpacking the following zip file:
> https://drive.google.com/file/d/0BxlwNhWxn8iTUGhhOXVKUGVRTUE/view?usp=sharing
> 2. Open the testing.html file in your browser.
>
> I'd like to create a new page that discusses test development to
> complement the current information, which is more focused on running tests.
>
> So, I'd like to cover topics such as new test suite, new test, modify
> test, and retire test.
>
> Does this makes sense?
>
> Thanks,
>
> Gunnar
>
> On Sun, Nov 22, 2015 at 10:58 PM, Sandhya Sundaresan <
> sandhya.sundaresan@esgyn.com> wrote:
>
>> Hi Gunnar,
>> There are a few details here.
>> https://wiki.trafodion.org/wiki/index.php/Test_Suites#Core
>>
>> Do you think we can keep this content and add to this ? I had added notes
>> on modifying an existing regression test but didn't add anything specific
>> for adding a new test suite or a new test. I can do that for the core
>> suite.
>> But this is the "old" website. Where should I add it now ?
>>
>> This same page contains notes on other dcs tests as well.
>> Thanks
>> Sandhya
>>
>>
>> -----Original Message-----
>> From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
>> Sent: Sunday, November 22, 2015 9:50 PM
>> To: dev@trafodion.incubator.apache.org
>> Subject: Tests
>>
>> Hi,
>>
>> Can someone explain how a contributor goes about creating a new test and a
>> new test library?
>>
>> SQL Core
>>
>> - Test library: For example, sql/regress/catman1
>> - Test: For example: cidefs
>>
>> DCS
>>
>> It seems that the main test is AppTest. dcs/src/test doesn't contain a
>> main
>> README file.
>>
>>
>> --
>> Thanks,
>>
>> Gunnar
>> *If you think you can you can, if you think you can't you're right.*
>>
>
>
>
> --
> Thanks,
>
> Gunnar
> *If you think you can you can, if you think you can't you're right.*
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Tests
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi,
The .html files are generated from markdown-formatted files. This
particular one hasn't been checked in yet. I'm providing the current
content here in case you want to modify it and send it back. :)
Thanks,
Gunnar
---
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
# Component Tests
Trafodion comes with several component-specific testing libraries.
## SQL Core
The SQL core components are written in a combination of C++ and Java.
You should ensure that the current set of regression tests pass each time
you add or modify a SQL feature.
If adding a new feature, then check that it is either covered by an
existing regression test or add a new test to an existing test suite.
### Test Suites
**Location**: <code>$MY_SQROOT/../sql/regress</code>
<table>
<tr>
<th>Directory</th>
<th>Usage</th>
</tr>
<tr>
<td><strong><code>catman1</code></strong></td>
<td>Tests the Catalog Manager.</td>
</tr>
<tr>
<td><strong><code>charsets</code></strong></td>
<td>Tests Character Sets.</td>
</tr>
<tr>
<td><strong><code>compGeneral</code></strong></td>
<td></td>
</tr>
<tr>
<td><strong><code>core</code></strong></td>
<td></td>
</tr>
<tr>
<td><strong><code>executor</code></strong></td>
<td>Tests the SQL Executor.</td>
</tr>
<tr>
<td><strong><code>fullstack2</code></strong></td>
<td></td>
</tr>
<tr>
<td><strong><code>hive</code></strong></td>
<td></td>
</tr>
<tr>
<td><strong><code>newregr</code></strong></td>
<td></td>
</tr>
<tr>
<td><strong><code>qat</code></strong></td>
<td></td>
</tr>
<tr>
<td><strong><code>seabase</code></strong></td>
<td></td>
</tr>
<tr>
<td><strong><code>tools</code></strong></td>
<td></td>
</tr>
<tr>
<td><strong><code>udr</code></strong></td>
<td></td>
</tr>
</table>
### Check Test Results
On completion, the test run prints out a test summary. All tests should
pass, or pass with known diffs.
Test results are written to the **<code>runregr-sb.log</code>** file in
each component's directory. Therefore, you can check the test results after
the fact as follows:
cd $MY_SQROOT/rundir
grep FAIL */runregr-sb.log
A successful test run shows no failures.
### Run Full Test Suite
This suite tests:
* SQL Compiler
* SQL Executor
* Transactions
* Foundation
Do the following:
cd $MY_SQHOME
. ./sqenv.sh
cd $MY_SQROOT/../sql/regress
tools/runallsb
### Run Individual Test Suites
You can select individual test suites as follows:
cd $MY_SQHOME
. ./sqenv.sh
cd $MY_SQROOT/../sql/regress
tools/runallsb <suite1> <suite2>
### Running an Individual Test
#### If You've Already Run the Test Suite
If you have already run the suite once, then you will have all your
directories set up and you can run one test as follows:
cd $MY_SQROOT/../sql/regress/<suite>
# You can add the following two exports to .bashrc or .profile for
convenience
export rundir=$MY_SQROOT/rundir
export scriptsdir=$MY_SQROOT/../sql/regress
# run the test
cd $rundir/<suite>
$scriptsdir/<suite>/runregr -sb <test>
**Example**
cd $rundir/executor
$scriptsdir/executor/runregr -sb TEST130
#### If You've Not Run the Test Suite
If you have not run any regression suites so far, then you will not have
the required sub directories set up. You manually create them for each
suite you want to run.
cd $MY_SQROOT/../sql/regress/<suite>
# You can add the following two exports to .bashrc or .profile for
convenience
export rundir=$MY_SQROOT/rundir
export scriptsdir=$MY_SQROOT/../sql/regress
mkdir $rundir
cd $rundir
# <suitename> should match the name of each directory in $scriptsdir
mkdir <suitename>
# run the test
cd $rundir/<suite>
$scriptsdir/<suite>/runregr -sb <test>
### Detecting Failures
If you see failures in any of your tests, you want to try running that
suite or test individually as detailed above.
Open up the DIFF file and correlate them to the LOG and EXPECTED files.
* DIFF files are in **<code>$rundir/\<suite name\></code>**.
* LOG files are in **<code>$rundir/\<suite name\></code>**.
* EXPECTED files are in **<code>$scriptsdir/\<suite name\></code>**.
To narrow down the failure, open up the test file (for example:
**<code>TEST130</code>**) in **<code>$scriptsdir/executor</code>**.
Recreate the problem with a smaller set of SQL commands and create a script
to run from **<code>sqlci</code>**. If it's an issue that can be recreated
only by running the whole suite, you can add a line to the test just before
the command that fails to include a **<code>wait</code>** or a
**<code>sleep</code>** **<code>sh sleep 60</code>** will make the test
pause and give you time to attach the **<code>sqlci</code>** process to the
debugger. (You can find the PID of the **<code>sqlci</code>** process using
**<code>sqps</code>** on the command line)
Introducing a **<code>wait</code>** in the test will wait forever until you
enter a character. This is another way to make the test pause to attach the
debugger to the **<code>sqlci</code>** process.
### Modifying an Existing Test
If you would like to add coverage for your new change, you can modify an
existing test.
Run the test after your modifications. If you are satisfied with your
results, you need to modify the **<code>EXPECTED\<test number\></code>**
file to reflect your new change. The standard way to do it is to copy the
**<code>LOG\<test number\></code>** file to **<code>EXPECTED\<test
number\></code>** file.
## Database Connectivity Services (DCS)
The DCS test suite is organized per the Maven standard.
## JDBC T4
The code is written in Java, and is built and unit tested using Maven. The
test suite organization and use follow Maven standards.
Instructions for setting up and running the test can be found in source
tree at <strong><code>dcs/src/test/jdbc_test</code></strong>.
## PyODBC Tests
The code is written for the Python 2.7 [<code>unittest</code>](
https://docs.python.org/2/library/unittest.html) framework.
It is run via the **<code>Testr</code>** and **<code>Tox</code>**.
Instructions for setting up and running the test can be found in source
tree at <strong><code>dcs/src/test/pytests</code></strong>.
# Functional Tests
## Phoenix
The Phoenix tests provides basic functional tests for Trafodion. These
tests were originally adapted from their counterpart at [salesforce.com](
salesforce.com).
The tests are executed using Maven with a Python wrapper. You can run them
the same way on your own workstation instance just like the way Jenkins
runs them. Do the following:
<!-- This part is done in raw HTML because it's too complex to do this
level of formatting in markdowm. -->
<ol>
<li>Prior to running Phoenix tests, you need to bring up your Trafodion
instance and DCS. You need to configure at least 2-4 servers for DCS. The
tests need at least two mxosrvrs as they make two connections at any given
time. We recommend configuring DCS with four mxosrvrs since we have seen
situations that mxosrvrs do not get released in time for the next
connection if there are only two mxosrvrs.</li>
<li><p>Run the Phoenix tests from source tree</p>
<p style="text-indent=20px">
<pre>
cd tests/phx
phoenix_test.py --target=<host>:<port> --user=dontcare
--pw=dontcare --targettype=TR --javahome=<jdk>
--jdbccp=<jdir>/jdbcT4.jar</pre>
</p>
<p style="text-indent=20px">
<ul>
<li><strong><host></strong>: your workstation name or IP
address.</li>
<li><strong><port></strong>: your DCS master port
number.</li>
<li><strong><jdk></strong>: the directory containing the
jdk1.7.0_21_64 or later version of the JDK.</li>
<li><strong><jdir></strong>: the directory containing your
JDBC T4 jar file. (export/lib if you downloaded a Trafodion binary
package.)</li>
</ul>
</p>
</li>
<li><p>Analyze the results. The test results can be found in
<strong><code>phoenix_test/target/surefire-reports</code></strong>. If
there are any failures, they would come with file names and line
numbers.</p>
<p>The source code can be found in
<strong><code>phoenix_test/src/test/java/com/hp/phoenix/end2end</code></strong>.</p>
<p>These are JDBC tests written in java.</p>
</li>
</ol>
On Mon, Nov 23, 2015 at 12:05 AM, Sandhya Sundaresan <
sandhya.sundaresan@esgyn.com> wrote:
> Makes sense. Will add some additional content to cover those missing
> topics.
> Thanks
> Sandhya
>
> -----Original Message-----
> From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
> Sent: Sunday, November 22, 2015 10:49 PM
> To: dev@trafodion.incubator.apache.org
> Subject: Re: Tests
>
> Hi Sandhya,
>
> I just finished porting this page including the note. You can review it by:
>
> 1. Downloading and unpacking the following zip file:
>
> https://drive.google.com/file/d/0BxlwNhWxn8iTUGhhOXVKUGVRTUE/view?usp=sharing
> 2. Open the testing.html file in your browser.
>
> I'd like to create a new page that discusses test development to complement
> the current information, which is more focused on running tests.
>
> So, I'd like to cover topics such as new test suite, new test, modify test,
> and retire test.
>
> Does this makes sense?
>
> Thanks,
>
> Gunnar
>
> On Sun, Nov 22, 2015 at 10:58 PM, Sandhya Sundaresan <
> sandhya.sundaresan@esgyn.com> wrote:
>
> > Hi Gunnar,
> > There are a few details here.
> > https://wiki.trafodion.org/wiki/index.php/Test_Suites#Core
> >
> > Do you think we can keep this content and add to this ? I had added
> > notes on modifying an existing regression test but didn't add anything
> > specific for adding a new test suite or a new test. I can do that for
> > the core suite.
> > But this is the "old" website. Where should I add it now ?
> >
> > This same page contains notes on other dcs tests as well.
> > Thanks
> > Sandhya
> >
> >
> > -----Original Message-----
> > From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
> > Sent: Sunday, November 22, 2015 9:50 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: Tests
> >
> > Hi,
> >
> > Can someone explain how a contributor goes about creating a new test
> > and a new test library?
> >
> > SQL Core
> >
> > - Test library: For example, sql/regress/catman1
> > - Test: For example: cidefs
> >
> > DCS
> >
> > It seems that the main test is AppTest. dcs/src/test doesn't contain a
> > main README file.
> >
> >
> > --
> > Thanks,
> >
> > Gunnar
> > *If you think you can you can, if you think you can't you're right.*
> >
>
>
>
> --
> Thanks,
>
> Gunnar
> *If you think you can you can, if you think you can't you're right.*
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
RE: Tests
Posted by Anuradha Hegde <an...@esgyn.com>.
HI Gunnar,
The package for Phoenix test should change from com.hp to org.trafodion.
Phoenix test works for both type 2 and type 4 drivers.
Instead of saying PyODBC tests, we should say ODBC Tests which are run using
the python frame work. Just like how we have given instructions for SQL and
JDBC, we should provide instructions for running ODBC tests
cd dcs/src/test/pytests
./config.sh -d <host>t:<port> -t <Location of your Linux ODBC driver tar
file>
tox -e py27
Thanks
Anu
-----Original Message-----
From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
Sent: Sunday, November 22, 2015 11:05 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Tests
Makes sense. Will add some additional content to cover those missing topics.
Thanks
Sandhya
-----Original Message-----
From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
Sent: Sunday, November 22, 2015 10:49 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Tests
Hi Sandhya,
I just finished porting this page including the note. You can review it by:
1. Downloading and unpacking the following zip file:
https://drive.google.com/file/d/0BxlwNhWxn8iTUGhhOXVKUGVRTUE/view?usp=sharing
2. Open the testing.html file in your browser.
I'd like to create a new page that discusses test development to complement
the current information, which is more focused on running tests.
So, I'd like to cover topics such as new test suite, new test, modify test,
and retire test.
Does this makes sense?
Thanks,
Gunnar
On Sun, Nov 22, 2015 at 10:58 PM, Sandhya Sundaresan <
sandhya.sundaresan@esgyn.com> wrote:
> Hi Gunnar,
> There are a few details here.
> https://wiki.trafodion.org/wiki/index.php/Test_Suites#Core
>
> Do you think we can keep this content and add to this ? I had added
> notes on modifying an existing regression test but didn't add anything
> specific for adding a new test suite or a new test. I can do that for
> the core suite.
> But this is the "old" website. Where should I add it now ?
>
> This same page contains notes on other dcs tests as well.
> Thanks
> Sandhya
>
>
> -----Original Message-----
> From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
> Sent: Sunday, November 22, 2015 9:50 PM
> To: dev@trafodion.incubator.apache.org
> Subject: Tests
>
> Hi,
>
> Can someone explain how a contributor goes about creating a new test
> and a new test library?
>
> SQL Core
>
> - Test library: For example, sql/regress/catman1
> - Test: For example: cidefs
>
> DCS
>
> It seems that the main test is AppTest. dcs/src/test doesn't contain a
> main README file.
>
>
> --
> Thanks,
>
> Gunnar
> *If you think you can you can, if you think you can't you're right.*
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
RE: Tests
Posted by Sandhya Sundaresan <sa...@esgyn.com>.
Makes sense. Will add some additional content to cover those missing topics.
Thanks
Sandhya
-----Original Message-----
From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
Sent: Sunday, November 22, 2015 10:49 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Tests
Hi Sandhya,
I just finished porting this page including the note. You can review it by:
1. Downloading and unpacking the following zip file:
https://drive.google.com/file/d/0BxlwNhWxn8iTUGhhOXVKUGVRTUE/view?usp=sharing
2. Open the testing.html file in your browser.
I'd like to create a new page that discusses test development to complement
the current information, which is more focused on running tests.
So, I'd like to cover topics such as new test suite, new test, modify test,
and retire test.
Does this makes sense?
Thanks,
Gunnar
On Sun, Nov 22, 2015 at 10:58 PM, Sandhya Sundaresan <
sandhya.sundaresan@esgyn.com> wrote:
> Hi Gunnar,
> There are a few details here.
> https://wiki.trafodion.org/wiki/index.php/Test_Suites#Core
>
> Do you think we can keep this content and add to this ? I had added
> notes on modifying an existing regression test but didn't add anything
> specific for adding a new test suite or a new test. I can do that for
> the core suite.
> But this is the "old" website. Where should I add it now ?
>
> This same page contains notes on other dcs tests as well.
> Thanks
> Sandhya
>
>
> -----Original Message-----
> From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
> Sent: Sunday, November 22, 2015 9:50 PM
> To: dev@trafodion.incubator.apache.org
> Subject: Tests
>
> Hi,
>
> Can someone explain how a contributor goes about creating a new test
> and a new test library?
>
> SQL Core
>
> - Test library: For example, sql/regress/catman1
> - Test: For example: cidefs
>
> DCS
>
> It seems that the main test is AppTest. dcs/src/test doesn't contain a
> main README file.
>
>
> --
> Thanks,
>
> Gunnar
> *If you think you can you can, if you think you can't you're right.*
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Tests
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi Sandhya,
I just finished porting this page including the note. You can review it by:
1. Downloading and unpacking the following zip file:
https://drive.google.com/file/d/0BxlwNhWxn8iTUGhhOXVKUGVRTUE/view?usp=sharing
2. Open the testing.html file in your browser.
I'd like to create a new page that discusses test development to complement
the current information, which is more focused on running tests.
So, I'd like to cover topics such as new test suite, new test, modify test,
and retire test.
Does this makes sense?
Thanks,
Gunnar
On Sun, Nov 22, 2015 at 10:58 PM, Sandhya Sundaresan <
sandhya.sundaresan@esgyn.com> wrote:
> Hi Gunnar,
> There are a few details here.
> https://wiki.trafodion.org/wiki/index.php/Test_Suites#Core
>
> Do you think we can keep this content and add to this ? I had added notes
> on modifying an existing regression test but didn't add anything specific
> for adding a new test suite or a new test. I can do that for the core
> suite.
> But this is the "old" website. Where should I add it now ?
>
> This same page contains notes on other dcs tests as well.
> Thanks
> Sandhya
>
>
> -----Original Message-----
> From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
> Sent: Sunday, November 22, 2015 9:50 PM
> To: dev@trafodion.incubator.apache.org
> Subject: Tests
>
> Hi,
>
> Can someone explain how a contributor goes about creating a new test and a
> new test library?
>
> SQL Core
>
> - Test library: For example, sql/regress/catman1
> - Test: For example: cidefs
>
> DCS
>
> It seems that the main test is AppTest. dcs/src/test doesn't contain a main
> README file.
>
>
> --
> Thanks,
>
> Gunnar
> *If you think you can you can, if you think you can't you're right.*
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
RE: Tests
Posted by Sandhya Sundaresan <sa...@esgyn.com>.
Hi Gunnar,
There are a few details here.
https://wiki.trafodion.org/wiki/index.php/Test_Suites#Core
Do you think we can keep this content and add to this ? I had added notes
on modifying an existing regression test but didn't add anything specific
for adding a new test suite or a new test. I can do that for the core suite.
But this is the "old" website. Where should I add it now ?
This same page contains notes on other dcs tests as well.
Thanks
Sandhya
-----Original Message-----
From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
Sent: Sunday, November 22, 2015 9:50 PM
To: dev@trafodion.incubator.apache.org
Subject: Tests
Hi,
Can someone explain how a contributor goes about creating a new test and a
new test library?
SQL Core
- Test library: For example, sql/regress/catman1
- Test: For example: cidefs
DCS
It seems that the main test is AppTest. dcs/src/test doesn't contain a main
README file.
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*