You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by dragon512 <gi...@git.apache.org> on 2017/02/17 18:30:50 UTC
[GitHub] trafficserver pull request #1465: Add new testing system to the test subdire...
GitHub user dragon512 opened a pull request:
https://github.com/apache/trafficserver/pull/1465
Add new testing system to the test subdirectory
First pass of new Testing system for Traffic server
Contain micro-server is the test/tools directory from the traffic-replay work
Contains some ported tests from the old TSQA
Contains some script to try to make it easy to bootstrap the testing system on most common Linux setup
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dragon512/trafficserver master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/trafficserver/pull/1465.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1465
----
commit 9db49d8e29b283977c54e7b919878e0bed4b700b
Author: Unknown <dr...@live.com>
Date: 2017-02-17T18:27:45Z
Add new testing system to the test subdirectory
First pass of new Testing system for Trafficserver
Contains some ported tests from the old TSQA
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by dragon512 <gi...@git.apache.org>.
Github user dragon512 commented on the issue:
https://github.com/apache/trafficserver/pull/1465
I should have added:
- the copyright headers�for the source files
- blurb to the notices file ( I hope this is correct @zwoop?)
- I have add some documentation @persiaAziz is going�to add to this for the origin server creation API
- minus the first commit the git�user name, etc is fixed. I will address this more once I squash the commits
todo:
Need the�system @jpeach was having�an issue with the bootstrap script so�I can try to address it. The main systems of Ubuntu/debian, RHEL6-7, Fedora should hopefully be working well enough.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by atsci <gi...@git.apache.org>.
Github user atsci commented on the issue:
https://github.com/apache/trafficserver/pull/1465
FreeBSD build *failed*! See https://ci.trafficserver.apache.org/job/freebsd-github/1581/ for details.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #1465: Add new testing system to the test subdire...
Posted by dragon512 <gi...@git.apache.org>.
GitHub user dragon512 reopened a pull request:
https://github.com/apache/trafficserver/pull/1465
Add new testing system to the test subdirectory
First pass of new Testing system for Traffic server
Contain micro-server is the test/tools directory from the traffic-replay work
Contains some ported tests from the old TSQA
Contains some script to try to make it easy to bootstrap the testing system on most common Linux setup
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dragon512/trafficserver master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/trafficserver/pull/1465.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1465
----
commit c9b675119b42810f0ad7eba76193d7c586c3a4fd
Author: Jason Kenny <dr...@live.com>
Date: 2017-02-22T16:30:10Z
Add new testing infra to ATS
Added new testing code
Added Apache license header
Add some getting started documentation
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by dragon512 <gi...@git.apache.org>.
Github user dragon512 commented on the issue:
https://github.com/apache/trafficserver/pull/1465
So copy paste header from the other source files in ATS.
There is no concern that the AuTest code ( ie the testing engine used) is MIT.. just asking.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by atsci <gi...@git.apache.org>.
Github user atsci commented on the issue:
https://github.com/apache/trafficserver/pull/1465
Linux build *successful*! See https://ci.trafficserver.apache.org/job/linux-github/1475/ for details.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by atsci <gi...@git.apache.org>.
Github user atsci commented on the issue:
https://github.com/apache/trafficserver/pull/1465
Intel CC build *successful*! See https://ci.trafficserver.apache.org/job/icc-github/14/ for details.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on the issue:
https://github.com/apache/trafficserver/pull/1465
I think it should be a top level subdirectory because it's not just for ci. We should expect developers to be running at least some of these tests locally, it was part of the basic design.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #1465: Add new testing system to the test subdire...
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/1465#discussion_r103799188
--- Diff: tests/getting_started.md ---
@@ -0,0 +1,197 @@
+
+# Getting Started
+
+This directory contains different tests for Apache Trafficserver. It is recommended that all test move to this common area under the correct location based on the type of test being added.
+
+## Layout
+The current layout is:
+
+**gold_tests/** - contains all the TSQA v4 based tests that run on the Reusable Gold Testing System (AuTest)
+**tools/** - contain programs used to help with testing.
+
+In the future a directory called **"unit/"** will be added for adding unit tests based on some standardized testing system.
+
+
+## Scripts
+
+To help with easy running of the tests, there is a autest.sh and bootstrap.py file.
+
+### autest.sh
+This file is a simple wrapper that will call the AuTest program in a python virtualenv. If the virtualenv is not setup it will try to install system. That will set up the Reusable Gold Testing System on most systems in a Python virtual environment. The wrapper add some basic options to the command to point to the location of the tests. Add --help for more details on options for running autest test system.
+
+### bootstrap.py
+This script should try to install python35 or better on the system, and needed python packages for running the tests.
+
+# Advance setup
+
+AuTest can be install manually instead of using the wrapper script. The advange of this is that it is often easier to debug issues with the testing system, or the tests. There are two ways this can be done.
+1. run the bootstrap script then source the path with a "source /env-test/bin/activate" command. At this point autest command should run without the wrapper script
--- End diff --
This really installs stuff in `/env-test`? That seems unfriendly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by bryancall <gi...@git.apache.org>.
Github user bryancall commented on the issue:
https://github.com/apache/trafficserver/pull/1465
Here is the full output from running the tests: https://da.gd/XOiCg
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by dragon512 <gi...@git.apache.org>.
Github user dragon512 commented on the issue:
https://github.com/apache/trafficserver/pull/1465
This is just starter documentation.. We will move it to the official sphinx doc later.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by zwoop <gi...@git.apache.org>.
Github user zwoop commented on the issue:
https://github.com/apache/trafficserver/pull/1465
MIT is fine. You should retain the MIT header in any imported source, and also update the NOTICE file accordingly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by atsci <gi...@git.apache.org>.
Github user atsci commented on the issue:
https://github.com/apache/trafficserver/pull/1465
clang-analyzer build *successful*! See https://ci.trafficserver.apache.org/job/clang-analyzer-github/146/ for details.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by dragon512 <gi...@git.apache.org>.
Github user dragon512 commented on the issue:
https://github.com/apache/trafficserver/pull/1465
@jpeach So I agree with everything you suggest except for moving�the test under the ci directory. The source layout is confusing enough as is.�hiding this under some location no one will look does not help people contribute to the trafficserver. We need to migrate test to a standard, expected location.�
What system did you test this one that did not install�virtual-env? This would help me address this problem. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #1465: Add new testing system to the test subdire...
Posted by dragon512 <gi...@git.apache.org>.
Github user dragon512 closed the pull request at:
https://github.com/apache/trafficserver/pull/1465
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by zwoop <gi...@git.apache.org>.
Github user zwoop commented on the issue:
https://github.com/apache/trafficserver/pull/1465
I'm going to land this. There will definitely be RAT report errors on this, but we can address those either with adding licenses when needed, or add to the RAT exclusions (e.g. exclude *.gold if that's appropriate).
Also, we need to clean up the NOTICES / LICENSE attribution; we must include the entire MIT license in LICENSE, and as such, we likely can and should remove the lines from NOTICES.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by persiaAziz <gi...@git.apache.org>.
Github user persiaAziz commented on the issue:
https://github.com/apache/trafficserver/pull/1465
Shouldn't the documentation be a inside the doc folder of ATS? Will this show up on the traffic server website?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by dragon512 <gi...@git.apache.org>.
Github user dragon512 commented on the issue:
https://github.com/apache/trafficserver/pull/1465
I just pushed a fix for CentOs and RHEL system to bootstrap better. I believe this should take care of the issue @jpeach reported given was running on one of those systems
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by bryancall <gi...@git.apache.org>.
Github user bryancall commented on the issue:
https://github.com/apache/trafficserver/pull/1465
[approve ci]
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #1465: Add new testing system to the test subdire...
Posted by dragon512 <gi...@git.apache.org>.
Github user dragon512 commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/1465#discussion_r103950549
--- Diff: tests/getting_started.md ---
@@ -0,0 +1,197 @@
+
+# Getting Started
+
+This directory contains different tests for Apache Trafficserver. It is recommended that all test move to this common area under the correct location based on the type of test being added.
+
+## Layout
+The current layout is:
+
+**gold_tests/** - contains all the TSQA v4 based tests that run on the Reusable Gold Testing System (AuTest)
+**tools/** - contain programs used to help with testing.
+
+In the future a directory called **"unit/"** will be added for adding unit tests based on some standardized testing system.
+
+
+## Scripts
+
+To help with easy running of the tests, there is a autest.sh and bootstrap.py file.
+
+### autest.sh
+This file is a simple wrapper that will call the AuTest program in a python virtualenv. If the virtualenv is not setup it will try to install system. That will set up the Reusable Gold Testing System on most systems in a Python virtual environment. The wrapper add some basic options to the command to point to the location of the tests. Add --help for more details on options for running autest test system.
+
+### bootstrap.py
+This script should try to install python35 or better on the system, and needed python packages for running the tests.
+
+# Advance setup
+
+AuTest can be install manually instead of using the wrapper script. The advange of this is that it is often easier to debug issues with the testing system, or the tests. There are two ways this can be done.
+1. run the bootstrap script then source the path with a "source /env-test/bin/activate" command. At this point autest command should run without the wrapper script
--- End diff --
no typo.. it is in ./env-test. I will correct
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #1465: Add new testing system to the test subdire...
Posted by zwoop <gi...@git.apache.org>.
Github user zwoop closed the pull request at:
https://github.com/apache/trafficserver/pull/1465
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by zwoop <gi...@git.apache.org>.
Github user zwoop commented on the issue:
https://github.com/apache/trafficserver/pull/1465
Should this be in the "ci" tree rather that its own top-level tree?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by atsci <gi...@git.apache.org>.
Github user atsci commented on the issue:
https://github.com/apache/trafficserver/pull/1465
FreeBSD build *successful*! See https://ci.trafficserver.apache.org/job/freebsd-github/1597/ for details.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by dragon512 <gi...@git.apache.org>.
Github user dragon512 commented on the issue:
https://github.com/apache/trafficserver/pull/1465
I am not sure why it should be under "ci" tree. That is stuff for the open source CI system. Why would tests for ATS go under the ci data file? Tests are for developer, not the ci system. Sure the ci system would run it, much like it build code outside the ci tree. We don't put our source under ci. With the TSQA code it is confusing that you look there, and what you should run. I want to clean this up.
The "tests" tree should contain all the test for ats, including unit tests. I had made arguments for clarify it would be much better is all the source was under a src/ tree much like the documentation ->doc/, examples -> example/, contribution->contrib/ and plugins -> plugins/ are under there own tree. If the concern is creating an extra directory at the top level. I would say that if we really want to clean this up we should move lib, proxy, mgmt, iocore, cmd under a src/ directory. I am not the only one that has echoed this point-of-view as such a layout would greatly clean up the tree and make it more obvious out-of-the-box where to start looking
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by bryancall <gi...@git.apache.org>.
Github user bryancall commented on the issue:
https://github.com/apache/trafficserver/pull/1465
@dragon512
I am seeing some errors when running the tests:
```
Total of 15 test
Unknown: 0
Exception: 0
Failed: 10
Warning: 0
kipped: 2
Passed: 3
```
Looking at the output from the errors:
```
- > User-Agent: curl/``
? ^^
+ > User-Agent: curl/7.51.0
? ^^^^^^
- ``
- < HTTP/1.1 404 Not Found on Accelerator
+ > Accept: */*
+ >
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by bryancall <gi...@git.apache.org>.
Github user bryancall commented on the issue:
https://github.com/apache/trafficserver/pull/1465
I am seeing 4 failed tests.
```
[bcall@homer tests]$ ./autest.sh --ats-bin /usr/local/bin
Running Test basic:. Passed
Running Test basic-cop:. Passed
Running Test basic-manager:. Passed
Running Test cache-generation-clear:FFFFFF Failed
Running Test cache-generation-disjoint:...... Passed
Running Test config:. Passed
Running Test copy_config:.. Passed
Running Test copy_config2:.. Passed
Running Test custom:........ Failed
Running Test custom_response: Skipped
Warning: Skipping test custom_response because:
This test fails at the moment as is turned off
Running Test disjoint-wait-for-cache: Skipped
Warning: Skipping test disjoint-wait-for-cache because:
This test fails at the moment as is turned off
Running Test header_rewrite:F Failed
Running Test http2:. Passed
Running Test remap_http:.FFF.. Failed
Running Test remap_https:...... Passed
Total of 15 test
Unknown: 0
Exception: 0
Failed: 4
Warning: 0
Skipped: 2
Passed: 9
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by atsci <gi...@git.apache.org>.
Github user atsci commented on the issue:
https://github.com/apache/trafficserver/pull/1465
Linux build *successful*! See https://ci.trafficserver.apache.org/job/linux-github/1491/ for details.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by bryancall <gi...@git.apache.org>.
Github user bryancall commented on the issue:
https://github.com/apache/trafficserver/pull/1465
@zwoop Did you run the tests?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #1465: Add new testing system to the test subdirectory
Posted by dragon512 <gi...@git.apache.org>.
Github user dragon512 commented on the issue:
https://github.com/apache/trafficserver/pull/1465
@bryancall So from what I can see at the moment. master is broken. 7.1 is working fine minus the custom logfile test. The remap_http and header-rewrite fail with
- < Proxy-Connection: keep-alive
? ------
+ < Connection: keep-alive
< Server: ATS/7.1.0
This shows ATS is returning something different (ie we had Proxy-Connection now it is Connection)
> GET http://www.example.com:80/ HTTP/1.1
> Host: www.example.com
> User-Agent: curl/7.50.1
> Accept: */*
+ >
- > Proxy-Connection: Keep-Alive
- ``
< HTTP/1.1 200 OK
< Date: Fri, 24 Feb 2017 19:45:35 GMT
< Age: 0
< Transfer-Encoding: chunked
- < Proxy-Connection: keep-alive
? ------
+ < Connection: keep-alive
< Server: ATS/7.1.0
<
{ [5 bytes data]
which says we changed the header order and case change of Keep-Alive to keep-alive.
ATS 6.2 did not error, but 6.2.1 errors the same as 7.1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---