You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by James Peach <jp...@apache.org> on 2017/07/23 18:59:24 UTC

autests broken?

Hi all,

I am unable to get autest tests to run succesfully on my Fedora26 box. I’m running the ci/regression script, which builds and runs all the tests. This doesn’t fail in jenkins so there must be some non-obvious thing wrong here.

AFAICT tsxs fails for some reason that is not captured by the test suite anywhere that I can find. Once that fails everything cascades.

Running Test tls_hooks:F Failed
Running Test tls_hooks10:F Failed
Running Test tls_hooks11:F Failed
Running Test tls_hooks12:F Failed
Running Test tls_hooks2:F Failed
Running Test tls_hooks3:F Failed
Running Test tls_hooks4:F Failed
Running Test tls_hooks6:F Failed
Running Test tls_hooks7:F Failed
Running Test tls_hooks8:F Failed
Running Test tls_hooks9:F Failed
...
 Test: tls_hooks: Failed
    File: tls_hooks.test.py
    Directory: /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks
   Starting Test tls_hooks : No issues found - Passed
      Reason: Started!
   Process: ts: Failed
...
     Setting up : RunCommand - Failed
        Reason: Failed to run command 'tsxs -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.so' because:
          Actual exit code '1' did not match expected value '0'
     Test : Checking that ReturnCode == None - Passed
        Reason: Returned Value: None == None
     Stream stderr : Checking that StdErrFile matches gold/ts-preaccept-1.gold - Failed
        Reason: File differences
         Gold File : /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks/gold/ts-preaccept-1.gold
         Data File : /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.stderr.txt
         - `` DIAG: (ssl_hook_test) Setup callbacks pa=1 sni=0 cert=0 cert_imm=0 pa_delay=0
         - `` DIAG: (ssl_hook_test) Pre accept callback 0 `` - event is good
         - ``
     Stream All : Pre accept message appears only once or twice - Failed
        Reason: Contents of /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.all.txt did not contains expression: "\A(?:(?!Pre accept callback 0).)*Pre accept callback 0.*(Pre accept callback 0)?(?!.*Pre accept callback 0).*\Z"
     file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
        Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log'
     file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
        Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log’




Re: autests broken?

Posted by Leif Hedstrom <zw...@apache.org>.

> On Jul 23, 2017, at 5:12 PM, James Peach <jp...@apache.org> wrote:
> 
> 
>> On Jul 23, 2017, at 3:39 PM, Leif Hedstrom <zw...@apache.org> wrote:
>> 
>> We should make it part of the Jenkins job that runs autest. Let me know what is needed, and I'll add it. Or part of make check or something.
>> 
>> Adding it to the normal build seems potentially confusing, such that people / package owners builds and installs it (but I know we can make them non-installed targets, so maybe not a huge deal).
> 
> I’d suggest an —-enable-test-plugins that works just like —enable-example-plugins.


+1

-- Leif 
> 
>> 
>>> On Jul 23, 2017, at 3:17 PM, James Peach <jp...@apache.org> wrote:
>>> 
>>> 
>>>> On Jul 23, 2017, at 2:03 PM, Alan M. Carroll <am...@network-geographics.com> wrote:
>>>> 
>>>> Susan fixed this in PR 2291.
>>> 
>>> Thanks. I verified that this fixed it for me.
>>> 
>>> For posterity, the error can be found here:
>>> 
>>> [jpeach@sk1 _setup_tmp_RunCommand_0]$ pwd
>>> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_setup_tmp_RunCommand_0
>>> [jpeach@sk1 _setup_tmp_RunCommand_0]$ cat stream.all.txt
>>> compiling /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.lo
>>> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc: In function 'void parse_callbacks(int, const char**, int&, int&, int&, int&, int&)':
>>> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc:135:15: error: 'index' was not declared in this scope
>>>       ptr = index(argv[i], '=');
>>>             ^~~~~
>>> tsxs: compilation failed: c++ -I/tmp/org.apache.trafficserver.23933/dst/include -std=c++11 -ggdb3 -pipe -Wall -Wextra -Wno-ignored-qualifiers -Wno-unused-parameter -Wno-format-truncation -Werror -Wno-invalid-offsetof -mcx16 -fpic -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.lo
>>> 
>>> We should integrate the test plugins into the build so that build problems are caught early and often.
>>> 
>>>> 
>>>>> You should be able to see the attempted compile output in one of the streams, such as .../_tmp_tls_hooks_tls_hooks_ts/streamm.all.txt, which should provide an indication of the reason for the failure. I know there have been some problems involving the flags stored in tsxs which you could be hitting. Does TS actually compile for you on Fedora26? There's an outstanding issue about failures in that regard (see #2266).
>>>> 
>>>>>> Hi all,
>>>> 
>>>>>> I am unable to get autest tests to run succesfully on my Fedora26 box. I’m running the ci/regression script, which builds and runs all the tests. This doesn’t fail in jenkins so there must be some non-obvious thing wrong here.
>>>> 
>>>>>> AFAICT tsxs fails for some reason that is not captured by the test suite anywhere that I can find. Once that fails everything cascades.
>>>> 
>>>>>> Running Test tls_hooks:F Failed
>>>>>> Running Test tls_hooks10:F Failed
>>>>>> Running Test tls_hooks11:F Failed
>>>>>> Running Test tls_hooks12:F Failed
>>>>>> Running Test tls_hooks2:F Failed
>>>>>> Running Test tls_hooks3:F Failed
>>>>>> Running Test tls_hooks4:F Failed
>>>>>> Running Test tls_hooks6:F Failed
>>>>>> Running Test tls_hooks7:F Failed
>>>>>> Running Test tls_hooks8:F Failed
>>>>>> Running Test tls_hooks9:F Failed
>>>>>> ...
>>>>>> Test: tls_hooks: Failed
>>>>>> File: tls_hooks.test.py
>>>>>> Directory: /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks
>>>>>> Starting Test tls_hooks : No issues found - Passed
>>>>>>   Reason: Started!
>>>>>> Process: ts: Failed
>>>>>> ...
>>>>>>  Setting up : RunCommand - Failed
>>>>>>     Reason: Failed to run command 'tsxs -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.so' because:
>>>>>>       Actual exit code '1' did not match expected value '0'
>>>>>>  Test : Checking that ReturnCode == None - Passed
>>>>>>     Reason: Returned Value: None == None
>>>>>>  Stream stderr : Checking that StdErrFile matches gold/ts-preaccept-1.gold - Failed
>>>>>>     Reason: File differences
>>>>>>      Gold File : /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks/gold/ts-preaccept-1.gold
>>>>>>      Data File : /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.stderr.txt
>>>>>>      - `` DIAG: (ssl_hook_test) Setup callbacks pa=1 sni=0 cert=0 cert_imm=0 pa_delay=0
>>>>>>      - `` DIAG: (ssl_hook_test) Pre accept callback 0 `` - event is good
>>>>>>      - ``
>>>>>>  Stream All : Pre accept message appears only once or twice - Failed
>>>>>>     Reason: Contents of /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.all.txt did not contains expression: "\A(?:(?!Pre accept callback 0).)*Pre accept callback 0.*(Pre accept callback 0)?(?!.*Pre accept callback 0).*\Z"
>>>>>>  file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>>>>>>     Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log'
>>>>>>  file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>>>>>>     Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log’
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>> 
> 


Re: autests broken?

Posted by James Peach <jp...@apache.org>.
> On Jul 23, 2017, at 3:39 PM, Leif Hedstrom <zw...@apache.org> wrote:
> 
> We should make it part of the Jenkins job that runs autest. Let me know what is needed, and I'll add it. Or part of make check or something.
> 
> Adding it to the normal build seems potentially confusing, such that people / package owners builds and installs it (but I know we can make them non-installed targets, so maybe not a huge deal).

I’d suggest an —-enable-test-plugins that works just like —enable-example-plugins.

> 
>> On Jul 23, 2017, at 3:17 PM, James Peach <jp...@apache.org> wrote:
>> 
>> 
>>> On Jul 23, 2017, at 2:03 PM, Alan M. Carroll <am...@network-geographics.com> wrote:
>>> 
>>> Susan fixed this in PR 2291.
>> 
>> Thanks. I verified that this fixed it for me.
>> 
>> For posterity, the error can be found here:
>> 
>> [jpeach@sk1 _setup_tmp_RunCommand_0]$ pwd
>> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_setup_tmp_RunCommand_0
>> [jpeach@sk1 _setup_tmp_RunCommand_0]$ cat stream.all.txt
>> compiling /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.lo
>> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc: In function 'void parse_callbacks(int, const char**, int&, int&, int&, int&, int&)':
>> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc:135:15: error: 'index' was not declared in this scope
>>        ptr = index(argv[i], '=');
>>              ^~~~~
>> tsxs: compilation failed: c++ -I/tmp/org.apache.trafficserver.23933/dst/include -std=c++11 -ggdb3 -pipe -Wall -Wextra -Wno-ignored-qualifiers -Wno-unused-parameter -Wno-format-truncation -Werror -Wno-invalid-offsetof -mcx16 -fpic -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.lo
>> 
>> We should integrate the test plugins into the build so that build problems are caught early and often.
>> 
>>> 
>>>> You should be able to see the attempted compile output in one of the streams, such as .../_tmp_tls_hooks_tls_hooks_ts/streamm.all.txt, which should provide an indication of the reason for the failure. I know there have been some problems involving the flags stored in tsxs which you could be hitting. Does TS actually compile for you on Fedora26? There's an outstanding issue about failures in that regard (see #2266).
>>> 
>>>>> Hi all,
>>> 
>>>>> I am unable to get autest tests to run succesfully on my Fedora26 box. I’m running the ci/regression script, which builds and runs all the tests. This doesn’t fail in jenkins so there must be some non-obvious thing wrong here.
>>> 
>>>>> AFAICT tsxs fails for some reason that is not captured by the test suite anywhere that I can find. Once that fails everything cascades.
>>> 
>>>>> Running Test tls_hooks:F Failed
>>>>> Running Test tls_hooks10:F Failed
>>>>> Running Test tls_hooks11:F Failed
>>>>> Running Test tls_hooks12:F Failed
>>>>> Running Test tls_hooks2:F Failed
>>>>> Running Test tls_hooks3:F Failed
>>>>> Running Test tls_hooks4:F Failed
>>>>> Running Test tls_hooks6:F Failed
>>>>> Running Test tls_hooks7:F Failed
>>>>> Running Test tls_hooks8:F Failed
>>>>> Running Test tls_hooks9:F Failed
>>>>> ...
>>>>> Test: tls_hooks: Failed
>>>>>  File: tls_hooks.test.py
>>>>>  Directory: /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks
>>>>> Starting Test tls_hooks : No issues found - Passed
>>>>>    Reason: Started!
>>>>> Process: ts: Failed
>>>>> ...
>>>>>   Setting up : RunCommand - Failed
>>>>>      Reason: Failed to run command 'tsxs -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.so' because:
>>>>>        Actual exit code '1' did not match expected value '0'
>>>>>   Test : Checking that ReturnCode == None - Passed
>>>>>      Reason: Returned Value: None == None
>>>>>   Stream stderr : Checking that StdErrFile matches gold/ts-preaccept-1.gold - Failed
>>>>>      Reason: File differences
>>>>>       Gold File : /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks/gold/ts-preaccept-1.gold
>>>>>       Data File : /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.stderr.txt
>>>>>       - `` DIAG: (ssl_hook_test) Setup callbacks pa=1 sni=0 cert=0 cert_imm=0 pa_delay=0
>>>>>       - `` DIAG: (ssl_hook_test) Pre accept callback 0 `` - event is good
>>>>>       - ``
>>>>>   Stream All : Pre accept message appears only once or twice - Failed
>>>>>      Reason: Contents of /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.all.txt did not contains expression: "\A(?:(?!Pre accept callback 0).)*Pre accept callback 0.*(Pre accept callback 0)?(?!.*Pre accept callback 0).*\Z"
>>>>>   file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>>>>>      Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log'
>>>>>   file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>>>>>      Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log’
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>> 
> 


Re: autests broken?

Posted by Leif Hedstrom <zw...@apache.org>.
We should make it part of the Jenkins job that runs autest. Let me know what is needed, and I'll add it. Or part of make check or something.

Adding it to the normal build seems potentially confusing, such that people / package owners builds and installs it (but I know we can make them non-installed targets, so maybe not a huge deal).

> On Jul 23, 2017, at 3:17 PM, James Peach <jp...@apache.org> wrote:
> 
> 
>> On Jul 23, 2017, at 2:03 PM, Alan M. Carroll <am...@network-geographics.com> wrote:
>> 
>> Susan fixed this in PR 2291.
> 
> Thanks. I verified that this fixed it for me.
> 
> For posterity, the error can be found here:
> 
> [jpeach@sk1 _setup_tmp_RunCommand_0]$ pwd
> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_setup_tmp_RunCommand_0
> [jpeach@sk1 _setup_tmp_RunCommand_0]$ cat stream.all.txt
>  compiling /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.lo
> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc: In function 'void parse_callbacks(int, const char**, int&, int&, int&, int&, int&)':
> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc:135:15: error: 'index' was not declared in this scope
>         ptr = index(argv[i], '=');
>               ^~~~~
> tsxs: compilation failed: c++ -I/tmp/org.apache.trafficserver.23933/dst/include -std=c++11 -ggdb3 -pipe -Wall -Wextra -Wno-ignored-qualifiers -Wno-unused-parameter -Wno-format-truncation -Werror -Wno-invalid-offsetof -mcx16 -fpic -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.lo
> 
> We should integrate the test plugins into the build so that build problems are caught early and often.
> 
>> 
>>> You should be able to see the attempted compile output in one of the streams, such as .../_tmp_tls_hooks_tls_hooks_ts/streamm.all.txt, which should provide an indication of the reason for the failure. I know there have been some problems involving the flags stored in tsxs which you could be hitting. Does TS actually compile for you on Fedora26? There's an outstanding issue about failures in that regard (see #2266).
>> 
>>>> Hi all,
>> 
>>>> I am unable to get autest tests to run succesfully on my Fedora26 box. I’m running the ci/regression script, which builds and runs all the tests. This doesn’t fail in jenkins so there must be some non-obvious thing wrong here.
>> 
>>>> AFAICT tsxs fails for some reason that is not captured by the test suite anywhere that I can find. Once that fails everything cascades.
>> 
>>>> Running Test tls_hooks:F Failed
>>>> Running Test tls_hooks10:F Failed
>>>> Running Test tls_hooks11:F Failed
>>>> Running Test tls_hooks12:F Failed
>>>> Running Test tls_hooks2:F Failed
>>>> Running Test tls_hooks3:F Failed
>>>> Running Test tls_hooks4:F Failed
>>>> Running Test tls_hooks6:F Failed
>>>> Running Test tls_hooks7:F Failed
>>>> Running Test tls_hooks8:F Failed
>>>> Running Test tls_hooks9:F Failed
>>>> ...
>>>> Test: tls_hooks: Failed
>>>>   File: tls_hooks.test.py
>>>>   Directory: /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks
>>>>  Starting Test tls_hooks : No issues found - Passed
>>>>     Reason: Started!
>>>>  Process: ts: Failed
>>>> ...
>>>>    Setting up : RunCommand - Failed
>>>>       Reason: Failed to run command 'tsxs -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.so' because:
>>>>         Actual exit code '1' did not match expected value '0'
>>>>    Test : Checking that ReturnCode == None - Passed
>>>>       Reason: Returned Value: None == None
>>>>    Stream stderr : Checking that StdErrFile matches gold/ts-preaccept-1.gold - Failed
>>>>       Reason: File differences
>>>>        Gold File : /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks/gold/ts-preaccept-1.gold
>>>>        Data File : /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.stderr.txt
>>>>        - `` DIAG: (ssl_hook_test) Setup callbacks pa=1 sni=0 cert=0 cert_imm=0 pa_delay=0
>>>>        - `` DIAG: (ssl_hook_test) Pre accept callback 0 `` - event is good
>>>>        - ``
>>>>    Stream All : Pre accept message appears only once or twice - Failed
>>>>       Reason: Contents of /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.all.txt did not contains expression: "\A(?:(?!Pre accept callback 0).)*Pre accept callback 0.*(Pre accept callback 0)?(?!.*Pre accept callback 0).*\Z"
>>>>    file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>>>>       Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log'
>>>>    file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>>>>       Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log’
>> 
>> 
>> 
>> 
>> 
>> 
>> 
> 


Re: autests broken?

Posted by James Peach <jp...@apache.org>.
> On Jul 23, 2017, at 2:03 PM, Alan M. Carroll <am...@network-geographics.com> wrote:
> 
> Susan fixed this in PR 2291.

Thanks. I verified that this fixed it for me.

For posterity, the error can be found here:

[jpeach@sk1 _setup_tmp_RunCommand_0]$ pwd
/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_setup_tmp_RunCommand_0
[jpeach@sk1 _setup_tmp_RunCommand_0]$ cat stream.all.txt
  compiling /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -> /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.lo
/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc: In function 'void parse_callbacks(int, const char**, int&, int&, int&, int&, int&)':
/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc:135:15: error: 'index' was not declared in this scope
         ptr = index(argv[i], '=');
               ^~~~~
tsxs: compilation failed: c++ -I/tmp/org.apache.trafficserver.23933/dst/include -std=c++11 -ggdb3 -pipe -Wall -Wextra -Wno-ignored-qualifiers -Wno-unused-parameter -Wno-format-truncation -Werror -Wno-invalid-offsetof -mcx16 -fpic -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.lo

We should integrate the test plugins into the build so that build problems are caught early and often.

> 
>> You should be able to see the attempted compile output in one of the streams, such as .../_tmp_tls_hooks_tls_hooks_ts/streamm.all.txt, which should provide an indication of the reason for the failure. I know there have been some problems involving the flags stored in tsxs which you could be hitting. Does TS actually compile for you on Fedora26? There's an outstanding issue about failures in that regard (see #2266).
> 
>>> Hi all,
> 
>>> I am unable to get autest tests to run succesfully on my Fedora26 box. I’m running the ci/regression script, which builds and runs all the tests. This doesn’t fail in jenkins so there must be some non-obvious thing wrong here.
> 
>>> AFAICT tsxs fails for some reason that is not captured by the test suite anywhere that I can find. Once that fails everything cascades.
> 
>>> Running Test tls_hooks:F Failed
>>> Running Test tls_hooks10:F Failed
>>> Running Test tls_hooks11:F Failed
>>> Running Test tls_hooks12:F Failed
>>> Running Test tls_hooks2:F Failed
>>> Running Test tls_hooks3:F Failed
>>> Running Test tls_hooks4:F Failed
>>> Running Test tls_hooks6:F Failed
>>> Running Test tls_hooks7:F Failed
>>> Running Test tls_hooks8:F Failed
>>> Running Test tls_hooks9:F Failed
>>> ...
>>> Test: tls_hooks: Failed
>>>    File: tls_hooks.test.py
>>>    Directory: /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks
>>>   Starting Test tls_hooks : No issues found - Passed
>>>      Reason: Started!
>>>   Process: ts: Failed
>>> ...
>>>     Setting up : RunCommand - Failed
>>>        Reason: Failed to run command 'tsxs -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.so' because:
>>>          Actual exit code '1' did not match expected value '0'
>>>     Test : Checking that ReturnCode == None - Passed
>>>        Reason: Returned Value: None == None
>>>     Stream stderr : Checking that StdErrFile matches gold/ts-preaccept-1.gold - Failed
>>>        Reason: File differences
>>>         Gold File : /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks/gold/ts-preaccept-1.gold
>>>         Data File : /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.stderr.txt
>>>         - `` DIAG: (ssl_hook_test) Setup callbacks pa=1 sni=0 cert=0 cert_imm=0 pa_delay=0
>>>         - `` DIAG: (ssl_hook_test) Pre accept callback 0 `` - event is good
>>>         - ``
>>>     Stream All : Pre accept message appears only once or twice - Failed
>>>        Reason: Contents of /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.all.txt did not contains expression: "\A(?:(?!Pre accept callback 0).)*Pre accept callback 0.*(Pre accept callback 0)?(?!.*Pre accept callback 0).*\Z"
>>>     file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>>>        Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log'
>>>     file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>>>        Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log’
> 
> 
> 
> 
> 
> 
> 


Re: autests broken?

Posted by "Alan M. Carroll" <am...@network-geographics.com>.
Susan fixed this in PR 2291.

> You should be able to see the attempted compile output in one of the streams, such as .../_tmp_tls_hooks_tls_hooks_ts/streamm.all.txt, which should provide an indication of the reason for the failure. I know there have been some problems involving the flags stored in tsxs which you could be hitting. Does TS actually compile for you on Fedora26? There's an outstanding issue about failures in that regard (see #2266).

>> Hi all,

>> I am unable to get autest tests to run succesfully on my Fedora26 box. I’m running the ci/regression script, which builds and runs all the tests. This doesn’t fail in jenkins so there must be some non-obvious thing wrong here.

>> AFAICT tsxs fails for some reason that is not captured by the test suite anywhere that I can find. Once that fails everything cascades.

>> Running Test tls_hooks:F Failed
>> Running Test tls_hooks10:F Failed
>> Running Test tls_hooks11:F Failed
>> Running Test tls_hooks12:F Failed
>> Running Test tls_hooks2:F Failed
>> Running Test tls_hooks3:F Failed
>> Running Test tls_hooks4:F Failed
>> Running Test tls_hooks6:F Failed
>> Running Test tls_hooks7:F Failed
>> Running Test tls_hooks8:F Failed
>> Running Test tls_hooks9:F Failed
>> ...
>>  Test: tls_hooks: Failed
>>     File: tls_hooks.test.py
>>     Directory: /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks
>>    Starting Test tls_hooks : No issues found - Passed
>>       Reason: Started!
>>    Process: ts: Failed
>> ...
>>      Setting up : RunCommand - Failed
>>         Reason: Failed to run command 'tsxs -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.so' because:
>>           Actual exit code '1' did not match expected value '0'
>>      Test : Checking that ReturnCode == None - Passed
>>         Reason: Returned Value: None == None
>>      Stream stderr : Checking that StdErrFile matches gold/ts-preaccept-1.gold - Failed
>>         Reason: File differences
>>          Gold File : /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks/gold/ts-preaccept-1.gold
>>          Data File : /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.stderr.txt
>>          - `` DIAG: (ssl_hook_test) Setup callbacks pa=1 sni=0 cert=0 cert_imm=0 pa_delay=0
>>          - `` DIAG: (ssl_hook_test) Pre accept callback 0 `` - event is good
>>          - ``
>>      Stream All : Pre accept message appears only once or twice - Failed
>>         Reason: Contents of /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.all.txt did not contains expression: "\A(?:(?!Pre accept callback 0).)*Pre accept callback 0.*(Pre accept callback 0)?(?!.*Pre accept callback 0).*\Z"
>>      file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>>         Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log'
>>      file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>>         Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log’








Re: autests broken?

Posted by "Alan M. Carroll" <am...@network-geographics.com>.
You should be able to see the attempted compile output in one of the streams, such as .../_tmp_tls_hooks_tls_hooks_ts/streamm.all.txt, which should provide an indication of the reason for the failure. I know there have been some problems involving the flags stored in tsxs which you could be hitting. Does TS actually compile for you on Fedora26? There's an outstanding issue about failures in that regard (see #2266).

> Hi all,

> I am unable to get autest tests to run succesfully on my Fedora26 box. I’m running the ci/regression script, which builds and runs all the tests. This doesn’t fail in jenkins so there must be some non-obvious thing wrong here.

> AFAICT tsxs fails for some reason that is not captured by the test suite anywhere that I can find. Once that fails everything cascades.

> Running Test tls_hooks:F Failed
> Running Test tls_hooks10:F Failed
> Running Test tls_hooks11:F Failed
> Running Test tls_hooks12:F Failed
> Running Test tls_hooks2:F Failed
> Running Test tls_hooks3:F Failed
> Running Test tls_hooks4:F Failed
> Running Test tls_hooks6:F Failed
> Running Test tls_hooks7:F Failed
> Running Test tls_hooks8:F Failed
> Running Test tls_hooks9:F Failed
> ...
>  Test: tls_hooks: Failed
>     File: tls_hooks.test.py
>     Directory: /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks
>    Starting Test tls_hooks : No issues found - Passed
>       Reason: Started!
>    Process: ts: Failed
> ...
>      Setting up : RunCommand - Failed
>         Reason: Failed to run command 'tsxs -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.so' because:
>           Actual exit code '1' did not match expected value '0'
>      Test : Checking that ReturnCode == None - Passed
>         Reason: Returned Value: None == None
>      Stream stderr : Checking that StdErrFile matches gold/ts-preaccept-1.gold - Failed
>         Reason: File differences
>          Gold File : /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks/gold/ts-preaccept-1.gold
>          Data File : /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.stderr.txt
>          - `` DIAG: (ssl_hook_test) Setup callbacks pa=1 sni=0 cert=0 cert_imm=0 pa_delay=0
>          - `` DIAG: (ssl_hook_test) Pre accept callback 0 `` - event is good
>          - ``
>      Stream All : Pre accept message appears only once or twice - Failed
>         Reason: Contents of /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.all.txt did not contains expression: "\A(?:(?!Pre accept callback 0).)*Pre accept callback 0.*(Pre accept callback 0)?(?!.*Pre accept callback 0).*\Z"
>      file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>         Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log'
>      file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
>         Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log’






Re: autests broken?

Posted by Jason Kenny <jk...@yahoo-inc.com.INVALID>.
No AuTest is most likely not broken. It is showing something else is wrong.
Jason

On Sunday, July 23, 2017, 1:59:32 PM CDT, James Peach <jp...@apache.org> wrote:

Hi all,

I am unable to get autest tests to run succesfully on my Fedora26 box. I’m running the ci/regression script, which builds and runs all the tests. This doesn’t fail in jenkins so there must be some non-obvious thing wrong here.

AFAICT tsxs fails for some reason that is not captured by the test suite anywhere that I can find. Once that fails everything cascades.

Running Test tls_hooks:F Failed
Running Test tls_hooks10:F Failed
Running Test tls_hooks11:F Failed
Running Test tls_hooks12:F Failed
Running Test tls_hooks2:F Failed
Running Test tls_hooks3:F Failed
Running Test tls_hooks4:F Failed
Running Test tls_hooks6:F Failed
Running Test tls_hooks7:F Failed
Running Test tls_hooks8:F Failed
Running Test tls_hooks9:F Failed
...
Test: tls_hooks: Failed
    File: tls_hooks.test.py
    Directory: /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks
  Starting Test tls_hooks : No issues found - Passed
      Reason: Started!
  Process: ts: Failed
...
    Setting up : RunCommand - Failed
        Reason: Failed to run command 'tsxs -c /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.cc -o /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/plugin/ssl_hook_test.so' because:
          Actual exit code '1' did not match expected value '0'
    Test : Checking that ReturnCode == None - Passed
        Reason: Returned Value: None == None
    Stream stderr : Checking that StdErrFile matches gold/ts-preaccept-1.gold - Failed
        Reason: File differences
        Gold File : /home/jpeach/src/trafficserver/tests/gold_tests/tls_hooks/gold/ts-preaccept-1.gold
        Data File : /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.stderr.txt
        - `` DIAG: (ssl_hook_test) Setup callbacks pa=1 sni=0 cert=0 cert_imm=0 pa_delay=0
        - `` DIAG: (ssl_hook_test) Pre accept callback 0 `` - event is good
        - ``
    Stream All : Pre accept message appears only once or twice - Failed
        Reason: Contents of /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/_tmp_tls_hooks_tls_hooks_ts/stream.all.txt did not contains expression: "\A(?:(?!Pre accept callback 0).)*Pre accept callback 0.*(Pre accept callback 0)?(?!.*Pre accept callback 0).*\Z"
    file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
        Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log'
    file /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log : diags.log should not contain errors - Failed
        Reason: Cannot read /home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log: [Errno 2] No such file or directory: '/home/jpeach/src/trafficserver/tests/_sandbox/tls_hooks/ts/log/diags.log’