Notes from W3C Testing Meeting Yesterday

[Filip Maj <fi...@adobe.com>]

Hey all,

I attended a W3C meeting at the Mozilla office in Mountain View yesterday.
It was a "Testing Stakeholders" meeting for members, fairly insightful to
see the current testing efforts underway at the W3C.

Meeting agenda (you need to be a W3C member to access):
https://www.w3.org/Member/wiki/Testing#Meeting_agenda

Meeting minutes (should be public):
http://www.w3.org/2013/01/29-webtest-minutes.html

http://www.w3.org/2013/01/30-webtest-minutes.html


My notes, for those interested:

Main problems listed:

1. User Agents using testing frameworks different from W3C's
(testharness.js [1])
2. User Agents each having a different, private set of tests which likely
overlap quite a bit with a) other UAs test base and b) W3C's test base
3. A lot of the W3C specifications have little to no tests at all

With these problems in mind, the following actions will be undertaken:

- A Testing Framework Task Force was created to investigate and help fix
problem #1 above. Timeline of 4 weeks. Tobie Langel spearheading effort.
I volunteered on behalf of the Apache Cordova community to help with this
as well.
- A couple UAs (Mozilla, Opera) volunteered to take a look at problem #2
with respect to their own test bases. Also a timeline of 4 weeks.
- Prioritization of the various W3C specifications was done by the group
to determine which specs would be looked at first to ensure best coverage.
Once #1 and #2 above are tackled and there is a better understanding of
the efforts needed, the third problem can be tackled (likely will need to
reconvene the meeting).

So, how does this affect us? I envision that down the road, once the test
suites for the various HTML5 (and other) specifications are settled, we
could consume them to help identify any "gaps" in our shims/apis. In this
context we are a User Agent vendor.

Additionally, the first two efforts (agreeing on a standard test
framework, and pulling out UA test suites into the open) affect us as
well. I will be keeping an eye out to see how viable integrating
mobile-spec tests into the W3C is. As well, any topics brought to my
attention regarding the test framework I will bring back to the dev list
here so that everyone on the list can get some form of input. Agreeing on
the test framework may be challenging but will help with getting the
various entities on the same page with regards to how to author tests as
well as help organizing which tests to author.

More to come as soon as the e-mails start pouring in.


[1] http://www.w3.org/2008/webapps/wiki/Harness