You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openoffice.apache.org by Raphael Bircher <r....@gmx.ch> on 2011/11/16 08:21:52 UTC
Re: Anybody interested in TestTool? I wrote a Java lib to replace
it.
Hi Liu
Am 16.11.11 03:29, schrieb Zhe Liu:
>
> Hi all,
>
> Vcl TestTool is used by GUI automated testing for OpenOffice.org, but it
> has many drawbacks. Too many errors, difficult to debug, maintain and
> execute parallelly. I found many people mentioned this problem. Libreoffice
> has abondoned the tool. But I think GUI automated testing still is
> valuable. It can test the product more like the actual users.
The problems with the VCLTestTool are well known. But to write a new one
is not trivial too. It needs a load of work to bring a new tool to the
productiv use. Well, we can maybe use the old testscripts as guide for
new testscripts. But the VCLTestTool as is works fine, and you can find
errors realy quick. Mainly not working dialoges, crashs and freese.
VCLTestTool is not so buggy as many people say. But yes, it needs
aditional work to bring it to a productive tool for the community.
> I has been working for long time on automated testing in Symphony project.
> I wrote a Java lib which can connect to the automation (automation module
> in source code) server in openoffice process and do GUI automated testing.
> The lib can be integrated with JUnit or other Java testing framework.
> Anybody interested in the lib? I can contribute it to community.
> Suggestion is welcome.
Yes, please contribut it. We can only compare your jave lib with the
VCLTestTool if we have the lib. It most not be finishd, but we can only
discouss her, if we see the things.
Well, I'm more interested in core development, but I'm willing to bring
my QA experience in to the community. So I'm still interested in
automated testing. And Yes, I also think that automated tests are
important in a project like Apache OpenOffice.org.
> PS, I am a newbie here. :-)
You are welcome
Greetings Raphael
--
My private Homepage: http://www.raphaelbircher.ch/
Re: Anybody interested in TestTool? I wrote a Java lib to replace
it.
Posted by Jürgen Schmidt <jo...@googlemail.com>.
Hi Liu,
On 11/16/11 9:38 AM, Andre Fischer wrote:
> Hi Liu,
>
> On 16.11.2011 09:16, Zhe Liu wrote:
>> Thank Raphael Bircher for the comment.
>> I used VCLTesttool several times and found that it's difficult to debug
>> and understand what's being done by scripts. Maybe because I don't know
>> Basic and the tool enough well. :-). I admit that VCLTestTool and these
>> large amounts of test scripts are very valuable and worth reusing.
>> Anyway, a chance to implement GUI testing using Java may attract more
>> people to contribute test scripts.
>
> It would certainly attract me. When working for Sun/Oracle I had the,
> ahm, pleasure to work with the VCL Testtool a couple of times (usually
> the colleagues from QA shielded us developers from it, but some bugs
> required us using the test tool directly.)
>
> Understanding the Basic code was never easy. The differences between C++
> and Basic are just too big. So a Java solution would be very welcome.
and even better is if somebody knows the code and is willing to maintain
it. I would really prefer to work on a new solution than investigating
in the old testtool that has many disadvantages. We can hopefully over
time build some knowledge base around this new test tool that help
others to get started with using it as well writing new tests.
I am looking forward to get more information about your library and how
we can use it for AOO.
Juergen
>
> Just my 2 cents.
>
> Regards,
> Andre
>
>> The lib has been used to test Lotus
>> Symphony for long time. I will contribute it ASAP after my company
>> approve it.
>>
>>
>> Inactive hide details for Raphael Bircher ---11/16/2011 03:22:28 PM---Hi
>> LiuRaphael Bircher ---11/16/2011 03:22:28 PM---Hi Liu
>>
>>
>> From:
>> Raphael Bircher <r....@gmx.ch>
>>
>> To:
>> ooo-dev@incubator.apache.org
>>
>> Date:
>> 11/16/2011 03:22 PM
>>
>> Subject:
>> Re: Anybody interested in TestTool? I wrote a Java lib to replace it.
>>
>> ------------------------------------------------------------------------
>>
>>
>>
>> Hi Liu
>>
>> Am 16.11.11 03:29, schrieb Zhe Liu:
>> >
>> > Hi all,
>> >
>> > Vcl TestTool is used by GUI automated testing for OpenOffice.org,
>> but it
>> > has many drawbacks. Too many errors, difficult to debug, maintain and
>> > execute parallelly. I found many people mentioned this problem.
>> Libreoffice
>> > has abondoned the tool. But I think GUI automated testing still is
>> > valuable. It can test the product more like the actual users.
>> The problems with the VCLTestTool are well known. But to write a new one
>> is not trivial too. It needs a load of work to bring a new tool to the
>> productiv use. Well, we can maybe use the old testscripts as guide for
>> new testscripts. But the VCLTestTool as is works fine, and you can find
>> errors realy quick. Mainly not working dialoges, crashs and freese.
>> VCLTestTool is not so buggy as many people say. But yes, it needs
>> aditional work to bring it to a productive tool for the community.
>> > I has been working for long time on automated testing in Symphony
>> project.
>> > I wrote a Java lib which can connect to the automation (automation
>> module
>> > in source code) server in openoffice process and do GUI automated
>> testing.
>> > The lib can be integrated with JUnit or other Java testing framework.
>> > Anybody interested in the lib? I can contribute it to community.
>> > Suggestion is welcome.
>> Yes, please contribut it. We can only compare your jave lib with the
>> VCLTestTool if we have the lib. It most not be finishd, but we can only
>> discouss her, if we see the things.
>>
>> Well, I'm more interested in core development, but I'm willing to bring
>> my QA experience in to the community. So I'm still interested in
>> automated testing. And Yes, I also think that automated tests are
>> important in a project like Apache OpenOffice.org.
>> > PS, I am a newbie here. :-)
>> You are welcome
>>
>>
>> Greetings Raphael
>>
>>
>> --
>> My private Homepage: http://www.raphaelbircher.ch/
>>
>>
>>
Re: Anybody interested in TestTool? I wrote a Java lib to replace
it.
Posted by Andre Fischer <af...@a-w-f.de>.
Hi Liu,
On 16.11.2011 09:16, Zhe Liu wrote:
> Thank Raphael Bircher for the comment.
> I used VCLTesttool several times and found that it's difficult to debug
> and understand what's being done by scripts. Maybe because I don't know
> Basic and the tool enough well. :-). I admit that VCLTestTool and these
> large amounts of test scripts are very valuable and worth reusing.
> Anyway, a chance to implement GUI testing using Java may attract more
> people to contribute test scripts.
It would certainly attract me. When working for Sun/Oracle I had the,
ahm, pleasure to work with the VCL Testtool a couple of times (usually
the colleagues from QA shielded us developers from it, but some bugs
required us using the test tool directly.)
Understanding the Basic code was never easy. The differences between
C++ and Basic are just too big. So a Java solution would be very welcome.
Just my 2 cents.
Regards,
Andre
> The lib has been used to test Lotus
> Symphony for long time. I will contribute it ASAP after my company
> approve it.
>
>
> Inactive hide details for Raphael Bircher ---11/16/2011 03:22:28 PM---Hi
> LiuRaphael Bircher ---11/16/2011 03:22:28 PM---Hi Liu
>
>
> From:
> Raphael Bircher <r....@gmx.ch>
>
> To:
> ooo-dev@incubator.apache.org
>
> Date:
> 11/16/2011 03:22 PM
>
> Subject:
> Re: Anybody interested in TestTool? I wrote a Java lib to replace it.
>
> ------------------------------------------------------------------------
>
>
>
> Hi Liu
>
> Am 16.11.11 03:29, schrieb Zhe Liu:
> >
> > Hi all,
> >
> > Vcl TestTool is used by GUI automated testing for OpenOffice.org, but it
> > has many drawbacks. Too many errors, difficult to debug, maintain and
> > execute parallelly. I found many people mentioned this problem.
> Libreoffice
> > has abondoned the tool. But I think GUI automated testing still is
> > valuable. It can test the product more like the actual users.
> The problems with the VCLTestTool are well known. But to write a new one
> is not trivial too. It needs a load of work to bring a new tool to the
> productiv use. Well, we can maybe use the old testscripts as guide for
> new testscripts. But the VCLTestTool as is works fine, and you can find
> errors realy quick. Mainly not working dialoges, crashs and freese.
> VCLTestTool is not so buggy as many people say. But yes, it needs
> aditional work to bring it to a productive tool for the community.
> > I has been working for long time on automated testing in Symphony
> project.
> > I wrote a Java lib which can connect to the automation (automation module
> > in source code) server in openoffice process and do GUI automated
> testing.
> > The lib can be integrated with JUnit or other Java testing framework.
> > Anybody interested in the lib? I can contribute it to community.
> > Suggestion is welcome.
> Yes, please contribut it. We can only compare your jave lib with the
> VCLTestTool if we have the lib. It most not be finishd, but we can only
> discouss her, if we see the things.
>
> Well, I'm more interested in core development, but I'm willing to bring
> my QA experience in to the community. So I'm still interested in
> automated testing. And Yes, I also think that automated tests are
> important in a project like Apache OpenOffice.org.
> > PS, I am a newbie here. :-)
> You are welcome
>
>
> Greetings Raphael
>
>
> --
> My private Homepage: http://www.raphaelbircher.ch/
>
>
>
Re: Anybody interested in TestTool? I wrote a Java lib to replace it.
Posted by Zhe Liu <li...@cn.ibm.com>.
Thank Raphael Bircher for the comment.
I used VCLTesttool several times and found that it's difficult to debug and
understand what's being done by scripts. Maybe because I don't know Basic
and the tool enough well. :-). I admit that VCLTestTool and these large
amounts of test scripts are very valuable and worth reusing. Anyway, a
chance to implement GUI testing using Java may attract more people to
contribute test scripts. The lib has been used to test Lotus Symphony for
long time. I will contribute it ASAP after my company approve it.
|------------>
| From: |
|------------>
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|Raphael Bircher <r....@gmx.ch> |
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| To: |
|------------>
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|ooo-dev@incubator.apache.org |
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| Date: |
|------------>
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|11/16/2011 03:22 PM |
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| Subject: |
|------------>
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|Re: Anybody interested in TestTool? I wrote a Java lib to replace it. |
>--------------------------------------------------------------------------------------------------------------------------------------------------|
Hi Liu
Am 16.11.11 03:29, schrieb Zhe Liu:
>
> Hi all,
>
> Vcl TestTool is used by GUI automated testing for OpenOffice.org, but it
> has many drawbacks. Too many errors, difficult to debug, maintain and
> execute parallelly. I found many people mentioned this problem.
Libreoffice
> has abondoned the tool. But I think GUI automated testing still is
> valuable. It can test the product more like the actual users.
The problems with the VCLTestTool are well known. But to write a new one
is not trivial too. It needs a load of work to bring a new tool to the
productiv use. Well, we can maybe use the old testscripts as guide for
new testscripts. But the VCLTestTool as is works fine, and you can find
errors realy quick. Mainly not working dialoges, crashs and freese.
VCLTestTool is not so buggy as many people say. But yes, it needs
aditional work to bring it to a productive tool for the community.
> I has been working for long time on automated testing in Symphony
project.
> I wrote a Java lib which can connect to the automation (automation module
> in source code) server in openoffice process and do GUI automated
testing.
> The lib can be integrated with JUnit or other Java testing framework.
> Anybody interested in the lib? I can contribute it to community.
> Suggestion is welcome.
Yes, please contribut it. We can only compare your jave lib with the
VCLTestTool if we have the lib. It most not be finishd, but we can only
discouss her, if we see the things.
Well, I'm more interested in core development, but I'm willing to bring
my QA experience in to the community. So I'm still interested in
automated testing. And Yes, I also think that automated tests are
important in a project like Apache OpenOffice.org.
> PS, I am a newbie here. :-)
You are welcome
Greetings Raphael
--
My private Homepage: http://www.raphaelbircher.ch/
Re: Anybody interested in TestTool? I wrote a Java lib to replace
it.
Posted by Mathias Bauer <Ma...@gmx.net>.
Am 18.11.2011 13:45, schrieb Joost Andrae:
> Hi,
>
> I prefer to have tests from several tiers, firstly having a UI test
> framework that is written and performed by testers who are not that
> experienced in writing tests in whatever programming language (BASIC is
> not a must have but it needs to be as easy as LOGO turle graphics)
> mostly to assure the quality of what has yet been tested and secondly
> having API based tests to stress test the applications' programming
> interface and thirdly having unit tests that are instanciated directly
> from within the C++ or JAVA source code to detect common coding bugs
> and at last to perform manual tests because UI based programs often show
> bugs when a real person uses the application (like we found several bugs
> in the past that would have never been found if we used the automation
> based tests only).
The language is of second order. The quality of the tool is what
matters. Here the vcl testtool leaves a lot to desire. And, as I wrote,
we can't improve it, fix bugs or add code for new requirements (e.g.
parallel test execution to utilize multi core computers, removal of
frustrating waits etc.) because we don't have the source code.
>> For me the VCLTesttool is dead. We shouldn't invest time there if we had
>> an alternative GUI testing option with available source code. So we
>> definitely should give the Java lib a try, if it could be made available
>> under a suitable license.
>
> For me VCL testtool is soumething from the past but it is still needed
> until we have a replacement.
I beg to differ. The test tool is for finding bugs in the UI code, for
everything else it's just a waste of time. As long as we don't do heavy
UI rewrites writing integration tests is far more effective.
The offer from Zhe Liu looks much more appealing than trying to revive
the vcl testtool zombie.
Regards,
Mathias
Re: Anybody interested in TestTool? I wrote a Java lib to replace
it.
Posted by Joost Andrae <Jo...@gmx.de>.
Hi,
I prefer to have tests from several tiers, firstly having a UI test
framework that is written and performed by testers who are not that
experienced in writing tests in whatever programming language (BASIC is
not a must have but it needs to be as easy as LOGO turle graphics)
mostly to assure the quality of what has yet been tested and secondly
having API based tests to stress test the applications' programming
interface and thirdly having unit tests that are instanciated directly
from within the C++ or JAVA source code to detect common coding bugs
and at last to perform manual tests because UI based programs often show
bugs when a real person uses the application (like we found several bugs
in the past that would have never been found if we used the automation
based tests only).
>
> For me the VCLTesttool is dead. We shouldn't invest time there if we had
> an alternative GUI testing option with available source code. So we
> definitely should give the Java lib a try, if it could be made available
> under a suitable license.
For me VCL testtool is soumething from the past but it is still needed
until we have a replacement.
>
> Until then we should concentrate on writing integration tests in C++ or
> Java. We already have some of them (called "complex tests") and they
> have proven to be the most effective tool for bug hunting that we have.
>
Kind regards, Joost
Re: Anybody interested in TestTool? I wrote a Java lib to replace
it.
Posted by Jürgen Schmidt <jo...@googlemail.com>.
On 11/18/11 12:21 PM, Mathias Bauer wrote:
> On 16.11.2011 08:21, Raphael Bircher wrote:
>> Hi Liu
>>
>> Am 16.11.11 03:29, schrieb Zhe Liu:
>>>
>>> Hi all,
>>>
>>> Vcl TestTool is used by GUI automated testing for OpenOffice.org, but it
>>> has many drawbacks. Too many errors, difficult to debug, maintain and
>>> execute parallelly. I found many people mentioned this problem.
>>> Libreoffice
>>> has abondoned the tool. But I think GUI automated testing still is
>>> valuable. It can test the product more like the actual users.
>> The problems with the VCLTestTool are well known. But to write a new one
>> is not trivial too. It needs a load of work to bring a new tool to the
>> productiv use. Well, we can maybe use the old testscripts as guide for
>> new testscripts. But the VCLTestTool as is works fine, and you can find
>> errors realy quick. Mainly not working dialoges, crashs and freese.
>> VCLTestTool is not so buggy as many people say. But yes, it needs
>> aditional work to bring it to a productive tool for the community.
>
> And here's the problem: to work on the testtool, you need the source
> code. And the source code is not available, at least not here at Apache.
> It is based on a very old version of OOo's source code that was written
> years ago. To build it, you will need large parts of the OOo source code
> in that old revision - no good idea, moreover, it's totally unclear if
> this old revision is covered by Oracle's code grant. At least it would
> require and additional IP review.
>
> Of course some testtool source code also exists in the current release,
> but it is unused and untested since years. I remember that even the
> latest larger code rework in that area (help IDs becoming Strings
> instead of integer numbers) was not done for the testtool, only for the
> testtool library that is loaded in OOo when tests are executed. Instead
> of that larger parts of the testtool source code where just commented
> out to please the compiler ("nobody needs that code, why invest time
> into it"). Sigh.
>
> Besides that, the available Linux version of the testtool does not run
> on at least Ubuntu 64Bit (with 32Bit libs installed), but probably also
> on other Linux versions. Sooner or later it won't run on more an more
> Linux and probably MacOS versions.
>
> For me the VCLTesttool is dead. We shouldn't invest time there if we had
> an alternative GUI testing option with available source code. So we
> definitely should give the Java lib a try, if it could be made available
> under a suitable license.
>
> Until then we should concentrate on writing integration tests in C++ or
> Java. We already have some of them (called "complex tests") and they
> have proven to be the most effective tool for bug hunting that we have.
>
> Some reasons why they are more helpful, at least ATM:
>
> - they run much faster than the UI based tests
> - they can be executed in parallel (as much as you want)
> - the gbuild build system supports them, so they can be automated
> without additional tooling (no "test launcher" necessary)
> - they are nearer to the code they test, so the root cause of a bug
> usually is found faster
> - they don't require to learn a strange Basic dialect, developers can
> write them in the same language that they use daily
> - it's common sense that testing through the UI should be done only if
> you want to test the UI itself, but not the code behind it
>
+1 to all what Mathias has said
We should focus on something new that we have under control.
Juergen
Re: Anybody interested in TestTool? I wrote a Java lib to replace
it.
Posted by Mathias Bauer <Ma...@gmx.net>.
On 16.11.2011 08:21, Raphael Bircher wrote:
> Hi Liu
>
> Am 16.11.11 03:29, schrieb Zhe Liu:
>>
>> Hi all,
>>
>> Vcl TestTool is used by GUI automated testing for OpenOffice.org, but it
>> has many drawbacks. Too many errors, difficult to debug, maintain and
>> execute parallelly. I found many people mentioned this problem.
>> Libreoffice
>> has abondoned the tool. But I think GUI automated testing still is
>> valuable. It can test the product more like the actual users.
> The problems with the VCLTestTool are well known. But to write a new one
> is not trivial too. It needs a load of work to bring a new tool to the
> productiv use. Well, we can maybe use the old testscripts as guide for
> new testscripts. But the VCLTestTool as is works fine, and you can find
> errors realy quick. Mainly not working dialoges, crashs and freese.
> VCLTestTool is not so buggy as many people say. But yes, it needs
> aditional work to bring it to a productive tool for the community.
And here's the problem: to work on the testtool, you need the source
code. And the source code is not available, at least not here at Apache.
It is based on a very old version of OOo's source code that was written
years ago. To build it, you will need large parts of the OOo source code
in that old revision - no good idea, moreover, it's totally unclear if
this old revision is covered by Oracle's code grant. At least it would
require and additional IP review.
Of course some testtool source code also exists in the current release,
but it is unused and untested since years. I remember that even the
latest larger code rework in that area (help IDs becoming Strings
instead of integer numbers) was not done for the testtool, only for the
testtool library that is loaded in OOo when tests are executed. Instead
of that larger parts of the testtool source code where just commented
out to please the compiler ("nobody needs that code, why invest time
into it"). Sigh.
Besides that, the available Linux version of the testtool does not run
on at least Ubuntu 64Bit (with 32Bit libs installed), but probably also
on other Linux versions. Sooner or later it won't run on more an more
Linux and probably MacOS versions.
For me the VCLTesttool is dead. We shouldn't invest time there if we had
an alternative GUI testing option with available source code. So we
definitely should give the Java lib a try, if it could be made available
under a suitable license.
Until then we should concentrate on writing integration tests in C++ or
Java. We already have some of them (called "complex tests") and they
have proven to be the most effective tool for bug hunting that we have.
Some reasons why they are more helpful, at least ATM:
- they run much faster than the UI based tests
- they can be executed in parallel (as much as you want)
- the gbuild build system supports them, so they can be automated
without additional tooling (no "test launcher" necessary)
- they are nearer to the code they test, so the root cause of a bug
usually is found faster
- they don't require to learn a strange Basic dialect, developers can
write them in the same language that they use daily
- it's common sense that testing through the UI should be done only if
you want to test the UI itself, but not the code behind it
Regards,
Mathias