You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by "Matthias Friedrich (JIRA)" <ji...@apache.org> on 2012/07/22 09:38:36 UTC
[jira] [Updated] (CRUNCH-24) Make test suite suitable for
continuous integration
[ https://issues.apache.org/jira/browse/CRUNCH-24?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Friedrich updated CRUNCH-24:
-------------------------------------
Attachment: 0001-CRUNCH-24-Clean-up-test-suite-output.patch
First installment which fixes all test suite output. Next is setting crunch.tmp.dir to a temporary directory that is cleaned automatically, but I can't summon the strength to fix the rest right now. Turnaround times are horrible but there's hope (additional RAM is in the mail) :)
> Make test suite suitable for continuous integration
> ---------------------------------------------------
>
> Key: CRUNCH-24
> URL: https://issues.apache.org/jira/browse/CRUNCH-24
> Project: Crunch
> Issue Type: Task
> Affects Versions: 0.3.0
> Reporter: Matthias Friedrich
> Assignee: Matthias Friedrich
> Fix For: 0.3.0
>
> Attachments: 0001-CRUNCH-24-Clean-up-test-suite-output.patch
>
>
> Right now the integration test suite leaves about 80 files behind on /tmp making it unsuitable for a shared continuous integration environment. Examples for these files are test case output ("output*") and Crunch's own temporary files ("crunch*", see CRUNCH-21).
> We have to delete these files or make sure they aren't created on /tmp in the first place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Updated] (CRUNCH-24) Make test suite suitable for continuous
integration
Posted by Rahul <rs...@xebia.com>.
Hi Matthias,
+1 for the second idea. I was thinking of creating a method there that
can set the required property in Configuration.
Then using TemporaryPath as @Rule component.
- Rahul
On 22-07-2012 16:35, Matthias Friedrich wrote:
> Hi Rahul,
>
> that would be really great :)
>
> My idea of solving this was to use the new self-cleaning TemporaryPath
> JUnit @Rule I added in the first patch and set "crunch.tmp.dir" to its
> root directory. Basically, this has to be done with each instantiation
> of MRPipeline (see MRPipelineTest for an example), so perhaps you can
> move something to a global utility method. Another idea would be to
> add a createConfig() method to TemporaryPath that sets "crunch.tmp.dir"
> for you.
>
> I tried assigning the issue to you but my only choices are myself and
> "Automatic".
>
> Thanks,
> Matthias
>
> On Sunday, 2012-07-22, Rahul wrote:
>> I can fix the issue of setting up a crunch.temp.dir in integration tests
>>
>> On 22-07-2012 13:08, Matthias Friedrich (JIRA) wrote:
>>> [ https://issues.apache.org/jira/browse/CRUNCH-24?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>>>
>>> Matthias Friedrich updated CRUNCH-24:
>>> -------------------------------------
>>>
>>> Attachment: 0001-CRUNCH-24-Clean-up-test-suite-output.patch
>>>
>>> First installment which fixes all test suite output. Next is setting crunch.tmp.dir to a temporary directory that is cleaned automatically, but I can't summon the strength to fix the rest right now. Turnaround times are horrible but there's hope (additional RAM is in the mail) :)
>>>> Make test suite suitable for continuous integration
>>>> ---------------------------------------------------
>>>>
>>>> Key: CRUNCH-24
>>>> URL: https://issues.apache.org/jira/browse/CRUNCH-24
>>>> Project: Crunch
>>>> Issue Type: Task
>>>> Affects Versions: 0.3.0
>>>> Reporter: Matthias Friedrich
>>>> Assignee: Matthias Friedrich
>>>> Fix For: 0.3.0
>>>>
>>>> Attachments: 0001-CRUNCH-24-Clean-up-test-suite-output.patch
>>>>
>>>>
>>>> Right now the integration test suite leaves about 80 files behind on /tmp making it unsuitable for a shared continuous integration environment. Examples for these files are test case output ("output*") and Crunch's own temporary files ("crunch*", see CRUNCH-21).
>>>> We have to delete these files or make sure they aren't created on /tmp in the first place.
>>> --
>>> This message is automatically generated by JIRA.
>>> If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
>>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>>
Re: [jira] [Updated] (CRUNCH-24) Make test suite suitable for
continuous integration
Posted by Matthias Friedrich <ma...@mafr.de>.
Hi Rahul,
that would be really great :)
My idea of solving this was to use the new self-cleaning TemporaryPath
JUnit @Rule I added in the first patch and set "crunch.tmp.dir" to its
root directory. Basically, this has to be done with each instantiation
of MRPipeline (see MRPipelineTest for an example), so perhaps you can
move something to a global utility method. Another idea would be to
add a createConfig() method to TemporaryPath that sets "crunch.tmp.dir"
for you.
I tried assigning the issue to you but my only choices are myself and
"Automatic".
Thanks,
Matthias
On Sunday, 2012-07-22, Rahul wrote:
> I can fix the issue of setting up a crunch.temp.dir in integration tests
>
> On 22-07-2012 13:08, Matthias Friedrich (JIRA) wrote:
> > [ https://issues.apache.org/jira/browse/CRUNCH-24?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
> >
> >Matthias Friedrich updated CRUNCH-24:
> >-------------------------------------
> >
> > Attachment: 0001-CRUNCH-24-Clean-up-test-suite-output.patch
> >
> >First installment which fixes all test suite output. Next is setting crunch.tmp.dir to a temporary directory that is cleaned automatically, but I can't summon the strength to fix the rest right now. Turnaround times are horrible but there's hope (additional RAM is in the mail) :)
> >>Make test suite suitable for continuous integration
> >>---------------------------------------------------
> >>
> >> Key: CRUNCH-24
> >> URL: https://issues.apache.org/jira/browse/CRUNCH-24
> >> Project: Crunch
> >> Issue Type: Task
> >> Affects Versions: 0.3.0
> >> Reporter: Matthias Friedrich
> >> Assignee: Matthias Friedrich
> >> Fix For: 0.3.0
> >>
> >> Attachments: 0001-CRUNCH-24-Clean-up-test-suite-output.patch
> >>
> >>
> >>Right now the integration test suite leaves about 80 files behind on /tmp making it unsuitable for a shared continuous integration environment. Examples for these files are test case output ("output*") and Crunch's own temporary files ("crunch*", see CRUNCH-21).
> >>We have to delete these files or make sure they aren't created on /tmp in the first place.
> >--
> >This message is automatically generated by JIRA.
> >If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> >For more information on JIRA, see: http://www.atlassian.com/software/jira
> >
>
Re: [jira] [Updated] (CRUNCH-24) Make test suite suitable for continuous
integration
Posted by Rahul <rs...@xebia.com>.
I can fix the issue of setting up a crunch.temp.dir in integration tests
On 22-07-2012 13:08, Matthias Friedrich (JIRA) wrote:
> [ https://issues.apache.org/jira/browse/CRUNCH-24?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Matthias Friedrich updated CRUNCH-24:
> -------------------------------------
>
> Attachment: 0001-CRUNCH-24-Clean-up-test-suite-output.patch
>
> First installment which fixes all test suite output. Next is setting crunch.tmp.dir to a temporary directory that is cleaned automatically, but I can't summon the strength to fix the rest right now. Turnaround times are horrible but there's hope (additional RAM is in the mail) :)
>
>> Make test suite suitable for continuous integration
>> ---------------------------------------------------
>>
>> Key: CRUNCH-24
>> URL: https://issues.apache.org/jira/browse/CRUNCH-24
>> Project: Crunch
>> Issue Type: Task
>> Affects Versions: 0.3.0
>> Reporter: Matthias Friedrich
>> Assignee: Matthias Friedrich
>> Fix For: 0.3.0
>>
>> Attachments: 0001-CRUNCH-24-Clean-up-test-suite-output.patch
>>
>>
>> Right now the integration test suite leaves about 80 files behind on /tmp making it unsuitable for a shared continuous integration environment. Examples for these files are test case output ("output*") and Crunch's own temporary files ("crunch*", see CRUNCH-21).
>> We have to delete these files or make sure they aren't created on /tmp in the first place.
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>