You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@bigtop.apache.org by Jay Vyas <ja...@gmail.com> on 2013/09/05 22:54:17 UTC
Proposal: Stress tests submodule
Hi bigtop:
Are there any filesystem stress tests? I.e
-running 100 small mapreduce jobs
-recursively deleting and recreating a directory multiple times from different clients, concurrently.
-attempting to streaming io into the same file concurrently from different tasks
I don't believe so - as most smokes/ are mapreduce oriented which is very structured.
Proposal: a stress/ submodule under test-artifacts which does some of these iterative metadata intensive operations in filesystem agnostic way.
Re: Proposal: Stress tests submodule
Posted by Jay Vyas <ja...@gmail.com>.
Thanks roman ! https://issues.apache.org/jira/browse/BIGTOP-1065 <--- ive
moved it to this jira thread.
On Sat, Sep 7, 2013 at 10:15 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Thu, Sep 5, 2013 at 1:54 PM, Jay Vyas <ja...@gmail.com> wrote:
> > Hi bigtop:
> >
> > Are there any filesystem stress tests? I.e
> > -running 100 small mapreduce jobs
> >
> > -recursively deleting and recreating a directory multiple times from
> different clients, concurrently.
> >
> > -attempting to streaming io into the same file concurrently from
> different tasks
> >
> > I don't believe so - as most smokes/ are mapreduce oriented which is
> very structured.
> >
> > Proposal: a stress/ submodule under test-artifacts which does some of
> these iterative
> > metadata intensive operations in filesystem agnostic way.
>
> +1 to the idea. Would love to see this in Bigtop. As usual -- filing
> a JIRA would be the best way to get this started.
>
> Thanks,
> Roman.
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Proposal: Stress tests submodule
Posted by Roman Shaposhnik <rv...@apache.org>.
On Thu, Sep 5, 2013 at 1:54 PM, Jay Vyas <ja...@gmail.com> wrote:
> Hi bigtop:
>
> Are there any filesystem stress tests? I.e
> -running 100 small mapreduce jobs
>
> -recursively deleting and recreating a directory multiple times from different clients, concurrently.
>
> -attempting to streaming io into the same file concurrently from different tasks
>
> I don't believe so - as most smokes/ are mapreduce oriented which is very structured.
>
> Proposal: a stress/ submodule under test-artifacts which does some of these iterative
> metadata intensive operations in filesystem agnostic way.
+1 to the idea. Would love to see this in Bigtop. As usual -- filing
a JIRA would be the best way to get this started.
Thanks,
Roman.