You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mxnet.apache.org by sandeep krishnamurthy <sa...@gmail.com> on 2019/04/29 20:56:54 UTC
[Proposal] MXNet operator benchmark library
Hello Community,
I am currently working on building a utility/library to help us easily do
individual operator benchmarking in MXNet. I have documented the proposal in
this cwiki
<https://cwiki.apache.org/confluence/display/MXNET/MXNet+Operator+Benchmarks>,
and staging the current development in this github repository
<https://github.com/sandeep-krishnamurthy/dl-operator-benchmark>. Proposal
is to get this library under incubator-mxnet/benchmark/
<https://github.com/apache/incubator-mxnet/tree/master/benchmark>. Please
do review and provide your feedback and suggestions.
Thanks to fellow MXNet community members - Lin, Sam, Rohit for providing
initial ideas and suggestion.
Best,
Sandeep
--
Sandeep Krishnamurthy
Re: [Proposal] MXNet operator benchmark library
Posted by Marco de Abreu <ma...@gmail.com>.
Great proposal!
sandeep krishnamurthy <sa...@gmail.com> schrieb am Di., 14. Mai
2019, 04:45:
> Hi Naveen,
>
> Thanks for your feedback and suggestions. I have updated the document
> addressing the feedback and concerns, alternate solutions pros/cons are
> added.
> https://cwiki.apache.org/confluence/display/MXNET/MXNet+Operator+Benchmarks
>
> Best,
> Sandeep
>
> On Mon, May 13, 2019 at 1:20 PM Naveen Swamy <mn...@gmail.com> wrote:
>
> > Sandeep,
> >
> > Thanks for initiating work on individual operator performance. However I
> > find the proposed approach(ie., a separate libary/framework) to
> unnecessary
> > and increases maintenance overhead for the project.
> > Also, have you considered alternate approaches to achieve the same goal.?
> >
> > Many of the requirements/motivations you have mentioned typically should
> be
> > covered in unit-tests(different data-types/ different dimensions), so
> > instead of having to rewrite for all operators measuring performance,
> > consider writing a @timeit routine(using Python decorators) which can be
> > called on individual unit tests. Also even if you call the performance
> > script from Python, typically you want to measure as close to the kernel
> as
> > possible and avoid any other variables.
> >
> > I left some comments on the doc itself.
> >
> > Happy to discuss further.
> >
> > -Naveen
> >
> >
> > On Mon, Apr 29, 2019 at 1:57 PM sandeep krishnamurthy <
> > sandeep.krishna98@gmail.com> wrote:
> >
> > > Hello Community,
> > >
> > > I am currently working on building a utility/library to help us easily
> do
> > > individual operator benchmarking in MXNet. I have documented the
> proposal
> > > in
> > > this cwiki
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/MXNET/MXNet+Operator+Benchmarks
> > > >,
> > > and staging the current development in this github repository
> > > <https://github.com/sandeep-krishnamurthy/dl-operator-benchmark>.
> > Proposal
> > > is to get this library under incubator-mxnet/benchmark/
> > > <https://github.com/apache/incubator-mxnet/tree/master/benchmark>.
> > Please
> > > do review and provide your feedback and suggestions.
> > >
> > > Thanks to fellow MXNet community members - Lin, Sam, Rohit for
> providing
> > > initial ideas and suggestion.
> > >
> > > Best,
> > > Sandeep
> > >
> > >
> > >
> > >
> > > --
> > > Sandeep Krishnamurthy
> > >
> >
>
>
> --
> Sandeep Krishnamurthy
>
Re: [Proposal] MXNet operator benchmark library
Posted by sandeep krishnamurthy <sa...@gmail.com>.
Hi Naveen,
Thanks for your feedback and suggestions. I have updated the document
addressing the feedback and concerns, alternate solutions pros/cons are
added.
https://cwiki.apache.org/confluence/display/MXNET/MXNet+Operator+Benchmarks
Best,
Sandeep
On Mon, May 13, 2019 at 1:20 PM Naveen Swamy <mn...@gmail.com> wrote:
> Sandeep,
>
> Thanks for initiating work on individual operator performance. However I
> find the proposed approach(ie., a separate libary/framework) to unnecessary
> and increases maintenance overhead for the project.
> Also, have you considered alternate approaches to achieve the same goal.?
>
> Many of the requirements/motivations you have mentioned typically should be
> covered in unit-tests(different data-types/ different dimensions), so
> instead of having to rewrite for all operators measuring performance,
> consider writing a @timeit routine(using Python decorators) which can be
> called on individual unit tests. Also even if you call the performance
> script from Python, typically you want to measure as close to the kernel as
> possible and avoid any other variables.
>
> I left some comments on the doc itself.
>
> Happy to discuss further.
>
> -Naveen
>
>
> On Mon, Apr 29, 2019 at 1:57 PM sandeep krishnamurthy <
> sandeep.krishna98@gmail.com> wrote:
>
> > Hello Community,
> >
> > I am currently working on building a utility/library to help us easily do
> > individual operator benchmarking in MXNet. I have documented the proposal
> > in
> > this cwiki
> > <
> >
> https://cwiki.apache.org/confluence/display/MXNET/MXNet+Operator+Benchmarks
> > >,
> > and staging the current development in this github repository
> > <https://github.com/sandeep-krishnamurthy/dl-operator-benchmark>.
> Proposal
> > is to get this library under incubator-mxnet/benchmark/
> > <https://github.com/apache/incubator-mxnet/tree/master/benchmark>.
> Please
> > do review and provide your feedback and suggestions.
> >
> > Thanks to fellow MXNet community members - Lin, Sam, Rohit for providing
> > initial ideas and suggestion.
> >
> > Best,
> > Sandeep
> >
> >
> >
> >
> > --
> > Sandeep Krishnamurthy
> >
>
--
Sandeep Krishnamurthy
Re: [Proposal] MXNet operator benchmark library
Posted by Naveen Swamy <mn...@gmail.com>.
Sandeep,
Thanks for initiating work on individual operator performance. However I
find the proposed approach(ie., a separate libary/framework) to unnecessary
and increases maintenance overhead for the project.
Also, have you considered alternate approaches to achieve the same goal.?
Many of the requirements/motivations you have mentioned typically should be
covered in unit-tests(different data-types/ different dimensions), so
instead of having to rewrite for all operators measuring performance,
consider writing a @timeit routine(using Python decorators) which can be
called on individual unit tests. Also even if you call the performance
script from Python, typically you want to measure as close to the kernel as
possible and avoid any other variables.
I left some comments on the doc itself.
Happy to discuss further.
-Naveen
On Mon, Apr 29, 2019 at 1:57 PM sandeep krishnamurthy <
sandeep.krishna98@gmail.com> wrote:
> Hello Community,
>
> I am currently working on building a utility/library to help us easily do
> individual operator benchmarking in MXNet. I have documented the proposal
> in
> this cwiki
> <
> https://cwiki.apache.org/confluence/display/MXNET/MXNet+Operator+Benchmarks
> >,
> and staging the current development in this github repository
> <https://github.com/sandeep-krishnamurthy/dl-operator-benchmark>. Proposal
> is to get this library under incubator-mxnet/benchmark/
> <https://github.com/apache/incubator-mxnet/tree/master/benchmark>. Please
> do review and provide your feedback and suggestions.
>
> Thanks to fellow MXNet community members - Lin, Sam, Rohit for providing
> initial ideas and suggestion.
>
> Best,
> Sandeep
>
>
>
>
> --
> Sandeep Krishnamurthy
>