You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ignite.apache.org by Oleg Ostanin <oo...@gridgain.com> on 2016/12/16 08:37:35 UTC

IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Hello Igniters!
I'm working on ticket IGNITE-4212 "Ignite Benchmarking Simplification and
Automation" and I'd like to ask your opinion about ignite-yardstick: where
do you think is the most appropriate place to put a compiled
ignite-yardstick module in the apache-ignite binary assembly? We can put it
in the libs/optional along with an others optional libraries, or we can
create a new directory named "tools" in the root directory and put
"ignite-yardstick" in it, or we can find another solution.

Best regards
Oleg

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Oleg, thanks for picking up this ticket!

I vote for placing the lib under “libs/optional” folder. Please keep in mind that we need to provide a script that will automate launching of benchmarks for users. In my view, the user needs only to define a name of a benchmark as a parameter to the script and script will launch it.

—
Denis

> On Dec 16, 2016, at 12:37 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> 
> Hello Igniters!
> I'm working on ticket IGNITE-4212 "Ignite Benchmarking Simplification and
> Automation" and I'd like to ask your opinion about ignite-yardstick: where
> do you think is the most appropriate place to put a compiled
> ignite-yardstick module in the apache-ignite binary assembly? We can put it
> in the libs/optional along with an others optional libraries, or we can
> create a new directory named "tools" in the root directory and put
> "ignite-yardstick" in it, or we can find another solution.
> 
> Best regards
> Oleg

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Oleg,

Thanks, appreciate this. It simplified the review process for me. I’ve left a valuable feedback in the ticket.
Please address them.
https://issues.apache.org/jira/browse/IGNITE-4212 <https://issues.apache.org/jira/browse/IGNITE-4212>

I’ll keep reviewing as soon as initial issues are fixed.

—
Denis

> On Feb 3, 2017, at 12:49 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> 
> Hi!
> 
> Denis, I've uploaded Ignite release binaries with my changes here:
> 
> https://drive.google.com/file/d/0B9teTzJ9iIvHYTdUMlRsYTdRckk/view?usp=sharing
> 
> On Thu, Feb 2, 2017 at 8:02 PM, Denis Magda <dm...@apache.org> wrote:
> 
>> Oleg, thanks,
>> 
>> Could you share Ignite release binaries with your patch applied? You
>> should be able to use TeamCity for that. Makes sense to see the new project
>> structure and walk through final version of the instructions.
>> 
>> —
>> Denis
>> 
>>> On Feb 2, 2017, at 2:30 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
>>> 
>>> Denis,
>>> 
>>> Yes, the same identical README.txt as in the pull request will be placed
>> in
>>> both benchmarks binaries and Ignite Yardstick module.
>>> 
>>> I've also created two DEVNOTES.txt with related building instructions.
>>> 
>>> On Wed, Feb 1, 2017 at 10:31 PM, Denis Magda <dm...@apache.org> wrote:
>>> 
>>>> Anton,
>>>> 
>>>> I’m absolutely fine to make a single README if its instructions will be
>>>> identical for benchmarks binaries delivered in Ignite binary releases
>> and
>>>> assembled with yardstick module directly. Oleg, is it feasible?
>>>> 
>>>> If to skip auto-generation idea then DEVNOTES split suggested by you is
>>>> the only option.
>>>> 
>>>> —
>>>> Denis
>>>> 
>>>>> On Feb 1, 2017, at 1:45 AM, Anton Vinogradov <avinogradov@gridgain.com
>>> 
>>>> wrote:
>>>>> 
>>>>> Denis,
>>>>> 
>>>>> I don't like autogeneration.
>>>>> 
>>>>> In my view, we have to keep one README and ret rid of "Installation
>>>>> instructions" inside it.
>>>>> README should explain how to work with assembled yardstick. So, same
>>>> README
>>>>> will be uses at sources and inside release assembly.
>>>>> 
>>>>> Also we have to have to split DEVNOTES to DEVNOTES and
>>>> DEVNOTES.standalone,
>>>>> to explain how to assembly yardstick from sources.
>>>>> 
>>>>> Thoughts?
>>>>> 
>>>>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org>
>> wrote:
>>>>> 
>>>>>> Oleg,
>>>>>> 
>>>>>> Thanks for the clarification.
>>>>>> 
>>>>>> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
>>>>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
>>>>>> instructions there that explain how to build and run benchmarks from
>>>>>> ‘modules/yardstick’. This existing files can refer to the sources and
>>>>>> compiled benchmarks that are in Ignite binary releases but this should
>>>> be a
>>>>>> couple of statements, no more.
>>>>>> 
>>>>>> As for the instructions related to the sources and binaries added to
>>>>>> Ignite binaries, preferably they need to be added to auto-generated
>>>>>> README.txt. *Anton*, is it feasible to do?
>>>>>> 
>>>>>> Finally, when apply the reviews notes please build and share Ignite
>>>>>> binaries with your patch. Want to see the new project structure and
>>>> final
>>>>>> version of the instructions. Presently I can’t merge your changes due
>> to
>>>>>> some conflicts.
>>>>>> 
>>>>>> *Anton*, please review modifications in the build procedures.
>>>>>> 
>>>>>> —
>>>>>> Denis
>>>>>> 
>>>>>>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com>
>>>> wrote:
>>>>>>> 
>>>>>>> Hi Denis,
>>>>>>> 
>>>>>>> Yes, we have included Ignite Yardstick source files with its pom.xml
>> in
>>>>>>> Ignite binary release. "Building from standalone sources" is the
>>>> building
>>>>>>> instruction for these source files. "Building from Ignite Sources" is
>>>> the
>>>>>>> instruction for building Ignite Yardstick from `modules/yardstick` in
>>>>>>> Ignite source files.
>>>>>>> 
>>>>>>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org>
>>>> wrote:
>>>>>>> 
>>>>>>>> Hi Oleg,
>>>>>>>> 
>>>>>>>> Great progress, thanks for keep driving this!
>>>>>>>> 
>>>>>>>> I’ve left some minor notes in GitHub’s pull-request. I have the
>>>>>> following
>>>>>>>> questions aside:
>>>>>>>> 
>>>>>>>> - What is the difference between "Building from standalone sources"
>>>> and
>>>>>>>> "Building from Ignite Sources"? In my understanding, a user
>> downloads
>>>>>>>> Apache Ignite release that has all the sources locally.
>>>>>>>> 
>>>>>>>> - I do remember we planned to add the benchmarks sources in a form
>> of
>>>> a
>>>>>>>> ready to be used project with its own pom.xml (similar to examples).
>>>> Did
>>>>>>>> you put this task off?
>>>>>>>> 
>>>>>>>> —
>>>>>>>> Denis
>>>>>>>> 
>>>>>>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Hi!
>>>>>>>>> 
>>>>>>>>> I've changed the README.txt and DEVNOTES.txt files. Also added a
>>>> simple
>>>>>>>>> config file for quick and easy start. Please take a look at them
>> and
>>>>>> tell
>>>>>>>>> me what you think.
>>>>>>>>> 
>>>>>>>>> https://github.com/apache/ignite/pull/1471
>>>>>>>>> 
>>>>>>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <
>> isuntsov@gridgain.com
>>>>> 
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Denis,
>>>>>>>>>> 
>>>>>>>>>> I think we can remove all configs except:
>>>>>>>>>> 
>>>>>>>>>> benchmark-multicast.properties
>>>>>>>>>> 
>>>>>>>>>> benchmark.properties
>>>>>>>>>> 
>>>>>>>>>> ignite-base-config.xml
>>>>>>>>>> 
>>>>>>>>>> ignite-localhost-config.xml
>>>>>>>>>> 
>>>>>>>>>> ignite-multicast-config.xml
>>>>>>>>>> 
>>>>>>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
>>>>>>>>>> 
>>>>>>>>>>> I would have only those configs that are useful. Ilya Suntsov,
>>>> basing
>>>>>>>> on
>>>>>>>>>>> your experience, please suggest which configs makes sense to
>>>> include
>>>>>>>> into
>>>>>>>>>>> every Ignite release.
>>>>>>>>>>> 
>>>>>>>>>>> Oleg, also please note that community decided to include not only
>>>> the
>>>>>>>>>>> benchmarking binaries but the sources as well into every Apache
>>>>>> Ignite
>>>>>>>>>>> release. I’ve update the ticket before. Hope you followed the
>>>>>>>> discussion
>>>>>>>>>> ;)
>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>>>>>>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
>>>>>>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>>>>>>>>>>> 
>>>>>>>>>>> —
>>>>>>>>>>> Denis
>>>>>>>>>>> 
>>>>>>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <
>> oostanin@gridgain.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> I mean removing those configs from binary assembly, not from
>>>>>>>>>> repository.
>>>>>>>>>>>> 
>>>>>>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
>>>>>> oostanin@gridgain.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hello Igniters.
>>>>>>>>>>>>> I think it would be better to remove some configuration files
>>>> from
>>>>>>>>>>>>> benchmarks/config:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ignite-base-load-config.xml
>>>>>>>>>>>>> ignite-cache-load-config.xml
>>>>>>>>>>>>> ignite-failover-base-config.xml
>>>>>>>>>>>>> ignite-failover-localhost-config.xml
>>>>>>>>>>>>> benchmark-cache-load.properties
>>>>>>>>>>>>> benchmark-cache-load-win.properties
>>>>>>>>>>>>> benchmark-failover.properties
>>>>>>>>>>>>> 
>>>>>>>>>>>>> because those configs do not relate to any of performance
>> tests.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <
>> dmagda@apache.org
>>>>> 
>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Summarized the discussion updating the ticket
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> —
>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Sergey,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
>>>>>> folder
>>>>>>>>>>> right
>>>>>>>>>>>>>>> under the root folder.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>>>>>>>>>> skozlov@gridgain.com
>>>>>>>>>>>> 
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Formatting has cut lines:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>>>> — examples
>>>>>>>>>>>>>>>> — extras
>>>>>>>>>>>>>>>> — benchmarks
>>>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>>>> — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>>>> — config
>>>>>>>>>>>>>>>> — libs (compiled benchmarks)
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>>>>>>>>>>> skozlov@gridgain.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Denis,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may include
>>>> more
>>>>>>>>>>>>>> things in
>>>>>>>>>>>>>>>>> future than yardstick only. It's why I suggest something
>> like
>>>>>>>>>> that:
>>>>>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>>>>> — examples
>>>>>>>>>>>>>>>>> — extras
>>>>>>>>>>>>>>>>>  — benchmarks
>>>>>>>>>>>>>>>>>      — bin
>>>>>>>>>>>>>>>>>      — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>>>>>      — config
>>>>>>>>>>>>>>>>>      — libs (compiled benchmarks)
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
>>>>>> dmagda@apache.org
>>>>>>>>> 
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
>>>>>>>> following
>>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>>>>>>> — examples
>>>>>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>>>>>> — benchmarks
>>>>>>>>>>>>>>>>>>  — bin
>>>>>>>>>>>>>>>>>>  — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>>>>>>  — config
>>>>>>>>>>>>>>>>>>  — libs (compiled benchmarks)
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
>>>> previously?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>>>>>>>>>> skozlov@gridgain.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin,
>>>> /config,
>>>>>>>>>>> /libs)
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> creates work/logs directory under yardstick root.
>>>>>>>>>> "libs/optional"
>>>>>>>>>>> is
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> optional modules but in general we can't say that for
>>>>>>>> Yardstick.
>>>>>>>>>>>>>> Also
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> may break the current user understanding of
>> "libs/optional"
>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>> place for additonal functionality activated by copying in
>>>>>>>>>> "libs".
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder
>> for
>>>>>>>>>>> anything
>>>>>>>>>>>>>>>>>>>> benchmark related. I am also against adding any
>> yardstick
>>>>>>>>>>> libraries
>>>>>>>>>>>>>>>>>> without
>>>>>>>>>>>>>>>>>>>> providing code.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> In my view, if the community wants to include benchmarks
>>>> in
>>>>>>>>>>>>>> releases,
>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
>>>>>> everything
>>>>>>>>>>>>>>>>>> benchmark
>>>>>>>>>>>>>>>>>>>> related, from code to all the dependent libraries, and
>>>>>>>>>>>>>> documentation
>>>>>>>>>>>>>>>>>>>> instructions.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra
>> for
>>>>>> me.
>>>>>>>>>> Why
>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
>>>>>> well?
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
>>>>>> “libs/optional”?
>>>>>>>>>> If
>>>>>>>>>>>>>>>>>> there’re
>>>>>>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
>>>>>>>> implementation.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
>>>>>>>>>>> skozlov@gridgain.com
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> What's about to introduce the new root folder called
>>>>>>>> 'extras'
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
>>>>>>>>>> binaries?
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
>>>>>>>>>>> dmagda@apache.org
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Dmitriy,
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
>>>> benchmarks
>>>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now
>> he/she
>>>>>>>> needs
>>>>>>>>>>> to
>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
>>>>>>>>>>> following
>>>>>>>>>>>>>>>>>>>> lengthy
>>>>>>>>>>>>>>>>>>>>>>> Yardstick guidance.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it
>>>> should
>>>>>> be
>>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script.
>>>> The
>>>>>>>>>>> script
>>>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
>>>>>>>>>> “libs/optional”
>>>>>>>>>>>>>>>>>> folder.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
>>>>>>>>>> benchmark
>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> add a
>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
>>>>>> guidance.
>>>>>>>>>>> So,
>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite
>>>> release.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>>>>>>>>>>>>>>>>>> oostanin@gridgain.com
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty
>>>> of
>>>>>>>>>>> useful
>>>>>>>>>>>>>>>>>>>>> Yardstick
>>>>>>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
>>>>>>>> performance.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs"
>> folder
>>>> at
>>>>>>>>>> all?
>>>>>>>>>>>>>> Do
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes,
>> then I
>>>>>>>>>> would
>>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
>>>>>>>>>> benchmarks
>>>>>>>>>>>>>>>>>> there.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy
>> Setrakyan <
>>>>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>>>>>>>>>>>>>>>>>>>> oostanin@gridgain.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
>>>>>> Benchmarking
>>>>>>>>>>>>>>>>>>>>> Simplification
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion
>> about
>>>>>>>>>>>>>>>>>>>> ignite-yardstick:
>>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to
>> put a
>>>>>>>>>>> compiled
>>>>>>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite
>> binary
>>>>>>>>>>>>>> assembly?
>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others
>> optional
>>>>>>>>>>>>>> libraries,
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
>>>>>>>>>> directory
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
>>>>>>>>>> solution.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>>>>>>>>>>>>>>>> Oleg
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Ilya Suntsov
>>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Sergey Kozlov <sk...@gridgain.com>.

But I suppose we should file a new ticket for that task

On Fri, Feb 3, 2017 at 12:35 PM, Sergey Kozlov <sk...@gridgain.com> wrote:

> Yes we can do it as new project but the task should be redesigned at least
> by security reasons (removing the credentials).
>
> On Fri, Feb 3, 2017 at 12:29 PM, Anton Vinogradov <
> avinogradov@gridgain.com> wrote:
>
>> BTW, it will be a good point to relocate Ignite assembly to
>> ci.ignite.apache.org :)
>>
>> On Fri, Feb 3, 2017 at 12:26 PM, Sergey Kozlov <sk...@gridgain.com>
>> wrote:
>>
>> > Hi
>> >
>> > Not sure that we've a release task on ci.ignite.apache.org
>> >
>> > On Fri, Feb 3, 2017 at 12:20 PM, Anton Vinogradov <
>> > avinogradov@gridgain.com>
>> > wrote:
>> >
>> > > Oleg, you can point to TC build artifacts next time.
>> > >
>> > > On Fri, Feb 3, 2017 at 11:49 AM, Oleg Ostanin <oo...@gridgain.com>
>> > > wrote:
>> > >
>> > > > Hi!
>> > > >
>> > > > Denis, I've uploaded Ignite release binaries with my changes here:
>> > > >
>> > > > https://drive.google.com/file/d/0B9teTzJ9iIvHYTdUMlRsYTdRckk/
>> > > > view?usp=sharing
>> > > >
>> > > > On Thu, Feb 2, 2017 at 8:02 PM, Denis Magda <dm...@apache.org>
>> wrote:
>> > > >
>> > > > > Oleg, thanks,
>> > > > >
>> > > > > Could you share Ignite release binaries with your patch applied?
>> You
>> > > > > should be able to use TeamCity for that. Makes sense to see the
>> new
>> > > > project
>> > > > > structure and walk through final version of the instructions.
>> > > > >
>> > > > > —
>> > > > > Denis
>> > > > >
>> > > > > > On Feb 2, 2017, at 2:30 AM, Oleg Ostanin <oostanin@gridgain.com
>> >
>> > > > wrote:
>> > > > > >
>> > > > > > Denis,
>> > > > > >
>> > > > > > Yes, the same identical README.txt as in the pull request will
>> be
>> > > > placed
>> > > > > in
>> > > > > > both benchmarks binaries and Ignite Yardstick module.
>> > > > > >
>> > > > > > I've also created two DEVNOTES.txt with related building
>> > > instructions.
>> > > > > >
>> > > > > > On Wed, Feb 1, 2017 at 10:31 PM, Denis Magda <dmagda@apache.org
>> >
>> > > > wrote:
>> > > > > >
>> > > > > >> Anton,
>> > > > > >>
>> > > > > >> I’m absolutely fine to make a single README if its instructions
>> > will
>> > > > be
>> > > > > >> identical for benchmarks binaries delivered in Ignite binary
>> > > releases
>> > > > > and
>> > > > > >> assembled with yardstick module directly. Oleg, is it feasible?
>> > > > > >>
>> > > > > >> If to skip auto-generation idea then DEVNOTES split suggested
>> by
>> > you
>> > > > is
>> > > > > >> the only option.
>> > > > > >>
>> > > > > >> —
>> > > > > >> Denis
>> > > > > >>
>> > > > > >>> On Feb 1, 2017, at 1:45 AM, Anton Vinogradov <
>> > > > avinogradov@gridgain.com
>> > > > > >
>> > > > > >> wrote:
>> > > > > >>>
>> > > > > >>> Denis,
>> > > > > >>>
>> > > > > >>> I don't like autogeneration.
>> > > > > >>>
>> > > > > >>> In my view, we have to keep one README and ret rid of
>> > "Installation
>> > > > > >>> instructions" inside it.
>> > > > > >>> README should explain how to work with assembled yardstick.
>> So,
>> > > same
>> > > > > >> README
>> > > > > >>> will be uses at sources and inside release assembly.
>> > > > > >>>
>> > > > > >>> Also we have to have to split DEVNOTES to DEVNOTES and
>> > > > > >> DEVNOTES.standalone,
>> > > > > >>> to explain how to assembly yardstick from sources.
>> > > > > >>>
>> > > > > >>> Thoughts?
>> > > > > >>>
>> > > > > >>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <
>> dmagda@apache.org>
>> > > > > wrote:
>> > > > > >>>
>> > > > > >>>> Oleg,
>> > > > > >>>>
>> > > > > >>>> Thanks for the clarification.
>> > > > > >>>>
>> > > > > >>>> My opinion is that we should leave
>> > ‘modules/yardstick/README.txt’
>> > > > and
>> > > > > >>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have
>> only
>> > > > those
>> > > > > >>>> instructions there that explain how to build and run
>> benchmarks
>> > > from
>> > > > > >>>> ‘modules/yardstick’. This existing files can refer to the
>> > sources
>> > > > and
>> > > > > >>>> compiled benchmarks that are in Ignite binary releases but
>> this
>> > > > should
>> > > > > >> be a
>> > > > > >>>> couple of statements, no more.
>> > > > > >>>>
>> > > > > >>>> As for the instructions related to the sources and binaries
>> > added
>> > > to
>> > > > > >>>> Ignite binaries, preferably they need to be added to
>> > > auto-generated
>> > > > > >>>> README.txt. *Anton*, is it feasible to do?
>> > > > > >>>>
>> > > > > >>>> Finally, when apply the reviews notes please build and share
>> > > Ignite
>> > > > > >>>> binaries with your patch. Want to see the new project
>> structure
>> > > and
>> > > > > >> final
>> > > > > >>>> version of the instructions. Presently I can’t merge your
>> > changes
>> > > > due
>> > > > > to
>> > > > > >>>> some conflicts.
>> > > > > >>>>
>> > > > > >>>> *Anton*, please review modifications in the build procedures.
>> > > > > >>>>
>> > > > > >>>> —
>> > > > > >>>> Denis
>> > > > > >>>>
>> > > > > >>>>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <
>> > oostanin@gridgain.com
>> > > >
>> > > > > >> wrote:
>> > > > > >>>>>
>> > > > > >>>>> Hi Denis,
>> > > > > >>>>>
>> > > > > >>>>> Yes, we have included Ignite Yardstick source files with its
>> > > > pom.xml
>> > > > > in
>> > > > > >>>>> Ignite binary release. "Building from standalone sources" is
>> > the
>> > > > > >> building
>> > > > > >>>>> instruction for these source files. "Building from Ignite
>> > > Sources"
>> > > > is
>> > > > > >> the
>> > > > > >>>>> instruction for building Ignite Yardstick from
>> > > `modules/yardstick`
>> > > > in
>> > > > > >>>>> Ignite source files.
>> > > > > >>>>>
>> > > > > >>>>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <
>> > dmagda@apache.org>
>> > > > > >> wrote:
>> > > > > >>>>>
>> > > > > >>>>>> Hi Oleg,
>> > > > > >>>>>>
>> > > > > >>>>>> Great progress, thanks for keep driving this!
>> > > > > >>>>>>
>> > > > > >>>>>> I’ve left some minor notes in GitHub’s pull-request. I have
>> > the
>> > > > > >>>> following
>> > > > > >>>>>> questions aside:
>> > > > > >>>>>>
>> > > > > >>>>>> - What is the difference between "Building from standalone
>> > > > sources"
>> > > > > >> and
>> > > > > >>>>>> "Building from Ignite Sources"? In my understanding, a user
>> > > > > downloads
>> > > > > >>>>>> Apache Ignite release that has all the sources locally.
>> > > > > >>>>>>
>> > > > > >>>>>> - I do remember we planned to add the benchmarks sources
>> in a
>> > > form
>> > > > > of
>> > > > > >> a
>> > > > > >>>>>> ready to be used project with its own pom.xml (similar to
>> > > > examples).
>> > > > > >> Did
>> > > > > >>>>>> you put this task off?
>> > > > > >>>>>>
>> > > > > >>>>>> —
>> > > > > >>>>>> Denis
>> > > > > >>>>>>
>> > > > > >>>>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <
>> > > oostanin@gridgain.com
>> > > > >
>> > > > > >>>> wrote:
>> > > > > >>>>>>>
>> > > > > >>>>>>> Hi!
>> > > > > >>>>>>>
>> > > > > >>>>>>> I've changed the README.txt and DEVNOTES.txt files. Also
>> > added
>> > > a
>> > > > > >> simple
>> > > > > >>>>>>> config file for quick and easy start. Please take a look
>> at
>> > > them
>> > > > > and
>> > > > > >>>> tell
>> > > > > >>>>>>> me what you think.
>> > > > > >>>>>>>
>> > > > > >>>>>>> https://github.com/apache/ignite/pull/1471
>> > > > > >>>>>>>
>> > > > > >>>>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <
>> > > > > isuntsov@gridgain.com
>> > > > > >>>
>> > > > > >>>>>> wrote:
>> > > > > >>>>>>>
>> > > > > >>>>>>>> Denis,
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> I think we can remove all configs except:
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> benchmark-multicast.properties
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> benchmark.properties
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> ignite-base-config.xml
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> ignite-localhost-config.xml
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> ignite-multicast-config.xml
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dmagda@apache.org
>> >:
>> > > > > >>>>>>>>
>> > > > > >>>>>>>>> I would have only those configs that are useful. Ilya
>> > > Suntsov,
>> > > > > >> basing
>> > > > > >>>>>> on
>> > > > > >>>>>>>>> your experience, please suggest which configs makes
>> sense
>> > to
>> > > > > >> include
>> > > > > >>>>>> into
>> > > > > >>>>>>>>> every Ignite release.
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>> Oleg, also please note that community decided to include
>> > not
>> > > > only
>> > > > > >> the
>> > > > > >>>>>>>>> benchmarking binaries but the sources as well into every
>> > > Apache
>> > > > > >>>> Ignite
>> > > > > >>>>>>>>> release. I’ve update the ticket before. Hope you
>> followed
>> > the
>> > > > > >>>>>> discussion
>> > > > > >>>>>>>> ;)
>> > > > > >>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>> > > > > >>>>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
>> > > > > >>>>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-
>> > > 15765151
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>> —
>> > > > > >>>>>>>>> Denis
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <
>> > > > > oostanin@gridgain.com>
>> > > > > >>>>>>>> wrote:
>> > > > > >>>>>>>>>>
>> > > > > >>>>>>>>>> I mean removing those configs from binary assembly, not
>> > from
>> > > > > >>>>>>>> repository.
>> > > > > >>>>>>>>>>
>> > > > > >>>>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
>> > > > > >>>> oostanin@gridgain.com>
>> > > > > >>>>>>>>> wrote:
>> > > > > >>>>>>>>>>
>> > > > > >>>>>>>>>>> Hello Igniters.
>> > > > > >>>>>>>>>>> I think it would be better to remove some
>> configuration
>> > > files
>> > > > > >> from
>> > > > > >>>>>>>>>>> benchmarks/config:
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>>> ignite-base-load-config.xml
>> > > > > >>>>>>>>>>> ignite-cache-load-config.xml
>> > > > > >>>>>>>>>>> ignite-failover-base-config.xml
>> > > > > >>>>>>>>>>> ignite-failover-localhost-config.xml
>> > > > > >>>>>>>>>>> benchmark-cache-load.properties
>> > > > > >>>>>>>>>>> benchmark-cache-load-win.properties
>> > > > > >>>>>>>>>>> benchmark-failover.properties
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>>> because those configs do not relate to any of
>> performance
>> > > > > tests.
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <
>> > > > > dmagda@apache.org
>> > > > > >>>
>> > > > > >>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>>>> Summarized the discussion updating the ticket
>> > > > > >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>> > > > > >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> —
>> > > > > >>>>>>>>>>>> Denis
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>> > > > > >>>>>>>>> dsetrakyan@apache.org>
>> > > > > >>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> Sergey,
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> I am not sure I like "extras". I am voting for
>> > > "benchmarks"
>> > > > > >>>> folder
>> > > > > >>>>>>>>> right
>> > > > > >>>>>>>>>>>>> under the root folder.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> D.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>> > > > > >>>>>>>> skozlov@gridgain.com
>> > > > > >>>>>>>>>>
>> > > > > >>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> Formatting has cut lines:
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> — apache_ignite_root_folder
>> > > > > >>>>>>>>>>>>>> — bin
>> > > > > >>>>>>>>>>>>>> — examples
>> > > > > >>>>>>>>>>>>>> — extras
>> > > > > >>>>>>>>>>>>>> — benchmarks
>> > > > > >>>>>>>>>>>>>> — bin
>> > > > > >>>>>>>>>>>>>> — src (benchmarks sources with pom.xml)
>> > > > > >>>>>>>>>>>>>> — config
>> > > > > >>>>>>>>>>>>>> — libs (compiled benchmarks)
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>> > > > > >>>>>>>>> skozlov@gridgain.com>
>> > > > > >>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> Denis,
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may
>> > > > include
>> > > > > >> more
>> > > > > >>>>>>>>>>>> things in
>> > > > > >>>>>>>>>>>>>>> future than yardstick only. It's why I suggest
>> > > something
>> > > > > like
>> > > > > >>>>>>>> that:
>> > > > > >>>>>>>>>>>>>>> — apache_ignite_root_folder
>> > > > > >>>>>>>>>>>>>>> — bin
>> > > > > >>>>>>>>>>>>>>> — examples
>> > > > > >>>>>>>>>>>>>>> — extras
>> > > > > >>>>>>>>>>>>>>>   — benchmarks
>> > > > > >>>>>>>>>>>>>>>       — bin
>> > > > > >>>>>>>>>>>>>>>       — src (benchmarks sources with pom.xml)
>> > > > > >>>>>>>>>>>>>>>       — config
>> > > > > >>>>>>>>>>>>>>>       — libs (compiled benchmarks)
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
>> > > > > >>>> dmagda@apache.org
>> > > > > >>>>>>>
>> > > > > >>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can
>> have
>> > > the
>> > > > > >>>>>> following
>> > > > > >>>>>>>>>>>>>>>> structure
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> — apache_ignite_root_folder
>> > > > > >>>>>>>>>>>>>>>> — examples
>> > > > > >>>>>>>>>>>>>>>> — bin
>> > > > > >>>>>>>>>>>>>>>> — benchmarks
>> > > > > >>>>>>>>>>>>>>>>   — bin
>> > > > > >>>>>>>>>>>>>>>>   — src (benchmarks sources with pom.xml)
>> > > > > >>>>>>>>>>>>>>>>   — config
>> > > > > >>>>>>>>>>>>>>>>   — libs (compiled benchmarks)
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
>> > > > > >> previously?
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> —
>> > > > > >>>>>>>>>>>>>>>> Denis
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>> > > > > >>>>>>>> skozlov@gridgain.com>
>> > > > > >>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> Yardstick requires own scripts/configurations
>> > (/bin,
>> > > > > >> /config,
>> > > > > >>>>>>>>> /libs)
>> > > > > >>>>>>>>>>>>>> and
>> > > > > >>>>>>>>>>>>>>>>> creates work/logs directory under yardstick
>> root.
>> > > > > >>>>>>>> "libs/optional"
>> > > > > >>>>>>>>> is
>> > > > > >>>>>>>>>>>>>> for
>> > > > > >>>>>>>>>>>>>>>>> optional modules but in general we can't say
>> that
>> > for
>> > > > > >>>>>> Yardstick.
>> > > > > >>>>>>>>>>>> Also
>> > > > > >>>>>>>>>>>>>> it
>> > > > > >>>>>>>>>>>>>>>>> may break the current user understanding of
>> > > > > "libs/optional"
>> > > > > >>>>>>>>>>>> directory
>> > > > > >>>>>>>>>>>>>> as
>> > > > > >>>>>>>>>>>>>>>>> place for additonal functionality activated by
>> > > copying
>> > > > in
>> > > > > >>>>>>>> "libs".
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy
>> Setrakyan
>> > <
>> > > > > >>>>>>>>>>>>>>>> dsetrakyan@apache.org>
>> > > > > >>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> I would be against using libs/optional or libs/
>> > > folder
>> > > > > for
>> > > > > >>>>>>>>> anything
>> > > > > >>>>>>>>>>>>>>>>>> benchmark related. I am also against adding any
>> > > > > yardstick
>> > > > > >>>>>>>>> libraries
>> > > > > >>>>>>>>>>>>>>>> without
>> > > > > >>>>>>>>>>>>>>>>>> providing code.
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> In my view, if the community wants to include
>> > > > benchmarks
>> > > > > >> in
>> > > > > >>>>>>>>>>>> releases,
>> > > > > >>>>>>>>>>>>>>>> then
>> > > > > >>>>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which
>> > provides
>> > > > > >>>> everything
>> > > > > >>>>>>>>>>>>>>>> benchmark
>> > > > > >>>>>>>>>>>>>>>>>> related, from code to all the dependent
>> libraries,
>> > > and
>> > > > > >>>>>>>>>>>> documentation
>> > > > > >>>>>>>>>>>>>>>>>> instructions.
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> D.
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
>> > > > > >>>>>>>> dmagda@apache.org>
>> > > > > >>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of
>> > > extra
>> > > > > for
>> > > > > >>>> me.
>> > > > > >>>>>>>> Why
>> > > > > >>>>>>>>>>>> do
>> > > > > >>>>>>>>>>>>>> we
>> > > > > >>>>>>>>>>>>>>>>>>> need this new folder if “libs/optional”
>> semantic
>> > > > works
>> > > > > >>>> well?
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
>> > > > > >>>> “libs/optional”?
>> > > > > >>>>>>>> If
>> > > > > >>>>>>>>>>>>>>>> there’re
>> > > > > >>>>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
>> > > > > >>>>>> implementation.
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> —
>> > > > > >>>>>>>>>>>>>>>>>>> Denis
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
>> > > > > >>>>>>>>> skozlov@gridgain.com
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> Hi
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> What's about to introduce the new root folder
>> > > called
>> > > > > >>>>>> 'extras'
>> > > > > >>>>>>>>>>>> with
>> > > > > >>>>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there
>> > > yardstick
>> > > > > >>>>>>>> binaries?
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis
>> Magda <
>> > > > > >>>>>>>>> dmagda@apache.org
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> Dmitriy,
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212
>> description (
>> > > > > >>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> > > > > >>>>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to
>> automate
>> > > > > >> benchmarks
>> > > > > >>>>>>>>>>>>>> execution
>> > > > > >>>>>>>>>>>>>>>>>> for
>> > > > > >>>>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release.
>> Now
>> > > > > he/she
>> > > > > >>>>>> needs
>> > > > > >>>>>>>>> to
>> > > > > >>>>>>>>>>>> go
>> > > > > >>>>>>>>>>>>>>>>>>> through
>> > > > > >>>>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run
>> > > > strictly
>> > > > > >>>>>>>>> following
>> > > > > >>>>>>>>>>>>>>>>>> lengthy
>> > > > > >>>>>>>>>>>>>>>>>>>>> Yardstick guidance.
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> Ideally, once a specific release is
>> downloaded
>> > it
>> > > > > >> should
>> > > > > >>>> be
>> > > > > >>>>>>>>>>>>>> possible
>> > > > > >>>>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use
>> > > > script.
>> > > > > >> The
>> > > > > >>>>>>>>> script
>> > > > > >>>>>>>>>>>>>>>> needs
>> > > > > >>>>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put
>> under
>> > > > > >>>>>>>> “libs/optional”
>> > > > > >>>>>>>>>>>>>>>> folder.
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an
>> > > existed
>> > > > > >>>>>>>> benchmark
>> > > > > >>>>>>>>> or
>> > > > > >>>>>>>>>>>>>>>> add a
>> > > > > >>>>>>>>>>>>>>>>>>> new
>> > > > > >>>>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed
>> > Yardstick
>> > > > > >>>> guidance.
>> > > > > >>>>>>>>> So,
>> > > > > >>>>>>>>>>>> no
>> > > > > >>>>>>>>>>>>>>>>>> need
>> > > > > >>>>>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of
>> > Ignite
>> > > > > >> release.
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> —
>> > > > > >>>>>>>>>>>>>>>>>>>>> Denis
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy
>> > Setrakyan <
>> > > > > >>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
>> > > > > >>>>>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg
>> Ostanin
>> > <
>> > > > > >>>>>>>>>>>>>>>> oostanin@gridgain.com
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to
>> run
>> > > > plenty
>> > > > > >> of
>> > > > > >>>>>>>>> useful
>> > > > > >>>>>>>>>>>>>>>>>>> Yardstick
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check
>> Ignite
>> > > > > >>>>>> performance.
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> In that case, why would it be under the
>> "libs"
>> > > > > folder
>> > > > > >> at
>> > > > > >>>>>>>> all?
>> > > > > >>>>>>>>>>>> Do
>> > > > > >>>>>>>>>>>>>> we
>> > > > > >>>>>>>>>>>>>>>>>>>>> really
>> > > > > >>>>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If
>> > yes,
>> > > > > then I
>> > > > > >>>>>>>> would
>> > > > > >>>>>>>>>>>>>>>> create a
>> > > > > >>>>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add
>> all
>> > > the
>> > > > > >>>>>>>> benchmarks
>> > > > > >>>>>>>>>>>>>>>> there.
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy
>> > > > > Setrakyan <
>> > > > > >>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module
>> do?
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg
>> > > Ostanin <
>> > > > > >>>>>>>>>>>>>>>>>>> oostanin@gridgain.com>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212
>> "Ignite
>> > > > > >>>> Benchmarking
>> > > > > >>>>>>>>>>>>>>>>>>> Simplification
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> and
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your
>> > opinion
>> > > > > about
>> > > > > >>>>>>>>>>>>>>>>>> ignite-yardstick:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> where
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate
>> place
>> > to
>> > > > > put a
>> > > > > >>>>>>>>> compiled
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the
>> > apache-ignite
>> > > > > binary
>> > > > > >>>>>>>>>>>> assembly?
>> > > > > >>>>>>>>>>>>>> We
>> > > > > >>>>>>>>>>>>>>>>>> can
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> put
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> it
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an
>> others
>> > > > > optional
>> > > > > >>>>>>>>>>>> libraries,
>> > > > > >>>>>>>>>>>>>> or
>> > > > > >>>>>>>>>>>>>>>>>> we
>> > > > > >>>>>>>>>>>>>>>>>>>>> can
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in
>> the
>> > > > root
>> > > > > >>>>>>>> directory
>> > > > > >>>>>>>>>>>> and
>> > > > > >>>>>>>>>>>>>>>> put
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find
>> > > > another
>> > > > > >>>>>>>> solution.
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Best regards
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Oleg
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> --
>> > > > > >>>>>>>>>>>>>>>>>>>> Sergey Kozlov
>> > > > > >>>>>>>>>>>>>>>>>>>> GridGain Systems
>> > > > > >>>>>>>>>>>>>>>>>>>> www.gridgain.com
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> --
>> > > > > >>>>>>>>>>>>>>>>> Sergey Kozlov
>> > > > > >>>>>>>>>>>>>>>>> GridGain Systems
>> > > > > >>>>>>>>>>>>>>>>> www.gridgain.com
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> --
>> > > > > >>>>>>>>>>>>>>> Sergey Kozlov
>> > > > > >>>>>>>>>>>>>>> GridGain Systems
>> > > > > >>>>>>>>>>>>>>> www.gridgain.com
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> --
>> > > > > >>>>>>>>>>>>>> Sergey Kozlov
>> > > > > >>>>>>>>>>>>>> GridGain Systems
>> > > > > >>>>>>>>>>>>>> www.gridgain.com
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> --
>> > > > > >>>>>>>> Ilya Suntsov
>> > > > > >>>>>>>>
>> > > > > >>>>>>
>> > > > > >>>>>>
>> > > > > >>>>
>> > > > > >>>>
>> > > > > >>
>> > > > > >>
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Sergey Kozlov
>> > GridGain Systems
>> > www.gridgain.com
>> >
>>
>
>
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Sergey Kozlov <sk...@gridgain.com>.

Yes we can do it as new project but the task should be redesigned at least
by security reasons (removing the credentials).

On Fri, Feb 3, 2017 at 12:29 PM, Anton Vinogradov <av...@gridgain.com>
wrote:

> BTW, it will be a good point to relocate Ignite assembly to
> ci.ignite.apache.org :)
>
> On Fri, Feb 3, 2017 at 12:26 PM, Sergey Kozlov <sk...@gridgain.com>
> wrote:
>
> > Hi
> >
> > Not sure that we've a release task on ci.ignite.apache.org
> >
> > On Fri, Feb 3, 2017 at 12:20 PM, Anton Vinogradov <
> > avinogradov@gridgain.com>
> > wrote:
> >
> > > Oleg, you can point to TC build artifacts next time.
> > >
> > > On Fri, Feb 3, 2017 at 11:49 AM, Oleg Ostanin <oo...@gridgain.com>
> > > wrote:
> > >
> > > > Hi!
> > > >
> > > > Denis, I've uploaded Ignite release binaries with my changes here:
> > > >
> > > > https://drive.google.com/file/d/0B9teTzJ9iIvHYTdUMlRsYTdRckk/
> > > > view?usp=sharing
> > > >
> > > > On Thu, Feb 2, 2017 at 8:02 PM, Denis Magda <dm...@apache.org>
> wrote:
> > > >
> > > > > Oleg, thanks,
> > > > >
> > > > > Could you share Ignite release binaries with your patch applied?
> You
> > > > > should be able to use TeamCity for that. Makes sense to see the new
> > > > project
> > > > > structure and walk through final version of the instructions.
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > > > On Feb 2, 2017, at 2:30 AM, Oleg Ostanin <oo...@gridgain.com>
> > > > wrote:
> > > > > >
> > > > > > Denis,
> > > > > >
> > > > > > Yes, the same identical README.txt as in the pull request will be
> > > > placed
> > > > > in
> > > > > > both benchmarks binaries and Ignite Yardstick module.
> > > > > >
> > > > > > I've also created two DEVNOTES.txt with related building
> > > instructions.
> > > > > >
> > > > > > On Wed, Feb 1, 2017 at 10:31 PM, Denis Magda <dm...@apache.org>
> > > > wrote:
> > > > > >
> > > > > >> Anton,
> > > > > >>
> > > > > >> I’m absolutely fine to make a single README if its instructions
> > will
> > > > be
> > > > > >> identical for benchmarks binaries delivered in Ignite binary
> > > releases
> > > > > and
> > > > > >> assembled with yardstick module directly. Oleg, is it feasible?
> > > > > >>
> > > > > >> If to skip auto-generation idea then DEVNOTES split suggested by
> > you
> > > > is
> > > > > >> the only option.
> > > > > >>
> > > > > >> —
> > > > > >> Denis
> > > > > >>
> > > > > >>> On Feb 1, 2017, at 1:45 AM, Anton Vinogradov <
> > > > avinogradov@gridgain.com
> > > > > >
> > > > > >> wrote:
> > > > > >>>
> > > > > >>> Denis,
> > > > > >>>
> > > > > >>> I don't like autogeneration.
> > > > > >>>
> > > > > >>> In my view, we have to keep one README and ret rid of
> > "Installation
> > > > > >>> instructions" inside it.
> > > > > >>> README should explain how to work with assembled yardstick. So,
> > > same
> > > > > >> README
> > > > > >>> will be uses at sources and inside release assembly.
> > > > > >>>
> > > > > >>> Also we have to have to split DEVNOTES to DEVNOTES and
> > > > > >> DEVNOTES.standalone,
> > > > > >>> to explain how to assembly yardstick from sources.
> > > > > >>>
> > > > > >>> Thoughts?
> > > > > >>>
> > > > > >>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <
> dmagda@apache.org>
> > > > > wrote:
> > > > > >>>
> > > > > >>>> Oleg,
> > > > > >>>>
> > > > > >>>> Thanks for the clarification.
> > > > > >>>>
> > > > > >>>> My opinion is that we should leave
> > ‘modules/yardstick/README.txt’
> > > > and
> > > > > >>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have
> only
> > > > those
> > > > > >>>> instructions there that explain how to build and run
> benchmarks
> > > from
> > > > > >>>> ‘modules/yardstick’. This existing files can refer to the
> > sources
> > > > and
> > > > > >>>> compiled benchmarks that are in Ignite binary releases but
> this
> > > > should
> > > > > >> be a
> > > > > >>>> couple of statements, no more.
> > > > > >>>>
> > > > > >>>> As for the instructions related to the sources and binaries
> > added
> > > to
> > > > > >>>> Ignite binaries, preferably they need to be added to
> > > auto-generated
> > > > > >>>> README.txt. *Anton*, is it feasible to do?
> > > > > >>>>
> > > > > >>>> Finally, when apply the reviews notes please build and share
> > > Ignite
> > > > > >>>> binaries with your patch. Want to see the new project
> structure
> > > and
> > > > > >> final
> > > > > >>>> version of the instructions. Presently I can’t merge your
> > changes
> > > > due
> > > > > to
> > > > > >>>> some conflicts.
> > > > > >>>>
> > > > > >>>> *Anton*, please review modifications in the build procedures.
> > > > > >>>>
> > > > > >>>> —
> > > > > >>>> Denis
> > > > > >>>>
> > > > > >>>>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <
> > oostanin@gridgain.com
> > > >
> > > > > >> wrote:
> > > > > >>>>>
> > > > > >>>>> Hi Denis,
> > > > > >>>>>
> > > > > >>>>> Yes, we have included Ignite Yardstick source files with its
> > > > pom.xml
> > > > > in
> > > > > >>>>> Ignite binary release. "Building from standalone sources" is
> > the
> > > > > >> building
> > > > > >>>>> instruction for these source files. "Building from Ignite
> > > Sources"
> > > > is
> > > > > >> the
> > > > > >>>>> instruction for building Ignite Yardstick from
> > > `modules/yardstick`
> > > > in
> > > > > >>>>> Ignite source files.
> > > > > >>>>>
> > > > > >>>>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <
> > dmagda@apache.org>
> > > > > >> wrote:
> > > > > >>>>>
> > > > > >>>>>> Hi Oleg,
> > > > > >>>>>>
> > > > > >>>>>> Great progress, thanks for keep driving this!
> > > > > >>>>>>
> > > > > >>>>>> I’ve left some minor notes in GitHub’s pull-request. I have
> > the
> > > > > >>>> following
> > > > > >>>>>> questions aside:
> > > > > >>>>>>
> > > > > >>>>>> - What is the difference between "Building from standalone
> > > > sources"
> > > > > >> and
> > > > > >>>>>> "Building from Ignite Sources"? In my understanding, a user
> > > > > downloads
> > > > > >>>>>> Apache Ignite release that has all the sources locally.
> > > > > >>>>>>
> > > > > >>>>>> - I do remember we planned to add the benchmarks sources in
> a
> > > form
> > > > > of
> > > > > >> a
> > > > > >>>>>> ready to be used project with its own pom.xml (similar to
> > > > examples).
> > > > > >> Did
> > > > > >>>>>> you put this task off?
> > > > > >>>>>>
> > > > > >>>>>> —
> > > > > >>>>>> Denis
> > > > > >>>>>>
> > > > > >>>>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <
> > > oostanin@gridgain.com
> > > > >
> > > > > >>>> wrote:
> > > > > >>>>>>>
> > > > > >>>>>>> Hi!
> > > > > >>>>>>>
> > > > > >>>>>>> I've changed the README.txt and DEVNOTES.txt files. Also
> > added
> > > a
> > > > > >> simple
> > > > > >>>>>>> config file for quick and easy start. Please take a look at
> > > them
> > > > > and
> > > > > >>>> tell
> > > > > >>>>>>> me what you think.
> > > > > >>>>>>>
> > > > > >>>>>>> https://github.com/apache/ignite/pull/1471
> > > > > >>>>>>>
> > > > > >>>>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <
> > > > > isuntsov@gridgain.com
> > > > > >>>
> > > > > >>>>>> wrote:
> > > > > >>>>>>>
> > > > > >>>>>>>> Denis,
> > > > > >>>>>>>>
> > > > > >>>>>>>> I think we can remove all configs except:
> > > > > >>>>>>>>
> > > > > >>>>>>>> benchmark-multicast.properties
> > > > > >>>>>>>>
> > > > > >>>>>>>> benchmark.properties
> > > > > >>>>>>>>
> > > > > >>>>>>>> ignite-base-config.xml
> > > > > >>>>>>>>
> > > > > >>>>>>>> ignite-localhost-config.xml
> > > > > >>>>>>>>
> > > > > >>>>>>>> ignite-multicast-config.xml
> > > > > >>>>>>>>
> > > > > >>>>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dmagda@apache.org
> >:
> > > > > >>>>>>>>
> > > > > >>>>>>>>> I would have only those configs that are useful. Ilya
> > > Suntsov,
> > > > > >> basing
> > > > > >>>>>> on
> > > > > >>>>>>>>> your experience, please suggest which configs makes sense
> > to
> > > > > >> include
> > > > > >>>>>> into
> > > > > >>>>>>>>> every Ignite release.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Oleg, also please note that community decided to include
> > not
> > > > only
> > > > > >> the
> > > > > >>>>>>>>> benchmarking binaries but the sources as well into every
> > > Apache
> > > > > >>>> Ignite
> > > > > >>>>>>>>> release. I’ve update the ticket before. Hope you followed
> > the
> > > > > >>>>>> discussion
> > > > > >>>>>>>> ;)
> > > > > >>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
> > > > > >>>>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
> > > > > >>>>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-
> > > 15765151
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> —
> > > > > >>>>>>>>> Denis
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <
> > > > > oostanin@gridgain.com>
> > > > > >>>>>>>> wrote:
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> I mean removing those configs from binary assembly, not
> > from
> > > > > >>>>>>>> repository.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
> > > > > >>>> oostanin@gridgain.com>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>> Hello Igniters.
> > > > > >>>>>>>>>>> I think it would be better to remove some configuration
> > > files
> > > > > >> from
> > > > > >>>>>>>>>>> benchmarks/config:
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> ignite-base-load-config.xml
> > > > > >>>>>>>>>>> ignite-cache-load-config.xml
> > > > > >>>>>>>>>>> ignite-failover-base-config.xml
> > > > > >>>>>>>>>>> ignite-failover-localhost-config.xml
> > > > > >>>>>>>>>>> benchmark-cache-load.properties
> > > > > >>>>>>>>>>> benchmark-cache-load-win.properties
> > > > > >>>>>>>>>>> benchmark-failover.properties
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> because those configs do not relate to any of
> performance
> > > > > tests.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <
> > > > > dmagda@apache.org
> > > > > >>>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>> Summarized the discussion updating the ticket
> > > > > >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> > > > > >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> —
> > > > > >>>>>>>>>>>> Denis
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> > > > > >>>>>>>>> dsetrakyan@apache.org>
> > > > > >>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> Sergey,
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> I am not sure I like "extras". I am voting for
> > > "benchmarks"
> > > > > >>>> folder
> > > > > >>>>>>>>> right
> > > > > >>>>>>>>>>>>> under the root folder.
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> D.
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
> > > > > >>>>>>>> skozlov@gridgain.com
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> Formatting has cut lines:
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> — apache_ignite_root_folder
> > > > > >>>>>>>>>>>>>> — bin
> > > > > >>>>>>>>>>>>>> — examples
> > > > > >>>>>>>>>>>>>> — extras
> > > > > >>>>>>>>>>>>>> — benchmarks
> > > > > >>>>>>>>>>>>>> — bin
> > > > > >>>>>>>>>>>>>> — src (benchmarks sources with pom.xml)
> > > > > >>>>>>>>>>>>>> — config
> > > > > >>>>>>>>>>>>>> — libs (compiled benchmarks)
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> > > > > >>>>>>>>> skozlov@gridgain.com>
> > > > > >>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> Denis,
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may
> > > > include
> > > > > >> more
> > > > > >>>>>>>>>>>> things in
> > > > > >>>>>>>>>>>>>>> future than yardstick only. It's why I suggest
> > > something
> > > > > like
> > > > > >>>>>>>> that:
> > > > > >>>>>>>>>>>>>>> — apache_ignite_root_folder
> > > > > >>>>>>>>>>>>>>> — bin
> > > > > >>>>>>>>>>>>>>> — examples
> > > > > >>>>>>>>>>>>>>> — extras
> > > > > >>>>>>>>>>>>>>>   — benchmarks
> > > > > >>>>>>>>>>>>>>>       — bin
> > > > > >>>>>>>>>>>>>>>       — src (benchmarks sources with pom.xml)
> > > > > >>>>>>>>>>>>>>>       — config
> > > > > >>>>>>>>>>>>>>>       — libs (compiled benchmarks)
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
> > > > > >>>> dmagda@apache.org
> > > > > >>>>>>>
> > > > > >>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can
> have
> > > the
> > > > > >>>>>> following
> > > > > >>>>>>>>>>>>>>>> structure
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> — apache_ignite_root_folder
> > > > > >>>>>>>>>>>>>>>> — examples
> > > > > >>>>>>>>>>>>>>>> — bin
> > > > > >>>>>>>>>>>>>>>> — benchmarks
> > > > > >>>>>>>>>>>>>>>>   — bin
> > > > > >>>>>>>>>>>>>>>>   — src (benchmarks sources with pom.xml)
> > > > > >>>>>>>>>>>>>>>>   — config
> > > > > >>>>>>>>>>>>>>>>   — libs (compiled benchmarks)
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
> > > > > >> previously?
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> —
> > > > > >>>>>>>>>>>>>>>> Denis
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
> > > > > >>>>>>>> skozlov@gridgain.com>
> > > > > >>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> Yardstick requires own scripts/configurations
> > (/bin,
> > > > > >> /config,
> > > > > >>>>>>>>> /libs)
> > > > > >>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>> creates work/logs directory under yardstick root.
> > > > > >>>>>>>> "libs/optional"
> > > > > >>>>>>>>> is
> > > > > >>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>> optional modules but in general we can't say that
> > for
> > > > > >>>>>> Yardstick.
> > > > > >>>>>>>>>>>> Also
> > > > > >>>>>>>>>>>>>> it
> > > > > >>>>>>>>>>>>>>>>> may break the current user understanding of
> > > > > "libs/optional"
> > > > > >>>>>>>>>>>> directory
> > > > > >>>>>>>>>>>>>> as
> > > > > >>>>>>>>>>>>>>>>> place for additonal functionality activated by
> > > copying
> > > > in
> > > > > >>>>>>>> "libs".
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy
> Setrakyan
> > <
> > > > > >>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> > > > > >>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> I would be against using libs/optional or libs/
> > > folder
> > > > > for
> > > > > >>>>>>>>> anything
> > > > > >>>>>>>>>>>>>>>>>> benchmark related. I am also against adding any
> > > > > yardstick
> > > > > >>>>>>>>> libraries
> > > > > >>>>>>>>>>>>>>>> without
> > > > > >>>>>>>>>>>>>>>>>> providing code.
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> In my view, if the community wants to include
> > > > benchmarks
> > > > > >> in
> > > > > >>>>>>>>>>>> releases,
> > > > > >>>>>>>>>>>>>>>> then
> > > > > >>>>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which
> > provides
> > > > > >>>> everything
> > > > > >>>>>>>>>>>>>>>> benchmark
> > > > > >>>>>>>>>>>>>>>>>> related, from code to all the dependent
> libraries,
> > > and
> > > > > >>>>>>>>>>>> documentation
> > > > > >>>>>>>>>>>>>>>>>> instructions.
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> D.
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
> > > > > >>>>>>>> dmagda@apache.org>
> > > > > >>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of
> > > extra
> > > > > for
> > > > > >>>> me.
> > > > > >>>>>>>> Why
> > > > > >>>>>>>>>>>> do
> > > > > >>>>>>>>>>>>>> we
> > > > > >>>>>>>>>>>>>>>>>>> need this new folder if “libs/optional”
> semantic
> > > > works
> > > > > >>>> well?
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
> > > > > >>>> “libs/optional”?
> > > > > >>>>>>>> If
> > > > > >>>>>>>>>>>>>>>> there’re
> > > > > >>>>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
> > > > > >>>>>> implementation.
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>> —
> > > > > >>>>>>>>>>>>>>>>>>> Denis
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> > > > > >>>>>>>>> skozlov@gridgain.com
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> Hi
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> What's about to introduce the new root folder
> > > called
> > > > > >>>>>> 'extras'
> > > > > >>>>>>>>>>>> with
> > > > > >>>>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there
> > > yardstick
> > > > > >>>>>>>> binaries?
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda
> <
> > > > > >>>>>>>>> dmagda@apache.org
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> Dmitriy,
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212
> description (
> > > > > >>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> > > > > >>>>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to
> automate
> > > > > >> benchmarks
> > > > > >>>>>>>>>>>>>> execution
> > > > > >>>>>>>>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release.
> Now
> > > > > he/she
> > > > > >>>>>> needs
> > > > > >>>>>>>>> to
> > > > > >>>>>>>>>>>> go
> > > > > >>>>>>>>>>>>>>>>>>> through
> > > > > >>>>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run
> > > > strictly
> > > > > >>>>>>>>> following
> > > > > >>>>>>>>>>>>>>>>>> lengthy
> > > > > >>>>>>>>>>>>>>>>>>>>> Yardstick guidance.
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> Ideally, once a specific release is
> downloaded
> > it
> > > > > >> should
> > > > > >>>> be
> > > > > >>>>>>>>>>>>>> possible
> > > > > >>>>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use
> > > > script.
> > > > > >> The
> > > > > >>>>>>>>> script
> > > > > >>>>>>>>>>>>>>>> needs
> > > > > >>>>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put
> under
> > > > > >>>>>>>> “libs/optional”
> > > > > >>>>>>>>>>>>>>>> folder.
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an
> > > existed
> > > > > >>>>>>>> benchmark
> > > > > >>>>>>>>> or
> > > > > >>>>>>>>>>>>>>>> add a
> > > > > >>>>>>>>>>>>>>>>>>> new
> > > > > >>>>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed
> > Yardstick
> > > > > >>>> guidance.
> > > > > >>>>>>>>> So,
> > > > > >>>>>>>>>>>> no
> > > > > >>>>>>>>>>>>>>>>>> need
> > > > > >>>>>>>>>>>>>>>>>>> to
> > > > > >>>>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of
> > Ignite
> > > > > >> release.
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> —
> > > > > >>>>>>>>>>>>>>>>>>>>> Denis
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy
> > Setrakyan <
> > > > > >>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> > > > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg
> Ostanin
> > <
> > > > > >>>>>>>>>>>>>>>> oostanin@gridgain.com
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to
> run
> > > > plenty
> > > > > >> of
> > > > > >>>>>>>>> useful
> > > > > >>>>>>>>>>>>>>>>>>> Yardstick
> > > > > >>>>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check
> Ignite
> > > > > >>>>>> performance.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>> In that case, why would it be under the
> "libs"
> > > > > folder
> > > > > >> at
> > > > > >>>>>>>> all?
> > > > > >>>>>>>>>>>> Do
> > > > > >>>>>>>>>>>>>> we
> > > > > >>>>>>>>>>>>>>>>>>>>> really
> > > > > >>>>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If
> > yes,
> > > > > then I
> > > > > >>>>>>>> would
> > > > > >>>>>>>>>>>>>>>> create a
> > > > > >>>>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add
> all
> > > the
> > > > > >>>>>>>> benchmarks
> > > > > >>>>>>>>>>>>>>>> there.
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy
> > > > > Setrakyan <
> > > > > >>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module
> do?
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg
> > > Ostanin <
> > > > > >>>>>>>>>>>>>>>>>>> oostanin@gridgain.com>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
> > > > > >>>> Benchmarking
> > > > > >>>>>>>>>>>>>>>>>>> Simplification
> > > > > >>>>>>>>>>>>>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your
> > opinion
> > > > > about
> > > > > >>>>>>>>>>>>>>>>>> ignite-yardstick:
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> where
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate
> place
> > to
> > > > > put a
> > > > > >>>>>>>>> compiled
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the
> > apache-ignite
> > > > > binary
> > > > > >>>>>>>>>>>> assembly?
> > > > > >>>>>>>>>>>>>> We
> > > > > >>>>>>>>>>>>>>>>>> can
> > > > > >>>>>>>>>>>>>>>>>>>>>>> put
> > > > > >>>>>>>>>>>>>>>>>>>>>>>> it
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others
> > > > > optional
> > > > > >>>>>>>>>>>> libraries,
> > > > > >>>>>>>>>>>>>> or
> > > > > >>>>>>>>>>>>>>>>>> we
> > > > > >>>>>>>>>>>>>>>>>>>>> can
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in
> the
> > > > root
> > > > > >>>>>>>> directory
> > > > > >>>>>>>>>>>> and
> > > > > >>>>>>>>>>>>>>>> put
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find
> > > > another
> > > > > >>>>>>>> solution.
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Best regards
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Oleg
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>>>>> Sergey Kozlov
> > > > > >>>>>>>>>>>>>>>>>>>> GridGain Systems
> > > > > >>>>>>>>>>>>>>>>>>>> www.gridgain.com
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>>>> Sergey Kozlov
> > > > > >>>>>>>>>>>>>>>>> GridGain Systems
> > > > > >>>>>>>>>>>>>>>>> www.gridgain.com
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>>> Sergey Kozlov
> > > > > >>>>>>>>>>>>>>> GridGain Systems
> > > > > >>>>>>>>>>>>>>> www.gridgain.com
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> --
> > > > > >>>>>>>>>>>>>> Sergey Kozlov
> > > > > >>>>>>>>>>>>>> GridGain Systems
> > > > > >>>>>>>>>>>>>> www.gridgain.com
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> --
> > > > > >>>>>>>> Ilya Suntsov
> > > > > >>>>>>>>
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Sergey Kozlov
> > GridGain Systems
> > www.gridgain.com
> >
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Anton Vinogradov <av...@gridgain.com>.

BTW, it will be a good point to relocate Ignite assembly to
ci.ignite.apache.org :)

On Fri, Feb 3, 2017 at 12:26 PM, Sergey Kozlov <sk...@gridgain.com> wrote:

> Hi
>
> Not sure that we've a release task on ci.ignite.apache.org
>
> On Fri, Feb 3, 2017 at 12:20 PM, Anton Vinogradov <
> avinogradov@gridgain.com>
> wrote:
>
> > Oleg, you can point to TC build artifacts next time.
> >
> > On Fri, Feb 3, 2017 at 11:49 AM, Oleg Ostanin <oo...@gridgain.com>
> > wrote:
> >
> > > Hi!
> > >
> > > Denis, I've uploaded Ignite release binaries with my changes here:
> > >
> > > https://drive.google.com/file/d/0B9teTzJ9iIvHYTdUMlRsYTdRckk/
> > > view?usp=sharing
> > >
> > > On Thu, Feb 2, 2017 at 8:02 PM, Denis Magda <dm...@apache.org> wrote:
> > >
> > > > Oleg, thanks,
> > > >
> > > > Could you share Ignite release binaries with your patch applied? You
> > > > should be able to use TeamCity for that. Makes sense to see the new
> > > project
> > > > structure and walk through final version of the instructions.
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Feb 2, 2017, at 2:30 AM, Oleg Ostanin <oo...@gridgain.com>
> > > wrote:
> > > > >
> > > > > Denis,
> > > > >
> > > > > Yes, the same identical README.txt as in the pull request will be
> > > placed
> > > > in
> > > > > both benchmarks binaries and Ignite Yardstick module.
> > > > >
> > > > > I've also created two DEVNOTES.txt with related building
> > instructions.
> > > > >
> > > > > On Wed, Feb 1, 2017 at 10:31 PM, Denis Magda <dm...@apache.org>
> > > wrote:
> > > > >
> > > > >> Anton,
> > > > >>
> > > > >> I’m absolutely fine to make a single README if its instructions
> will
> > > be
> > > > >> identical for benchmarks binaries delivered in Ignite binary
> > releases
> > > > and
> > > > >> assembled with yardstick module directly. Oleg, is it feasible?
> > > > >>
> > > > >> If to skip auto-generation idea then DEVNOTES split suggested by
> you
> > > is
> > > > >> the only option.
> > > > >>
> > > > >> —
> > > > >> Denis
> > > > >>
> > > > >>> On Feb 1, 2017, at 1:45 AM, Anton Vinogradov <
> > > avinogradov@gridgain.com
> > > > >
> > > > >> wrote:
> > > > >>>
> > > > >>> Denis,
> > > > >>>
> > > > >>> I don't like autogeneration.
> > > > >>>
> > > > >>> In my view, we have to keep one README and ret rid of
> "Installation
> > > > >>> instructions" inside it.
> > > > >>> README should explain how to work with assembled yardstick. So,
> > same
> > > > >> README
> > > > >>> will be uses at sources and inside release assembly.
> > > > >>>
> > > > >>> Also we have to have to split DEVNOTES to DEVNOTES and
> > > > >> DEVNOTES.standalone,
> > > > >>> to explain how to assembly yardstick from sources.
> > > > >>>
> > > > >>> Thoughts?
> > > > >>>
> > > > >>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org>
> > > > wrote:
> > > > >>>
> > > > >>>> Oleg,
> > > > >>>>
> > > > >>>> Thanks for the clarification.
> > > > >>>>
> > > > >>>> My opinion is that we should leave
> ‘modules/yardstick/README.txt’
> > > and
> > > > >>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only
> > > those
> > > > >>>> instructions there that explain how to build and run benchmarks
> > from
> > > > >>>> ‘modules/yardstick’. This existing files can refer to the
> sources
> > > and
> > > > >>>> compiled benchmarks that are in Ignite binary releases but this
> > > should
> > > > >> be a
> > > > >>>> couple of statements, no more.
> > > > >>>>
> > > > >>>> As for the instructions related to the sources and binaries
> added
> > to
> > > > >>>> Ignite binaries, preferably they need to be added to
> > auto-generated
> > > > >>>> README.txt. *Anton*, is it feasible to do?
> > > > >>>>
> > > > >>>> Finally, when apply the reviews notes please build and share
> > Ignite
> > > > >>>> binaries with your patch. Want to see the new project structure
> > and
> > > > >> final
> > > > >>>> version of the instructions. Presently I can’t merge your
> changes
> > > due
> > > > to
> > > > >>>> some conflicts.
> > > > >>>>
> > > > >>>> *Anton*, please review modifications in the build procedures.
> > > > >>>>
> > > > >>>> —
> > > > >>>> Denis
> > > > >>>>
> > > > >>>>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <
> oostanin@gridgain.com
> > >
> > > > >> wrote:
> > > > >>>>>
> > > > >>>>> Hi Denis,
> > > > >>>>>
> > > > >>>>> Yes, we have included Ignite Yardstick source files with its
> > > pom.xml
> > > > in
> > > > >>>>> Ignite binary release. "Building from standalone sources" is
> the
> > > > >> building
> > > > >>>>> instruction for these source files. "Building from Ignite
> > Sources"
> > > is
> > > > >> the
> > > > >>>>> instruction for building Ignite Yardstick from
> > `modules/yardstick`
> > > in
> > > > >>>>> Ignite source files.
> > > > >>>>>
> > > > >>>>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <
> dmagda@apache.org>
> > > > >> wrote:
> > > > >>>>>
> > > > >>>>>> Hi Oleg,
> > > > >>>>>>
> > > > >>>>>> Great progress, thanks for keep driving this!
> > > > >>>>>>
> > > > >>>>>> I’ve left some minor notes in GitHub’s pull-request. I have
> the
> > > > >>>> following
> > > > >>>>>> questions aside:
> > > > >>>>>>
> > > > >>>>>> - What is the difference between "Building from standalone
> > > sources"
> > > > >> and
> > > > >>>>>> "Building from Ignite Sources"? In my understanding, a user
> > > > downloads
> > > > >>>>>> Apache Ignite release that has all the sources locally.
> > > > >>>>>>
> > > > >>>>>> - I do remember we planned to add the benchmarks sources in a
> > form
> > > > of
> > > > >> a
> > > > >>>>>> ready to be used project with its own pom.xml (similar to
> > > examples).
> > > > >> Did
> > > > >>>>>> you put this task off?
> > > > >>>>>>
> > > > >>>>>> —
> > > > >>>>>> Denis
> > > > >>>>>>
> > > > >>>>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <
> > oostanin@gridgain.com
> > > >
> > > > >>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>> Hi!
> > > > >>>>>>>
> > > > >>>>>>> I've changed the README.txt and DEVNOTES.txt files. Also
> added
> > a
> > > > >> simple
> > > > >>>>>>> config file for quick and easy start. Please take a look at
> > them
> > > > and
> > > > >>>> tell
> > > > >>>>>>> me what you think.
> > > > >>>>>>>
> > > > >>>>>>> https://github.com/apache/ignite/pull/1471
> > > > >>>>>>>
> > > > >>>>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <
> > > > isuntsov@gridgain.com
> > > > >>>
> > > > >>>>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> Denis,
> > > > >>>>>>>>
> > > > >>>>>>>> I think we can remove all configs except:
> > > > >>>>>>>>
> > > > >>>>>>>> benchmark-multicast.properties
> > > > >>>>>>>>
> > > > >>>>>>>> benchmark.properties
> > > > >>>>>>>>
> > > > >>>>>>>> ignite-base-config.xml
> > > > >>>>>>>>
> > > > >>>>>>>> ignite-localhost-config.xml
> > > > >>>>>>>>
> > > > >>>>>>>> ignite-multicast-config.xml
> > > > >>>>>>>>
> > > > >>>>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
> > > > >>>>>>>>
> > > > >>>>>>>>> I would have only those configs that are useful. Ilya
> > Suntsov,
> > > > >> basing
> > > > >>>>>> on
> > > > >>>>>>>>> your experience, please suggest which configs makes sense
> to
> > > > >> include
> > > > >>>>>> into
> > > > >>>>>>>>> every Ignite release.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Oleg, also please note that community decided to include
> not
> > > only
> > > > >> the
> > > > >>>>>>>>> benchmarking binaries but the sources as well into every
> > Apache
> > > > >>>> Ignite
> > > > >>>>>>>>> release. I’ve update the ticket before. Hope you followed
> the
> > > > >>>>>> discussion
> > > > >>>>>>>> ;)
> > > > >>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
> > > > >>>>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
> > > > >>>>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-
> > 15765151
> > > > >>>>>>>>>
> > > > >>>>>>>>> —
> > > > >>>>>>>>> Denis
> > > > >>>>>>>>>
> > > > >>>>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <
> > > > oostanin@gridgain.com>
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> I mean removing those configs from binary assembly, not
> from
> > > > >>>>>>>> repository.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
> > > > >>>> oostanin@gridgain.com>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>> Hello Igniters.
> > > > >>>>>>>>>>> I think it would be better to remove some configuration
> > files
> > > > >> from
> > > > >>>>>>>>>>> benchmarks/config:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> ignite-base-load-config.xml
> > > > >>>>>>>>>>> ignite-cache-load-config.xml
> > > > >>>>>>>>>>> ignite-failover-base-config.xml
> > > > >>>>>>>>>>> ignite-failover-localhost-config.xml
> > > > >>>>>>>>>>> benchmark-cache-load.properties
> > > > >>>>>>>>>>> benchmark-cache-load-win.properties
> > > > >>>>>>>>>>> benchmark-failover.properties
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> because those configs do not relate to any of performance
> > > > tests.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <
> > > > dmagda@apache.org
> > > > >>>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>> Summarized the discussion updating the ticket
> > > > >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> > > > >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> —
> > > > >>>>>>>>>>>> Denis
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> > > > >>>>>>>>> dsetrakyan@apache.org>
> > > > >>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> Sergey,
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> I am not sure I like "extras". I am voting for
> > "benchmarks"
> > > > >>>> folder
> > > > >>>>>>>>> right
> > > > >>>>>>>>>>>>> under the root folder.
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> D.
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
> > > > >>>>>>>> skozlov@gridgain.com
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> Formatting has cut lines:
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> — apache_ignite_root_folder
> > > > >>>>>>>>>>>>>> — bin
> > > > >>>>>>>>>>>>>> — examples
> > > > >>>>>>>>>>>>>> — extras
> > > > >>>>>>>>>>>>>> — benchmarks
> > > > >>>>>>>>>>>>>> — bin
> > > > >>>>>>>>>>>>>> — src (benchmarks sources with pom.xml)
> > > > >>>>>>>>>>>>>> — config
> > > > >>>>>>>>>>>>>> — libs (compiled benchmarks)
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> > > > >>>>>>>>> skozlov@gridgain.com>
> > > > >>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> Denis,
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may
> > > include
> > > > >> more
> > > > >>>>>>>>>>>> things in
> > > > >>>>>>>>>>>>>>> future than yardstick only. It's why I suggest
> > something
> > > > like
> > > > >>>>>>>> that:
> > > > >>>>>>>>>>>>>>> — apache_ignite_root_folder
> > > > >>>>>>>>>>>>>>> — bin
> > > > >>>>>>>>>>>>>>> — examples
> > > > >>>>>>>>>>>>>>> — extras
> > > > >>>>>>>>>>>>>>>   — benchmarks
> > > > >>>>>>>>>>>>>>>       — bin
> > > > >>>>>>>>>>>>>>>       — src (benchmarks sources with pom.xml)
> > > > >>>>>>>>>>>>>>>       — config
> > > > >>>>>>>>>>>>>>>       — libs (compiled benchmarks)
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
> > > > >>>> dmagda@apache.org
> > > > >>>>>>>
> > > > >>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have
> > the
> > > > >>>>>> following
> > > > >>>>>>>>>>>>>>>> structure
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> — apache_ignite_root_folder
> > > > >>>>>>>>>>>>>>>> — examples
> > > > >>>>>>>>>>>>>>>> — bin
> > > > >>>>>>>>>>>>>>>> — benchmarks
> > > > >>>>>>>>>>>>>>>>   — bin
> > > > >>>>>>>>>>>>>>>>   — src (benchmarks sources with pom.xml)
> > > > >>>>>>>>>>>>>>>>   — config
> > > > >>>>>>>>>>>>>>>>   — libs (compiled benchmarks)
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
> > > > >> previously?
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> —
> > > > >>>>>>>>>>>>>>>> Denis
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
> > > > >>>>>>>> skozlov@gridgain.com>
> > > > >>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> Yardstick requires own scripts/configurations
> (/bin,
> > > > >> /config,
> > > > >>>>>>>>> /libs)
> > > > >>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>> creates work/logs directory under yardstick root.
> > > > >>>>>>>> "libs/optional"
> > > > >>>>>>>>> is
> > > > >>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>> optional modules but in general we can't say that
> for
> > > > >>>>>> Yardstick.
> > > > >>>>>>>>>>>> Also
> > > > >>>>>>>>>>>>>> it
> > > > >>>>>>>>>>>>>>>>> may break the current user understanding of
> > > > "libs/optional"
> > > > >>>>>>>>>>>> directory
> > > > >>>>>>>>>>>>>> as
> > > > >>>>>>>>>>>>>>>>> place for additonal functionality activated by
> > copying
> > > in
> > > > >>>>>>>> "libs".
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan
> <
> > > > >>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> > > > >>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> I would be against using libs/optional or libs/
> > folder
> > > > for
> > > > >>>>>>>>> anything
> > > > >>>>>>>>>>>>>>>>>> benchmark related. I am also against adding any
> > > > yardstick
> > > > >>>>>>>>> libraries
> > > > >>>>>>>>>>>>>>>> without
> > > > >>>>>>>>>>>>>>>>>> providing code.
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> In my view, if the community wants to include
> > > benchmarks
> > > > >> in
> > > > >>>>>>>>>>>> releases,
> > > > >>>>>>>>>>>>>>>> then
> > > > >>>>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which
> provides
> > > > >>>> everything
> > > > >>>>>>>>>>>>>>>> benchmark
> > > > >>>>>>>>>>>>>>>>>> related, from code to all the dependent libraries,
> > and
> > > > >>>>>>>>>>>> documentation
> > > > >>>>>>>>>>>>>>>>>> instructions.
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> D.
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
> > > > >>>>>>>> dmagda@apache.org>
> > > > >>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of
> > extra
> > > > for
> > > > >>>> me.
> > > > >>>>>>>> Why
> > > > >>>>>>>>>>>> do
> > > > >>>>>>>>>>>>>> we
> > > > >>>>>>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic
> > > works
> > > > >>>> well?
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
> > > > >>>> “libs/optional”?
> > > > >>>>>>>> If
> > > > >>>>>>>>>>>>>>>> there’re
> > > > >>>>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
> > > > >>>>>> implementation.
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>> —
> > > > >>>>>>>>>>>>>>>>>>> Denis
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> > > > >>>>>>>>> skozlov@gridgain.com
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> Hi
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> What's about to introduce the new root folder
> > called
> > > > >>>>>> 'extras'
> > > > >>>>>>>>>>>> with
> > > > >>>>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there
> > yardstick
> > > > >>>>>>>> binaries?
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
> > > > >>>>>>>>> dmagda@apache.org
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> Dmitriy,
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
> > > > >>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> > > > >>>>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
> > > > >> benchmarks
> > > > >>>>>>>>>>>>>> execution
> > > > >>>>>>>>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now
> > > > he/she
> > > > >>>>>> needs
> > > > >>>>>>>>> to
> > > > >>>>>>>>>>>> go
> > > > >>>>>>>>>>>>>>>>>>> through
> > > > >>>>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run
> > > strictly
> > > > >>>>>>>>> following
> > > > >>>>>>>>>>>>>>>>>> lengthy
> > > > >>>>>>>>>>>>>>>>>>>>> Yardstick guidance.
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded
> it
> > > > >> should
> > > > >>>> be
> > > > >>>>>>>>>>>>>> possible
> > > > >>>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use
> > > script.
> > > > >> The
> > > > >>>>>>>>> script
> > > > >>>>>>>>>>>>>>>> needs
> > > > >>>>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
> > > > >>>>>>>> “libs/optional”
> > > > >>>>>>>>>>>>>>>> folder.
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an
> > existed
> > > > >>>>>>>> benchmark
> > > > >>>>>>>>> or
> > > > >>>>>>>>>>>>>>>> add a
> > > > >>>>>>>>>>>>>>>>>>> new
> > > > >>>>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed
> Yardstick
> > > > >>>> guidance.
> > > > >>>>>>>>> So,
> > > > >>>>>>>>>>>> no
> > > > >>>>>>>>>>>>>>>>>> need
> > > > >>>>>>>>>>>>>>>>>>> to
> > > > >>>>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of
> Ignite
> > > > >> release.
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> —
> > > > >>>>>>>>>>>>>>>>>>>>> Denis
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy
> Setrakyan <
> > > > >>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> > > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin
> <
> > > > >>>>>>>>>>>>>>>> oostanin@gridgain.com
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run
> > > plenty
> > > > >> of
> > > > >>>>>>>>> useful
> > > > >>>>>>>>>>>>>>>>>>> Yardstick
> > > > >>>>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
> > > > >>>>>> performance.
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs"
> > > > folder
> > > > >> at
> > > > >>>>>>>> all?
> > > > >>>>>>>>>>>> Do
> > > > >>>>>>>>>>>>>> we
> > > > >>>>>>>>>>>>>>>>>>>>> really
> > > > >>>>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If
> yes,
> > > > then I
> > > > >>>>>>>> would
> > > > >>>>>>>>>>>>>>>> create a
> > > > >>>>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all
> > the
> > > > >>>>>>>> benchmarks
> > > > >>>>>>>>>>>>>>>> there.
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy
> > > > Setrakyan <
> > > > >>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg
> > Ostanin <
> > > > >>>>>>>>>>>>>>>>>>> oostanin@gridgain.com>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
> > > > >>>> Benchmarking
> > > > >>>>>>>>>>>>>>>>>>> Simplification
> > > > >>>>>>>>>>>>>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your
> opinion
> > > > about
> > > > >>>>>>>>>>>>>>>>>> ignite-yardstick:
> > > > >>>>>>>>>>>>>>>>>>>>>>>> where
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place
> to
> > > > put a
> > > > >>>>>>>>> compiled
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the
> apache-ignite
> > > > binary
> > > > >>>>>>>>>>>> assembly?
> > > > >>>>>>>>>>>>>> We
> > > > >>>>>>>>>>>>>>>>>> can
> > > > >>>>>>>>>>>>>>>>>>>>>>> put
> > > > >>>>>>>>>>>>>>>>>>>>>>>> it
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others
> > > > optional
> > > > >>>>>>>>>>>> libraries,
> > > > >>>>>>>>>>>>>> or
> > > > >>>>>>>>>>>>>>>>>> we
> > > > >>>>>>>>>>>>>>>>>>>>> can
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the
> > > root
> > > > >>>>>>>> directory
> > > > >>>>>>>>>>>> and
> > > > >>>>>>>>>>>>>>>> put
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find
> > > another
> > > > >>>>>>>> solution.
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> Best regards
> > > > >>>>>>>>>>>>>>>>>>>>>>>>> Oleg
> > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>>>>> Sergey Kozlov
> > > > >>>>>>>>>>>>>>>>>>>> GridGain Systems
> > > > >>>>>>>>>>>>>>>>>>>> www.gridgain.com
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>>>> Sergey Kozlov
> > > > >>>>>>>>>>>>>>>>> GridGain Systems
> > > > >>>>>>>>>>>>>>>>> www.gridgain.com
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>>> Sergey Kozlov
> > > > >>>>>>>>>>>>>>> GridGain Systems
> > > > >>>>>>>>>>>>>>> www.gridgain.com
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> --
> > > > >>>>>>>>>>>>>> Sergey Kozlov
> > > > >>>>>>>>>>>>>> GridGain Systems
> > > > >>>>>>>>>>>>>> www.gridgain.com
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> --
> > > > >>>>>>>> Ilya Suntsov
> > > > >>>>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>
> > > > >>>>
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> >
>
>
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Sergey Kozlov <sk...@gridgain.com>.

Hi

Not sure that we've a release task on ci.ignite.apache.org

On Fri, Feb 3, 2017 at 12:20 PM, Anton Vinogradov <av...@gridgain.com>
wrote:

> Oleg, you can point to TC build artifacts next time.
>
> On Fri, Feb 3, 2017 at 11:49 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
>
> > Hi!
> >
> > Denis, I've uploaded Ignite release binaries with my changes here:
> >
> > https://drive.google.com/file/d/0B9teTzJ9iIvHYTdUMlRsYTdRckk/
> > view?usp=sharing
> >
> > On Thu, Feb 2, 2017 at 8:02 PM, Denis Magda <dm...@apache.org> wrote:
> >
> > > Oleg, thanks,
> > >
> > > Could you share Ignite release binaries with your patch applied? You
> > > should be able to use TeamCity for that. Makes sense to see the new
> > project
> > > structure and walk through final version of the instructions.
> > >
> > > —
> > > Denis
> > >
> > > > On Feb 2, 2017, at 2:30 AM, Oleg Ostanin <oo...@gridgain.com>
> > wrote:
> > > >
> > > > Denis,
> > > >
> > > > Yes, the same identical README.txt as in the pull request will be
> > placed
> > > in
> > > > both benchmarks binaries and Ignite Yardstick module.
> > > >
> > > > I've also created two DEVNOTES.txt with related building
> instructions.
> > > >
> > > > On Wed, Feb 1, 2017 at 10:31 PM, Denis Magda <dm...@apache.org>
> > wrote:
> > > >
> > > >> Anton,
> > > >>
> > > >> I’m absolutely fine to make a single README if its instructions will
> > be
> > > >> identical for benchmarks binaries delivered in Ignite binary
> releases
> > > and
> > > >> assembled with yardstick module directly. Oleg, is it feasible?
> > > >>
> > > >> If to skip auto-generation idea then DEVNOTES split suggested by you
> > is
> > > >> the only option.
> > > >>
> > > >> —
> > > >> Denis
> > > >>
> > > >>> On Feb 1, 2017, at 1:45 AM, Anton Vinogradov <
> > avinogradov@gridgain.com
> > > >
> > > >> wrote:
> > > >>>
> > > >>> Denis,
> > > >>>
> > > >>> I don't like autogeneration.
> > > >>>
> > > >>> In my view, we have to keep one README and ret rid of "Installation
> > > >>> instructions" inside it.
> > > >>> README should explain how to work with assembled yardstick. So,
> same
> > > >> README
> > > >>> will be uses at sources and inside release assembly.
> > > >>>
> > > >>> Also we have to have to split DEVNOTES to DEVNOTES and
> > > >> DEVNOTES.standalone,
> > > >>> to explain how to assembly yardstick from sources.
> > > >>>
> > > >>> Thoughts?
> > > >>>
> > > >>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org>
> > > wrote:
> > > >>>
> > > >>>> Oleg,
> > > >>>>
> > > >>>> Thanks for the clarification.
> > > >>>>
> > > >>>> My opinion is that we should leave ‘modules/yardstick/README.txt’
> > and
> > > >>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only
> > those
> > > >>>> instructions there that explain how to build and run benchmarks
> from
> > > >>>> ‘modules/yardstick’. This existing files can refer to the sources
> > and
> > > >>>> compiled benchmarks that are in Ignite binary releases but this
> > should
> > > >> be a
> > > >>>> couple of statements, no more.
> > > >>>>
> > > >>>> As for the instructions related to the sources and binaries added
> to
> > > >>>> Ignite binaries, preferably they need to be added to
> auto-generated
> > > >>>> README.txt. *Anton*, is it feasible to do?
> > > >>>>
> > > >>>> Finally, when apply the reviews notes please build and share
> Ignite
> > > >>>> binaries with your patch. Want to see the new project structure
> and
> > > >> final
> > > >>>> version of the instructions. Presently I can’t merge your changes
> > due
> > > to
> > > >>>> some conflicts.
> > > >>>>
> > > >>>> *Anton*, please review modifications in the build procedures.
> > > >>>>
> > > >>>> —
> > > >>>> Denis
> > > >>>>
> > > >>>>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oostanin@gridgain.com
> >
> > > >> wrote:
> > > >>>>>
> > > >>>>> Hi Denis,
> > > >>>>>
> > > >>>>> Yes, we have included Ignite Yardstick source files with its
> > pom.xml
> > > in
> > > >>>>> Ignite binary release. "Building from standalone sources" is the
> > > >> building
> > > >>>>> instruction for these source files. "Building from Ignite
> Sources"
> > is
> > > >> the
> > > >>>>> instruction for building Ignite Yardstick from
> `modules/yardstick`
> > in
> > > >>>>> Ignite source files.
> > > >>>>>
> > > >>>>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org>
> > > >> wrote:
> > > >>>>>
> > > >>>>>> Hi Oleg,
> > > >>>>>>
> > > >>>>>> Great progress, thanks for keep driving this!
> > > >>>>>>
> > > >>>>>> I’ve left some minor notes in GitHub’s pull-request. I have the
> > > >>>> following
> > > >>>>>> questions aside:
> > > >>>>>>
> > > >>>>>> - What is the difference between "Building from standalone
> > sources"
> > > >> and
> > > >>>>>> "Building from Ignite Sources"? In my understanding, a user
> > > downloads
> > > >>>>>> Apache Ignite release that has all the sources locally.
> > > >>>>>>
> > > >>>>>> - I do remember we planned to add the benchmarks sources in a
> form
> > > of
> > > >> a
> > > >>>>>> ready to be used project with its own pom.xml (similar to
> > examples).
> > > >> Did
> > > >>>>>> you put this task off?
> > > >>>>>>
> > > >>>>>> —
> > > >>>>>> Denis
> > > >>>>>>
> > > >>>>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <
> oostanin@gridgain.com
> > >
> > > >>>> wrote:
> > > >>>>>>>
> > > >>>>>>> Hi!
> > > >>>>>>>
> > > >>>>>>> I've changed the README.txt and DEVNOTES.txt files. Also added
> a
> > > >> simple
> > > >>>>>>> config file for quick and easy start. Please take a look at
> them
> > > and
> > > >>>> tell
> > > >>>>>>> me what you think.
> > > >>>>>>>
> > > >>>>>>> https://github.com/apache/ignite/pull/1471
> > > >>>>>>>
> > > >>>>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <
> > > isuntsov@gridgain.com
> > > >>>
> > > >>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> Denis,
> > > >>>>>>>>
> > > >>>>>>>> I think we can remove all configs except:
> > > >>>>>>>>
> > > >>>>>>>> benchmark-multicast.properties
> > > >>>>>>>>
> > > >>>>>>>> benchmark.properties
> > > >>>>>>>>
> > > >>>>>>>> ignite-base-config.xml
> > > >>>>>>>>
> > > >>>>>>>> ignite-localhost-config.xml
> > > >>>>>>>>
> > > >>>>>>>> ignite-multicast-config.xml
> > > >>>>>>>>
> > > >>>>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
> > > >>>>>>>>
> > > >>>>>>>>> I would have only those configs that are useful. Ilya
> Suntsov,
> > > >> basing
> > > >>>>>> on
> > > >>>>>>>>> your experience, please suggest which configs makes sense to
> > > >> include
> > > >>>>>> into
> > > >>>>>>>>> every Ignite release.
> > > >>>>>>>>>
> > > >>>>>>>>> Oleg, also please note that community decided to include not
> > only
> > > >> the
> > > >>>>>>>>> benchmarking binaries but the sources as well into every
> Apache
> > > >>>> Ignite
> > > >>>>>>>>> release. I’ve update the ticket before. Hope you followed the
> > > >>>>>> discussion
> > > >>>>>>>> ;)
> > > >>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
> > > >>>>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
> > > >>>>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-
> 15765151
> > > >>>>>>>>>
> > > >>>>>>>>> —
> > > >>>>>>>>> Denis
> > > >>>>>>>>>
> > > >>>>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <
> > > oostanin@gridgain.com>
> > > >>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>> I mean removing those configs from binary assembly, not from
> > > >>>>>>>> repository.
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
> > > >>>> oostanin@gridgain.com>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Hello Igniters.
> > > >>>>>>>>>>> I think it would be better to remove some configuration
> files
> > > >> from
> > > >>>>>>>>>>> benchmarks/config:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> ignite-base-load-config.xml
> > > >>>>>>>>>>> ignite-cache-load-config.xml
> > > >>>>>>>>>>> ignite-failover-base-config.xml
> > > >>>>>>>>>>> ignite-failover-localhost-config.xml
> > > >>>>>>>>>>> benchmark-cache-load.properties
> > > >>>>>>>>>>> benchmark-cache-load-win.properties
> > > >>>>>>>>>>> benchmark-failover.properties
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> because those configs do not relate to any of performance
> > > tests.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <
> > > dmagda@apache.org
> > > >>>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> Summarized the discussion updating the ticket
> > > >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> > > >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> —
> > > >>>>>>>>>>>> Denis
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> > > >>>>>>>>> dsetrakyan@apache.org>
> > > >>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Sergey,
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I am not sure I like "extras". I am voting for
> "benchmarks"
> > > >>>> folder
> > > >>>>>>>>> right
> > > >>>>>>>>>>>>> under the root folder.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> D.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
> > > >>>>>>>> skozlov@gridgain.com
> > > >>>>>>>>>>
> > > >>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Formatting has cut lines:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> — apache_ignite_root_folder
> > > >>>>>>>>>>>>>> — bin
> > > >>>>>>>>>>>>>> — examples
> > > >>>>>>>>>>>>>> — extras
> > > >>>>>>>>>>>>>> — benchmarks
> > > >>>>>>>>>>>>>> — bin
> > > >>>>>>>>>>>>>> — src (benchmarks sources with pom.xml)
> > > >>>>>>>>>>>>>> — config
> > > >>>>>>>>>>>>>> — libs (compiled benchmarks)
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> > > >>>>>>>>> skozlov@gridgain.com>
> > > >>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Denis,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may
> > include
> > > >> more
> > > >>>>>>>>>>>> things in
> > > >>>>>>>>>>>>>>> future than yardstick only. It's why I suggest
> something
> > > like
> > > >>>>>>>> that:
> > > >>>>>>>>>>>>>>> — apache_ignite_root_folder
> > > >>>>>>>>>>>>>>> — bin
> > > >>>>>>>>>>>>>>> — examples
> > > >>>>>>>>>>>>>>> — extras
> > > >>>>>>>>>>>>>>>   — benchmarks
> > > >>>>>>>>>>>>>>>       — bin
> > > >>>>>>>>>>>>>>>       — src (benchmarks sources with pom.xml)
> > > >>>>>>>>>>>>>>>       — config
> > > >>>>>>>>>>>>>>>       — libs (compiled benchmarks)
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
> > > >>>> dmagda@apache.org
> > > >>>>>>>
> > > >>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have
> the
> > > >>>>>> following
> > > >>>>>>>>>>>>>>>> structure
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> — apache_ignite_root_folder
> > > >>>>>>>>>>>>>>>> — examples
> > > >>>>>>>>>>>>>>>> — bin
> > > >>>>>>>>>>>>>>>> — benchmarks
> > > >>>>>>>>>>>>>>>>   — bin
> > > >>>>>>>>>>>>>>>>   — src (benchmarks sources with pom.xml)
> > > >>>>>>>>>>>>>>>>   — config
> > > >>>>>>>>>>>>>>>>   — libs (compiled benchmarks)
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
> > > >> previously?
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> —
> > > >>>>>>>>>>>>>>>> Denis
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
> > > >>>>>>>> skozlov@gridgain.com>
> > > >>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin,
> > > >> /config,
> > > >>>>>>>>> /libs)
> > > >>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>> creates work/logs directory under yardstick root.
> > > >>>>>>>> "libs/optional"
> > > >>>>>>>>> is
> > > >>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>> optional modules but in general we can't say that for
> > > >>>>>> Yardstick.
> > > >>>>>>>>>>>> Also
> > > >>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>> may break the current user understanding of
> > > "libs/optional"
> > > >>>>>>>>>>>> directory
> > > >>>>>>>>>>>>>> as
> > > >>>>>>>>>>>>>>>>> place for additonal functionality activated by
> copying
> > in
> > > >>>>>>>> "libs".
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> > > >>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> > > >>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> I would be against using libs/optional or libs/
> folder
> > > for
> > > >>>>>>>>> anything
> > > >>>>>>>>>>>>>>>>>> benchmark related. I am also against adding any
> > > yardstick
> > > >>>>>>>>> libraries
> > > >>>>>>>>>>>>>>>> without
> > > >>>>>>>>>>>>>>>>>> providing code.
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> In my view, if the community wants to include
> > benchmarks
> > > >> in
> > > >>>>>>>>>>>> releases,
> > > >>>>>>>>>>>>>>>> then
> > > >>>>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
> > > >>>> everything
> > > >>>>>>>>>>>>>>>> benchmark
> > > >>>>>>>>>>>>>>>>>> related, from code to all the dependent libraries,
> and
> > > >>>>>>>>>>>> documentation
> > > >>>>>>>>>>>>>>>>>> instructions.
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> D.
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
> > > >>>>>>>> dmagda@apache.org>
> > > >>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of
> extra
> > > for
> > > >>>> me.
> > > >>>>>>>> Why
> > > >>>>>>>>>>>> do
> > > >>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic
> > works
> > > >>>> well?
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
> > > >>>> “libs/optional”?
> > > >>>>>>>> If
> > > >>>>>>>>>>>>>>>> there’re
> > > >>>>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
> > > >>>>>> implementation.
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>> —
> > > >>>>>>>>>>>>>>>>>>> Denis
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> > > >>>>>>>>> skozlov@gridgain.com
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> Hi
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> What's about to introduce the new root folder
> called
> > > >>>>>> 'extras'
> > > >>>>>>>>>>>> with
> > > >>>>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there
> yardstick
> > > >>>>>>>> binaries?
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
> > > >>>>>>>>> dmagda@apache.org
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Dmitriy,
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
> > > >>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> > > >>>>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
> > > >> benchmarks
> > > >>>>>>>>>>>>>> execution
> > > >>>>>>>>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now
> > > he/she
> > > >>>>>> needs
> > > >>>>>>>>> to
> > > >>>>>>>>>>>> go
> > > >>>>>>>>>>>>>>>>>>> through
> > > >>>>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run
> > strictly
> > > >>>>>>>>> following
> > > >>>>>>>>>>>>>>>>>> lengthy
> > > >>>>>>>>>>>>>>>>>>>>> Yardstick guidance.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it
> > > >> should
> > > >>>> be
> > > >>>>>>>>>>>>>> possible
> > > >>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use
> > script.
> > > >> The
> > > >>>>>>>>> script
> > > >>>>>>>>>>>>>>>> needs
> > > >>>>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
> > > >>>>>>>> “libs/optional”
> > > >>>>>>>>>>>>>>>> folder.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an
> existed
> > > >>>>>>>> benchmark
> > > >>>>>>>>> or
> > > >>>>>>>>>>>>>>>> add a
> > > >>>>>>>>>>>>>>>>>>> new
> > > >>>>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
> > > >>>> guidance.
> > > >>>>>>>>> So,
> > > >>>>>>>>>>>> no
> > > >>>>>>>>>>>>>>>>>> need
> > > >>>>>>>>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite
> > > >> release.
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> —
> > > >>>>>>>>>>>>>>>>>>>>> Denis
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> > > >>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> > > >>>>>>>>>>>>>>>> oostanin@gridgain.com
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run
> > plenty
> > > >> of
> > > >>>>>>>>> useful
> > > >>>>>>>>>>>>>>>>>>> Yardstick
> > > >>>>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
> > > >>>>>> performance.
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs"
> > > folder
> > > >> at
> > > >>>>>>>> all?
> > > >>>>>>>>>>>> Do
> > > >>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>>>> really
> > > >>>>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes,
> > > then I
> > > >>>>>>>> would
> > > >>>>>>>>>>>>>>>> create a
> > > >>>>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all
> the
> > > >>>>>>>> benchmarks
> > > >>>>>>>>>>>>>>>> there.
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy
> > > Setrakyan <
> > > >>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg
> Ostanin <
> > > >>>>>>>>>>>>>>>>>>> oostanin@gridgain.com>
> > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
> > > >>>>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
> > > >>>> Benchmarking
> > > >>>>>>>>>>>>>>>>>>> Simplification
> > > >>>>>>>>>>>>>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion
> > > about
> > > >>>>>>>>>>>>>>>>>> ignite-yardstick:
> > > >>>>>>>>>>>>>>>>>>>>>>>> where
> > > >>>>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to
> > > put a
> > > >>>>>>>>> compiled
> > > >>>>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite
> > > binary
> > > >>>>>>>>>>>> assembly?
> > > >>>>>>>>>>>>>> We
> > > >>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>>> put
> > > >>>>>>>>>>>>>>>>>>>>>>>> it
> > > >>>>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others
> > > optional
> > > >>>>>>>>>>>> libraries,
> > > >>>>>>>>>>>>>> or
> > > >>>>>>>>>>>>>>>>>> we
> > > >>>>>>>>>>>>>>>>>>>>> can
> > > >>>>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the
> > root
> > > >>>>>>>> directory
> > > >>>>>>>>>>>> and
> > > >>>>>>>>>>>>>>>> put
> > > >>>>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find
> > another
> > > >>>>>>>> solution.
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Best regards
> > > >>>>>>>>>>>>>>>>>>>>>>>>> Oleg
> > > >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>>>>> Sergey Kozlov
> > > >>>>>>>>>>>>>>>>>>>> GridGain Systems
> > > >>>>>>>>>>>>>>>>>>>> www.gridgain.com
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>> Sergey Kozlov
> > > >>>>>>>>>>>>>>>>> GridGain Systems
> > > >>>>>>>>>>>>>>>>> www.gridgain.com
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>> Sergey Kozlov
> > > >>>>>>>>>>>>>>> GridGain Systems
> > > >>>>>>>>>>>>>>> www.gridgain.com
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>> Sergey Kozlov
> > > >>>>>>>>>>>>>> GridGain Systems
> > > >>>>>>>>>>>>>> www.gridgain.com
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> Ilya Suntsov
> > > >>>>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>>
> > > >>
> > > >>
> > >
> > >
> >
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Anton Vinogradov <av...@gridgain.com>.

Oleg, you can point to TC build artifacts next time.

On Fri, Feb 3, 2017 at 11:49 AM, Oleg Ostanin <oo...@gridgain.com> wrote:

> Hi!
>
> Denis, I've uploaded Ignite release binaries with my changes here:
>
> https://drive.google.com/file/d/0B9teTzJ9iIvHYTdUMlRsYTdRckk/
> view?usp=sharing
>
> On Thu, Feb 2, 2017 at 8:02 PM, Denis Magda <dm...@apache.org> wrote:
>
> > Oleg, thanks,
> >
> > Could you share Ignite release binaries with your patch applied? You
> > should be able to use TeamCity for that. Makes sense to see the new
> project
> > structure and walk through final version of the instructions.
> >
> > —
> > Denis
> >
> > > On Feb 2, 2017, at 2:30 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
> > >
> > > Denis,
> > >
> > > Yes, the same identical README.txt as in the pull request will be
> placed
> > in
> > > both benchmarks binaries and Ignite Yardstick module.
> > >
> > > I've also created two DEVNOTES.txt with related building instructions.
> > >
> > > On Wed, Feb 1, 2017 at 10:31 PM, Denis Magda <dm...@apache.org>
> wrote:
> > >
> > >> Anton,
> > >>
> > >> I’m absolutely fine to make a single README if its instructions will
> be
> > >> identical for benchmarks binaries delivered in Ignite binary releases
> > and
> > >> assembled with yardstick module directly. Oleg, is it feasible?
> > >>
> > >> If to skip auto-generation idea then DEVNOTES split suggested by you
> is
> > >> the only option.
> > >>
> > >> —
> > >> Denis
> > >>
> > >>> On Feb 1, 2017, at 1:45 AM, Anton Vinogradov <
> avinogradov@gridgain.com
> > >
> > >> wrote:
> > >>>
> > >>> Denis,
> > >>>
> > >>> I don't like autogeneration.
> > >>>
> > >>> In my view, we have to keep one README and ret rid of "Installation
> > >>> instructions" inside it.
> > >>> README should explain how to work with assembled yardstick. So, same
> > >> README
> > >>> will be uses at sources and inside release assembly.
> > >>>
> > >>> Also we have to have to split DEVNOTES to DEVNOTES and
> > >> DEVNOTES.standalone,
> > >>> to explain how to assembly yardstick from sources.
> > >>>
> > >>> Thoughts?
> > >>>
> > >>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org>
> > wrote:
> > >>>
> > >>>> Oleg,
> > >>>>
> > >>>> Thanks for the clarification.
> > >>>>
> > >>>> My opinion is that we should leave ‘modules/yardstick/README.txt’
> and
> > >>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only
> those
> > >>>> instructions there that explain how to build and run benchmarks from
> > >>>> ‘modules/yardstick’. This existing files can refer to the sources
> and
> > >>>> compiled benchmarks that are in Ignite binary releases but this
> should
> > >> be a
> > >>>> couple of statements, no more.
> > >>>>
> > >>>> As for the instructions related to the sources and binaries added to
> > >>>> Ignite binaries, preferably they need to be added to auto-generated
> > >>>> README.txt. *Anton*, is it feasible to do?
> > >>>>
> > >>>> Finally, when apply the reviews notes please build and share Ignite
> > >>>> binaries with your patch. Want to see the new project structure and
> > >> final
> > >>>> version of the instructions. Presently I can’t merge your changes
> due
> > to
> > >>>> some conflicts.
> > >>>>
> > >>>> *Anton*, please review modifications in the build procedures.
> > >>>>
> > >>>> —
> > >>>> Denis
> > >>>>
> > >>>>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com>
> > >> wrote:
> > >>>>>
> > >>>>> Hi Denis,
> > >>>>>
> > >>>>> Yes, we have included Ignite Yardstick source files with its
> pom.xml
> > in
> > >>>>> Ignite binary release. "Building from standalone sources" is the
> > >> building
> > >>>>> instruction for these source files. "Building from Ignite Sources"
> is
> > >> the
> > >>>>> instruction for building Ignite Yardstick from `modules/yardstick`
> in
> > >>>>> Ignite source files.
> > >>>>>
> > >>>>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org>
> > >> wrote:
> > >>>>>
> > >>>>>> Hi Oleg,
> > >>>>>>
> > >>>>>> Great progress, thanks for keep driving this!
> > >>>>>>
> > >>>>>> I’ve left some minor notes in GitHub’s pull-request. I have the
> > >>>> following
> > >>>>>> questions aside:
> > >>>>>>
> > >>>>>> - What is the difference between "Building from standalone
> sources"
> > >> and
> > >>>>>> "Building from Ignite Sources"? In my understanding, a user
> > downloads
> > >>>>>> Apache Ignite release that has all the sources locally.
> > >>>>>>
> > >>>>>> - I do remember we planned to add the benchmarks sources in a form
> > of
> > >> a
> > >>>>>> ready to be used project with its own pom.xml (similar to
> examples).
> > >> Did
> > >>>>>> you put this task off?
> > >>>>>>
> > >>>>>> —
> > >>>>>> Denis
> > >>>>>>
> > >>>>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oostanin@gridgain.com
> >
> > >>>> wrote:
> > >>>>>>>
> > >>>>>>> Hi!
> > >>>>>>>
> > >>>>>>> I've changed the README.txt and DEVNOTES.txt files. Also added a
> > >> simple
> > >>>>>>> config file for quick and easy start. Please take a look at them
> > and
> > >>>> tell
> > >>>>>>> me what you think.
> > >>>>>>>
> > >>>>>>> https://github.com/apache/ignite/pull/1471
> > >>>>>>>
> > >>>>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <
> > isuntsov@gridgain.com
> > >>>
> > >>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Denis,
> > >>>>>>>>
> > >>>>>>>> I think we can remove all configs except:
> > >>>>>>>>
> > >>>>>>>> benchmark-multicast.properties
> > >>>>>>>>
> > >>>>>>>> benchmark.properties
> > >>>>>>>>
> > >>>>>>>> ignite-base-config.xml
> > >>>>>>>>
> > >>>>>>>> ignite-localhost-config.xml
> > >>>>>>>>
> > >>>>>>>> ignite-multicast-config.xml
> > >>>>>>>>
> > >>>>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
> > >>>>>>>>
> > >>>>>>>>> I would have only those configs that are useful. Ilya Suntsov,
> > >> basing
> > >>>>>> on
> > >>>>>>>>> your experience, please suggest which configs makes sense to
> > >> include
> > >>>>>> into
> > >>>>>>>>> every Ignite release.
> > >>>>>>>>>
> > >>>>>>>>> Oleg, also please note that community decided to include not
> only
> > >> the
> > >>>>>>>>> benchmarking binaries but the sources as well into every Apache
> > >>>> Ignite
> > >>>>>>>>> release. I’ve update the ticket before. Hope you followed the
> > >>>>>> discussion
> > >>>>>>>> ;)
> > >>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
> > >>>>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
> > >>>>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
> > >>>>>>>>>
> > >>>>>>>>> —
> > >>>>>>>>> Denis
> > >>>>>>>>>
> > >>>>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <
> > oostanin@gridgain.com>
> > >>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> I mean removing those configs from binary assembly, not from
> > >>>>>>>> repository.
> > >>>>>>>>>>
> > >>>>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
> > >>>> oostanin@gridgain.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Hello Igniters.
> > >>>>>>>>>>> I think it would be better to remove some configuration files
> > >> from
> > >>>>>>>>>>> benchmarks/config:
> > >>>>>>>>>>>
> > >>>>>>>>>>> ignite-base-load-config.xml
> > >>>>>>>>>>> ignite-cache-load-config.xml
> > >>>>>>>>>>> ignite-failover-base-config.xml
> > >>>>>>>>>>> ignite-failover-localhost-config.xml
> > >>>>>>>>>>> benchmark-cache-load.properties
> > >>>>>>>>>>> benchmark-cache-load-win.properties
> > >>>>>>>>>>> benchmark-failover.properties
> > >>>>>>>>>>>
> > >>>>>>>>>>> because those configs do not relate to any of performance
> > tests.
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <
> > dmagda@apache.org
> > >>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Summarized the discussion updating the ticket
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> —
> > >>>>>>>>>>>> Denis
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> > >>>>>>>>> dsetrakyan@apache.org>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Sergey,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
> > >>>> folder
> > >>>>>>>>> right
> > >>>>>>>>>>>>> under the root folder.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> D.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
> > >>>>>>>> skozlov@gridgain.com
> > >>>>>>>>>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Formatting has cut lines:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> — apache_ignite_root_folder
> > >>>>>>>>>>>>>> — bin
> > >>>>>>>>>>>>>> — examples
> > >>>>>>>>>>>>>> — extras
> > >>>>>>>>>>>>>> — benchmarks
> > >>>>>>>>>>>>>> — bin
> > >>>>>>>>>>>>>> — src (benchmarks sources with pom.xml)
> > >>>>>>>>>>>>>> — config
> > >>>>>>>>>>>>>> — libs (compiled benchmarks)
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> > >>>>>>>>> skozlov@gridgain.com>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Denis,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may
> include
> > >> more
> > >>>>>>>>>>>> things in
> > >>>>>>>>>>>>>>> future than yardstick only. It's why I suggest something
> > like
> > >>>>>>>> that:
> > >>>>>>>>>>>>>>> — apache_ignite_root_folder
> > >>>>>>>>>>>>>>> — bin
> > >>>>>>>>>>>>>>> — examples
> > >>>>>>>>>>>>>>> — extras
> > >>>>>>>>>>>>>>>   — benchmarks
> > >>>>>>>>>>>>>>>       — bin
> > >>>>>>>>>>>>>>>       — src (benchmarks sources with pom.xml)
> > >>>>>>>>>>>>>>>       — config
> > >>>>>>>>>>>>>>>       — libs (compiled benchmarks)
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
> > >>>> dmagda@apache.org
> > >>>>>>>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
> > >>>>>> following
> > >>>>>>>>>>>>>>>> structure
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> — apache_ignite_root_folder
> > >>>>>>>>>>>>>>>> — examples
> > >>>>>>>>>>>>>>>> — bin
> > >>>>>>>>>>>>>>>> — benchmarks
> > >>>>>>>>>>>>>>>>   — bin
> > >>>>>>>>>>>>>>>>   — src (benchmarks sources with pom.xml)
> > >>>>>>>>>>>>>>>>   — config
> > >>>>>>>>>>>>>>>>   — libs (compiled benchmarks)
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
> > >> previously?
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> —
> > >>>>>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
> > >>>>>>>> skozlov@gridgain.com>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin,
> > >> /config,
> > >>>>>>>>> /libs)
> > >>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>> creates work/logs directory under yardstick root.
> > >>>>>>>> "libs/optional"
> > >>>>>>>>> is
> > >>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>> optional modules but in general we can't say that for
> > >>>>>> Yardstick.
> > >>>>>>>>>>>> Also
> > >>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>> may break the current user understanding of
> > "libs/optional"
> > >>>>>>>>>>>> directory
> > >>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>> place for additonal functionality activated by copying
> in
> > >>>>>>>> "libs".
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> > >>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> > >>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder
> > for
> > >>>>>>>>> anything
> > >>>>>>>>>>>>>>>>>> benchmark related. I am also against adding any
> > yardstick
> > >>>>>>>>> libraries
> > >>>>>>>>>>>>>>>> without
> > >>>>>>>>>>>>>>>>>> providing code.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> In my view, if the community wants to include
> benchmarks
> > >> in
> > >>>>>>>>>>>> releases,
> > >>>>>>>>>>>>>>>> then
> > >>>>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
> > >>>> everything
> > >>>>>>>>>>>>>>>> benchmark
> > >>>>>>>>>>>>>>>>>> related, from code to all the dependent libraries, and
> > >>>>>>>>>>>> documentation
> > >>>>>>>>>>>>>>>>>> instructions.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> D.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
> > >>>>>>>> dmagda@apache.org>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra
> > for
> > >>>> me.
> > >>>>>>>> Why
> > >>>>>>>>>>>> do
> > >>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic
> works
> > >>>> well?
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
> > >>>> “libs/optional”?
> > >>>>>>>> If
> > >>>>>>>>>>>>>>>> there’re
> > >>>>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
> > >>>>>> implementation.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> —
> > >>>>>>>>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> > >>>>>>>>> skozlov@gridgain.com
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Hi
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> What's about to introduce the new root folder called
> > >>>>>> 'extras'
> > >>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
> > >>>>>>>> binaries?
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
> > >>>>>>>>> dmagda@apache.org
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Dmitriy,
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
> > >>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> > >>>>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
> > >> benchmarks
> > >>>>>>>>>>>>>> execution
> > >>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now
> > he/she
> > >>>>>> needs
> > >>>>>>>>> to
> > >>>>>>>>>>>> go
> > >>>>>>>>>>>>>>>>>>> through
> > >>>>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run
> strictly
> > >>>>>>>>> following
> > >>>>>>>>>>>>>>>>>> lengthy
> > >>>>>>>>>>>>>>>>>>>>> Yardstick guidance.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it
> > >> should
> > >>>> be
> > >>>>>>>>>>>>>> possible
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use
> script.
> > >> The
> > >>>>>>>>> script
> > >>>>>>>>>>>>>>>> needs
> > >>>>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
> > >>>>>>>> “libs/optional”
> > >>>>>>>>>>>>>>>> folder.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
> > >>>>>>>> benchmark
> > >>>>>>>>> or
> > >>>>>>>>>>>>>>>> add a
> > >>>>>>>>>>>>>>>>>>> new
> > >>>>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
> > >>>> guidance.
> > >>>>>>>>> So,
> > >>>>>>>>>>>> no
> > >>>>>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite
> > >> release.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> —
> > >>>>>>>>>>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> > >>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> > >>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> > >>>>>>>>>>>>>>>> oostanin@gridgain.com
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run
> plenty
> > >> of
> > >>>>>>>>> useful
> > >>>>>>>>>>>>>>>>>>> Yardstick
> > >>>>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
> > >>>>>> performance.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs"
> > folder
> > >> at
> > >>>>>>>> all?
> > >>>>>>>>>>>> Do
> > >>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>> really
> > >>>>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes,
> > then I
> > >>>>>>>> would
> > >>>>>>>>>>>>>>>> create a
> > >>>>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
> > >>>>>>>> benchmarks
> > >>>>>>>>>>>>>>>> there.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy
> > Setrakyan <
> > >>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> > >>>>>>>>>>>>>>>>>>> oostanin@gridgain.com>
> > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
> > >>>>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
> > >>>> Benchmarking
> > >>>>>>>>>>>>>>>>>>> Simplification
> > >>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion
> > about
> > >>>>>>>>>>>>>>>>>> ignite-yardstick:
> > >>>>>>>>>>>>>>>>>>>>>>>> where
> > >>>>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to
> > put a
> > >>>>>>>>> compiled
> > >>>>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite
> > binary
> > >>>>>>>>>>>> assembly?
> > >>>>>>>>>>>>>> We
> > >>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>> put
> > >>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others
> > optional
> > >>>>>>>>>>>> libraries,
> > >>>>>>>>>>>>>> or
> > >>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the
> root
> > >>>>>>>> directory
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>> put
> > >>>>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find
> another
> > >>>>>>>> solution.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Best regards
> > >>>>>>>>>>>>>>>>>>>>>>>>> Oleg
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>> Sergey Kozlov
> > >>>>>>>>>>>>>>>>>>>> GridGain Systems
> > >>>>>>>>>>>>>>>>>>>> www.gridgain.com
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>> Sergey Kozlov
> > >>>>>>>>>>>>>>>>> GridGain Systems
> > >>>>>>>>>>>>>>>>> www.gridgain.com
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>> Sergey Kozlov
> > >>>>>>>>>>>>>>> GridGain Systems
> > >>>>>>>>>>>>>>> www.gridgain.com
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>> Sergey Kozlov
> > >>>>>>>>>>>>>> GridGain Systems
> > >>>>>>>>>>>>>> www.gridgain.com
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Ilya Suntsov
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>>>
> > >>
> > >>
> >
> >
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

Hi!

Denis, I've uploaded Ignite release binaries with my changes here:

https://drive.google.com/file/d/0B9teTzJ9iIvHYTdUMlRsYTdRckk/view?usp=sharing

On Thu, Feb 2, 2017 at 8:02 PM, Denis Magda <dm...@apache.org> wrote:

> Oleg, thanks,
>
> Could you share Ignite release binaries with your patch applied? You
> should be able to use TeamCity for that. Makes sense to see the new project
> structure and walk through final version of the instructions.
>
> —
> Denis
>
> > On Feb 2, 2017, at 2:30 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> >
> > Denis,
> >
> > Yes, the same identical README.txt as in the pull request will be placed
> in
> > both benchmarks binaries and Ignite Yardstick module.
> >
> > I've also created two DEVNOTES.txt with related building instructions.
> >
> > On Wed, Feb 1, 2017 at 10:31 PM, Denis Magda <dm...@apache.org> wrote:
> >
> >> Anton,
> >>
> >> I’m absolutely fine to make a single README if its instructions will be
> >> identical for benchmarks binaries delivered in Ignite binary releases
> and
> >> assembled with yardstick module directly. Oleg, is it feasible?
> >>
> >> If to skip auto-generation idea then DEVNOTES split suggested by you is
> >> the only option.
> >>
> >> —
> >> Denis
> >>
> >>> On Feb 1, 2017, at 1:45 AM, Anton Vinogradov <avinogradov@gridgain.com
> >
> >> wrote:
> >>>
> >>> Denis,
> >>>
> >>> I don't like autogeneration.
> >>>
> >>> In my view, we have to keep one README and ret rid of "Installation
> >>> instructions" inside it.
> >>> README should explain how to work with assembled yardstick. So, same
> >> README
> >>> will be uses at sources and inside release assembly.
> >>>
> >>> Also we have to have to split DEVNOTES to DEVNOTES and
> >> DEVNOTES.standalone,
> >>> to explain how to assembly yardstick from sources.
> >>>
> >>> Thoughts?
> >>>
> >>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org>
> wrote:
> >>>
> >>>> Oleg,
> >>>>
> >>>> Thanks for the clarification.
> >>>>
> >>>> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
> >>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
> >>>> instructions there that explain how to build and run benchmarks from
> >>>> ‘modules/yardstick’. This existing files can refer to the sources and
> >>>> compiled benchmarks that are in Ignite binary releases but this should
> >> be a
> >>>> couple of statements, no more.
> >>>>
> >>>> As for the instructions related to the sources and binaries added to
> >>>> Ignite binaries, preferably they need to be added to auto-generated
> >>>> README.txt. *Anton*, is it feasible to do?
> >>>>
> >>>> Finally, when apply the reviews notes please build and share Ignite
> >>>> binaries with your patch. Want to see the new project structure and
> >> final
> >>>> version of the instructions. Presently I can’t merge your changes due
> to
> >>>> some conflicts.
> >>>>
> >>>> *Anton*, please review modifications in the build procedures.
> >>>>
> >>>> —
> >>>> Denis
> >>>>
> >>>>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com>
> >> wrote:
> >>>>>
> >>>>> Hi Denis,
> >>>>>
> >>>>> Yes, we have included Ignite Yardstick source files with its pom.xml
> in
> >>>>> Ignite binary release. "Building from standalone sources" is the
> >> building
> >>>>> instruction for these source files. "Building from Ignite Sources" is
> >> the
> >>>>> instruction for building Ignite Yardstick from `modules/yardstick` in
> >>>>> Ignite source files.
> >>>>>
> >>>>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org>
> >> wrote:
> >>>>>
> >>>>>> Hi Oleg,
> >>>>>>
> >>>>>> Great progress, thanks for keep driving this!
> >>>>>>
> >>>>>> I’ve left some minor notes in GitHub’s pull-request. I have the
> >>>> following
> >>>>>> questions aside:
> >>>>>>
> >>>>>> - What is the difference between "Building from standalone sources"
> >> and
> >>>>>> "Building from Ignite Sources"? In my understanding, a user
> downloads
> >>>>>> Apache Ignite release that has all the sources locally.
> >>>>>>
> >>>>>> - I do remember we planned to add the benchmarks sources in a form
> of
> >> a
> >>>>>> ready to be used project with its own pom.xml (similar to examples).
> >> Did
> >>>>>> you put this task off?
> >>>>>>
> >>>>>> —
> >>>>>> Denis
> >>>>>>
> >>>>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>> Hi!
> >>>>>>>
> >>>>>>> I've changed the README.txt and DEVNOTES.txt files. Also added a
> >> simple
> >>>>>>> config file for quick and easy start. Please take a look at them
> and
> >>>> tell
> >>>>>>> me what you think.
> >>>>>>>
> >>>>>>> https://github.com/apache/ignite/pull/1471
> >>>>>>>
> >>>>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <
> isuntsov@gridgain.com
> >>>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Denis,
> >>>>>>>>
> >>>>>>>> I think we can remove all configs except:
> >>>>>>>>
> >>>>>>>> benchmark-multicast.properties
> >>>>>>>>
> >>>>>>>> benchmark.properties
> >>>>>>>>
> >>>>>>>> ignite-base-config.xml
> >>>>>>>>
> >>>>>>>> ignite-localhost-config.xml
> >>>>>>>>
> >>>>>>>> ignite-multicast-config.xml
> >>>>>>>>
> >>>>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
> >>>>>>>>
> >>>>>>>>> I would have only those configs that are useful. Ilya Suntsov,
> >> basing
> >>>>>> on
> >>>>>>>>> your experience, please suggest which configs makes sense to
> >> include
> >>>>>> into
> >>>>>>>>> every Ignite release.
> >>>>>>>>>
> >>>>>>>>> Oleg, also please note that community decided to include not only
> >> the
> >>>>>>>>> benchmarking binaries but the sources as well into every Apache
> >>>> Ignite
> >>>>>>>>> release. I’ve update the ticket before. Hope you followed the
> >>>>>> discussion
> >>>>>>>> ;)
> >>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
> >>>>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
> >>>>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
> >>>>>>>>>
> >>>>>>>>> —
> >>>>>>>>> Denis
> >>>>>>>>>
> >>>>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <
> oostanin@gridgain.com>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I mean removing those configs from binary assembly, not from
> >>>>>>>> repository.
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
> >>>> oostanin@gridgain.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hello Igniters.
> >>>>>>>>>>> I think it would be better to remove some configuration files
> >> from
> >>>>>>>>>>> benchmarks/config:
> >>>>>>>>>>>
> >>>>>>>>>>> ignite-base-load-config.xml
> >>>>>>>>>>> ignite-cache-load-config.xml
> >>>>>>>>>>> ignite-failover-base-config.xml
> >>>>>>>>>>> ignite-failover-localhost-config.xml
> >>>>>>>>>>> benchmark-cache-load.properties
> >>>>>>>>>>> benchmark-cache-load-win.properties
> >>>>>>>>>>> benchmark-failover.properties
> >>>>>>>>>>>
> >>>>>>>>>>> because those configs do not relate to any of performance
> tests.
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <
> dmagda@apache.org
> >>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Summarized the discussion updating the ticket
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> >>>>>>>>>>>>
> >>>>>>>>>>>> —
> >>>>>>>>>>>> Denis
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> >>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sergey,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
> >>>> folder
> >>>>>>>>> right
> >>>>>>>>>>>>> under the root folder.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> D.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
> >>>>>>>> skozlov@gridgain.com
> >>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Formatting has cut lines:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>>>>> — bin
> >>>>>>>>>>>>>> — examples
> >>>>>>>>>>>>>> — extras
> >>>>>>>>>>>>>> — benchmarks
> >>>>>>>>>>>>>> — bin
> >>>>>>>>>>>>>> — src (benchmarks sources with pom.xml)
> >>>>>>>>>>>>>> — config
> >>>>>>>>>>>>>> — libs (compiled benchmarks)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> >>>>>>>>> skozlov@gridgain.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Denis,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may include
> >> more
> >>>>>>>>>>>> things in
> >>>>>>>>>>>>>>> future than yardstick only. It's why I suggest something
> like
> >>>>>>>> that:
> >>>>>>>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>>>>>> — bin
> >>>>>>>>>>>>>>> — examples
> >>>>>>>>>>>>>>> — extras
> >>>>>>>>>>>>>>>   — benchmarks
> >>>>>>>>>>>>>>>       — bin
> >>>>>>>>>>>>>>>       — src (benchmarks sources with pom.xml)
> >>>>>>>>>>>>>>>       — config
> >>>>>>>>>>>>>>>       — libs (compiled benchmarks)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
> >>>> dmagda@apache.org
> >>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
> >>>>>> following
> >>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>>>>>>> — examples
> >>>>>>>>>>>>>>>> — bin
> >>>>>>>>>>>>>>>> — benchmarks
> >>>>>>>>>>>>>>>>   — bin
> >>>>>>>>>>>>>>>>   — src (benchmarks sources with pom.xml)
> >>>>>>>>>>>>>>>>   — config
> >>>>>>>>>>>>>>>>   — libs (compiled benchmarks)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
> >> previously?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
> >>>>>>>> skozlov@gridgain.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin,
> >> /config,
> >>>>>>>>> /libs)
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> creates work/logs directory under yardstick root.
> >>>>>>>> "libs/optional"
> >>>>>>>>> is
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> optional modules but in general we can't say that for
> >>>>>> Yardstick.
> >>>>>>>>>>>> Also
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> may break the current user understanding of
> "libs/optional"
> >>>>>>>>>>>> directory
> >>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>> place for additonal functionality activated by copying in
> >>>>>>>> "libs".
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder
> for
> >>>>>>>>> anything
> >>>>>>>>>>>>>>>>>> benchmark related. I am also against adding any
> yardstick
> >>>>>>>>> libraries
> >>>>>>>>>>>>>>>> without
> >>>>>>>>>>>>>>>>>> providing code.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> In my view, if the community wants to include benchmarks
> >> in
> >>>>>>>>>>>> releases,
> >>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
> >>>> everything
> >>>>>>>>>>>>>>>> benchmark
> >>>>>>>>>>>>>>>>>> related, from code to all the dependent libraries, and
> >>>>>>>>>>>> documentation
> >>>>>>>>>>>>>>>>>> instructions.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
> >>>>>>>> dmagda@apache.org>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra
> for
> >>>> me.
> >>>>>>>> Why
> >>>>>>>>>>>> do
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
> >>>> well?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
> >>>> “libs/optional”?
> >>>>>>>> If
> >>>>>>>>>>>>>>>> there’re
> >>>>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
> >>>>>> implementation.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> >>>>>>>>> skozlov@gridgain.com
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Hi
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> What's about to introduce the new root folder called
> >>>>>> 'extras'
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
> >>>>>>>> binaries?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
> >>>>>>>>> dmagda@apache.org
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Dmitriy,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
> >>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
> >> benchmarks
> >>>>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now
> he/she
> >>>>>> needs
> >>>>>>>>> to
> >>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
> >>>>>>>>> following
> >>>>>>>>>>>>>>>>>> lengthy
> >>>>>>>>>>>>>>>>>>>>> Yardstick guidance.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it
> >> should
> >>>> be
> >>>>>>>>>>>>>> possible
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script.
> >> The
> >>>>>>>>> script
> >>>>>>>>>>>>>>>> needs
> >>>>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
> >>>>>>>> “libs/optional”
> >>>>>>>>>>>>>>>> folder.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
> >>>>>>>> benchmark
> >>>>>>>>> or
> >>>>>>>>>>>>>>>> add a
> >>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
> >>>> guidance.
> >>>>>>>>> So,
> >>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite
> >> release.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> >>>>>>>>>>>>>>>> oostanin@gridgain.com
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty
> >> of
> >>>>>>>>> useful
> >>>>>>>>>>>>>>>>>>> Yardstick
> >>>>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
> >>>>>> performance.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs"
> folder
> >> at
> >>>>>>>> all?
> >>>>>>>>>>>> Do
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> really
> >>>>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes,
> then I
> >>>>>>>> would
> >>>>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
> >>>>>>>> benchmarks
> >>>>>>>>>>>>>>>> there.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy
> Setrakyan <
> >>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> >>>>>>>>>>>>>>>>>>> oostanin@gridgain.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
> >>>>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
> >>>> Benchmarking
> >>>>>>>>>>>>>>>>>>> Simplification
> >>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion
> about
> >>>>>>>>>>>>>>>>>> ignite-yardstick:
> >>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to
> put a
> >>>>>>>>> compiled
> >>>>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite
> binary
> >>>>>>>>>>>> assembly?
> >>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>> put
> >>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others
> optional
> >>>>>>>>>>>> libraries,
> >>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
> >>>>>>>> directory
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> put
> >>>>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
> >>>>>>>> solution.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Best regards
> >>>>>>>>>>>>>>>>>>>>>>>>> Oleg
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Ilya Suntsov
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
> >>
>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Oleg, thanks,

Could you share Ignite release binaries with your patch applied? You should be able to use TeamCity for that. Makes sense to see the new project structure and walk through final version of the instructions.

—
Denis

> On Feb 2, 2017, at 2:30 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> 
> Denis,
> 
> Yes, the same identical README.txt as in the pull request will be placed in
> both benchmarks binaries and Ignite Yardstick module.
> 
> I've also created two DEVNOTES.txt with related building instructions.
> 
> On Wed, Feb 1, 2017 at 10:31 PM, Denis Magda <dm...@apache.org> wrote:
> 
>> Anton,
>> 
>> I’m absolutely fine to make a single README if its instructions will be
>> identical for benchmarks binaries delivered in Ignite binary releases and
>> assembled with yardstick module directly. Oleg, is it feasible?
>> 
>> If to skip auto-generation idea then DEVNOTES split suggested by you is
>> the only option.
>> 
>> —
>> Denis
>> 
>>> On Feb 1, 2017, at 1:45 AM, Anton Vinogradov <av...@gridgain.com>
>> wrote:
>>> 
>>> Denis,
>>> 
>>> I don't like autogeneration.
>>> 
>>> In my view, we have to keep one README and ret rid of "Installation
>>> instructions" inside it.
>>> README should explain how to work with assembled yardstick. So, same
>> README
>>> will be uses at sources and inside release assembly.
>>> 
>>> Also we have to have to split DEVNOTES to DEVNOTES and
>> DEVNOTES.standalone,
>>> to explain how to assembly yardstick from sources.
>>> 
>>> Thoughts?
>>> 
>>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org> wrote:
>>> 
>>>> Oleg,
>>>> 
>>>> Thanks for the clarification.
>>>> 
>>>> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
>>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
>>>> instructions there that explain how to build and run benchmarks from
>>>> ‘modules/yardstick’. This existing files can refer to the sources and
>>>> compiled benchmarks that are in Ignite binary releases but this should
>> be a
>>>> couple of statements, no more.
>>>> 
>>>> As for the instructions related to the sources and binaries added to
>>>> Ignite binaries, preferably they need to be added to auto-generated
>>>> README.txt. *Anton*, is it feasible to do?
>>>> 
>>>> Finally, when apply the reviews notes please build and share Ignite
>>>> binaries with your patch. Want to see the new project structure and
>> final
>>>> version of the instructions. Presently I can’t merge your changes due to
>>>> some conflicts.
>>>> 
>>>> *Anton*, please review modifications in the build procedures.
>>>> 
>>>> —
>>>> Denis
>>>> 
>>>>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com>
>> wrote:
>>>>> 
>>>>> Hi Denis,
>>>>> 
>>>>> Yes, we have included Ignite Yardstick source files with its pom.xml in
>>>>> Ignite binary release. "Building from standalone sources" is the
>> building
>>>>> instruction for these source files. "Building from Ignite Sources" is
>> the
>>>>> instruction for building Ignite Yardstick from `modules/yardstick` in
>>>>> Ignite source files.
>>>>> 
>>>>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org>
>> wrote:
>>>>> 
>>>>>> Hi Oleg,
>>>>>> 
>>>>>> Great progress, thanks for keep driving this!
>>>>>> 
>>>>>> I’ve left some minor notes in GitHub’s pull-request. I have the
>>>> following
>>>>>> questions aside:
>>>>>> 
>>>>>> - What is the difference between "Building from standalone sources"
>> and
>>>>>> "Building from Ignite Sources"? In my understanding, a user downloads
>>>>>> Apache Ignite release that has all the sources locally.
>>>>>> 
>>>>>> - I do remember we planned to add the benchmarks sources in a form of
>> a
>>>>>> ready to be used project with its own pom.xml (similar to examples).
>> Did
>>>>>> you put this task off?
>>>>>> 
>>>>>> —
>>>>>> Denis
>>>>>> 
>>>>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
>>>> wrote:
>>>>>>> 
>>>>>>> Hi!
>>>>>>> 
>>>>>>> I've changed the README.txt and DEVNOTES.txt files. Also added a
>> simple
>>>>>>> config file for quick and easy start. Please take a look at them and
>>>> tell
>>>>>>> me what you think.
>>>>>>> 
>>>>>>> https://github.com/apache/ignite/pull/1471
>>>>>>> 
>>>>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <isuntsov@gridgain.com
>>> 
>>>>>> wrote:
>>>>>>> 
>>>>>>>> Denis,
>>>>>>>> 
>>>>>>>> I think we can remove all configs except:
>>>>>>>> 
>>>>>>>> benchmark-multicast.properties
>>>>>>>> 
>>>>>>>> benchmark.properties
>>>>>>>> 
>>>>>>>> ignite-base-config.xml
>>>>>>>> 
>>>>>>>> ignite-localhost-config.xml
>>>>>>>> 
>>>>>>>> ignite-multicast-config.xml
>>>>>>>> 
>>>>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
>>>>>>>> 
>>>>>>>>> I would have only those configs that are useful. Ilya Suntsov,
>> basing
>>>>>> on
>>>>>>>>> your experience, please suggest which configs makes sense to
>> include
>>>>>> into
>>>>>>>>> every Ignite release.
>>>>>>>>> 
>>>>>>>>> Oleg, also please note that community decided to include not only
>> the
>>>>>>>>> benchmarking binaries but the sources as well into every Apache
>>>> Ignite
>>>>>>>>> release. I’ve update the ticket before. Hope you followed the
>>>>>> discussion
>>>>>>>> ;)
>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>>>>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
>>>>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>>>>>>>>> 
>>>>>>>>> —
>>>>>>>>> Denis
>>>>>>>>> 
>>>>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> I mean removing those configs from binary assembly, not from
>>>>>>>> repository.
>>>>>>>>>> 
>>>>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
>>>> oostanin@gridgain.com>
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hello Igniters.
>>>>>>>>>>> I think it would be better to remove some configuration files
>> from
>>>>>>>>>>> benchmarks/config:
>>>>>>>>>>> 
>>>>>>>>>>> ignite-base-load-config.xml
>>>>>>>>>>> ignite-cache-load-config.xml
>>>>>>>>>>> ignite-failover-base-config.xml
>>>>>>>>>>> ignite-failover-localhost-config.xml
>>>>>>>>>>> benchmark-cache-load.properties
>>>>>>>>>>> benchmark-cache-load-win.properties
>>>>>>>>>>> benchmark-failover.properties
>>>>>>>>>>> 
>>>>>>>>>>> because those configs do not relate to any of performance tests.
>>>>>>>>>>> 
>>>>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dmagda@apache.org
>>> 
>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Summarized the discussion updating the ticket
>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>>>>>>>>>>>> 
>>>>>>>>>>>> —
>>>>>>>>>>>> Denis
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Sergey,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
>>>> folder
>>>>>>>>> right
>>>>>>>>>>>>> under the root folder.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> D.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>>>>>>>> skozlov@gridgain.com
>>>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Formatting has cut lines:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>> — examples
>>>>>>>>>>>>>> — extras
>>>>>>>>>>>>>> — benchmarks
>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>> — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>> — config
>>>>>>>>>>>>>> — libs (compiled benchmarks)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>>>>>>>>> skozlov@gridgain.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Denis,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may include
>> more
>>>>>>>>>>>> things in
>>>>>>>>>>>>>>> future than yardstick only. It's why I suggest something like
>>>>>>>> that:
>>>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>>> — examples
>>>>>>>>>>>>>>> — extras
>>>>>>>>>>>>>>>   — benchmarks
>>>>>>>>>>>>>>>       — bin
>>>>>>>>>>>>>>>       — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>>>       — config
>>>>>>>>>>>>>>>       — libs (compiled benchmarks)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
>>>> dmagda@apache.org
>>>>>>> 
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
>>>>>> following
>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>>>>> — examples
>>>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>>>> — benchmarks
>>>>>>>>>>>>>>>>   — bin
>>>>>>>>>>>>>>>>   — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>>>>   — config
>>>>>>>>>>>>>>>>   — libs (compiled benchmarks)
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
>> previously?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>>>>>>>> skozlov@gridgain.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin,
>> /config,
>>>>>>>>> /libs)
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> creates work/logs directory under yardstick root.
>>>>>>>> "libs/optional"
>>>>>>>>> is
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> optional modules but in general we can't say that for
>>>>>> Yardstick.
>>>>>>>>>>>> Also
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> may break the current user understanding of "libs/optional"
>>>>>>>>>>>> directory
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>> place for additonal functionality activated by copying in
>>>>>>>> "libs".
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder for
>>>>>>>>> anything
>>>>>>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
>>>>>>>>> libraries
>>>>>>>>>>>>>>>> without
>>>>>>>>>>>>>>>>>> providing code.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> In my view, if the community wants to include benchmarks
>> in
>>>>>>>>>>>> releases,
>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
>>>> everything
>>>>>>>>>>>>>>>> benchmark
>>>>>>>>>>>>>>>>>> related, from code to all the dependent libraries, and
>>>>>>>>>>>> documentation
>>>>>>>>>>>>>>>>>> instructions.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for
>>>> me.
>>>>>>>> Why
>>>>>>>>>>>> do
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
>>>> well?
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
>>>> “libs/optional”?
>>>>>>>> If
>>>>>>>>>>>>>>>> there’re
>>>>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
>>>>>> implementation.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
>>>>>>>>> skozlov@gridgain.com
>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> What's about to introduce the new root folder called
>>>>>> 'extras'
>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
>>>>>>>> binaries?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
>>>>>>>>> dmagda@apache.org
>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Dmitriy,
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
>> benchmarks
>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
>>>>>> needs
>>>>>>>>> to
>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
>>>>>>>>> following
>>>>>>>>>>>>>>>>>> lengthy
>>>>>>>>>>>>>>>>>>>>> Yardstick guidance.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it
>> should
>>>> be
>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script.
>> The
>>>>>>>>> script
>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
>>>>>>>> “libs/optional”
>>>>>>>>>>>>>>>> folder.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
>>>>>>>> benchmark
>>>>>>>>> or
>>>>>>>>>>>>>>>> add a
>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
>>>> guidance.
>>>>>>>>> So,
>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite
>> release.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>>>>>>>>>>>>>>>> oostanin@gridgain.com
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty
>> of
>>>>>>>>> useful
>>>>>>>>>>>>>>>>>>> Yardstick
>>>>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
>>>>>> performance.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder
>> at
>>>>>>>> all?
>>>>>>>>>>>> Do
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
>>>>>>>> would
>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
>>>>>>>> benchmarks
>>>>>>>>>>>>>>>> there.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>>>>>>>>>>>>>>>>>> oostanin@gridgain.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
>>>> Benchmarking
>>>>>>>>>>>>>>>>>>> Simplification
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>>>>>>>>>>>>>>>>>> ignite-yardstick:
>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
>>>>>>>>> compiled
>>>>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
>>>>>>>>>>>> assembly?
>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
>>>>>>>>>>>> libraries,
>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
>>>>>>>> directory
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
>>>>>>>> solution.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>>>>>>>>>>>>>> Oleg
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Ilya Suntsov
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

Denis,

Yes, the same identical README.txt as in the pull request will be placed in
both benchmarks binaries and Ignite Yardstick module.

I've also created two DEVNOTES.txt with related building instructions.

On Wed, Feb 1, 2017 at 10:31 PM, Denis Magda <dm...@apache.org> wrote:

> Anton,
>
> I’m absolutely fine to make a single README if its instructions will be
> identical for benchmarks binaries delivered in Ignite binary releases and
> assembled with yardstick module directly. Oleg, is it feasible?
>
> If to skip auto-generation idea then DEVNOTES split suggested by you is
> the only option.
>
> —
> Denis
>
> > On Feb 1, 2017, at 1:45 AM, Anton Vinogradov <av...@gridgain.com>
> wrote:
> >
> > Denis,
> >
> > I don't like autogeneration.
> >
> > In my view, we have to keep one README and ret rid of "Installation
> > instructions" inside it.
> > README should explain how to work with assembled yardstick. So, same
> README
> > will be uses at sources and inside release assembly.
> >
> > Also we have to have to split DEVNOTES to DEVNOTES and
> DEVNOTES.standalone,
> > to explain how to assembly yardstick from sources.
> >
> > Thoughts?
> >
> > On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org> wrote:
> >
> >> Oleg,
> >>
> >> Thanks for the clarification.
> >>
> >> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
> >> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
> >> instructions there that explain how to build and run benchmarks from
> >> ‘modules/yardstick’. This existing files can refer to the sources and
> >> compiled benchmarks that are in Ignite binary releases but this should
> be a
> >> couple of statements, no more.
> >>
> >> As for the instructions related to the sources and binaries added to
> >> Ignite binaries, preferably they need to be added to auto-generated
> >> README.txt. *Anton*, is it feasible to do?
> >>
> >> Finally, when apply the reviews notes please build and share Ignite
> >> binaries with your patch. Want to see the new project structure and
> final
> >> version of the instructions. Presently I can’t merge your changes due to
> >> some conflicts.
> >>
> >> *Anton*, please review modifications in the build procedures.
> >>
> >> —
> >> Denis
> >>
> >>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
> >>>
> >>> Hi Denis,
> >>>
> >>> Yes, we have included Ignite Yardstick source files with its pom.xml in
> >>> Ignite binary release. "Building from standalone sources" is the
> building
> >>> instruction for these source files. "Building from Ignite Sources" is
> the
> >>> instruction for building Ignite Yardstick from `modules/yardstick` in
> >>> Ignite source files.
> >>>
> >>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org>
> wrote:
> >>>
> >>>> Hi Oleg,
> >>>>
> >>>> Great progress, thanks for keep driving this!
> >>>>
> >>>> I’ve left some minor notes in GitHub’s pull-request. I have the
> >> following
> >>>> questions aside:
> >>>>
> >>>> - What is the difference between "Building from standalone sources"
> and
> >>>> "Building from Ignite Sources"? In my understanding, a user downloads
> >>>> Apache Ignite release that has all the sources locally.
> >>>>
> >>>> - I do remember we planned to add the benchmarks sources in a form of
> a
> >>>> ready to be used project with its own pom.xml (similar to examples).
> Did
> >>>> you put this task off?
> >>>>
> >>>> —
> >>>> Denis
> >>>>
> >>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
> >> wrote:
> >>>>>
> >>>>> Hi!
> >>>>>
> >>>>> I've changed the README.txt and DEVNOTES.txt files. Also added a
> simple
> >>>>> config file for quick and easy start. Please take a look at them and
> >> tell
> >>>>> me what you think.
> >>>>>
> >>>>> https://github.com/apache/ignite/pull/1471
> >>>>>
> >>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <isuntsov@gridgain.com
> >
> >>>> wrote:
> >>>>>
> >>>>>> Denis,
> >>>>>>
> >>>>>> I think we can remove all configs except:
> >>>>>>
> >>>>>> benchmark-multicast.properties
> >>>>>>
> >>>>>> benchmark.properties
> >>>>>>
> >>>>>> ignite-base-config.xml
> >>>>>>
> >>>>>> ignite-localhost-config.xml
> >>>>>>
> >>>>>> ignite-multicast-config.xml
> >>>>>>
> >>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
> >>>>>>
> >>>>>>> I would have only those configs that are useful. Ilya Suntsov,
> basing
> >>>> on
> >>>>>>> your experience, please suggest which configs makes sense to
> include
> >>>> into
> >>>>>>> every Ignite release.
> >>>>>>>
> >>>>>>> Oleg, also please note that community decided to include not only
> the
> >>>>>>> benchmarking binaries but the sources as well into every Apache
> >> Ignite
> >>>>>>> release. I’ve update the ticket before. Hope you followed the
> >>>> discussion
> >>>>>> ;)
> >>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
> >>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
> >>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
> >>>>>>>
> >>>>>>> —
> >>>>>>> Denis
> >>>>>>>
> >>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> I mean removing those configs from binary assembly, not from
> >>>>>> repository.
> >>>>>>>>
> >>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
> >> oostanin@gridgain.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hello Igniters.
> >>>>>>>>> I think it would be better to remove some configuration files
> from
> >>>>>>>>> benchmarks/config:
> >>>>>>>>>
> >>>>>>>>> ignite-base-load-config.xml
> >>>>>>>>> ignite-cache-load-config.xml
> >>>>>>>>> ignite-failover-base-config.xml
> >>>>>>>>> ignite-failover-localhost-config.xml
> >>>>>>>>> benchmark-cache-load.properties
> >>>>>>>>> benchmark-cache-load-win.properties
> >>>>>>>>> benchmark-failover.properties
> >>>>>>>>>
> >>>>>>>>> because those configs do not relate to any of performance tests.
> >>>>>>>>>
> >>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dmagda@apache.org
> >
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Summarized the discussion updating the ticket
> >>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> >>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> >>>>>>>>>>
> >>>>>>>>>> —
> >>>>>>>>>> Denis
> >>>>>>>>>>
> >>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> >>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Sergey,
> >>>>>>>>>>>
> >>>>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
> >> folder
> >>>>>>> right
> >>>>>>>>>>> under the root folder.
> >>>>>>>>>>>
> >>>>>>>>>>> D.
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
> >>>>>> skozlov@gridgain.com
> >>>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Formatting has cut lines:
> >>>>>>>>>>>>
> >>>>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>>> — bin
> >>>>>>>>>>>> — examples
> >>>>>>>>>>>> — extras
> >>>>>>>>>>>> — benchmarks
> >>>>>>>>>>>> — bin
> >>>>>>>>>>>> — src (benchmarks sources with pom.xml)
> >>>>>>>>>>>> — config
> >>>>>>>>>>>> — libs (compiled benchmarks)
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> >>>>>>> skozlov@gridgain.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Denis,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may include
> more
> >>>>>>>>>> things in
> >>>>>>>>>>>>> future than yardstick only. It's why I suggest something like
> >>>>>> that:
> >>>>>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>>>> — bin
> >>>>>>>>>>>>> — examples
> >>>>>>>>>>>>> — extras
> >>>>>>>>>>>>>    — benchmarks
> >>>>>>>>>>>>>        — bin
> >>>>>>>>>>>>>        — src (benchmarks sources with pom.xml)
> >>>>>>>>>>>>>        — config
> >>>>>>>>>>>>>        — libs (compiled benchmarks)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
> >> dmagda@apache.org
> >>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
> >>>> following
> >>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>>>>> — examples
> >>>>>>>>>>>>>> — bin
> >>>>>>>>>>>>>> — benchmarks
> >>>>>>>>>>>>>>    — bin
> >>>>>>>>>>>>>>    — src (benchmarks sources with pom.xml)
> >>>>>>>>>>>>>>    — config
> >>>>>>>>>>>>>>    — libs (compiled benchmarks)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
> previously?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> —
> >>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
> >>>>>> skozlov@gridgain.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin,
> /config,
> >>>>>>> /libs)
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>> creates work/logs directory under yardstick root.
> >>>>>> "libs/optional"
> >>>>>>> is
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>> optional modules but in general we can't say that for
> >>>> Yardstick.
> >>>>>>>>>> Also
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>> may break the current user understanding of "libs/optional"
> >>>>>>>>>> directory
> >>>>>>>>>>>> as
> >>>>>>>>>>>>>>> place for additonal functionality activated by copying in
> >>>>>> "libs".
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder for
> >>>>>>> anything
> >>>>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
> >>>>>>> libraries
> >>>>>>>>>>>>>> without
> >>>>>>>>>>>>>>>> providing code.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> In my view, if the community wants to include benchmarks
> in
> >>>>>>>>>> releases,
> >>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
> >> everything
> >>>>>>>>>>>>>> benchmark
> >>>>>>>>>>>>>>>> related, from code to all the dependent libraries, and
> >>>>>>>>>> documentation
> >>>>>>>>>>>>>>>> instructions.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
> >>>>>> dmagda@apache.org>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for
> >> me.
> >>>>>> Why
> >>>>>>>>>> do
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
> >> well?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
> >> “libs/optional”?
> >>>>>> If
> >>>>>>>>>>>>>> there’re
> >>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
> >>>> implementation.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> >>>>>>> skozlov@gridgain.com
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> What's about to introduce the new root folder called
> >>>> 'extras'
> >>>>>>>>>> with
> >>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
> >>>>>> binaries?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
> >>>>>>> dmagda@apache.org
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Dmitriy,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
> >>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
> benchmarks
> >>>>>>>>>>>> execution
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
> >>>> needs
> >>>>>>> to
> >>>>>>>>>> go
> >>>>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
> >>>>>>> following
> >>>>>>>>>>>>>>>> lengthy
> >>>>>>>>>>>>>>>>>>> Yardstick guidance.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it
> should
> >> be
> >>>>>>>>>>>> possible
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script.
> The
> >>>>>>> script
> >>>>>>>>>>>>>> needs
> >>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
> >>>>>> “libs/optional”
> >>>>>>>>>>>>>> folder.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
> >>>>>> benchmark
> >>>>>>> or
> >>>>>>>>>>>>>> add a
> >>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
> >> guidance.
> >>>>>>> So,
> >>>>>>>>>> no
> >>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite
> release.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> >>>>>>>>>>>>>> oostanin@gridgain.com
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty
> of
> >>>>>>> useful
> >>>>>>>>>>>>>>>>> Yardstick
> >>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
> >>>> performance.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder
> at
> >>>>>> all?
> >>>>>>>>>> Do
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> really
> >>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
> >>>>>> would
> >>>>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
> >>>>>> benchmarks
> >>>>>>>>>>>>>> there.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> >>>>>>>>>>>>>>>>> oostanin@gridgain.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
> >>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
> >> Benchmarking
> >>>>>>>>>>>>>>>>> Simplification
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
> >>>>>>>>>>>>>>>> ignite-yardstick:
> >>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
> >>>>>>> compiled
> >>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
> >>>>>>>>>> assembly?
> >>>>>>>>>>>> We
> >>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>> put
> >>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
> >>>>>>>>>> libraries,
> >>>>>>>>>>>> or
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
> >>>>>> directory
> >>>>>>>>>> and
> >>>>>>>>>>>>>> put
> >>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
> >>>>>> solution.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Best regards
> >>>>>>>>>>>>>>>>>>>>>>> Oleg
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Ilya Suntsov
> >>>>>>
> >>>>
> >>>>
> >>
> >>
>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Anton,

I’m absolutely fine to make a single README if its instructions will be identical for benchmarks binaries delivered in Ignite binary releases and assembled with yardstick module directly. Oleg, is it feasible?

If to skip auto-generation idea then DEVNOTES split suggested by you is the only option. 

—
Denis

> On Feb 1, 2017, at 1:45 AM, Anton Vinogradov <av...@gridgain.com> wrote:
> 
> Denis,
> 
> I don't like autogeneration.
> 
> In my view, we have to keep one README and ret rid of "Installation
> instructions" inside it.
> README should explain how to work with assembled yardstick. So, same README
> will be uses at sources and inside release assembly.
> 
> Also we have to have to split DEVNOTES to DEVNOTES and DEVNOTES.standalone,
> to explain how to assembly yardstick from sources.
> 
> Thoughts?
> 
> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org> wrote:
> 
>> Oleg,
>> 
>> Thanks for the clarification.
>> 
>> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
>> instructions there that explain how to build and run benchmarks from
>> ‘modules/yardstick’. This existing files can refer to the sources and
>> compiled benchmarks that are in Ignite binary releases but this should be a
>> couple of statements, no more.
>> 
>> As for the instructions related to the sources and binaries added to
>> Ignite binaries, preferably they need to be added to auto-generated
>> README.txt. *Anton*, is it feasible to do?
>> 
>> Finally, when apply the reviews notes please build and share Ignite
>> binaries with your patch. Want to see the new project structure and final
>> version of the instructions. Presently I can’t merge your changes due to
>> some conflicts.
>> 
>> *Anton*, please review modifications in the build procedures.
>> 
>> —
>> Denis
>> 
>>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
>>> 
>>> Hi Denis,
>>> 
>>> Yes, we have included Ignite Yardstick source files with its pom.xml in
>>> Ignite binary release. "Building from standalone sources" is the building
>>> instruction for these source files. "Building from Ignite Sources" is the
>>> instruction for building Ignite Yardstick from `modules/yardstick` in
>>> Ignite source files.
>>> 
>>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org> wrote:
>>> 
>>>> Hi Oleg,
>>>> 
>>>> Great progress, thanks for keep driving this!
>>>> 
>>>> I’ve left some minor notes in GitHub’s pull-request. I have the
>> following
>>>> questions aside:
>>>> 
>>>> - What is the difference between "Building from standalone sources" and
>>>> "Building from Ignite Sources"? In my understanding, a user downloads
>>>> Apache Ignite release that has all the sources locally.
>>>> 
>>>> - I do remember we planned to add the benchmarks sources in a form of a
>>>> ready to be used project with its own pom.xml (similar to examples). Did
>>>> you put this task off?
>>>> 
>>>> —
>>>> Denis
>>>> 
>>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
>> wrote:
>>>>> 
>>>>> Hi!
>>>>> 
>>>>> I've changed the README.txt and DEVNOTES.txt files. Also added a simple
>>>>> config file for quick and easy start. Please take a look at them and
>> tell
>>>>> me what you think.
>>>>> 
>>>>> https://github.com/apache/ignite/pull/1471
>>>>> 
>>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <is...@gridgain.com>
>>>> wrote:
>>>>> 
>>>>>> Denis,
>>>>>> 
>>>>>> I think we can remove all configs except:
>>>>>> 
>>>>>> benchmark-multicast.properties
>>>>>> 
>>>>>> benchmark.properties
>>>>>> 
>>>>>> ignite-base-config.xml
>>>>>> 
>>>>>> ignite-localhost-config.xml
>>>>>> 
>>>>>> ignite-multicast-config.xml
>>>>>> 
>>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
>>>>>> 
>>>>>>> I would have only those configs that are useful. Ilya Suntsov, basing
>>>> on
>>>>>>> your experience, please suggest which configs makes sense to include
>>>> into
>>>>>>> every Ignite release.
>>>>>>> 
>>>>>>> Oleg, also please note that community decided to include not only the
>>>>>>> benchmarking binaries but the sources as well into every Apache
>> Ignite
>>>>>>> release. I’ve update the ticket before. Hope you followed the
>>>> discussion
>>>>>> ;)
>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
>>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>>>>>>> 
>>>>>>> —
>>>>>>> Denis
>>>>>>> 
>>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
>>>>>> wrote:
>>>>>>>> 
>>>>>>>> I mean removing those configs from binary assembly, not from
>>>>>> repository.
>>>>>>>> 
>>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
>> oostanin@gridgain.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hello Igniters.
>>>>>>>>> I think it would be better to remove some configuration files from
>>>>>>>>> benchmarks/config:
>>>>>>>>> 
>>>>>>>>> ignite-base-load-config.xml
>>>>>>>>> ignite-cache-load-config.xml
>>>>>>>>> ignite-failover-base-config.xml
>>>>>>>>> ignite-failover-localhost-config.xml
>>>>>>>>> benchmark-cache-load.properties
>>>>>>>>> benchmark-cache-load-win.properties
>>>>>>>>> benchmark-failover.properties
>>>>>>>>> 
>>>>>>>>> because those configs do not relate to any of performance tests.
>>>>>>>>> 
>>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org>
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Summarized the discussion updating the ticket
>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>>>>>>>>>> 
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>> 
>>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Sergey,
>>>>>>>>>>> 
>>>>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
>> folder
>>>>>>> right
>>>>>>>>>>> under the root folder.
>>>>>>>>>>> 
>>>>>>>>>>> D.
>>>>>>>>>>> 
>>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>>>>>> skozlov@gridgain.com
>>>>>>>> 
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Formatting has cut lines:
>>>>>>>>>>>> 
>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>> — bin
>>>>>>>>>>>> — examples
>>>>>>>>>>>> — extras
>>>>>>>>>>>> — benchmarks
>>>>>>>>>>>> — bin
>>>>>>>>>>>> — src (benchmarks sources with pom.xml)
>>>>>>>>>>>> — config
>>>>>>>>>>>> — libs (compiled benchmarks)
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>>>>>>> skozlov@gridgain.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Denis,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may include more
>>>>>>>>>> things in
>>>>>>>>>>>>> future than yardstick only. It's why I suggest something like
>>>>>> that:
>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>> — bin
>>>>>>>>>>>>> — examples
>>>>>>>>>>>>> — extras
>>>>>>>>>>>>>    — benchmarks
>>>>>>>>>>>>>        — bin
>>>>>>>>>>>>>        — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>        — config
>>>>>>>>>>>>>        — libs (compiled benchmarks)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
>> dmagda@apache.org
>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
>>>> following
>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>>> — examples
>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>> — benchmarks
>>>>>>>>>>>>>>    — bin
>>>>>>>>>>>>>>    — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>>    — config
>>>>>>>>>>>>>>    — libs (compiled benchmarks)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met previously?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> —
>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>>>>>> skozlov@gridgain.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin, /config,
>>>>>>> /libs)
>>>>>>>>>>>> and
>>>>>>>>>>>>>>> creates work/logs directory under yardstick root.
>>>>>> "libs/optional"
>>>>>>> is
>>>>>>>>>>>> for
>>>>>>>>>>>>>>> optional modules but in general we can't say that for
>>>> Yardstick.
>>>>>>>>>> Also
>>>>>>>>>>>> it
>>>>>>>>>>>>>>> may break the current user understanding of "libs/optional"
>>>>>>>>>> directory
>>>>>>>>>>>> as
>>>>>>>>>>>>>>> place for additonal functionality activated by copying in
>>>>>> "libs".
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder for
>>>>>>> anything
>>>>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
>>>>>>> libraries
>>>>>>>>>>>>>> without
>>>>>>>>>>>>>>>> providing code.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> In my view, if the community wants to include benchmarks in
>>>>>>>>>> releases,
>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
>> everything
>>>>>>>>>>>>>> benchmark
>>>>>>>>>>>>>>>> related, from code to all the dependent libraries, and
>>>>>>>>>> documentation
>>>>>>>>>>>>>>>> instructions.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for
>> me.
>>>>>> Why
>>>>>>>>>> do
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
>> well?
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
>> “libs/optional”?
>>>>>> If
>>>>>>>>>>>>>> there’re
>>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
>>>> implementation.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
>>>>>>> skozlov@gridgain.com
>>>>>>>>>>> 
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> What's about to introduce the new root folder called
>>>> 'extras'
>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
>>>>>> binaries?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
>>>>>>> dmagda@apache.org
>>>>>>>>>>> 
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Dmitriy,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate benchmarks
>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
>>>> needs
>>>>>>> to
>>>>>>>>>> go
>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
>>>>>>> following
>>>>>>>>>>>>>>>> lengthy
>>>>>>>>>>>>>>>>>>> Yardstick guidance.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it should
>> be
>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The
>>>>>>> script
>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
>>>>>> “libs/optional”
>>>>>>>>>>>>>> folder.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
>>>>>> benchmark
>>>>>>> or
>>>>>>>>>>>>>> add a
>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
>> guidance.
>>>>>>> So,
>>>>>>>>>> no
>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite release.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>>>>>>>>>>>>>> oostanin@gridgain.com
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of
>>>>>>> useful
>>>>>>>>>>>>>>>>> Yardstick
>>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
>>>> performance.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder at
>>>>>> all?
>>>>>>>>>> Do
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
>>>>>> would
>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
>>>>>> benchmarks
>>>>>>>>>>>>>> there.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>>>>>>>>>>>>>>>> oostanin@gridgain.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
>> Benchmarking
>>>>>>>>>>>>>>>>> Simplification
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>>>>>>>>>>>>>>>> ignite-yardstick:
>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
>>>>>>> compiled
>>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
>>>>>>>>>> assembly?
>>>>>>>>>>>> We
>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
>>>>>>>>>> libraries,
>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
>>>>>> directory
>>>>>>>>>> and
>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
>>>>>> solution.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>>>>>>>>>>>> Oleg
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Ilya Suntsov
>>>>>> 
>>>> 
>>>> 
>> 
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Anton Vinogradov <av...@gridgain.com>.

Denis,

I don't like autogeneration.

In my view, we have to keep one README and ret rid of "Installation
instructions" inside it.
README should explain how to work with assembled yardstick. So, same README
will be uses at sources and inside release assembly.

Also we have to have to split DEVNOTES to DEVNOTES and DEVNOTES.standalone,
to explain how to assembly yardstick from sources.

Thoughts?

On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org> wrote:

> Oleg,
>
> Thanks for the clarification.
>
> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
> instructions there that explain how to build and run benchmarks from
> ‘modules/yardstick’. This existing files can refer to the sources and
> compiled benchmarks that are in Ignite binary releases but this should be a
> couple of statements, no more.
>
> As for the instructions related to the sources and binaries added to
> Ignite binaries, preferably they need to be added to auto-generated
> README.txt. *Anton*, is it feasible to do?
>
> Finally, when apply the reviews notes please build and share Ignite
> binaries with your patch. Want to see the new project structure and final
> version of the instructions. Presently I can’t merge your changes due to
> some conflicts.
>
> *Anton*, please review modifications in the build procedures.
>
> —
> Denis
>
> > On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> >
> > Hi Denis,
> >
> > Yes, we have included Ignite Yardstick source files with its pom.xml in
> > Ignite binary release. "Building from standalone sources" is the building
> > instruction for these source files. "Building from Ignite Sources" is the
> > instruction for building Ignite Yardstick from `modules/yardstick` in
> > Ignite source files.
> >
> > On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org> wrote:
> >
> >> Hi Oleg,
> >>
> >> Great progress, thanks for keep driving this!
> >>
> >> I’ve left some minor notes in GitHub’s pull-request. I have the
> following
> >> questions aside:
> >>
> >> - What is the difference between "Building from standalone sources" and
> >> "Building from Ignite Sources"? In my understanding, a user downloads
> >> Apache Ignite release that has all the sources locally.
> >>
> >> - I do remember we planned to add the benchmarks sources in a form of a
> >> ready to be used project with its own pom.xml (similar to examples). Did
> >> you put this task off?
> >>
> >> —
> >> Denis
> >>
> >>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
> >>>
> >>> Hi!
> >>>
> >>> I've changed the README.txt and DEVNOTES.txt files. Also added a simple
> >>> config file for quick and easy start. Please take a look at them and
> tell
> >>> me what you think.
> >>>
> >>> https://github.com/apache/ignite/pull/1471
> >>>
> >>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <is...@gridgain.com>
> >> wrote:
> >>>
> >>>> Denis,
> >>>>
> >>>> I think we can remove all configs except:
> >>>>
> >>>> benchmark-multicast.properties
> >>>>
> >>>> benchmark.properties
> >>>>
> >>>> ignite-base-config.xml
> >>>>
> >>>> ignite-localhost-config.xml
> >>>>
> >>>> ignite-multicast-config.xml
> >>>>
> >>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
> >>>>
> >>>>> I would have only those configs that are useful. Ilya Suntsov, basing
> >> on
> >>>>> your experience, please suggest which configs makes sense to include
> >> into
> >>>>> every Ignite release.
> >>>>>
> >>>>> Oleg, also please note that community decided to include not only the
> >>>>> benchmarking binaries but the sources as well into every Apache
> Ignite
> >>>>> release. I’ve update the ticket before. Hope you followed the
> >> discussion
> >>>> ;)
> >>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
> >>>>> focusedCommentId=15765151&page=com.atlassian.jira.
> >>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
> >>>>>
> >>>>> —
> >>>>> Denis
> >>>>>
> >>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
> >>>> wrote:
> >>>>>>
> >>>>>> I mean removing those configs from binary assembly, not from
> >>>> repository.
> >>>>>>
> >>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
> oostanin@gridgain.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Hello Igniters.
> >>>>>>> I think it would be better to remove some configuration files from
> >>>>>>> benchmarks/config:
> >>>>>>>
> >>>>>>> ignite-base-load-config.xml
> >>>>>>> ignite-cache-load-config.xml
> >>>>>>> ignite-failover-base-config.xml
> >>>>>>> ignite-failover-localhost-config.xml
> >>>>>>> benchmark-cache-load.properties
> >>>>>>> benchmark-cache-load-win.properties
> >>>>>>> benchmark-failover.properties
> >>>>>>>
> >>>>>>> because those configs do not relate to any of performance tests.
> >>>>>>>
> >>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Summarized the discussion updating the ticket
> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> >>>>>>>>
> >>>>>>>> —
> >>>>>>>> Denis
> >>>>>>>>
> >>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> >>>>> dsetrakyan@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Sergey,
> >>>>>>>>>
> >>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
> folder
> >>>>> right
> >>>>>>>>> under the root folder.
> >>>>>>>>>
> >>>>>>>>> D.
> >>>>>>>>>
> >>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
> >>>> skozlov@gridgain.com
> >>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Formatting has cut lines:
> >>>>>>>>>>
> >>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>> — bin
> >>>>>>>>>> — examples
> >>>>>>>>>> — extras
> >>>>>>>>>> — benchmarks
> >>>>>>>>>>  — bin
> >>>>>>>>>>  — src (benchmarks sources with pom.xml)
> >>>>>>>>>>  — config
> >>>>>>>>>>  — libs (compiled benchmarks)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> >>>>> skozlov@gridgain.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Denis,
> >>>>>>>>>>>
> >>>>>>>>>>> Mostly yes. But I look ahead and think that we may include more
> >>>>>>>> things in
> >>>>>>>>>>> future than yardstick only. It's why I suggest something like
> >>>> that:
> >>>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>> — bin
> >>>>>>>>>>> — examples
> >>>>>>>>>>> — extras
> >>>>>>>>>>>     — benchmarks
> >>>>>>>>>>>         — bin
> >>>>>>>>>>>         — src (benchmarks sources with pom.xml)
> >>>>>>>>>>>         — config
> >>>>>>>>>>>         — libs (compiled benchmarks)
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
> dmagda@apache.org
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
> >> following
> >>>>>>>>>>>> structure
> >>>>>>>>>>>>
> >>>>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>>> — examples
> >>>>>>>>>>>> — bin
> >>>>>>>>>>>> — benchmarks
> >>>>>>>>>>>>     — bin
> >>>>>>>>>>>>     — src (benchmarks sources with pom.xml)
> >>>>>>>>>>>>     — config
> >>>>>>>>>>>>     — libs (compiled benchmarks)
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sergey, will it cover all the use case you’ve met previously?
> >>>>>>>>>>>>
> >>>>>>>>>>>> —
> >>>>>>>>>>>> Denis
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
> >>>> skozlov@gridgain.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin, /config,
> >>>>> /libs)
> >>>>>>>>>> and
> >>>>>>>>>>>>> creates work/logs directory under yardstick root.
> >>>> "libs/optional"
> >>>>> is
> >>>>>>>>>> for
> >>>>>>>>>>>>> optional modules but in general we can't say that for
> >> Yardstick.
> >>>>>>>> Also
> >>>>>>>>>> it
> >>>>>>>>>>>>> may break the current user understanding of "libs/optional"
> >>>>>>>> directory
> >>>>>>>>>> as
> >>>>>>>>>>>>> place for additonal functionality activated by copying in
> >>>> "libs".
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder for
> >>>>> anything
> >>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
> >>>>> libraries
> >>>>>>>>>>>> without
> >>>>>>>>>>>>>> providing code.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> In my view, if the community wants to include benchmarks in
> >>>>>>>> releases,
> >>>>>>>>>>>> then
> >>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
> everything
> >>>>>>>>>>>> benchmark
> >>>>>>>>>>>>>> related, from code to all the dependent libraries, and
> >>>>>>>> documentation
> >>>>>>>>>>>>>> instructions.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
> >>>> dmagda@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for
> me.
> >>>> Why
> >>>>>>>> do
> >>>>>>>>>> we
> >>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
> well?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Is there anyone else who is concerned about
> “libs/optional”?
> >>>> If
> >>>>>>>>>>>> there’re
> >>>>>>>>>>>>>>> not, I would agree on this and get down to the
> >> implementation.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> >>>>> skozlov@gridgain.com
> >>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> What's about to introduce the new root folder called
> >> 'extras'
> >>>>>>>> with
> >>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
> >>>> binaries?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
> >>>>> dmagda@apache.org
> >>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Dmitriy,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
> >>>>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate benchmarks
> >>>>>>>>>> execution
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
> >> needs
> >>>>> to
> >>>>>>>> go
> >>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
> >>>>> following
> >>>>>>>>>>>>>> lengthy
> >>>>>>>>>>>>>>>>> Yardstick guidance.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it should
> be
> >>>>>>>>>> possible
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The
> >>>>> script
> >>>>>>>>>>>> needs
> >>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
> >>>> “libs/optional”
> >>>>>>>>>>>> folder.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
> >>>> benchmark
> >>>>> or
> >>>>>>>>>>>> add a
> >>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
> guidance.
> >>>>> So,
> >>>>>>>> no
> >>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite release.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> >>>>>>>>>>>> oostanin@gridgain.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of
> >>>>> useful
> >>>>>>>>>>>>>>> Yardstick
> >>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
> >> performance.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder at
> >>>> all?
> >>>>>>>> Do
> >>>>>>>>>> we
> >>>>>>>>>>>>>>>>> really
> >>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
> >>>> would
> >>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
> >>>> benchmarks
> >>>>>>>>>>>> there.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> >>>>>>>>>>>>>>> oostanin@gridgain.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hello Igniters!
> >>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
> Benchmarking
> >>>>>>>>>>>>>>> Simplification
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
> >>>>>>>>>>>>>> ignite-yardstick:
> >>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
> >>>>> compiled
> >>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
> >>>>>>>> assembly?
> >>>>>>>>>> We
> >>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>> put
> >>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
> >>>>>>>> libraries,
> >>>>>>>>>> or
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
> >>>> directory
> >>>>>>>> and
> >>>>>>>>>>>> put
> >>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
> >>>> solution.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Best regards
> >>>>>>>>>>>>>>>>>>>>> Oleg
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>> GridGain Systems
> >>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Sergey Kozlov
> >>>>>>>>>> GridGain Systems
> >>>>>>>>>> www.gridgain.com
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Ilya Suntsov
> >>>>
> >>
> >>
>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Oleg,

Thanks a lot, we’re almost done. Please address the latest comments from my side.
https://issues.apache.org/jira/browse/IGNITE-4212 <https://issues.apache.org/jira/browse/IGNITE-4212>

Sergey K., please take over from this point trying to use the feature created by Oleg for Ignite 1.9 benchmarking purposes.
If you’re happy with it then Oleg is free to merge the changes into 1.9 and release them there.

—
Denis

> On Feb 10, 2017, at 2:50 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> 
> I'm sorry, there wasn't any problem with the build, it was local problem on
> my machine.
> 
> On Fri, Feb 10, 2017 at 11:54 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
> 
>> Hi.
>> 
>> I just discovered that some functionality does not work correctly. I will
>> present fixed version later.
>> 
>> On Fri, Feb 10, 2017 at 12:43 AM, Oleg Ostanin <oo...@gridgain.com>
>> wrote:
>> 
>>> https://drive.google.com/open?id=0B9teTzJ9iIvHckl6eVQ3N0R4a3c
>>> 
>>> On Fri, Feb 10, 2017 at 12:43 AM, Oleg Ostanin <oo...@gridgain.com>
>>> wrote:
>>> 
>>>> I've fixed README.txt and created new DEVNOTES.txt. Also renamed sample
>>>> benchmark configuration file to benchmark-sample.properties.
>>>> Please look for the results:
>>>> 
>>>> 
>>>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org> wrote:
>>>> 
>>>>> Oleg,
>>>>> 
>>>>> Thanks for the clarification.
>>>>> 
>>>>> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
>>>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
>>>>> instructions there that explain how to build and run benchmarks from
>>>>> ‘modules/yardstick’. This existing files can refer to the sources and
>>>>> compiled benchmarks that are in Ignite binary releases but this should be a
>>>>> couple of statements, no more.
>>>>> 
>>>>> As for the instructions related to the sources and binaries added to
>>>>> Ignite binaries, preferably they need to be added to auto-generated
>>>>> README.txt. *Anton*, is it feasible to do?
>>>>> 
>>>>> Finally, when apply the reviews notes please build and share Ignite
>>>>> binaries with your patch. Want to see the new project structure and final
>>>>> version of the instructions. Presently I can’t merge your changes due to
>>>>> some conflicts.
>>>>> 
>>>>> *Anton*, please review modifications in the build procedures.
>>>>> 
>>>>> —
>>>>> Denis
>>>>> 
>>>>>> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com>
>>>>> wrote:
>>>>>> 
>>>>>> Hi Denis,
>>>>>> 
>>>>>> Yes, we have included Ignite Yardstick source files with its pom.xml
>>>>> in
>>>>>> Ignite binary release. "Building from standalone sources" is the
>>>>> building
>>>>>> instruction for these source files. "Building from Ignite Sources" is
>>>>> the
>>>>>> instruction for building Ignite Yardstick from `modules/yardstick` in
>>>>>> Ignite source files.
>>>>>> 
>>>>>> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org>
>>>>> wrote:
>>>>>> 
>>>>>>> Hi Oleg,
>>>>>>> 
>>>>>>> Great progress, thanks for keep driving this!
>>>>>>> 
>>>>>>> I’ve left some minor notes in GitHub’s pull-request. I have the
>>>>> following
>>>>>>> questions aside:
>>>>>>> 
>>>>>>> - What is the difference between "Building from standalone sources"
>>>>> and
>>>>>>> "Building from Ignite Sources"? In my understanding, a user downloads
>>>>>>> Apache Ignite release that has all the sources locally.
>>>>>>> 
>>>>>>> - I do remember we planned to add the benchmarks sources in a form
>>>>> of a
>>>>>>> ready to be used project with its own pom.xml (similar to examples).
>>>>> Did
>>>>>>> you put this task off?
>>>>>>> 
>>>>>>> —
>>>>>>> Denis
>>>>>>> 
>>>>>>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hi!
>>>>>>>> 
>>>>>>>> I've changed the README.txt and DEVNOTES.txt files. Also added a
>>>>> simple
>>>>>>>> config file for quick and easy start. Please take a look at them
>>>>> and tell
>>>>>>>> me what you think.
>>>>>>>> 
>>>>>>>> https://github.com/apache/ignite/pull/1471
>>>>>>>> 
>>>>>>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <
>>>>> isuntsov@gridgain.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Denis,
>>>>>>>>> 
>>>>>>>>> I think we can remove all configs except:
>>>>>>>>> 
>>>>>>>>> benchmark-multicast.properties
>>>>>>>>> 
>>>>>>>>> benchmark.properties
>>>>>>>>> 
>>>>>>>>> ignite-base-config.xml
>>>>>>>>> 
>>>>>>>>> ignite-localhost-config.xml
>>>>>>>>> 
>>>>>>>>> ignite-multicast-config.xml
>>>>>>>>> 
>>>>>>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
>>>>>>>>> 
>>>>>>>>>> I would have only those configs that are useful. Ilya Suntsov,
>>>>> basing
>>>>>>> on
>>>>>>>>>> your experience, please suggest which configs makes sense to
>>>>> include
>>>>>>> into
>>>>>>>>>> every Ignite release.
>>>>>>>>>> 
>>>>>>>>>> Oleg, also please note that community decided to include not only
>>>>> the
>>>>>>>>>> benchmarking binaries but the sources as well into every Apache
>>>>> Ignite
>>>>>>>>>> release. I’ve update the ticket before. Hope you followed the
>>>>>>> discussion
>>>>>>>>> ;)
>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>>>>>>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
>>>>>>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>>>>>>>>>> 
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>> 
>>>>>>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oostanin@gridgain.com
>>>>>> 
>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> I mean removing those configs from binary assembly, not from
>>>>>>>>> repository.
>>>>>>>>>>> 
>>>>>>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
>>>>> oostanin@gridgain.com>
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Hello Igniters.
>>>>>>>>>>>> I think it would be better to remove some configuration files
>>>>> from
>>>>>>>>>>>> benchmarks/config:
>>>>>>>>>>>> 
>>>>>>>>>>>> ignite-base-load-config.xml
>>>>>>>>>>>> ignite-cache-load-config.xml
>>>>>>>>>>>> ignite-failover-base-config.xml
>>>>>>>>>>>> ignite-failover-localhost-config.xml
>>>>>>>>>>>> benchmark-cache-load.properties
>>>>>>>>>>>> benchmark-cache-load-win.properties
>>>>>>>>>>>> benchmark-failover.properties
>>>>>>>>>>>> 
>>>>>>>>>>>> because those configs do not relate to any of performance tests.
>>>>>>>>>>>> 
>>>>>>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <
>>>>> dmagda@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Summarized the discussion updating the ticket
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>>>>>>>>>>>>> 
>>>>>>>>>>>>> —
>>>>>>>>>>>>> Denis
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Sergey,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
>>>>> folder
>>>>>>>>>> right
>>>>>>>>>>>>>> under the root folder.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>>>>>>>>> skozlov@gridgain.com
>>>>>>>>>>> 
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Formatting has cut lines:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>>> — examples
>>>>>>>>>>>>>>> — extras
>>>>>>>>>>>>>>> — benchmarks
>>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>>> — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>>> — config
>>>>>>>>>>>>>>> — libs (compiled benchmarks)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>>>>>>>>>> skozlov@gridgain.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Denis,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Mostly yes. But I look ahead and think that we may include
>>>>> more
>>>>>>>>>>>>> things in
>>>>>>>>>>>>>>>> future than yardstick only. It's why I suggest something
>>>>> like
>>>>>>>>> that:
>>>>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>>>> — examples
>>>>>>>>>>>>>>>> — extras
>>>>>>>>>>>>>>>>    — benchmarks
>>>>>>>>>>>>>>>>        — bin
>>>>>>>>>>>>>>>>        — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>>>>        — config
>>>>>>>>>>>>>>>>        — libs (compiled benchmarks)
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
>>>>> dmagda@apache.org
>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
>>>>>>> following
>>>>>>>>>>>>>>>>> structure
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>>>>>>> — examples
>>>>>>>>>>>>>>>>> — bin
>>>>>>>>>>>>>>>>> — benchmarks
>>>>>>>>>>>>>>>>>    — bin
>>>>>>>>>>>>>>>>>    — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>>>>>>    — config
>>>>>>>>>>>>>>>>>    — libs (compiled benchmarks)
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
>>>>> previously?
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>>>>>>>>> skozlov@gridgain.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin,
>>>>> /config,
>>>>>>>>>> /libs)
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> creates work/logs directory under yardstick root.
>>>>>>>>> "libs/optional"
>>>>>>>>>> is
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> optional modules but in general we can't say that for
>>>>>>> Yardstick.
>>>>>>>>>>>>> Also
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> may break the current user understanding of
>>>>> "libs/optional"
>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>> place for additonal functionality activated by copying in
>>>>>>>>> "libs".
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder
>>>>> for
>>>>>>>>>> anything
>>>>>>>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
>>>>>>>>>> libraries
>>>>>>>>>>>>>>>>> without
>>>>>>>>>>>>>>>>>>> providing code.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> In my view, if the community wants to include benchmarks
>>>>> in
>>>>>>>>>>>>> releases,
>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
>>>>> everything
>>>>>>>>>>>>>>>>> benchmark
>>>>>>>>>>>>>>>>>>> related, from code to all the dependent libraries, and
>>>>>>>>>>>>> documentation
>>>>>>>>>>>>>>>>>>> instructions.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra
>>>>> for me.
>>>>>>>>> Why
>>>>>>>>>>>>> do
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
>>>>> well?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Is there anyone else who is concerned about
>>>>> “libs/optional”?
>>>>>>>>> If
>>>>>>>>>>>>>>>>> there’re
>>>>>>>>>>>>>>>>>>>> not, I would agree on this and get down to the
>>>>>>> implementation.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
>>>>>>>>>> skozlov@gridgain.com
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> What's about to introduce the new root folder called
>>>>>>> 'extras'
>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
>>>>>>>>> binaries?
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
>>>>>>>>>> dmagda@apache.org
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Dmitriy,
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
>>>>> benchmarks
>>>>>>>>>>>>>>> execution
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
>>>>>>> needs
>>>>>>>>>> to
>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
>>>>>>>>>> following
>>>>>>>>>>>>>>>>>>> lengthy
>>>>>>>>>>>>>>>>>>>>>> Yardstick guidance.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it
>>>>> should be
>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script.
>>>>> The
>>>>>>>>>> script
>>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
>>>>>>>>> “libs/optional”
>>>>>>>>>>>>>>>>> folder.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
>>>>>>>>> benchmark
>>>>>>>>>> or
>>>>>>>>>>>>>>>>> add a
>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
>>>>> guidance.
>>>>>>>>>> So,
>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite
>>>>> release.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>>>>>>>>>>>>>>>>> oostanin@gridgain.com
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty
>>>>> of
>>>>>>>>>> useful
>>>>>>>>>>>>>>>>>>>> Yardstick
>>>>>>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
>>>>>>> performance.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs"
>>>>> folder at
>>>>>>>>> all?
>>>>>>>>>>>>> Do
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then
>>>>> I
>>>>>>>>> would
>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
>>>>>>>>> benchmarks
>>>>>>>>>>>>>>>>> there.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan
>>>>> <
>>>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>>>>>>>>>>>>>>>>>>> oostanin@gridgain.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
>>>>> Benchmarking
>>>>>>>>>>>>>>>>>>>> Simplification
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>>>>>>>>>>>>>>>>>>> ignite-yardstick:
>>>>>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put
>>>>> a
>>>>>>>>>> compiled
>>>>>>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite
>>>>> binary
>>>>>>>>>>>>> assembly?
>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
>>>>>>>>>>>>> libraries,
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
>>>>>>>>> directory
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
>>>>>>>>> solution.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>>>>>>>>>>>>>>> Oleg
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Ilya Suntsov
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

I'm sorry, there wasn't any problem with the build, it was local problem on
my machine.

On Fri, Feb 10, 2017 at 11:54 AM, Oleg Ostanin <oo...@gridgain.com>
wrote:

> Hi.
>
> I just discovered that some functionality does not work correctly. I will
> present fixed version later.
>
> On Fri, Feb 10, 2017 at 12:43 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
>
>> https://drive.google.com/open?id=0B9teTzJ9iIvHckl6eVQ3N0R4a3c
>>
>> On Fri, Feb 10, 2017 at 12:43 AM, Oleg Ostanin <oo...@gridgain.com>
>> wrote:
>>
>>> I've fixed README.txt and created new DEVNOTES.txt. Also renamed sample
>>> benchmark configuration file to benchmark-sample.properties.
>>> Please look for the results:
>>>
>>>
>>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org> wrote:
>>>
>>>> Oleg,
>>>>
>>>> Thanks for the clarification.
>>>>
>>>> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
>>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
>>>> instructions there that explain how to build and run benchmarks from
>>>> ‘modules/yardstick’. This existing files can refer to the sources and
>>>> compiled benchmarks that are in Ignite binary releases but this should be a
>>>> couple of statements, no more.
>>>>
>>>> As for the instructions related to the sources and binaries added to
>>>> Ignite binaries, preferably they need to be added to auto-generated
>>>> README.txt. *Anton*, is it feasible to do?
>>>>
>>>> Finally, when apply the reviews notes please build and share Ignite
>>>> binaries with your patch. Want to see the new project structure and final
>>>> version of the instructions. Presently I can’t merge your changes due to
>>>> some conflicts.
>>>>
>>>> *Anton*, please review modifications in the build procedures.
>>>>
>>>> —
>>>> Denis
>>>>
>>>> > On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com>
>>>> wrote:
>>>> >
>>>> > Hi Denis,
>>>> >
>>>> > Yes, we have included Ignite Yardstick source files with its pom.xml
>>>> in
>>>> > Ignite binary release. "Building from standalone sources" is the
>>>> building
>>>> > instruction for these source files. "Building from Ignite Sources" is
>>>> the
>>>> > instruction for building Ignite Yardstick from `modules/yardstick` in
>>>> > Ignite source files.
>>>> >
>>>> > On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org>
>>>> wrote:
>>>> >
>>>> >> Hi Oleg,
>>>> >>
>>>> >> Great progress, thanks for keep driving this!
>>>> >>
>>>> >> I’ve left some minor notes in GitHub’s pull-request. I have the
>>>> following
>>>> >> questions aside:
>>>> >>
>>>> >> - What is the difference between "Building from standalone sources"
>>>> and
>>>> >> "Building from Ignite Sources"? In my understanding, a user downloads
>>>> >> Apache Ignite release that has all the sources locally.
>>>> >>
>>>> >> - I do remember we planned to add the benchmarks sources in a form
>>>> of a
>>>> >> ready to be used project with its own pom.xml (similar to examples).
>>>> Did
>>>> >> you put this task off?
>>>> >>
>>>> >> —
>>>> >> Denis
>>>> >>
>>>> >>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
>>>> wrote:
>>>> >>>
>>>> >>> Hi!
>>>> >>>
>>>> >>> I've changed the README.txt and DEVNOTES.txt files. Also added a
>>>> simple
>>>> >>> config file for quick and easy start. Please take a look at them
>>>> and tell
>>>> >>> me what you think.
>>>> >>>
>>>> >>> https://github.com/apache/ignite/pull/1471
>>>> >>>
>>>> >>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <
>>>> isuntsov@gridgain.com>
>>>> >> wrote:
>>>> >>>
>>>> >>>> Denis,
>>>> >>>>
>>>> >>>> I think we can remove all configs except:
>>>> >>>>
>>>> >>>> benchmark-multicast.properties
>>>> >>>>
>>>> >>>> benchmark.properties
>>>> >>>>
>>>> >>>> ignite-base-config.xml
>>>> >>>>
>>>> >>>> ignite-localhost-config.xml
>>>> >>>>
>>>> >>>> ignite-multicast-config.xml
>>>> >>>>
>>>> >>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
>>>> >>>>
>>>> >>>>> I would have only those configs that are useful. Ilya Suntsov,
>>>> basing
>>>> >> on
>>>> >>>>> your experience, please suggest which configs makes sense to
>>>> include
>>>> >> into
>>>> >>>>> every Ignite release.
>>>> >>>>>
>>>> >>>>> Oleg, also please note that community decided to include not only
>>>> the
>>>> >>>>> benchmarking binaries but the sources as well into every Apache
>>>> Ignite
>>>> >>>>> release. I’ve update the ticket before. Hope you followed the
>>>> >> discussion
>>>> >>>> ;)
>>>> >>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>>>> >>>>> focusedCommentId=15765151&page=com.atlassian.jira.
>>>> >>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>>>> >>>>>
>>>> >>>>> —
>>>> >>>>> Denis
>>>> >>>>>
>>>> >>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oostanin@gridgain.com
>>>> >
>>>> >>>> wrote:
>>>> >>>>>>
>>>> >>>>>> I mean removing those configs from binary assembly, not from
>>>> >>>> repository.
>>>> >>>>>>
>>>> >>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
>>>> oostanin@gridgain.com>
>>>> >>>>> wrote:
>>>> >>>>>>
>>>> >>>>>>> Hello Igniters.
>>>> >>>>>>> I think it would be better to remove some configuration files
>>>> from
>>>> >>>>>>> benchmarks/config:
>>>> >>>>>>>
>>>> >>>>>>> ignite-base-load-config.xml
>>>> >>>>>>> ignite-cache-load-config.xml
>>>> >>>>>>> ignite-failover-base-config.xml
>>>> >>>>>>> ignite-failover-localhost-config.xml
>>>> >>>>>>> benchmark-cache-load.properties
>>>> >>>>>>> benchmark-cache-load-win.properties
>>>> >>>>>>> benchmark-failover.properties
>>>> >>>>>>>
>>>> >>>>>>> because those configs do not relate to any of performance tests.
>>>> >>>>>>>
>>>> >>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <
>>>> dmagda@apache.org>
>>>> >>>>> wrote:
>>>> >>>>>>>
>>>> >>>>>>>> Summarized the discussion updating the ticket
>>>> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>>>> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>>>> >>>>>>>>
>>>> >>>>>>>> —
>>>> >>>>>>>> Denis
>>>> >>>>>>>>
>>>> >>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>>>> >>>>> dsetrakyan@apache.org>
>>>> >>>>>>>> wrote:
>>>> >>>>>>>>>
>>>> >>>>>>>>> Sergey,
>>>> >>>>>>>>>
>>>> >>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
>>>> folder
>>>> >>>>> right
>>>> >>>>>>>>> under the root folder.
>>>> >>>>>>>>>
>>>> >>>>>>>>> D.
>>>> >>>>>>>>>
>>>> >>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>>>> >>>> skozlov@gridgain.com
>>>> >>>>>>
>>>> >>>>>>>>> wrote:
>>>> >>>>>>>>>
>>>> >>>>>>>>>> Formatting has cut lines:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> — apache_ignite_root_folder
>>>> >>>>>>>>>> — bin
>>>> >>>>>>>>>> — examples
>>>> >>>>>>>>>> — extras
>>>> >>>>>>>>>> — benchmarks
>>>> >>>>>>>>>>  — bin
>>>> >>>>>>>>>>  — src (benchmarks sources with pom.xml)
>>>> >>>>>>>>>>  — config
>>>> >>>>>>>>>>  — libs (compiled benchmarks)
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>>>> >>>>> skozlov@gridgain.com>
>>>> >>>>>>>>>> wrote:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>> Denis,
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Mostly yes. But I look ahead and think that we may include
>>>> more
>>>> >>>>>>>> things in
>>>> >>>>>>>>>>> future than yardstick only. It's why I suggest something
>>>> like
>>>> >>>> that:
>>>> >>>>>>>>>>> — apache_ignite_root_folder
>>>> >>>>>>>>>>> — bin
>>>> >>>>>>>>>>> — examples
>>>> >>>>>>>>>>> — extras
>>>> >>>>>>>>>>>     — benchmarks
>>>> >>>>>>>>>>>         — bin
>>>> >>>>>>>>>>>         — src (benchmarks sources with pom.xml)
>>>> >>>>>>>>>>>         — config
>>>> >>>>>>>>>>>         — libs (compiled benchmarks)
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
>>>> dmagda@apache.org
>>>> >>>
>>>> >>>>>>>> wrote:
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
>>>> >> following
>>>> >>>>>>>>>>>> structure
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> — apache_ignite_root_folder
>>>> >>>>>>>>>>>> — examples
>>>> >>>>>>>>>>>> — bin
>>>> >>>>>>>>>>>> — benchmarks
>>>> >>>>>>>>>>>>     — bin
>>>> >>>>>>>>>>>>     — src (benchmarks sources with pom.xml)
>>>> >>>>>>>>>>>>     — config
>>>> >>>>>>>>>>>>     — libs (compiled benchmarks)
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
>>>> previously?
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> —
>>>> >>>>>>>>>>>> Denis
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>>>> >>>> skozlov@gridgain.com>
>>>> >>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin,
>>>> /config,
>>>> >>>>> /libs)
>>>> >>>>>>>>>> and
>>>> >>>>>>>>>>>>> creates work/logs directory under yardstick root.
>>>> >>>> "libs/optional"
>>>> >>>>> is
>>>> >>>>>>>>>> for
>>>> >>>>>>>>>>>>> optional modules but in general we can't say that for
>>>> >> Yardstick.
>>>> >>>>>>>> Also
>>>> >>>>>>>>>> it
>>>> >>>>>>>>>>>>> may break the current user understanding of
>>>> "libs/optional"
>>>> >>>>>>>> directory
>>>> >>>>>>>>>> as
>>>> >>>>>>>>>>>>> place for additonal functionality activated by copying in
>>>> >>>> "libs".
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>>>> >>>>>>>>>>>> dsetrakyan@apache.org>
>>>> >>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder
>>>> for
>>>> >>>>> anything
>>>> >>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
>>>> >>>>> libraries
>>>> >>>>>>>>>>>> without
>>>> >>>>>>>>>>>>>> providing code.
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> In my view, if the community wants to include benchmarks
>>>> in
>>>> >>>>>>>> releases,
>>>> >>>>>>>>>>>> then
>>>> >>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
>>>> everything
>>>> >>>>>>>>>>>> benchmark
>>>> >>>>>>>>>>>>>> related, from code to all the dependent libraries, and
>>>> >>>>>>>> documentation
>>>> >>>>>>>>>>>>>> instructions.
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> D.
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
>>>> >>>> dmagda@apache.org>
>>>> >>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra
>>>> for me.
>>>> >>>> Why
>>>> >>>>>>>> do
>>>> >>>>>>>>>> we
>>>> >>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
>>>> well?
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Is there anyone else who is concerned about
>>>> “libs/optional”?
>>>> >>>> If
>>>> >>>>>>>>>>>> there’re
>>>> >>>>>>>>>>>>>>> not, I would agree on this and get down to the
>>>> >> implementation.
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> —
>>>> >>>>>>>>>>>>>>> Denis
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
>>>> >>>>> skozlov@gridgain.com
>>>> >>>>>>>>>
>>>> >>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> Hi
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> What's about to introduce the new root folder called
>>>> >> 'extras'
>>>> >>>>>>>> with
>>>> >>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
>>>> >>>> binaries?
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
>>>> >>>>> dmagda@apache.org
>>>> >>>>>>>>>
>>>> >>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> Dmitriy,
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
>>>> >>>>>>>>>>>>>>> https://issues.apache.org/
>>>> >>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
>>>> benchmarks
>>>> >>>>>>>>>> execution
>>>> >>>>>>>>>>>>>> for
>>>> >>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
>>>> >> needs
>>>> >>>>> to
>>>> >>>>>>>> go
>>>> >>>>>>>>>>>>>>> through
>>>> >>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
>>>> >>>>> following
>>>> >>>>>>>>>>>>>> lengthy
>>>> >>>>>>>>>>>>>>>>> Yardstick guidance.
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it
>>>> should be
>>>> >>>>>>>>>> possible
>>>> >>>>>>>>>>>>>> to
>>>> >>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script.
>>>> The
>>>> >>>>> script
>>>> >>>>>>>>>>>> needs
>>>> >>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
>>>> >>>> “libs/optional”
>>>> >>>>>>>>>>>> folder.
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
>>>> >>>> benchmark
>>>> >>>>> or
>>>> >>>>>>>>>>>> add a
>>>> >>>>>>>>>>>>>>> new
>>>> >>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
>>>> guidance.
>>>> >>>>> So,
>>>> >>>>>>>> no
>>>> >>>>>>>>>>>>>> need
>>>> >>>>>>>>>>>>>>> to
>>>> >>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite
>>>> release.
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> —
>>>> >>>>>>>>>>>>>>>>> Denis
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>>>> >>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>> >>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>>>> >>>>>>>>>>>> oostanin@gridgain.com
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty
>>>> of
>>>> >>>>> useful
>>>> >>>>>>>>>>>>>>> Yardstick
>>>> >>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
>>>> >> performance.
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs"
>>>> folder at
>>>> >>>> all?
>>>> >>>>>>>> Do
>>>> >>>>>>>>>> we
>>>> >>>>>>>>>>>>>>>>> really
>>>> >>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then
>>>> I
>>>> >>>> would
>>>> >>>>>>>>>>>> create a
>>>> >>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
>>>> >>>> benchmarks
>>>> >>>>>>>>>>>> there.
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan
>>>> <
>>>> >>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>>> >>>>>>>>>>>>>>> oostanin@gridgain.com>
>>>> >>>>>>>>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> Hello Igniters!
>>>> >>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
>>>> Benchmarking
>>>> >>>>>>>>>>>>>>> Simplification
>>>> >>>>>>>>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>>>> >>>>>>>>>>>>>> ignite-yardstick:
>>>> >>>>>>>>>>>>>>>>>>>> where
>>>> >>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put
>>>> a
>>>> >>>>> compiled
>>>> >>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite
>>>> binary
>>>> >>>>>>>> assembly?
>>>> >>>>>>>>>> We
>>>> >>>>>>>>>>>>>> can
>>>> >>>>>>>>>>>>>>>>>>> put
>>>> >>>>>>>>>>>>>>>>>>>> it
>>>> >>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
>>>> >>>>>>>> libraries,
>>>> >>>>>>>>>> or
>>>> >>>>>>>>>>>>>> we
>>>> >>>>>>>>>>>>>>>>> can
>>>> >>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
>>>> >>>> directory
>>>> >>>>>>>> and
>>>> >>>>>>>>>>>> put
>>>> >>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
>>>> >>>> solution.
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>> Best regards
>>>> >>>>>>>>>>>>>>>>>>>>> Oleg
>>>> >>>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>>>>> Sergey Kozlov
>>>> >>>>>>>>>>>>>>>> GridGain Systems
>>>> >>>>>>>>>>>>>>>> www.gridgain.com
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> --
>>>> >>>>>>>>>>>>> Sergey Kozlov
>>>> >>>>>>>>>>>>> GridGain Systems
>>>> >>>>>>>>>>>>> www.gridgain.com
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> --
>>>> >>>>>>>>>>> Sergey Kozlov
>>>> >>>>>>>>>>> GridGain Systems
>>>> >>>>>>>>>>> www.gridgain.com
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> --
>>>> >>>>>>>>>> Sergey Kozlov
>>>> >>>>>>>>>> GridGain Systems
>>>> >>>>>>>>>> www.gridgain.com
>>>> >>>>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> --
>>>> >>>> Ilya Suntsov
>>>> >>>>
>>>> >>
>>>> >>
>>>>
>>>>
>>>
>>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

Hi.

I just discovered that some functionality does not work correctly. I will
present fixed version later.

On Fri, Feb 10, 2017 at 12:43 AM, Oleg Ostanin <oo...@gridgain.com>
wrote:

> https://drive.google.com/open?id=0B9teTzJ9iIvHckl6eVQ3N0R4a3c
>
> On Fri, Feb 10, 2017 at 12:43 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
>
>> I've fixed README.txt and created new DEVNOTES.txt. Also renamed sample
>> benchmark configuration file to benchmark-sample.properties.
>> Please look for the results:
>>
>>
>> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org> wrote:
>>
>>> Oleg,
>>>
>>> Thanks for the clarification.
>>>
>>> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
>>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
>>> instructions there that explain how to build and run benchmarks from
>>> ‘modules/yardstick’. This existing files can refer to the sources and
>>> compiled benchmarks that are in Ignite binary releases but this should be a
>>> couple of statements, no more.
>>>
>>> As for the instructions related to the sources and binaries added to
>>> Ignite binaries, preferably they need to be added to auto-generated
>>> README.txt. *Anton*, is it feasible to do?
>>>
>>> Finally, when apply the reviews notes please build and share Ignite
>>> binaries with your patch. Want to see the new project structure and final
>>> version of the instructions. Presently I can’t merge your changes due to
>>> some conflicts.
>>>
>>> *Anton*, please review modifications in the build procedures.
>>>
>>> —
>>> Denis
>>>
>>> > On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com>
>>> wrote:
>>> >
>>> > Hi Denis,
>>> >
>>> > Yes, we have included Ignite Yardstick source files with its pom.xml in
>>> > Ignite binary release. "Building from standalone sources" is the
>>> building
>>> > instruction for these source files. "Building from Ignite Sources" is
>>> the
>>> > instruction for building Ignite Yardstick from `modules/yardstick` in
>>> > Ignite source files.
>>> >
>>> > On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org>
>>> wrote:
>>> >
>>> >> Hi Oleg,
>>> >>
>>> >> Great progress, thanks for keep driving this!
>>> >>
>>> >> I’ve left some minor notes in GitHub’s pull-request. I have the
>>> following
>>> >> questions aside:
>>> >>
>>> >> - What is the difference between "Building from standalone sources"
>>> and
>>> >> "Building from Ignite Sources"? In my understanding, a user downloads
>>> >> Apache Ignite release that has all the sources locally.
>>> >>
>>> >> - I do remember we planned to add the benchmarks sources in a form of
>>> a
>>> >> ready to be used project with its own pom.xml (similar to examples).
>>> Did
>>> >> you put this task off?
>>> >>
>>> >> —
>>> >> Denis
>>> >>
>>> >>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
>>> wrote:
>>> >>>
>>> >>> Hi!
>>> >>>
>>> >>> I've changed the README.txt and DEVNOTES.txt files. Also added a
>>> simple
>>> >>> config file for quick and easy start. Please take a look at them and
>>> tell
>>> >>> me what you think.
>>> >>>
>>> >>> https://github.com/apache/ignite/pull/1471
>>> >>>
>>> >>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <isuntsov@gridgain.com
>>> >
>>> >> wrote:
>>> >>>
>>> >>>> Denis,
>>> >>>>
>>> >>>> I think we can remove all configs except:
>>> >>>>
>>> >>>> benchmark-multicast.properties
>>> >>>>
>>> >>>> benchmark.properties
>>> >>>>
>>> >>>> ignite-base-config.xml
>>> >>>>
>>> >>>> ignite-localhost-config.xml
>>> >>>>
>>> >>>> ignite-multicast-config.xml
>>> >>>>
>>> >>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
>>> >>>>
>>> >>>>> I would have only those configs that are useful. Ilya Suntsov,
>>> basing
>>> >> on
>>> >>>>> your experience, please suggest which configs makes sense to
>>> include
>>> >> into
>>> >>>>> every Ignite release.
>>> >>>>>
>>> >>>>> Oleg, also please note that community decided to include not only
>>> the
>>> >>>>> benchmarking binaries but the sources as well into every Apache
>>> Ignite
>>> >>>>> release. I’ve update the ticket before. Hope you followed the
>>> >> discussion
>>> >>>> ;)
>>> >>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>>> >>>>> focusedCommentId=15765151&page=com.atlassian.jira.
>>> >>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>>> >>>>>
>>> >>>>> —
>>> >>>>> Denis
>>> >>>>>
>>> >>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
>>> >>>> wrote:
>>> >>>>>>
>>> >>>>>> I mean removing those configs from binary assembly, not from
>>> >>>> repository.
>>> >>>>>>
>>> >>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
>>> oostanin@gridgain.com>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>>> Hello Igniters.
>>> >>>>>>> I think it would be better to remove some configuration files
>>> from
>>> >>>>>>> benchmarks/config:
>>> >>>>>>>
>>> >>>>>>> ignite-base-load-config.xml
>>> >>>>>>> ignite-cache-load-config.xml
>>> >>>>>>> ignite-failover-base-config.xml
>>> >>>>>>> ignite-failover-localhost-config.xml
>>> >>>>>>> benchmark-cache-load.properties
>>> >>>>>>> benchmark-cache-load-win.properties
>>> >>>>>>> benchmark-failover.properties
>>> >>>>>>>
>>> >>>>>>> because those configs do not relate to any of performance tests.
>>> >>>>>>>
>>> >>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dmagda@apache.org
>>> >
>>> >>>>> wrote:
>>> >>>>>>>
>>> >>>>>>>> Summarized the discussion updating the ticket
>>> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>>> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>>> >>>>>>>>
>>> >>>>>>>> —
>>> >>>>>>>> Denis
>>> >>>>>>>>
>>> >>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>>> >>>>> dsetrakyan@apache.org>
>>> >>>>>>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>> Sergey,
>>> >>>>>>>>>
>>> >>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
>>> folder
>>> >>>>> right
>>> >>>>>>>>> under the root folder.
>>> >>>>>>>>>
>>> >>>>>>>>> D.
>>> >>>>>>>>>
>>> >>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>>> >>>> skozlov@gridgain.com
>>> >>>>>>
>>> >>>>>>>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>>> Formatting has cut lines:
>>> >>>>>>>>>>
>>> >>>>>>>>>> — apache_ignite_root_folder
>>> >>>>>>>>>> — bin
>>> >>>>>>>>>> — examples
>>> >>>>>>>>>> — extras
>>> >>>>>>>>>> — benchmarks
>>> >>>>>>>>>>  — bin
>>> >>>>>>>>>>  — src (benchmarks sources with pom.xml)
>>> >>>>>>>>>>  — config
>>> >>>>>>>>>>  — libs (compiled benchmarks)
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>>> >>>>> skozlov@gridgain.com>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>>> Denis,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Mostly yes. But I look ahead and think that we may include
>>> more
>>> >>>>>>>> things in
>>> >>>>>>>>>>> future than yardstick only. It's why I suggest something like
>>> >>>> that:
>>> >>>>>>>>>>> — apache_ignite_root_folder
>>> >>>>>>>>>>> — bin
>>> >>>>>>>>>>> — examples
>>> >>>>>>>>>>> — extras
>>> >>>>>>>>>>>     — benchmarks
>>> >>>>>>>>>>>         — bin
>>> >>>>>>>>>>>         — src (benchmarks sources with pom.xml)
>>> >>>>>>>>>>>         — config
>>> >>>>>>>>>>>         — libs (compiled benchmarks)
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
>>> dmagda@apache.org
>>> >>>
>>> >>>>>>>> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
>>> >> following
>>> >>>>>>>>>>>> structure
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> — apache_ignite_root_folder
>>> >>>>>>>>>>>> — examples
>>> >>>>>>>>>>>> — bin
>>> >>>>>>>>>>>> — benchmarks
>>> >>>>>>>>>>>>     — bin
>>> >>>>>>>>>>>>     — src (benchmarks sources with pom.xml)
>>> >>>>>>>>>>>>     — config
>>> >>>>>>>>>>>>     — libs (compiled benchmarks)
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Sergey, will it cover all the use case you’ve met
>>> previously?
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> —
>>> >>>>>>>>>>>> Denis
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>>> >>>> skozlov@gridgain.com>
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin,
>>> /config,
>>> >>>>> /libs)
>>> >>>>>>>>>> and
>>> >>>>>>>>>>>>> creates work/logs directory under yardstick root.
>>> >>>> "libs/optional"
>>> >>>>> is
>>> >>>>>>>>>> for
>>> >>>>>>>>>>>>> optional modules but in general we can't say that for
>>> >> Yardstick.
>>> >>>>>>>> Also
>>> >>>>>>>>>> it
>>> >>>>>>>>>>>>> may break the current user understanding of "libs/optional"
>>> >>>>>>>> directory
>>> >>>>>>>>>> as
>>> >>>>>>>>>>>>> place for additonal functionality activated by copying in
>>> >>>> "libs".
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>>> >>>>>>>>>>>> dsetrakyan@apache.org>
>>> >>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder for
>>> >>>>> anything
>>> >>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
>>> >>>>> libraries
>>> >>>>>>>>>>>> without
>>> >>>>>>>>>>>>>> providing code.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> In my view, if the community wants to include benchmarks
>>> in
>>> >>>>>>>> releases,
>>> >>>>>>>>>>>> then
>>> >>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
>>> everything
>>> >>>>>>>>>>>> benchmark
>>> >>>>>>>>>>>>>> related, from code to all the dependent libraries, and
>>> >>>>>>>> documentation
>>> >>>>>>>>>>>>>> instructions.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> D.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
>>> >>>> dmagda@apache.org>
>>> >>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for
>>> me.
>>> >>>> Why
>>> >>>>>>>> do
>>> >>>>>>>>>> we
>>> >>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
>>> well?
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Is there anyone else who is concerned about
>>> “libs/optional”?
>>> >>>> If
>>> >>>>>>>>>>>> there’re
>>> >>>>>>>>>>>>>>> not, I would agree on this and get down to the
>>> >> implementation.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> —
>>> >>>>>>>>>>>>>>> Denis
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
>>> >>>>> skozlov@gridgain.com
>>> >>>>>>>>>
>>> >>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Hi
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> What's about to introduce the new root folder called
>>> >> 'extras'
>>> >>>>>>>> with
>>> >>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
>>> >>>> binaries?
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
>>> >>>>> dmagda@apache.org
>>> >>>>>>>>>
>>> >>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Dmitriy,
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
>>> >>>>>>>>>>>>>>> https://issues.apache.org/
>>> >>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
>>> benchmarks
>>> >>>>>>>>>> execution
>>> >>>>>>>>>>>>>> for
>>> >>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
>>> >> needs
>>> >>>>> to
>>> >>>>>>>> go
>>> >>>>>>>>>>>>>>> through
>>> >>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
>>> >>>>> following
>>> >>>>>>>>>>>>>> lengthy
>>> >>>>>>>>>>>>>>>>> Yardstick guidance.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it
>>> should be
>>> >>>>>>>>>> possible
>>> >>>>>>>>>>>>>> to
>>> >>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script.
>>> The
>>> >>>>> script
>>> >>>>>>>>>>>> needs
>>> >>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
>>> >>>> “libs/optional”
>>> >>>>>>>>>>>> folder.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
>>> >>>> benchmark
>>> >>>>> or
>>> >>>>>>>>>>>> add a
>>> >>>>>>>>>>>>>>> new
>>> >>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
>>> guidance.
>>> >>>>> So,
>>> >>>>>>>> no
>>> >>>>>>>>>>>>>> need
>>> >>>>>>>>>>>>>>> to
>>> >>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite
>>> release.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> —
>>> >>>>>>>>>>>>>>>>> Denis
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>>> >>>>>>>>>>>>>> dsetrakyan@apache.org>
>>> >>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>>> >>>>>>>>>>>> oostanin@gridgain.com
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty
>>> of
>>> >>>>> useful
>>> >>>>>>>>>>>>>>> Yardstick
>>> >>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
>>> >> performance.
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder
>>> at
>>> >>>> all?
>>> >>>>>>>> Do
>>> >>>>>>>>>> we
>>> >>>>>>>>>>>>>>>>> really
>>> >>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
>>> >>>> would
>>> >>>>>>>>>>>> create a
>>> >>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
>>> >>>> benchmarks
>>> >>>>>>>>>>>> there.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>>> >>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>> >>>>>>>>>>>>>>> oostanin@gridgain.com>
>>> >>>>>>>>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Hello Igniters!
>>> >>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
>>> Benchmarking
>>> >>>>>>>>>>>>>>> Simplification
>>> >>>>>>>>>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>>> >>>>>>>>>>>>>> ignite-yardstick:
>>> >>>>>>>>>>>>>>>>>>>> where
>>> >>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
>>> >>>>> compiled
>>> >>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
>>> >>>>>>>> assembly?
>>> >>>>>>>>>> We
>>> >>>>>>>>>>>>>> can
>>> >>>>>>>>>>>>>>>>>>> put
>>> >>>>>>>>>>>>>>>>>>>> it
>>> >>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
>>> >>>>>>>> libraries,
>>> >>>>>>>>>> or
>>> >>>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>>>>> can
>>> >>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
>>> >>>> directory
>>> >>>>>>>> and
>>> >>>>>>>>>>>> put
>>> >>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
>>> >>>> solution.
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>> Best regards
>>> >>>>>>>>>>>>>>>>>>>>> Oleg
>>> >>>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> --
>>> >>>>>>>>>>>>>>>> Sergey Kozlov
>>> >>>>>>>>>>>>>>>> GridGain Systems
>>> >>>>>>>>>>>>>>>> www.gridgain.com
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> --
>>> >>>>>>>>>>>>> Sergey Kozlov
>>> >>>>>>>>>>>>> GridGain Systems
>>> >>>>>>>>>>>>> www.gridgain.com
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> --
>>> >>>>>>>>>>> Sergey Kozlov
>>> >>>>>>>>>>> GridGain Systems
>>> >>>>>>>>>>> www.gridgain.com
>>> >>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> --
>>> >>>>>>>>>> Sergey Kozlov
>>> >>>>>>>>>> GridGain Systems
>>> >>>>>>>>>> www.gridgain.com
>>> >>>>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> Ilya Suntsov
>>> >>>>
>>> >>
>>> >>
>>>
>>>
>>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

https://drive.google.com/open?id=0B9teTzJ9iIvHckl6eVQ3N0R4a3c

On Fri, Feb 10, 2017 at 12:43 AM, Oleg Ostanin <oo...@gridgain.com>
wrote:

> I've fixed README.txt and created new DEVNOTES.txt. Also renamed sample
> benchmark configuration file to benchmark-sample.properties.
> Please look for the results:
>
>
> On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org> wrote:
>
>> Oleg,
>>
>> Thanks for the clarification.
>>
>> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
>> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
>> instructions there that explain how to build and run benchmarks from
>> ‘modules/yardstick’. This existing files can refer to the sources and
>> compiled benchmarks that are in Ignite binary releases but this should be a
>> couple of statements, no more.
>>
>> As for the instructions related to the sources and binaries added to
>> Ignite binaries, preferably they need to be added to auto-generated
>> README.txt. *Anton*, is it feasible to do?
>>
>> Finally, when apply the reviews notes please build and share Ignite
>> binaries with your patch. Want to see the new project structure and final
>> version of the instructions. Presently I can’t merge your changes due to
>> some conflicts.
>>
>> *Anton*, please review modifications in the build procedures.
>>
>> —
>> Denis
>>
>> > On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com>
>> wrote:
>> >
>> > Hi Denis,
>> >
>> > Yes, we have included Ignite Yardstick source files with its pom.xml in
>> > Ignite binary release. "Building from standalone sources" is the
>> building
>> > instruction for these source files. "Building from Ignite Sources" is
>> the
>> > instruction for building Ignite Yardstick from `modules/yardstick` in
>> > Ignite source files.
>> >
>> > On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org> wrote:
>> >
>> >> Hi Oleg,
>> >>
>> >> Great progress, thanks for keep driving this!
>> >>
>> >> I’ve left some minor notes in GitHub’s pull-request. I have the
>> following
>> >> questions aside:
>> >>
>> >> - What is the difference between "Building from standalone sources" and
>> >> "Building from Ignite Sources"? In my understanding, a user downloads
>> >> Apache Ignite release that has all the sources locally.
>> >>
>> >> - I do remember we planned to add the benchmarks sources in a form of a
>> >> ready to be used project with its own pom.xml (similar to examples).
>> Did
>> >> you put this task off?
>> >>
>> >> —
>> >> Denis
>> >>
>> >>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
>> wrote:
>> >>>
>> >>> Hi!
>> >>>
>> >>> I've changed the README.txt and DEVNOTES.txt files. Also added a
>> simple
>> >>> config file for quick and easy start. Please take a look at them and
>> tell
>> >>> me what you think.
>> >>>
>> >>> https://github.com/apache/ignite/pull/1471
>> >>>
>> >>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <is...@gridgain.com>
>> >> wrote:
>> >>>
>> >>>> Denis,
>> >>>>
>> >>>> I think we can remove all configs except:
>> >>>>
>> >>>> benchmark-multicast.properties
>> >>>>
>> >>>> benchmark.properties
>> >>>>
>> >>>> ignite-base-config.xml
>> >>>>
>> >>>> ignite-localhost-config.xml
>> >>>>
>> >>>> ignite-multicast-config.xml
>> >>>>
>> >>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
>> >>>>
>> >>>>> I would have only those configs that are useful. Ilya Suntsov,
>> basing
>> >> on
>> >>>>> your experience, please suggest which configs makes sense to include
>> >> into
>> >>>>> every Ignite release.
>> >>>>>
>> >>>>> Oleg, also please note that community decided to include not only
>> the
>> >>>>> benchmarking binaries but the sources as well into every Apache
>> Ignite
>> >>>>> release. I’ve update the ticket before. Hope you followed the
>> >> discussion
>> >>>> ;)
>> >>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>> >>>>> focusedCommentId=15765151&page=com.atlassian.jira.
>> >>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>> >>>>>
>> >>>>> —
>> >>>>> Denis
>> >>>>>
>> >>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
>> >>>> wrote:
>> >>>>>>
>> >>>>>> I mean removing those configs from binary assembly, not from
>> >>>> repository.
>> >>>>>>
>> >>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
>> oostanin@gridgain.com>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>>> Hello Igniters.
>> >>>>>>> I think it would be better to remove some configuration files from
>> >>>>>>> benchmarks/config:
>> >>>>>>>
>> >>>>>>> ignite-base-load-config.xml
>> >>>>>>> ignite-cache-load-config.xml
>> >>>>>>> ignite-failover-base-config.xml
>> >>>>>>> ignite-failover-localhost-config.xml
>> >>>>>>> benchmark-cache-load.properties
>> >>>>>>> benchmark-cache-load-win.properties
>> >>>>>>> benchmark-failover.properties
>> >>>>>>>
>> >>>>>>> because those configs do not relate to any of performance tests.
>> >>>>>>>
>> >>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org>
>> >>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Summarized the discussion updating the ticket
>> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>> >>>>>>>>
>> >>>>>>>> —
>> >>>>>>>> Denis
>> >>>>>>>>
>> >>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>> >>>>> dsetrakyan@apache.org>
>> >>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>> Sergey,
>> >>>>>>>>>
>> >>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
>> folder
>> >>>>> right
>> >>>>>>>>> under the root folder.
>> >>>>>>>>>
>> >>>>>>>>> D.
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>> >>>> skozlov@gridgain.com
>> >>>>>>
>> >>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Formatting has cut lines:
>> >>>>>>>>>>
>> >>>>>>>>>> — apache_ignite_root_folder
>> >>>>>>>>>> — bin
>> >>>>>>>>>> — examples
>> >>>>>>>>>> — extras
>> >>>>>>>>>> — benchmarks
>> >>>>>>>>>>  — bin
>> >>>>>>>>>>  — src (benchmarks sources with pom.xml)
>> >>>>>>>>>>  — config
>> >>>>>>>>>>  — libs (compiled benchmarks)
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>> >>>>> skozlov@gridgain.com>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Denis,
>> >>>>>>>>>>>
>> >>>>>>>>>>> Mostly yes. But I look ahead and think that we may include
>> more
>> >>>>>>>> things in
>> >>>>>>>>>>> future than yardstick only. It's why I suggest something like
>> >>>> that:
>> >>>>>>>>>>> — apache_ignite_root_folder
>> >>>>>>>>>>> — bin
>> >>>>>>>>>>> — examples
>> >>>>>>>>>>> — extras
>> >>>>>>>>>>>     — benchmarks
>> >>>>>>>>>>>         — bin
>> >>>>>>>>>>>         — src (benchmarks sources with pom.xml)
>> >>>>>>>>>>>         — config
>> >>>>>>>>>>>         — libs (compiled benchmarks)
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
>> dmagda@apache.org
>> >>>
>> >>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
>> >> following
>> >>>>>>>>>>>> structure
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> — apache_ignite_root_folder
>> >>>>>>>>>>>> — examples
>> >>>>>>>>>>>> — bin
>> >>>>>>>>>>>> — benchmarks
>> >>>>>>>>>>>>     — bin
>> >>>>>>>>>>>>     — src (benchmarks sources with pom.xml)
>> >>>>>>>>>>>>     — config
>> >>>>>>>>>>>>     — libs (compiled benchmarks)
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Sergey, will it cover all the use case you’ve met previously?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> —
>> >>>>>>>>>>>> Denis
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>> >>>> skozlov@gridgain.com>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin,
>> /config,
>> >>>>> /libs)
>> >>>>>>>>>> and
>> >>>>>>>>>>>>> creates work/logs directory under yardstick root.
>> >>>> "libs/optional"
>> >>>>> is
>> >>>>>>>>>> for
>> >>>>>>>>>>>>> optional modules but in general we can't say that for
>> >> Yardstick.
>> >>>>>>>> Also
>> >>>>>>>>>> it
>> >>>>>>>>>>>>> may break the current user understanding of "libs/optional"
>> >>>>>>>> directory
>> >>>>>>>>>> as
>> >>>>>>>>>>>>> place for additonal functionality activated by copying in
>> >>>> "libs".
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>> >>>>>>>>>>>> dsetrakyan@apache.org>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder for
>> >>>>> anything
>> >>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
>> >>>>> libraries
>> >>>>>>>>>>>> without
>> >>>>>>>>>>>>>> providing code.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> In my view, if the community wants to include benchmarks in
>> >>>>>>>> releases,
>> >>>>>>>>>>>> then
>> >>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
>> everything
>> >>>>>>>>>>>> benchmark
>> >>>>>>>>>>>>>> related, from code to all the dependent libraries, and
>> >>>>>>>> documentation
>> >>>>>>>>>>>>>> instructions.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> D.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
>> >>>> dmagda@apache.org>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for
>> me.
>> >>>> Why
>> >>>>>>>> do
>> >>>>>>>>>> we
>> >>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
>> well?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Is there anyone else who is concerned about
>> “libs/optional”?
>> >>>> If
>> >>>>>>>>>>>> there’re
>> >>>>>>>>>>>>>>> not, I would agree on this and get down to the
>> >> implementation.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> —
>> >>>>>>>>>>>>>>> Denis
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
>> >>>>> skozlov@gridgain.com
>> >>>>>>>>>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Hi
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> What's about to introduce the new root folder called
>> >> 'extras'
>> >>>>>>>> with
>> >>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
>> >>>> binaries?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
>> >>>>> dmagda@apache.org
>> >>>>>>>>>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Dmitriy,
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
>> >>>>>>>>>>>>>>> https://issues.apache.org/
>> >>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate
>> benchmarks
>> >>>>>>>>>> execution
>> >>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
>> >> needs
>> >>>>> to
>> >>>>>>>> go
>> >>>>>>>>>>>>>>> through
>> >>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
>> >>>>> following
>> >>>>>>>>>>>>>> lengthy
>> >>>>>>>>>>>>>>>>> Yardstick guidance.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it
>> should be
>> >>>>>>>>>> possible
>> >>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The
>> >>>>> script
>> >>>>>>>>>>>> needs
>> >>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
>> >>>> “libs/optional”
>> >>>>>>>>>>>> folder.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
>> >>>> benchmark
>> >>>>> or
>> >>>>>>>>>>>> add a
>> >>>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
>> guidance.
>> >>>>> So,
>> >>>>>>>> no
>> >>>>>>>>>>>>>> need
>> >>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite
>> release.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> —
>> >>>>>>>>>>>>>>>>> Denis
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>> >>>>>>>>>>>>>> dsetrakyan@apache.org>
>> >>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>> >>>>>>>>>>>> oostanin@gridgain.com
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of
>> >>>>> useful
>> >>>>>>>>>>>>>>> Yardstick
>> >>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
>> >> performance.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder
>> at
>> >>>> all?
>> >>>>>>>> Do
>> >>>>>>>>>> we
>> >>>>>>>>>>>>>>>>> really
>> >>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
>> >>>> would
>> >>>>>>>>>>>> create a
>> >>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
>> >>>> benchmarks
>> >>>>>>>>>>>> there.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>> >>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>> >>>>>>>>>>>>>>> oostanin@gridgain.com>
>> >>>>>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Hello Igniters!
>> >>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
>> Benchmarking
>> >>>>>>>>>>>>>>> Simplification
>> >>>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>> >>>>>>>>>>>>>> ignite-yardstick:
>> >>>>>>>>>>>>>>>>>>>> where
>> >>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
>> >>>>> compiled
>> >>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
>> >>>>>>>> assembly?
>> >>>>>>>>>> We
>> >>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>> put
>> >>>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
>> >>>>>>>> libraries,
>> >>>>>>>>>> or
>> >>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
>> >>>> directory
>> >>>>>>>> and
>> >>>>>>>>>>>> put
>> >>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
>> >>>> solution.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Best regards
>> >>>>>>>>>>>>>>>>>>>>> Oleg
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>> Sergey Kozlov
>> >>>>>>>>>>>>>>>> GridGain Systems
>> >>>>>>>>>>>>>>>> www.gridgain.com
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> Sergey Kozlov
>> >>>>>>>>>>>>> GridGain Systems
>> >>>>>>>>>>>>> www.gridgain.com
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> Sergey Kozlov
>> >>>>>>>>>>> GridGain Systems
>> >>>>>>>>>>> www.gridgain.com
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> Sergey Kozlov
>> >>>>>>>>>> GridGain Systems
>> >>>>>>>>>> www.gridgain.com
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Ilya Suntsov
>> >>>>
>> >>
>> >>
>>
>>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

I've fixed README.txt and created new DEVNOTES.txt. Also renamed sample
benchmark configuration file to benchmark-sample.properties.
Please look for the results:


On Tue, Jan 31, 2017 at 9:18 PM, Denis Magda <dm...@apache.org> wrote:

> Oleg,
>
> Thanks for the clarification.
>
> My opinion is that we should leave ‘modules/yardstick/README.txt’ and
> ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those
> instructions there that explain how to build and run benchmarks from
> ‘modules/yardstick’. This existing files can refer to the sources and
> compiled benchmarks that are in Ignite binary releases but this should be a
> couple of statements, no more.
>
> As for the instructions related to the sources and binaries added to
> Ignite binaries, preferably they need to be added to auto-generated
> README.txt. *Anton*, is it feasible to do?
>
> Finally, when apply the reviews notes please build and share Ignite
> binaries with your patch. Want to see the new project structure and final
> version of the instructions. Presently I can’t merge your changes due to
> some conflicts.
>
> *Anton*, please review modifications in the build procedures.
>
> —
> Denis
>
> > On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> >
> > Hi Denis,
> >
> > Yes, we have included Ignite Yardstick source files with its pom.xml in
> > Ignite binary release. "Building from standalone sources" is the building
> > instruction for these source files. "Building from Ignite Sources" is the
> > instruction for building Ignite Yardstick from `modules/yardstick` in
> > Ignite source files.
> >
> > On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org> wrote:
> >
> >> Hi Oleg,
> >>
> >> Great progress, thanks for keep driving this!
> >>
> >> I’ve left some minor notes in GitHub’s pull-request. I have the
> following
> >> questions aside:
> >>
> >> - What is the difference between "Building from standalone sources" and
> >> "Building from Ignite Sources"? In my understanding, a user downloads
> >> Apache Ignite release that has all the sources locally.
> >>
> >> - I do remember we planned to add the benchmarks sources in a form of a
> >> ready to be used project with its own pom.xml (similar to examples). Did
> >> you put this task off?
> >>
> >> —
> >> Denis
> >>
> >>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
> >>>
> >>> Hi!
> >>>
> >>> I've changed the README.txt and DEVNOTES.txt files. Also added a simple
> >>> config file for quick and easy start. Please take a look at them and
> tell
> >>> me what you think.
> >>>
> >>> https://github.com/apache/ignite/pull/1471
> >>>
> >>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <is...@gridgain.com>
> >> wrote:
> >>>
> >>>> Denis,
> >>>>
> >>>> I think we can remove all configs except:
> >>>>
> >>>> benchmark-multicast.properties
> >>>>
> >>>> benchmark.properties
> >>>>
> >>>> ignite-base-config.xml
> >>>>
> >>>> ignite-localhost-config.xml
> >>>>
> >>>> ignite-multicast-config.xml
> >>>>
> >>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
> >>>>
> >>>>> I would have only those configs that are useful. Ilya Suntsov, basing
> >> on
> >>>>> your experience, please suggest which configs makes sense to include
> >> into
> >>>>> every Ignite release.
> >>>>>
> >>>>> Oleg, also please note that community decided to include not only the
> >>>>> benchmarking binaries but the sources as well into every Apache
> Ignite
> >>>>> release. I’ve update the ticket before. Hope you followed the
> >> discussion
> >>>> ;)
> >>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
> >>>>> focusedCommentId=15765151&page=com.atlassian.jira.
> >>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
> >>>>>
> >>>>> —
> >>>>> Denis
> >>>>>
> >>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
> >>>> wrote:
> >>>>>>
> >>>>>> I mean removing those configs from binary assembly, not from
> >>>> repository.
> >>>>>>
> >>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <
> oostanin@gridgain.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Hello Igniters.
> >>>>>>> I think it would be better to remove some configuration files from
> >>>>>>> benchmarks/config:
> >>>>>>>
> >>>>>>> ignite-base-load-config.xml
> >>>>>>> ignite-cache-load-config.xml
> >>>>>>> ignite-failover-base-config.xml
> >>>>>>> ignite-failover-localhost-config.xml
> >>>>>>> benchmark-cache-load.properties
> >>>>>>> benchmark-cache-load-win.properties
> >>>>>>> benchmark-failover.properties
> >>>>>>>
> >>>>>>> because those configs do not relate to any of performance tests.
> >>>>>>>
> >>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Summarized the discussion updating the ticket
> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> >>>>>>>>
> >>>>>>>> —
> >>>>>>>> Denis
> >>>>>>>>
> >>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> >>>>> dsetrakyan@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Sergey,
> >>>>>>>>>
> >>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks"
> folder
> >>>>> right
> >>>>>>>>> under the root folder.
> >>>>>>>>>
> >>>>>>>>> D.
> >>>>>>>>>
> >>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
> >>>> skozlov@gridgain.com
> >>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Formatting has cut lines:
> >>>>>>>>>>
> >>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>> — bin
> >>>>>>>>>> — examples
> >>>>>>>>>> — extras
> >>>>>>>>>> — benchmarks
> >>>>>>>>>>  — bin
> >>>>>>>>>>  — src (benchmarks sources with pom.xml)
> >>>>>>>>>>  — config
> >>>>>>>>>>  — libs (compiled benchmarks)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> >>>>> skozlov@gridgain.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Denis,
> >>>>>>>>>>>
> >>>>>>>>>>> Mostly yes. But I look ahead and think that we may include more
> >>>>>>>> things in
> >>>>>>>>>>> future than yardstick only. It's why I suggest something like
> >>>> that:
> >>>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>> — bin
> >>>>>>>>>>> — examples
> >>>>>>>>>>> — extras
> >>>>>>>>>>>     — benchmarks
> >>>>>>>>>>>         — bin
> >>>>>>>>>>>         — src (benchmarks sources with pom.xml)
> >>>>>>>>>>>         — config
> >>>>>>>>>>>         — libs (compiled benchmarks)
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <
> dmagda@apache.org
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
> >> following
> >>>>>>>>>>>> structure
> >>>>>>>>>>>>
> >>>>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>>> — examples
> >>>>>>>>>>>> — bin
> >>>>>>>>>>>> — benchmarks
> >>>>>>>>>>>>     — bin
> >>>>>>>>>>>>     — src (benchmarks sources with pom.xml)
> >>>>>>>>>>>>     — config
> >>>>>>>>>>>>     — libs (compiled benchmarks)
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sergey, will it cover all the use case you’ve met previously?
> >>>>>>>>>>>>
> >>>>>>>>>>>> —
> >>>>>>>>>>>> Denis
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
> >>>> skozlov@gridgain.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin, /config,
> >>>>> /libs)
> >>>>>>>>>> and
> >>>>>>>>>>>>> creates work/logs directory under yardstick root.
> >>>> "libs/optional"
> >>>>> is
> >>>>>>>>>> for
> >>>>>>>>>>>>> optional modules but in general we can't say that for
> >> Yardstick.
> >>>>>>>> Also
> >>>>>>>>>> it
> >>>>>>>>>>>>> may break the current user understanding of "libs/optional"
> >>>>>>>> directory
> >>>>>>>>>> as
> >>>>>>>>>>>>> place for additonal functionality activated by copying in
> >>>> "libs".
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder for
> >>>>> anything
> >>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
> >>>>> libraries
> >>>>>>>>>>>> without
> >>>>>>>>>>>>>> providing code.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> In my view, if the community wants to include benchmarks in
> >>>>>>>> releases,
> >>>>>>>>>>>> then
> >>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides
> everything
> >>>>>>>>>>>> benchmark
> >>>>>>>>>>>>>> related, from code to all the dependent libraries, and
> >>>>>>>> documentation
> >>>>>>>>>>>>>> instructions.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
> >>>> dmagda@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for
> me.
> >>>> Why
> >>>>>>>> do
> >>>>>>>>>> we
> >>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works
> well?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Is there anyone else who is concerned about
> “libs/optional”?
> >>>> If
> >>>>>>>>>>>> there’re
> >>>>>>>>>>>>>>> not, I would agree on this and get down to the
> >> implementation.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> >>>>> skozlov@gridgain.com
> >>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> What's about to introduce the new root folder called
> >> 'extras'
> >>>>>>>> with
> >>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
> >>>> binaries?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
> >>>>> dmagda@apache.org
> >>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Dmitriy,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
> >>>>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate benchmarks
> >>>>>>>>>> execution
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
> >> needs
> >>>>> to
> >>>>>>>> go
> >>>>>>>>>>>>>>> through
> >>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
> >>>>> following
> >>>>>>>>>>>>>> lengthy
> >>>>>>>>>>>>>>>>> Yardstick guidance.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it should
> be
> >>>>>>>>>> possible
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The
> >>>>> script
> >>>>>>>>>>>> needs
> >>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
> >>>> “libs/optional”
> >>>>>>>>>>>> folder.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
> >>>> benchmark
> >>>>> or
> >>>>>>>>>>>> add a
> >>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick
> guidance.
> >>>>> So,
> >>>>>>>> no
> >>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite release.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> >>>>>>>>>>>> oostanin@gridgain.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of
> >>>>> useful
> >>>>>>>>>>>>>>> Yardstick
> >>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
> >> performance.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder at
> >>>> all?
> >>>>>>>> Do
> >>>>>>>>>> we
> >>>>>>>>>>>>>>>>> really
> >>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
> >>>> would
> >>>>>>>>>>>> create a
> >>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
> >>>> benchmarks
> >>>>>>>>>>>> there.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> >>>>>>>>>>>>>>> oostanin@gridgain.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hello Igniters!
> >>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite
> Benchmarking
> >>>>>>>>>>>>>>> Simplification
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
> >>>>>>>>>>>>>> ignite-yardstick:
> >>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
> >>>>> compiled
> >>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
> >>>>>>>> assembly?
> >>>>>>>>>> We
> >>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>> put
> >>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
> >>>>>>>> libraries,
> >>>>>>>>>> or
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
> >>>> directory
> >>>>>>>> and
> >>>>>>>>>>>> put
> >>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
> >>>> solution.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Best regards
> >>>>>>>>>>>>>>>>>>>>> Oleg
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>> GridGain Systems
> >>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Sergey Kozlov
> >>>>>>>>>> GridGain Systems
> >>>>>>>>>> www.gridgain.com
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Ilya Suntsov
> >>>>
> >>
> >>
>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Oleg,

Thanks for the clarification.

My opinion is that we should leave ‘modules/yardstick/README.txt’ and ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those instructions there that explain how to build and run benchmarks from ‘modules/yardstick’. This existing files can refer to the sources and compiled benchmarks that are in Ignite binary releases but this should be a couple of statements, no more.

As for the instructions related to the sources and binaries added to Ignite binaries, preferably they need to be added to auto-generated README.txt. *Anton*, is it feasible to do?
 
Finally, when apply the reviews notes please build and share Ignite binaries with your patch. Want to see the new project structure and final version of the instructions. Presently I can’t merge your changes due to some conflicts.

*Anton*, please review modifications in the build procedures.

—
Denis

> On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> 
> Hi Denis,
> 
> Yes, we have included Ignite Yardstick source files with its pom.xml in
> Ignite binary release. "Building from standalone sources" is the building
> instruction for these source files. "Building from Ignite Sources" is the
> instruction for building Ignite Yardstick from `modules/yardstick` in
> Ignite source files.
> 
> On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org> wrote:
> 
>> Hi Oleg,
>> 
>> Great progress, thanks for keep driving this!
>> 
>> I’ve left some minor notes in GitHub’s pull-request. I have the following
>> questions aside:
>> 
>> - What is the difference between "Building from standalone sources" and
>> "Building from Ignite Sources"? In my understanding, a user downloads
>> Apache Ignite release that has all the sources locally.
>> 
>> - I do remember we planned to add the benchmarks sources in a form of a
>> ready to be used project with its own pom.xml (similar to examples). Did
>> you put this task off?
>> 
>> —
>> Denis
>> 
>>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
>>> 
>>> Hi!
>>> 
>>> I've changed the README.txt and DEVNOTES.txt files. Also added a simple
>>> config file for quick and easy start. Please take a look at them and tell
>>> me what you think.
>>> 
>>> https://github.com/apache/ignite/pull/1471
>>> 
>>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <is...@gridgain.com>
>> wrote:
>>> 
>>>> Denis,
>>>> 
>>>> I think we can remove all configs except:
>>>> 
>>>> benchmark-multicast.properties
>>>> 
>>>> benchmark.properties
>>>> 
>>>> ignite-base-config.xml
>>>> 
>>>> ignite-localhost-config.xml
>>>> 
>>>> ignite-multicast-config.xml
>>>> 
>>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
>>>> 
>>>>> I would have only those configs that are useful. Ilya Suntsov, basing
>> on
>>>>> your experience, please suggest which configs makes sense to include
>> into
>>>>> every Ignite release.
>>>>> 
>>>>> Oleg, also please note that community decided to include not only the
>>>>> benchmarking binaries but the sources as well into every Apache Ignite
>>>>> release. I’ve update the ticket before. Hope you followed the
>> discussion
>>>> ;)
>>>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>>>>> focusedCommentId=15765151&page=com.atlassian.jira.
>>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>>>>> 
>>>>> —
>>>>> Denis
>>>>> 
>>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
>>>> wrote:
>>>>>> 
>>>>>> I mean removing those configs from binary assembly, not from
>>>> repository.
>>>>>> 
>>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <oo...@gridgain.com>
>>>>> wrote:
>>>>>> 
>>>>>>> Hello Igniters.
>>>>>>> I think it would be better to remove some configuration files from
>>>>>>> benchmarks/config:
>>>>>>> 
>>>>>>> ignite-base-load-config.xml
>>>>>>> ignite-cache-load-config.xml
>>>>>>> ignite-failover-base-config.xml
>>>>>>> ignite-failover-localhost-config.xml
>>>>>>> benchmark-cache-load.properties
>>>>>>> benchmark-cache-load-win.properties
>>>>>>> benchmark-failover.properties
>>>>>>> 
>>>>>>> because those configs do not relate to any of performance tests.
>>>>>>> 
>>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org>
>>>>> wrote:
>>>>>>> 
>>>>>>>> Summarized the discussion updating the ticket
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>>>>>>>> 
>>>>>>>> —
>>>>>>>> Denis
>>>>>>>> 
>>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>>>>> dsetrakyan@apache.org>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Sergey,
>>>>>>>>> 
>>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks" folder
>>>>> right
>>>>>>>>> under the root folder.
>>>>>>>>> 
>>>>>>>>> D.
>>>>>>>>> 
>>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>>>> skozlov@gridgain.com
>>>>>> 
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Formatting has cut lines:
>>>>>>>>>> 
>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>> — bin
>>>>>>>>>> — examples
>>>>>>>>>> — extras
>>>>>>>>>> — benchmarks
>>>>>>>>>>  — bin
>>>>>>>>>>  — src (benchmarks sources with pom.xml)
>>>>>>>>>>  — config
>>>>>>>>>>  — libs (compiled benchmarks)
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>>>>> skozlov@gridgain.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Denis,
>>>>>>>>>>> 
>>>>>>>>>>> Mostly yes. But I look ahead and think that we may include more
>>>>>>>> things in
>>>>>>>>>>> future than yardstick only. It's why I suggest something like
>>>> that:
>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>> — bin
>>>>>>>>>>> — examples
>>>>>>>>>>> — extras
>>>>>>>>>>>     — benchmarks
>>>>>>>>>>>         — bin
>>>>>>>>>>>         — src (benchmarks sources with pom.xml)
>>>>>>>>>>>         — config
>>>>>>>>>>>         — libs (compiled benchmarks)
>>>>>>>>>>> 
>>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dmagda@apache.org
>>> 
>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
>> following
>>>>>>>>>>>> structure
>>>>>>>>>>>> 
>>>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>>> — examples
>>>>>>>>>>>> — bin
>>>>>>>>>>>> — benchmarks
>>>>>>>>>>>>     — bin
>>>>>>>>>>>>     — src (benchmarks sources with pom.xml)
>>>>>>>>>>>>     — config
>>>>>>>>>>>>     — libs (compiled benchmarks)
>>>>>>>>>>>> 
>>>>>>>>>>>> Sergey, will it cover all the use case you’ve met previously?
>>>>>>>>>>>> 
>>>>>>>>>>>> —
>>>>>>>>>>>> Denis
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>>>> skozlov@gridgain.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin, /config,
>>>>> /libs)
>>>>>>>>>> and
>>>>>>>>>>>>> creates work/logs directory under yardstick root.
>>>> "libs/optional"
>>>>> is
>>>>>>>>>> for
>>>>>>>>>>>>> optional modules but in general we can't say that for
>> Yardstick.
>>>>>>>> Also
>>>>>>>>>> it
>>>>>>>>>>>>> may break the current user understanding of "libs/optional"
>>>>>>>> directory
>>>>>>>>>> as
>>>>>>>>>>>>> place for additonal functionality activated by copying in
>>>> "libs".
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder for
>>>>> anything
>>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
>>>>> libraries
>>>>>>>>>>>> without
>>>>>>>>>>>>>> providing code.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> In my view, if the community wants to include benchmarks in
>>>>>>>> releases,
>>>>>>>>>>>> then
>>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides everything
>>>>>>>>>>>> benchmark
>>>>>>>>>>>>>> related, from code to all the dependent libraries, and
>>>>>>>> documentation
>>>>>>>>>>>>>> instructions.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
>>>> dmagda@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for me.
>>>> Why
>>>>>>>> do
>>>>>>>>>> we
>>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works well?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Is there anyone else who is concerned about “libs/optional”?
>>>> If
>>>>>>>>>>>> there’re
>>>>>>>>>>>>>>> not, I would agree on this and get down to the
>> implementation.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
>>>>> skozlov@gridgain.com
>>>>>>>>> 
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> What's about to introduce the new root folder called
>> 'extras'
>>>>>>>> with
>>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
>>>> binaries?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
>>>>> dmagda@apache.org
>>>>>>>>> 
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Dmitriy,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate benchmarks
>>>>>>>>>> execution
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
>> needs
>>>>> to
>>>>>>>> go
>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
>>>>> following
>>>>>>>>>>>>>> lengthy
>>>>>>>>>>>>>>>>> Yardstick guidance.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it should be
>>>>>>>>>> possible
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The
>>>>> script
>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
>>>> “libs/optional”
>>>>>>>>>>>> folder.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
>>>> benchmark
>>>>> or
>>>>>>>>>>>> add a
>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick guidance.
>>>>> So,
>>>>>>>> no
>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite release.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>>>>>>>>>>>> oostanin@gridgain.com
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of
>>>>> useful
>>>>>>>>>>>>>>> Yardstick
>>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
>> performance.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder at
>>>> all?
>>>>>>>> Do
>>>>>>>>>> we
>>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
>>>> would
>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
>>>> benchmarks
>>>>>>>>>>>> there.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>>>>>>>>>>>>>> oostanin@gridgain.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
>>>>>>>>>>>>>>> Simplification
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>>>>>>>>>>>>>> ignite-yardstick:
>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
>>>>> compiled
>>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
>>>>>>>> assembly?
>>>>>>>>>> We
>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
>>>>>>>> libraries,
>>>>>>>>>> or
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
>>>> directory
>>>>>>>> and
>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
>>>> solution.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>>>>>>>>>> Oleg
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>> GridGain Systems
>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Sergey Kozlov
>>>>>>>>>> GridGain Systems
>>>>>>>>>> www.gridgain.com
>>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> Ilya Suntsov
>>>> 
>> 
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

Hi Denis,

Yes, we have included Ignite Yardstick source files with its pom.xml in
Ignite binary release. "Building from standalone sources" is the building
instruction for these source files. "Building from Ignite Sources" is the
instruction for building Ignite Yardstick from `modules/yardstick` in
Ignite source files.

On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dm...@apache.org> wrote:

> Hi Oleg,
>
> Great progress, thanks for keep driving this!
>
> I’ve left some minor notes in GitHub’s pull-request. I have the following
> questions aside:
>
> - What is the difference between "Building from standalone sources" and
> "Building from Ignite Sources"? In my understanding, a user downloads
> Apache Ignite release that has all the sources locally.
>
> - I do remember we planned to add the benchmarks sources in a form of a
> ready to be used project with its own pom.xml (similar to examples). Did
> you put this task off?
>
> —
> Denis
>
> > On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> >
> > Hi!
> >
> > I've changed the README.txt and DEVNOTES.txt files. Also added a simple
> > config file for quick and easy start. Please take a look at them and tell
> > me what you think.
> >
> > https://github.com/apache/ignite/pull/1471
> >
> > On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <is...@gridgain.com>
> wrote:
> >
> >> Denis,
> >>
> >> I think we can remove all configs except:
> >>
> >> benchmark-multicast.properties
> >>
> >> benchmark.properties
> >>
> >> ignite-base-config.xml
> >>
> >> ignite-localhost-config.xml
> >>
> >> ignite-multicast-config.xml
> >>
> >> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
> >>
> >>> I would have only those configs that are useful. Ilya Suntsov, basing
> on
> >>> your experience, please suggest which configs makes sense to include
> into
> >>> every Ignite release.
> >>>
> >>> Oleg, also please note that community decided to include not only the
> >>> benchmarking binaries but the sources as well into every Apache Ignite
> >>> release. I’ve update the ticket before. Hope you followed the
> discussion
> >> ;)
> >>> https://issues.apache.org/jira/browse/IGNITE-4212?
> >>> focusedCommentId=15765151&page=com.atlassian.jira.
> >>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
> >>>
> >>> —
> >>> Denis
> >>>
> >>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
> >> wrote:
> >>>>
> >>>> I mean removing those configs from binary assembly, not from
> >> repository.
> >>>>
> >>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <oo...@gridgain.com>
> >>> wrote:
> >>>>
> >>>>> Hello Igniters.
> >>>>> I think it would be better to remove some configuration files from
> >>>>> benchmarks/config:
> >>>>>
> >>>>> ignite-base-load-config.xml
> >>>>> ignite-cache-load-config.xml
> >>>>> ignite-failover-base-config.xml
> >>>>> ignite-failover-localhost-config.xml
> >>>>> benchmark-cache-load.properties
> >>>>> benchmark-cache-load-win.properties
> >>>>> benchmark-failover.properties
> >>>>>
> >>>>> because those configs do not relate to any of performance tests.
> >>>>>
> >>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org>
> >>> wrote:
> >>>>>
> >>>>>> Summarized the discussion updating the ticket
> >>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> >>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> >>>>>>
> >>>>>> —
> >>>>>> Denis
> >>>>>>
> >>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> >>> dsetrakyan@apache.org>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Sergey,
> >>>>>>>
> >>>>>>> I am not sure I like "extras". I am voting for "benchmarks" folder
> >>> right
> >>>>>>> under the root folder.
> >>>>>>>
> >>>>>>> D.
> >>>>>>>
> >>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
> >> skozlov@gridgain.com
> >>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Formatting has cut lines:
> >>>>>>>>
> >>>>>>>> — apache_ignite_root_folder
> >>>>>>>> — bin
> >>>>>>>> — examples
> >>>>>>>> — extras
> >>>>>>>> — benchmarks
> >>>>>>>>   — bin
> >>>>>>>>   — src (benchmarks sources with pom.xml)
> >>>>>>>>   — config
> >>>>>>>>   — libs (compiled benchmarks)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> >>> skozlov@gridgain.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Denis,
> >>>>>>>>>
> >>>>>>>>> Mostly yes. But I look ahead and think that we may include more
> >>>>>> things in
> >>>>>>>>> future than yardstick only. It's why I suggest something like
> >> that:
> >>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>  — bin
> >>>>>>>>>  — examples
> >>>>>>>>>  — extras
> >>>>>>>>>      — benchmarks
> >>>>>>>>>          — bin
> >>>>>>>>>          — src (benchmarks sources with pom.xml)
> >>>>>>>>>          — config
> >>>>>>>>>          — libs (compiled benchmarks)
> >>>>>>>>>
> >>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dmagda@apache.org
> >
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the
> following
> >>>>>>>>>> structure
> >>>>>>>>>>
> >>>>>>>>>> — apache_ignite_root_folder
> >>>>>>>>>>  — examples
> >>>>>>>>>>  — bin
> >>>>>>>>>>  — benchmarks
> >>>>>>>>>>      — bin
> >>>>>>>>>>      — src (benchmarks sources with pom.xml)
> >>>>>>>>>>      — config
> >>>>>>>>>>      — libs (compiled benchmarks)
> >>>>>>>>>>
> >>>>>>>>>> Sergey, will it cover all the use case you’ve met previously?
> >>>>>>>>>>
> >>>>>>>>>> —
> >>>>>>>>>> Denis
> >>>>>>>>>>
> >>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
> >> skozlov@gridgain.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Yardstick requires own scripts/configurations (/bin, /config,
> >>> /libs)
> >>>>>>>> and
> >>>>>>>>>>> creates work/logs directory under yardstick root.
> >> "libs/optional"
> >>> is
> >>>>>>>> for
> >>>>>>>>>>> optional modules but in general we can't say that for
> Yardstick.
> >>>>>> Also
> >>>>>>>> it
> >>>>>>>>>>> may break the current user understanding of "libs/optional"
> >>>>>> directory
> >>>>>>>> as
> >>>>>>>>>>> place for additonal functionality activated by copying in
> >> "libs".
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> >>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> I would be against using libs/optional or libs/ folder for
> >>> anything
> >>>>>>>>>>>> benchmark related. I am also against adding any yardstick
> >>> libraries
> >>>>>>>>>> without
> >>>>>>>>>>>> providing code.
> >>>>>>>>>>>>
> >>>>>>>>>>>> In my view, if the community wants to include benchmarks in
> >>>>>> releases,
> >>>>>>>>>> then
> >>>>>>>>>>>> we should add a "benchmarks" folder, which provides everything
> >>>>>>>>>> benchmark
> >>>>>>>>>>>> related, from code to all the dependent libraries, and
> >>>>>> documentation
> >>>>>>>>>>>> instructions.
> >>>>>>>>>>>>
> >>>>>>>>>>>> D.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
> >> dmagda@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for me.
> >> Why
> >>>>>> do
> >>>>>>>> we
> >>>>>>>>>>>>> need this new folder if “libs/optional” semantic works well?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Is there anyone else who is concerned about “libs/optional”?
> >> If
> >>>>>>>>>> there’re
> >>>>>>>>>>>>> not, I would agree on this and get down to the
> implementation.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> —
> >>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> >>> skozlov@gridgain.com
> >>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What's about to introduce the new root folder called
> 'extras'
> >>>>>> with
> >>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
> >> binaries?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
> >>> dmagda@apache.org
> >>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Dmitriy,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
> >>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The whole purpose of the ticket is to automate benchmarks
> >>>>>>>> execution
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she
> needs
> >>> to
> >>>>>> go
> >>>>>>>>>>>>> through
> >>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
> >>> following
> >>>>>>>>>>>> lengthy
> >>>>>>>>>>>>>>> Yardstick guidance.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it should be
> >>>>>>>> possible
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The
> >>> script
> >>>>>>>>>> needs
> >>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
> >> “libs/optional”
> >>>>>>>>>> folder.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If someone wants to modify the source of an existed
> >> benchmark
> >>> or
> >>>>>>>>>> add a
> >>>>>>>>>>>>> new
> >>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick guidance.
> >>> So,
> >>>>>> no
> >>>>>>>>>>>> need
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite release.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> >>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> >>>>>>>>>> oostanin@gridgain.com
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of
> >>> useful
> >>>>>>>>>>>>> Yardstick
> >>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite
> performance.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder at
> >> all?
> >>>>>> Do
> >>>>>>>> we
> >>>>>>>>>>>>>>> really
> >>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
> >> would
> >>>>>>>>>> create a
> >>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
> >> benchmarks
> >>>>>>>>>> there.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> >>>>>>>>>>>>> oostanin@gridgain.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hello Igniters!
> >>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
> >>>>>>>>>>>>> Simplification
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
> >>>>>>>>>>>> ignite-yardstick:
> >>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
> >>> compiled
> >>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
> >>>>>> assembly?
> >>>>>>>> We
> >>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>> put
> >>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
> >>>>>> libraries,
> >>>>>>>> or
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
> >> directory
> >>>>>> and
> >>>>>>>>>> put
> >>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
> >> solution.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Best regards
> >>>>>>>>>>>>>>>>>>> Oleg
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>>>>> GridGain Systems
> >>>>>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>> GridGain Systems
> >>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Sergey Kozlov
> >>>>>>>>> GridGain Systems
> >>>>>>>>> www.gridgain.com
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Sergey Kozlov
> >>>>>>>> GridGain Systems
> >>>>>>>> www.gridgain.com
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Ilya Suntsov
> >>
>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Hi Oleg,

Great progress, thanks for keep driving this!

I’ve left some minor notes in GitHub’s pull-request. I have the following questions aside:

- What is the difference between "Building from standalone sources" and "Building from Ignite Sources"? In my understanding, a user downloads Apache Ignite release that has all the sources locally.

- I do remember we planned to add the benchmarks sources in a form of a ready to be used project with its own pom.xml (similar to examples). Did you put this task off?

—
Denis

> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> 
> Hi!
> 
> I've changed the README.txt and DEVNOTES.txt files. Also added a simple
> config file for quick and easy start. Please take a look at them and tell
> me what you think.
> 
> https://github.com/apache/ignite/pull/1471
> 
> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <is...@gridgain.com> wrote:
> 
>> Denis,
>> 
>> I think we can remove all configs except:
>> 
>> benchmark-multicast.properties
>> 
>> benchmark.properties
>> 
>> ignite-base-config.xml
>> 
>> ignite-localhost-config.xml
>> 
>> ignite-multicast-config.xml
>> 
>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
>> 
>>> I would have only those configs that are useful. Ilya Suntsov, basing on
>>> your experience, please suggest which configs makes sense to include into
>>> every Ignite release.
>>> 
>>> Oleg, also please note that community decided to include not only the
>>> benchmarking binaries but the sources as well into every Apache Ignite
>>> release. I’ve update the ticket before. Hope you followed the discussion
>> ;)
>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>>> focusedCommentId=15765151&page=com.atlassian.jira.
>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>>> 
>>> —
>>> Denis
>>> 
>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
>> wrote:
>>>> 
>>>> I mean removing those configs from binary assembly, not from
>> repository.
>>>> 
>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <oo...@gridgain.com>
>>> wrote:
>>>> 
>>>>> Hello Igniters.
>>>>> I think it would be better to remove some configuration files from
>>>>> benchmarks/config:
>>>>> 
>>>>> ignite-base-load-config.xml
>>>>> ignite-cache-load-config.xml
>>>>> ignite-failover-base-config.xml
>>>>> ignite-failover-localhost-config.xml
>>>>> benchmark-cache-load.properties
>>>>> benchmark-cache-load-win.properties
>>>>> benchmark-failover.properties
>>>>> 
>>>>> because those configs do not relate to any of performance tests.
>>>>> 
>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org>
>>> wrote:
>>>>> 
>>>>>> Summarized the discussion updating the ticket
>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>>>>>> 
>>>>>> —
>>>>>> Denis
>>>>>> 
>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>>> dsetrakyan@apache.org>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Sergey,
>>>>>>> 
>>>>>>> I am not sure I like "extras". I am voting for "benchmarks" folder
>>> right
>>>>>>> under the root folder.
>>>>>>> 
>>>>>>> D.
>>>>>>> 
>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>> skozlov@gridgain.com
>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Formatting has cut lines:
>>>>>>>> 
>>>>>>>> — apache_ignite_root_folder
>>>>>>>> — bin
>>>>>>>> — examples
>>>>>>>> — extras
>>>>>>>> — benchmarks
>>>>>>>>   — bin
>>>>>>>>   — src (benchmarks sources with pom.xml)
>>>>>>>>   — config
>>>>>>>>   — libs (compiled benchmarks)
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>>> skozlov@gridgain.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Denis,
>>>>>>>>> 
>>>>>>>>> Mostly yes. But I look ahead and think that we may include more
>>>>>> things in
>>>>>>>>> future than yardstick only. It's why I suggest something like
>> that:
>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>  — bin
>>>>>>>>>  — examples
>>>>>>>>>  — extras
>>>>>>>>>      — benchmarks
>>>>>>>>>          — bin
>>>>>>>>>          — src (benchmarks sources with pom.xml)
>>>>>>>>>          — config
>>>>>>>>>          — libs (compiled benchmarks)
>>>>>>>>> 
>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dm...@apache.org>
>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the following
>>>>>>>>>> structure
>>>>>>>>>> 
>>>>>>>>>> — apache_ignite_root_folder
>>>>>>>>>>  — examples
>>>>>>>>>>  — bin
>>>>>>>>>>  — benchmarks
>>>>>>>>>>      — bin
>>>>>>>>>>      — src (benchmarks sources with pom.xml)
>>>>>>>>>>      — config
>>>>>>>>>>      — libs (compiled benchmarks)
>>>>>>>>>> 
>>>>>>>>>> Sergey, will it cover all the use case you’ve met previously?
>>>>>>>>>> 
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>> 
>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>> skozlov@gridgain.com>
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin, /config,
>>> /libs)
>>>>>>>> and
>>>>>>>>>>> creates work/logs directory under yardstick root.
>> "libs/optional"
>>> is
>>>>>>>> for
>>>>>>>>>>> optional modules but in general we can't say that for Yardstick.
>>>>>> Also
>>>>>>>> it
>>>>>>>>>>> may break the current user understanding of "libs/optional"
>>>>>> directory
>>>>>>>> as
>>>>>>>>>>> place for additonal functionality activated by copying in
>> "libs".
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> I would be against using libs/optional or libs/ folder for
>>> anything
>>>>>>>>>>>> benchmark related. I am also against adding any yardstick
>>> libraries
>>>>>>>>>> without
>>>>>>>>>>>> providing code.
>>>>>>>>>>>> 
>>>>>>>>>>>> In my view, if the community wants to include benchmarks in
>>>>>> releases,
>>>>>>>>>> then
>>>>>>>>>>>> we should add a "benchmarks" folder, which provides everything
>>>>>>>>>> benchmark
>>>>>>>>>>>> related, from code to all the dependent libraries, and
>>>>>> documentation
>>>>>>>>>>>> instructions.
>>>>>>>>>>>> 
>>>>>>>>>>>> D.
>>>>>>>>>>>> 
>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
>> dmagda@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for me.
>> Why
>>>>>> do
>>>>>>>> we
>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works well?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Is there anyone else who is concerned about “libs/optional”?
>> If
>>>>>>>>>> there’re
>>>>>>>>>>>>> not, I would agree on this and get down to the implementation.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> —
>>>>>>>>>>>>> Denis
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
>>> skozlov@gridgain.com
>>>>>>> 
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> What's about to introduce the new root folder called 'extras'
>>>>>> with
>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
>> binaries?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
>>> dmagda@apache.org
>>>>>>> 
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Dmitriy,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>>>>>> jira/browse/IGNITE-4212).
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate benchmarks
>>>>>>>> execution
>>>>>>>>>>>> for
>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she needs
>>> to
>>>>>> go
>>>>>>>>>>>>> through
>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly
>>> following
>>>>>>>>>>>> lengthy
>>>>>>>>>>>>>>> Yardstick guidance.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it should be
>>>>>>>> possible
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The
>>> script
>>>>>>>>>> needs
>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under
>> “libs/optional”
>>>>>>>>>> folder.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> If someone wants to modify the source of an existed
>> benchmark
>>> or
>>>>>>>>>> add a
>>>>>>>>>>>>> new
>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick guidance.
>>> So,
>>>>>> no
>>>>>>>>>>>> need
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite release.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>>>>>>>>>> oostanin@gridgain.com
>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of
>>> useful
>>>>>>>>>>>>> Yardstick
>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite performance.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder at
>> all?
>>>>>> Do
>>>>>>>> we
>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
>> would
>>>>>>>>>> create a
>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the
>> benchmarks
>>>>>>>>>> there.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>>>>>>>>>>>> oostanin@gridgain.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
>>>>>>>>>>>>> Simplification
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>>>>>>>>>>>> ignite-yardstick:
>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
>>> compiled
>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
>>>>>> assembly?
>>>>>>>> We
>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
>>>>>> libraries,
>>>>>>>> or
>>>>>>>>>>>> we
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root
>> directory
>>>>>> and
>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
>> solution.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>>>>>>>> Oleg
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>>>>> GridGain Systems
>>>>>>>>>>>>>> www.gridgain.com
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>> GridGain Systems
>>>>>>>>>>> www.gridgain.com
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Sergey Kozlov
>>>>>>>>> GridGain Systems
>>>>>>>>> www.gridgain.com
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Sergey Kozlov
>>>>>>>> GridGain Systems
>>>>>>>> www.gridgain.com
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>> 
>>> 
>> 
>> 
>> --
>> Ilya Suntsov
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

Hi!

I've changed the README.txt and DEVNOTES.txt files. Also added a simple
config file for quick and easy start. Please take a look at them and tell
me what you think.

https://github.com/apache/ignite/pull/1471

On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <is...@gridgain.com> wrote:

> Denis,
>
> I think we can remove all configs except:
>
> benchmark-multicast.properties
>
> benchmark.properties
>
> ignite-base-config.xml
>
> ignite-localhost-config.xml
>
> ignite-multicast-config.xml
>
> 2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:
>
> > I would have only those configs that are useful. Ilya Suntsov, basing on
> > your experience, please suggest which configs makes sense to include into
> > every Ignite release.
> >
> > Oleg, also please note that community decided to include not only the
> > benchmarking binaries but the sources as well into every Apache Ignite
> > release. I’ve update the ticket before. Hope you followed the discussion
> ;)
> > https://issues.apache.org/jira/browse/IGNITE-4212?
> > focusedCommentId=15765151&page=com.atlassian.jira.
> > plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
> >
> > —
> > Denis
> >
> > > On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
> > >
> > > I mean removing those configs from binary assembly, not from
> repository.
> > >
> > > On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <oo...@gridgain.com>
> > wrote:
> > >
> > >> Hello Igniters.
> > >> I think it would be better to remove some configuration files from
> > >> benchmarks/config:
> > >>
> > >> ignite-base-load-config.xml
> > >> ignite-cache-load-config.xml
> > >> ignite-failover-base-config.xml
> > >> ignite-failover-localhost-config.xml
> > >> benchmark-cache-load.properties
> > >> benchmark-cache-load-win.properties
> > >> benchmark-failover.properties
> > >>
> > >> because those configs do not relate to any of performance tests.
> > >>
> > >> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org>
> > wrote:
> > >>
> > >>> Summarized the discussion updating the ticket
> > >>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> > >>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> > >>>
> > >>> —
> > >>> Denis
> > >>>
> > >>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > >>> wrote:
> > >>>>
> > >>>> Sergey,
> > >>>>
> > >>>> I am not sure I like "extras". I am voting for "benchmarks" folder
> > right
> > >>>> under the root folder.
> > >>>>
> > >>>> D.
> > >>>>
> > >>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
> skozlov@gridgain.com
> > >
> > >>>> wrote:
> > >>>>
> > >>>>> Formatting has cut lines:
> > >>>>>
> > >>>>> — apache_ignite_root_folder
> > >>>>> — bin
> > >>>>> — examples
> > >>>>> — extras
> > >>>>>  — benchmarks
> > >>>>>    — bin
> > >>>>>    — src (benchmarks sources with pom.xml)
> > >>>>>    — config
> > >>>>>    — libs (compiled benchmarks)
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> > skozlov@gridgain.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Denis,
> > >>>>>>
> > >>>>>> Mostly yes. But I look ahead and think that we may include more
> > >>> things in
> > >>>>>> future than yardstick only. It's why I suggest something like
> that:
> > >>>>>> — apache_ignite_root_folder
> > >>>>>>   — bin
> > >>>>>>   — examples
> > >>>>>>   — extras
> > >>>>>>       — benchmarks
> > >>>>>>           — bin
> > >>>>>>           — src (benchmarks sources with pom.xml)
> > >>>>>>           — config
> > >>>>>>           — libs (compiled benchmarks)
> > >>>>>>
> > >>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dm...@apache.org>
> > >>> wrote:
> > >>>>>>
> > >>>>>>> Well, if to refer to Dmitriy suggestion we can have the following
> > >>>>>>> structure
> > >>>>>>>
> > >>>>>>> — apache_ignite_root_folder
> > >>>>>>>   — examples
> > >>>>>>>   — bin
> > >>>>>>>   — benchmarks
> > >>>>>>>       — bin
> > >>>>>>>       — src (benchmarks sources with pom.xml)
> > >>>>>>>       — config
> > >>>>>>>       — libs (compiled benchmarks)
> > >>>>>>>
> > >>>>>>> Sergey, will it cover all the use case you’ve met previously?
> > >>>>>>>
> > >>>>>>> —
> > >>>>>>> Denis
> > >>>>>>>
> > >>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
> skozlov@gridgain.com>
> > >>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> Yardstick requires own scripts/configurations (/bin, /config,
> > /libs)
> > >>>>> and
> > >>>>>>>> creates work/logs directory under yardstick root.
> "libs/optional"
> > is
> > >>>>> for
> > >>>>>>>> optional modules but in general we can't say that for Yardstick.
> > >>> Also
> > >>>>> it
> > >>>>>>>> may break the current user understanding of "libs/optional"
> > >>> directory
> > >>>>> as
> > >>>>>>>> place for additonal functionality activated by copying in
> "libs".
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> > >>>>>>> dsetrakyan@apache.org>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> I would be against using libs/optional or libs/ folder for
> > anything
> > >>>>>>>>> benchmark related. I am also against adding any yardstick
> > libraries
> > >>>>>>> without
> > >>>>>>>>> providing code.
> > >>>>>>>>>
> > >>>>>>>>> In my view, if the community wants to include benchmarks in
> > >>> releases,
> > >>>>>>> then
> > >>>>>>>>> we should add a "benchmarks" folder, which provides everything
> > >>>>>>> benchmark
> > >>>>>>>>> related, from code to all the dependent libraries, and
> > >>> documentation
> > >>>>>>>>> instructions.
> > >>>>>>>>>
> > >>>>>>>>> D.
> > >>>>>>>>>
> > >>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <
> dmagda@apache.org>
> > >>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Actually, “libs/optional” is already a kind of extra for me.
> Why
> > >>> do
> > >>>>> we
> > >>>>>>>>>> need this new folder if “libs/optional” semantic works well?
> > >>>>>>>>>>
> > >>>>>>>>>> Is there anyone else who is concerned about “libs/optional”?
> If
> > >>>>>>> there’re
> > >>>>>>>>>> not, I would agree on this and get down to the implementation.
> > >>>>>>>>>>
> > >>>>>>>>>> —
> > >>>>>>>>>> Denis
> > >>>>>>>>>>
> > >>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> > skozlov@gridgain.com
> > >>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Hi
> > >>>>>>>>>>>
> > >>>>>>>>>>> What's about to introduce the new root folder called 'extras'
> > >>> with
> > >>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick
> binaries?
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
> > dmagda@apache.org
> > >>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Dmitriy,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Please have a look at IGNITE-4212 description (
> > >>>>>>>>>> https://issues.apache.org/
> > >>>>>>>>>>>> jira/browse/IGNITE-4212).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> The whole purpose of the ticket is to automate benchmarks
> > >>>>> execution
> > >>>>>>>>> for
> > >>>>>>>>>>>> the end user for a specific Ignite release. Now he/she needs
> > to
> > >>> go
> > >>>>>>>>>> through
> > >>>>>>>>>>>> a number of steps like build, configure, run strictly
> > following
> > >>>>>>>>> lengthy
> > >>>>>>>>>>>> Yardstick guidance.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Ideally, once a specific release is downloaded it should be
> > >>>>> possible
> > >>>>>>>>> to
> > >>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The
> > script
> > >>>>>>> needs
> > >>>>>>>>>>>> benchmarks' lib which makes sense to put under
> “libs/optional”
> > >>>>>>> folder.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> If someone wants to modify the source of an existed
> benchmark
> > or
> > >>>>>>> add a
> > >>>>>>>>>> new
> > >>>>>>>>>>>> one then he/she needs to follow existed Yardstick guidance.
> > So,
> > >>> no
> > >>>>>>>>> need
> > >>>>>>>>>> to
> > >>>>>>>>>>>> release benchmarks’s sources as a part of Ignite release.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> —
> > >>>>>>>>>>>> Denis
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> > >>>>>>>>> dsetrakyan@apache.org>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> > >>>>>>> oostanin@gridgain.com
> > >>>>>>>>>>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of
> > useful
> > >>>>>>>>>> Yardstick
> > >>>>>>>>>>>>>> benchmarks, which can be used to check Ignite performance.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> In that case, why would it be under the "libs" folder at
> all?
> > >>> Do
> > >>>>> we
> > >>>>>>>>>>>> really
> > >>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I
> would
> > >>>>>>> create a
> > >>>>>>>>>>>>> benchmarks folder under "examples" and add all the
> benchmarks
> > >>>>>>> there.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> > >>>>>>>>>>>> dsetrakyan@apache.org
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> > >>>>>>>>>> oostanin@gridgain.com>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Hello Igniters!
> > >>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
> > >>>>>>>>>> Simplification
> > >>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
> > >>>>>>>>> ignite-yardstick:
> > >>>>>>>>>>>>>>> where
> > >>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
> > compiled
> > >>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
> > >>> assembly?
> > >>>>> We
> > >>>>>>>>> can
> > >>>>>>>>>>>>>> put
> > >>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>> in the libs/optional along with an others optional
> > >>> libraries,
> > >>>>> or
> > >>>>>>>>> we
> > >>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>> create a new directory named "tools" in the root
> directory
> > >>> and
> > >>>>>>> put
> > >>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another
> solution.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Best regards
> > >>>>>>>>>>>>>>>> Oleg
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> Sergey Kozlov
> > >>>>>>>>>>> GridGain Systems
> > >>>>>>>>>>> www.gridgain.com
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Sergey Kozlov
> > >>>>>>>> GridGain Systems
> > >>>>>>>> www.gridgain.com
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Sergey Kozlov
> > >>>>>> GridGain Systems
> > >>>>>> www.gridgain.com
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Sergey Kozlov
> > >>>>> GridGain Systems
> > >>>>> www.gridgain.com
> > >>>>>
> > >>>
> > >>>
> > >>
> >
> >
>
>
> --
> Ilya Suntsov
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Ilya Suntsov <is...@gridgain.com>.

Denis,

I think we can remove all configs except:

benchmark-multicast.properties

benchmark.properties

ignite-base-config.xml

ignite-localhost-config.xml

ignite-multicast-config.xml

2016-12-28 2:49 GMT+03:00 Denis Magda <dm...@apache.org>:

> I would have only those configs that are useful. Ilya Suntsov, basing on
> your experience, please suggest which configs makes sense to include into
> every Ignite release.
>
> Oleg, also please note that community decided to include not only the
> benchmarking binaries but the sources as well into every Apache Ignite
> release. I’ve update the ticket before. Hope you followed the discussion ;)
> https://issues.apache.org/jira/browse/IGNITE-4212?
> focusedCommentId=15765151&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>
> —
> Denis
>
> > On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> >
> > I mean removing those configs from binary assembly, not from repository.
> >
> > On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
> >
> >> Hello Igniters.
> >> I think it would be better to remove some configuration files from
> >> benchmarks/config:
> >>
> >> ignite-base-load-config.xml
> >> ignite-cache-load-config.xml
> >> ignite-failover-base-config.xml
> >> ignite-failover-localhost-config.xml
> >> benchmark-cache-load.properties
> >> benchmark-cache-load-win.properties
> >> benchmark-failover.properties
> >>
> >> because those configs do not relate to any of performance tests.
> >>
> >> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org>
> wrote:
> >>
> >>> Summarized the discussion updating the ticket
> >>> https://issues.apache.org/jira/browse/IGNITE-4212# <
> >>> https://issues.apache.org/jira/browse/IGNITE-4212#>
> >>>
> >>> —
> >>> Denis
> >>>
> >>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> >>> wrote:
> >>>>
> >>>> Sergey,
> >>>>
> >>>> I am not sure I like "extras". I am voting for "benchmarks" folder
> right
> >>>> under the root folder.
> >>>>
> >>>> D.
> >>>>
> >>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <skozlov@gridgain.com
> >
> >>>> wrote:
> >>>>
> >>>>> Formatting has cut lines:
> >>>>>
> >>>>> — apache_ignite_root_folder
> >>>>> — bin
> >>>>> — examples
> >>>>> — extras
> >>>>>  — benchmarks
> >>>>>    — bin
> >>>>>    — src (benchmarks sources with pom.xml)
> >>>>>    — config
> >>>>>    — libs (compiled benchmarks)
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
> skozlov@gridgain.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Denis,
> >>>>>>
> >>>>>> Mostly yes. But I look ahead and think that we may include more
> >>> things in
> >>>>>> future than yardstick only. It's why I suggest something like that:
> >>>>>> — apache_ignite_root_folder
> >>>>>>   — bin
> >>>>>>   — examples
> >>>>>>   — extras
> >>>>>>       — benchmarks
> >>>>>>           — bin
> >>>>>>           — src (benchmarks sources with pom.xml)
> >>>>>>           — config
> >>>>>>           — libs (compiled benchmarks)
> >>>>>>
> >>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dm...@apache.org>
> >>> wrote:
> >>>>>>
> >>>>>>> Well, if to refer to Dmitriy suggestion we can have the following
> >>>>>>> structure
> >>>>>>>
> >>>>>>> — apache_ignite_root_folder
> >>>>>>>   — examples
> >>>>>>>   — bin
> >>>>>>>   — benchmarks
> >>>>>>>       — bin
> >>>>>>>       — src (benchmarks sources with pom.xml)
> >>>>>>>       — config
> >>>>>>>       — libs (compiled benchmarks)
> >>>>>>>
> >>>>>>> Sergey, will it cover all the use case you’ve met previously?
> >>>>>>>
> >>>>>>> —
> >>>>>>> Denis
> >>>>>>>
> >>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <sk...@gridgain.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Yardstick requires own scripts/configurations (/bin, /config,
> /libs)
> >>>>> and
> >>>>>>>> creates work/logs directory under yardstick root. "libs/optional"
> is
> >>>>> for
> >>>>>>>> optional modules but in general we can't say that for Yardstick.
> >>> Also
> >>>>> it
> >>>>>>>> may break the current user understanding of "libs/optional"
> >>> directory
> >>>>> as
> >>>>>>>> place for additonal functionality activated by copying in "libs".
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> >>>>>>> dsetrakyan@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I would be against using libs/optional or libs/ folder for
> anything
> >>>>>>>>> benchmark related. I am also against adding any yardstick
> libraries
> >>>>>>> without
> >>>>>>>>> providing code.
> >>>>>>>>>
> >>>>>>>>> In my view, if the community wants to include benchmarks in
> >>> releases,
> >>>>>>> then
> >>>>>>>>> we should add a "benchmarks" folder, which provides everything
> >>>>>>> benchmark
> >>>>>>>>> related, from code to all the dependent libraries, and
> >>> documentation
> >>>>>>>>> instructions.
> >>>>>>>>>
> >>>>>>>>> D.
> >>>>>>>>>
> >>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Actually, “libs/optional” is already a kind of extra for me. Why
> >>> do
> >>>>> we
> >>>>>>>>>> need this new folder if “libs/optional” semantic works well?
> >>>>>>>>>>
> >>>>>>>>>> Is there anyone else who is concerned about “libs/optional”? If
> >>>>>>> there’re
> >>>>>>>>>> not, I would agree on this and get down to the implementation.
> >>>>>>>>>>
> >>>>>>>>>> —
> >>>>>>>>>> Denis
> >>>>>>>>>>
> >>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <
> skozlov@gridgain.com
> >>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi
> >>>>>>>>>>>
> >>>>>>>>>>> What's about to introduce the new root folder called 'extras'
> >>> with
> >>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <
> dmagda@apache.org
> >>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Dmitriy,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please have a look at IGNITE-4212 description (
> >>>>>>>>>> https://issues.apache.org/
> >>>>>>>>>>>> jira/browse/IGNITE-4212).
> >>>>>>>>>>>>
> >>>>>>>>>>>> The whole purpose of the ticket is to automate benchmarks
> >>>>> execution
> >>>>>>>>> for
> >>>>>>>>>>>> the end user for a specific Ignite release. Now he/she needs
> to
> >>> go
> >>>>>>>>>> through
> >>>>>>>>>>>> a number of steps like build, configure, run strictly
> following
> >>>>>>>>> lengthy
> >>>>>>>>>>>> Yardstick guidance.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Ideally, once a specific release is downloaded it should be
> >>>>> possible
> >>>>>>>>> to
> >>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The
> script
> >>>>>>> needs
> >>>>>>>>>>>> benchmarks' lib which makes sense to put under “libs/optional”
> >>>>>>> folder.
> >>>>>>>>>>>>
> >>>>>>>>>>>> If someone wants to modify the source of an existed benchmark
> or
> >>>>>>> add a
> >>>>>>>>>> new
> >>>>>>>>>>>> one then he/she needs to follow existed Yardstick guidance.
> So,
> >>> no
> >>>>>>>>> need
> >>>>>>>>>> to
> >>>>>>>>>>>> release benchmarks’s sources as a part of Ignite release.
> >>>>>>>>>>>>
> >>>>>>>>>>>> —
> >>>>>>>>>>>> Denis
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> >>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> >>>>>>> oostanin@gridgain.com
> >>>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of
> useful
> >>>>>>>>>> Yardstick
> >>>>>>>>>>>>>> benchmarks, which can be used to check Ignite performance.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> In that case, why would it be under the "libs" folder at all?
> >>> Do
> >>>>> we
> >>>>>>>>>>>> really
> >>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I would
> >>>>>>> create a
> >>>>>>>>>>>>> benchmarks folder under "examples" and add all the benchmarks
> >>>>>>> there.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> >>>>>>>>>> oostanin@gridgain.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hello Igniters!
> >>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
> >>>>>>>>>> Simplification
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
> >>>>>>>>> ignite-yardstick:
> >>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>> do you think is the most appropriate place to put a
> compiled
> >>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
> >>> assembly?
> >>>>> We
> >>>>>>>>> can
> >>>>>>>>>>>>>> put
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>> in the libs/optional along with an others optional
> >>> libraries,
> >>>>> or
> >>>>>>>>> we
> >>>>>>>>>>>> can
> >>>>>>>>>>>>>>>> create a new directory named "tools" in the root directory
> >>> and
> >>>>>>> put
> >>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Best regards
> >>>>>>>>>>>>>>>> Oleg
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Sergey Kozlov
> >>>>>>>>>>> GridGain Systems
> >>>>>>>>>>> www.gridgain.com
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Sergey Kozlov
> >>>>>>>> GridGain Systems
> >>>>>>>> www.gridgain.com
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Sergey Kozlov
> >>>>>> GridGain Systems
> >>>>>> www.gridgain.com
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Sergey Kozlov
> >>>>> GridGain Systems
> >>>>> www.gridgain.com
> >>>>>
> >>>
> >>>
> >>
>
>


-- 
Ilya Suntsov

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

I would have only those configs that are useful. Ilya Suntsov, basing on your experience, please suggest which configs makes sense to include into every Ignite release.

Oleg, also please note that community decided to include not only the benchmarking binaries but the sources as well into every Apache Ignite release. I’ve update the ticket before. Hope you followed the discussion ;)
https://issues.apache.org/jira/browse/IGNITE-4212?focusedCommentId=15765151&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15765151

—
Denis

> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> 
> I mean removing those configs from binary assembly, not from repository.
> 
> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <oo...@gridgain.com> wrote:
> 
>> Hello Igniters.
>> I think it would be better to remove some configuration files from
>> benchmarks/config:
>> 
>> ignite-base-load-config.xml
>> ignite-cache-load-config.xml
>> ignite-failover-base-config.xml
>> ignite-failover-localhost-config.xml
>> benchmark-cache-load.properties
>> benchmark-cache-load-win.properties
>> benchmark-failover.properties
>> 
>> because those configs do not relate to any of performance tests.
>> 
>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org> wrote:
>> 
>>> Summarized the discussion updating the ticket
>>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>>> 
>>> —
>>> Denis
>>> 
>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <ds...@apache.org>
>>> wrote:
>>>> 
>>>> Sergey,
>>>> 
>>>> I am not sure I like "extras". I am voting for "benchmarks" folder right
>>>> under the root folder.
>>>> 
>>>> D.
>>>> 
>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <sk...@gridgain.com>
>>>> wrote:
>>>> 
>>>>> Formatting has cut lines:
>>>>> 
>>>>> — apache_ignite_root_folder
>>>>> — bin
>>>>> — examples
>>>>> — extras
>>>>>  — benchmarks
>>>>>    — bin
>>>>>    — src (benchmarks sources with pom.xml)
>>>>>    — config
>>>>>    — libs (compiled benchmarks)
>>>>> 
>>>>> 
>>>>> 
>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <sk...@gridgain.com>
>>>>> wrote:
>>>>> 
>>>>>> Denis,
>>>>>> 
>>>>>> Mostly yes. But I look ahead and think that we may include more
>>> things in
>>>>>> future than yardstick only. It's why I suggest something like that:
>>>>>> — apache_ignite_root_folder
>>>>>>   — bin
>>>>>>   — examples
>>>>>>   — extras
>>>>>>       — benchmarks
>>>>>>           — bin
>>>>>>           — src (benchmarks sources with pom.xml)
>>>>>>           — config
>>>>>>           — libs (compiled benchmarks)
>>>>>> 
>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dm...@apache.org>
>>> wrote:
>>>>>> 
>>>>>>> Well, if to refer to Dmitriy suggestion we can have the following
>>>>>>> structure
>>>>>>> 
>>>>>>> — apache_ignite_root_folder
>>>>>>>   — examples
>>>>>>>   — bin
>>>>>>>   — benchmarks
>>>>>>>       — bin
>>>>>>>       — src (benchmarks sources with pom.xml)
>>>>>>>       — config
>>>>>>>       — libs (compiled benchmarks)
>>>>>>> 
>>>>>>> Sergey, will it cover all the use case you’ve met previously?
>>>>>>> 
>>>>>>> —
>>>>>>> Denis
>>>>>>> 
>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <sk...@gridgain.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Yardstick requires own scripts/configurations (/bin, /config, /libs)
>>>>> and
>>>>>>>> creates work/logs directory under yardstick root. "libs/optional" is
>>>>> for
>>>>>>>> optional modules but in general we can't say that for Yardstick.
>>> Also
>>>>> it
>>>>>>>> may break the current user understanding of "libs/optional"
>>> directory
>>>>> as
>>>>>>>> place for additonal functionality activated by copying in "libs".
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>>>>>>> dsetrakyan@apache.org>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> I would be against using libs/optional or libs/ folder for anything
>>>>>>>>> benchmark related. I am also against adding any yardstick libraries
>>>>>>> without
>>>>>>>>> providing code.
>>>>>>>>> 
>>>>>>>>> In my view, if the community wants to include benchmarks in
>>> releases,
>>>>>>> then
>>>>>>>>> we should add a "benchmarks" folder, which provides everything
>>>>>>> benchmark
>>>>>>>>> related, from code to all the dependent libraries, and
>>> documentation
>>>>>>>>> instructions.
>>>>>>>>> 
>>>>>>>>> D.
>>>>>>>>> 
>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org>
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for me. Why
>>> do
>>>>> we
>>>>>>>>>> need this new folder if “libs/optional” semantic works well?
>>>>>>>>>> 
>>>>>>>>>> Is there anyone else who is concerned about “libs/optional”? If
>>>>>>> there’re
>>>>>>>>>> not, I would agree on this and get down to the implementation.
>>>>>>>>>> 
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>> 
>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <skozlov@gridgain.com
>>>> 
>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi
>>>>>>>>>>> 
>>>>>>>>>>> What's about to introduce the new root folder called 'extras'
>>> with
>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dmagda@apache.org
>>>> 
>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Dmitriy,
>>>>>>>>>>>> 
>>>>>>>>>>>> Please have a look at IGNITE-4212 description (
>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>>> jira/browse/IGNITE-4212).
>>>>>>>>>>>> 
>>>>>>>>>>>> The whole purpose of the ticket is to automate benchmarks
>>>>> execution
>>>>>>>>> for
>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she needs to
>>> go
>>>>>>>>>> through
>>>>>>>>>>>> a number of steps like build, configure, run strictly following
>>>>>>>>> lengthy
>>>>>>>>>>>> Yardstick guidance.
>>>>>>>>>>>> 
>>>>>>>>>>>> Ideally, once a specific release is downloaded it should be
>>>>> possible
>>>>>>>>> to
>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The script
>>>>>>> needs
>>>>>>>>>>>> benchmarks' lib which makes sense to put under “libs/optional”
>>>>>>> folder.
>>>>>>>>>>>> 
>>>>>>>>>>>> If someone wants to modify the source of an existed benchmark or
>>>>>>> add a
>>>>>>>>>> new
>>>>>>>>>>>> one then he/she needs to follow existed Yardstick guidance. So,
>>> no
>>>>>>>>> need
>>>>>>>>>> to
>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite release.
>>>>>>>>>>>> 
>>>>>>>>>>>> —
>>>>>>>>>>>> Denis
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>>>>>>> oostanin@gridgain.com
>>>>>>>>>> 
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
>>>>>>>>>> Yardstick
>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite performance.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> In that case, why would it be under the "libs" folder at all?
>>> Do
>>>>> we
>>>>>>>>>>>> really
>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I would
>>>>>>> create a
>>>>>>>>>>>>> benchmarks folder under "examples" and add all the benchmarks
>>>>>>> there.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>>>>>>>>> oostanin@gridgain.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
>>>>>>>>>> Simplification
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>>>>>>>>> ignite-yardstick:
>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a compiled
>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
>>> assembly?
>>>>> We
>>>>>>>>> can
>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> in the libs/optional along with an others optional
>>> libraries,
>>>>> or
>>>>>>>>> we
>>>>>>>>>>>> can
>>>>>>>>>>>>>>>> create a new directory named "tools" in the root directory
>>> and
>>>>>>> put
>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>>>>> Oleg
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Sergey Kozlov
>>>>>>>>>>> GridGain Systems
>>>>>>>>>>> www.gridgain.com
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Sergey Kozlov
>>>>>>>> GridGain Systems
>>>>>>>> www.gridgain.com
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Sergey Kozlov
>>>>>> GridGain Systems
>>>>>> www.gridgain.com
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Sergey Kozlov
>>>>> GridGain Systems
>>>>> www.gridgain.com
>>>>> 
>>> 
>>> 
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

I mean removing those configs from binary assembly, not from repository.

On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <oo...@gridgain.com> wrote:

> Hello Igniters.
> I think it would be better to remove some configuration files from
> benchmarks/config:
>
> ignite-base-load-config.xml
> ignite-cache-load-config.xml
> ignite-failover-base-config.xml
> ignite-failover-localhost-config.xml
> benchmark-cache-load.properties
> benchmark-cache-load-win.properties
> benchmark-failover.properties
>
> because those configs do not relate to any of performance tests.
>
> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org> wrote:
>
>> Summarized the discussion updating the ticket
>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>>
>> —
>> Denis
>>
>> > On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>> >
>> > Sergey,
>> >
>> > I am not sure I like "extras". I am voting for "benchmarks" folder right
>> > under the root folder.
>> >
>> > D.
>> >
>> > On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <sk...@gridgain.com>
>> > wrote:
>> >
>> >> Formatting has cut lines:
>> >>
>> >> — apache_ignite_root_folder
>> >>  — bin
>> >>  — examples
>> >>  — extras
>> >>   — benchmarks
>> >>     — bin
>> >>     — src (benchmarks sources with pom.xml)
>> >>     — config
>> >>     — libs (compiled benchmarks)
>> >>
>> >>
>> >>
>> >> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <sk...@gridgain.com>
>> >> wrote:
>> >>
>> >>> Denis,
>> >>>
>> >>> Mostly yes. But I look ahead and think that we may include more
>> things in
>> >>> future than yardstick only. It's why I suggest something like that:
>> >>> — apache_ignite_root_folder
>> >>>    — bin
>> >>>    — examples
>> >>>    — extras
>> >>>        — benchmarks
>> >>>            — bin
>> >>>            — src (benchmarks sources with pom.xml)
>> >>>            — config
>> >>>            — libs (compiled benchmarks)
>> >>>
>> >>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dm...@apache.org>
>> wrote:
>> >>>
>> >>>> Well, if to refer to Dmitriy suggestion we can have the following
>> >>>> structure
>> >>>>
>> >>>> — apache_ignite_root_folder
>> >>>>    — examples
>> >>>>    — bin
>> >>>>    — benchmarks
>> >>>>        — bin
>> >>>>        — src (benchmarks sources with pom.xml)
>> >>>>        — config
>> >>>>        — libs (compiled benchmarks)
>> >>>>
>> >>>> Sergey, will it cover all the use case you’ve met previously?
>> >>>>
>> >>>> —
>> >>>> Denis
>> >>>>
>> >>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <sk...@gridgain.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Yardstick requires own scripts/configurations (/bin, /config, /libs)
>> >> and
>> >>>>> creates work/logs directory under yardstick root. "libs/optional" is
>> >> for
>> >>>>> optional modules but in general we can't say that for Yardstick.
>> Also
>> >> it
>> >>>>> may break the current user understanding of "libs/optional"
>> directory
>> >> as
>> >>>>> place for additonal functionality activated by copying in "libs".
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>> >>>> dsetrakyan@apache.org>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> I would be against using libs/optional or libs/ folder for anything
>> >>>>>> benchmark related. I am also against adding any yardstick libraries
>> >>>> without
>> >>>>>> providing code.
>> >>>>>>
>> >>>>>> In my view, if the community wants to include benchmarks in
>> releases,
>> >>>> then
>> >>>>>> we should add a "benchmarks" folder, which provides everything
>> >>>> benchmark
>> >>>>>> related, from code to all the dependent libraries, and
>> documentation
>> >>>>>> instructions.
>> >>>>>>
>> >>>>>> D.
>> >>>>>>
>> >>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org>
>> >>>> wrote:
>> >>>>>>
>> >>>>>>> Actually, “libs/optional” is already a kind of extra for me. Why
>> do
>> >> we
>> >>>>>>> need this new folder if “libs/optional” semantic works well?
>> >>>>>>>
>> >>>>>>> Is there anyone else who is concerned about “libs/optional”? If
>> >>>> there’re
>> >>>>>>> not, I would agree on this and get down to the implementation.
>> >>>>>>>
>> >>>>>>> —
>> >>>>>>> Denis
>> >>>>>>>
>> >>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <skozlov@gridgain.com
>> >
>> >>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Hi
>> >>>>>>>>
>> >>>>>>>> What's about to introduce the new root folder called 'extras'
>> with
>> >>>>>>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dmagda@apache.org
>> >
>> >>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Dmitriy,
>> >>>>>>>>>
>> >>>>>>>>> Please have a look at IGNITE-4212 description (
>> >>>>>>> https://issues.apache.org/
>> >>>>>>>>> jira/browse/IGNITE-4212).
>> >>>>>>>>>
>> >>>>>>>>> The whole purpose of the ticket is to automate benchmarks
>> >> execution
>> >>>>>> for
>> >>>>>>>>> the end user for a specific Ignite release. Now he/she needs to
>> go
>> >>>>>>> through
>> >>>>>>>>> a number of steps like build, configure, run strictly following
>> >>>>>> lengthy
>> >>>>>>>>> Yardstick guidance.
>> >>>>>>>>>
>> >>>>>>>>> Ideally, once a specific release is downloaded it should be
>> >> possible
>> >>>>>> to
>> >>>>>>>>> run a concrete benchmark with a ready-to-use script. The script
>> >>>> needs
>> >>>>>>>>> benchmarks' lib which makes sense to put under “libs/optional”
>> >>>> folder.
>> >>>>>>>>>
>> >>>>>>>>> If someone wants to modify the source of an existed benchmark or
>> >>>> add a
>> >>>>>>> new
>> >>>>>>>>> one then he/she needs to follow existed Yardstick guidance. So,
>> no
>> >>>>>> need
>> >>>>>>> to
>> >>>>>>>>> release benchmarks’s sources as a part of Ignite release.
>> >>>>>>>>>
>> >>>>>>>>> —
>> >>>>>>>>> Denis
>> >>>>>>>>>
>> >>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>> >>>>>> dsetrakyan@apache.org>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>> >>>> oostanin@gridgain.com
>> >>>>>>>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
>> >>>>>>> Yardstick
>> >>>>>>>>>>> benchmarks, which can be used to check Ignite performance.
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> In that case, why would it be under the "libs" folder at all?
>> Do
>> >> we
>> >>>>>>>>> really
>> >>>>>>>>>> need to include benchmarks into Ignite? If yes, then I would
>> >>>> create a
>> >>>>>>>>>> benchmarks folder under "examples" and add all the benchmarks
>> >>>> there.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>> >>>>>>>>> dsetrakyan@apache.org
>> >>>>>>>>>>>>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>> >>>>>>> oostanin@gridgain.com>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Hello Igniters!
>> >>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
>> >>>>>>> Simplification
>> >>>>>>>>>>> and
>> >>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>> >>>>>> ignite-yardstick:
>> >>>>>>>>>>>> where
>> >>>>>>>>>>>>> do you think is the most appropriate place to put a compiled
>> >>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary
>> assembly?
>> >> We
>> >>>>>> can
>> >>>>>>>>>>> put
>> >>>>>>>>>>>> it
>> >>>>>>>>>>>>> in the libs/optional along with an others optional
>> libraries,
>> >> or
>> >>>>>> we
>> >>>>>>>>> can
>> >>>>>>>>>>>>> create a new directory named "tools" in the root directory
>> and
>> >>>> put
>> >>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Best regards
>> >>>>>>>>>>>>> Oleg
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Sergey Kozlov
>> >>>>>>>> GridGain Systems
>> >>>>>>>> www.gridgain.com
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Sergey Kozlov
>> >>>>> GridGain Systems
>> >>>>> www.gridgain.com
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> Sergey Kozlov
>> >>> GridGain Systems
>> >>> www.gridgain.com
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Sergey Kozlov
>> >> GridGain Systems
>> >> www.gridgain.com
>> >>
>>
>>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

Hello Igniters.
I think it would be better to remove some configuration files from
benchmarks/config:

ignite-base-load-config.xml
ignite-cache-load-config.xml
ignite-failover-base-config.xml
ignite-failover-localhost-config.xml
benchmark-cache-load.properties
benchmark-cache-load-win.properties
benchmark-failover.properties

because those configs do not relate to any of performance tests.

On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dm...@apache.org> wrote:

> Summarized the discussion updating the ticket
> https://issues.apache.org/jira/browse/IGNITE-4212# <
> https://issues.apache.org/jira/browse/IGNITE-4212#>
>
> —
> Denis
>
> > On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >
> > Sergey,
> >
> > I am not sure I like "extras". I am voting for "benchmarks" folder right
> > under the root folder.
> >
> > D.
> >
> > On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <sk...@gridgain.com>
> > wrote:
> >
> >> Formatting has cut lines:
> >>
> >> — apache_ignite_root_folder
> >>  — bin
> >>  — examples
> >>  — extras
> >>   — benchmarks
> >>     — bin
> >>     — src (benchmarks sources with pom.xml)
> >>     — config
> >>     — libs (compiled benchmarks)
> >>
> >>
> >>
> >> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <sk...@gridgain.com>
> >> wrote:
> >>
> >>> Denis,
> >>>
> >>> Mostly yes. But I look ahead and think that we may include more things
> in
> >>> future than yardstick only. It's why I suggest something like that:
> >>> — apache_ignite_root_folder
> >>>    — bin
> >>>    — examples
> >>>    — extras
> >>>        — benchmarks
> >>>            — bin
> >>>            — src (benchmarks sources with pom.xml)
> >>>            — config
> >>>            — libs (compiled benchmarks)
> >>>
> >>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dm...@apache.org>
> wrote:
> >>>
> >>>> Well, if to refer to Dmitriy suggestion we can have the following
> >>>> structure
> >>>>
> >>>> — apache_ignite_root_folder
> >>>>    — examples
> >>>>    — bin
> >>>>    — benchmarks
> >>>>        — bin
> >>>>        — src (benchmarks sources with pom.xml)
> >>>>        — config
> >>>>        — libs (compiled benchmarks)
> >>>>
> >>>> Sergey, will it cover all the use case you’ve met previously?
> >>>>
> >>>> —
> >>>> Denis
> >>>>
> >>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <sk...@gridgain.com>
> >>>> wrote:
> >>>>>
> >>>>> Yardstick requires own scripts/configurations (/bin, /config, /libs)
> >> and
> >>>>> creates work/logs directory under yardstick root. "libs/optional" is
> >> for
> >>>>> optional modules but in general we can't say that for Yardstick. Also
> >> it
> >>>>> may break the current user understanding of "libs/optional" directory
> >> as
> >>>>> place for additonal functionality activated by copying in "libs".
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> >>>> dsetrakyan@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>>> I would be against using libs/optional or libs/ folder for anything
> >>>>>> benchmark related. I am also against adding any yardstick libraries
> >>>> without
> >>>>>> providing code.
> >>>>>>
> >>>>>> In my view, if the community wants to include benchmarks in
> releases,
> >>>> then
> >>>>>> we should add a "benchmarks" folder, which provides everything
> >>>> benchmark
> >>>>>> related, from code to all the dependent libraries, and documentation
> >>>>>> instructions.
> >>>>>>
> >>>>>> D.
> >>>>>>
> >>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org>
> >>>> wrote:
> >>>>>>
> >>>>>>> Actually, “libs/optional” is already a kind of extra for me. Why do
> >> we
> >>>>>>> need this new folder if “libs/optional” semantic works well?
> >>>>>>>
> >>>>>>> Is there anyone else who is concerned about “libs/optional”? If
> >>>> there’re
> >>>>>>> not, I would agree on this and get down to the implementation.
> >>>>>>>
> >>>>>>> —
> >>>>>>> Denis
> >>>>>>>
> >>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hi
> >>>>>>>>
> >>>>>>>> What's about to introduce the new root folder called 'extras' with
> >>>>>>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Dmitriy,
> >>>>>>>>>
> >>>>>>>>> Please have a look at IGNITE-4212 description (
> >>>>>>> https://issues.apache.org/
> >>>>>>>>> jira/browse/IGNITE-4212).
> >>>>>>>>>
> >>>>>>>>> The whole purpose of the ticket is to automate benchmarks
> >> execution
> >>>>>> for
> >>>>>>>>> the end user for a specific Ignite release. Now he/she needs to
> go
> >>>>>>> through
> >>>>>>>>> a number of steps like build, configure, run strictly following
> >>>>>> lengthy
> >>>>>>>>> Yardstick guidance.
> >>>>>>>>>
> >>>>>>>>> Ideally, once a specific release is downloaded it should be
> >> possible
> >>>>>> to
> >>>>>>>>> run a concrete benchmark with a ready-to-use script. The script
> >>>> needs
> >>>>>>>>> benchmarks' lib which makes sense to put under “libs/optional”
> >>>> folder.
> >>>>>>>>>
> >>>>>>>>> If someone wants to modify the source of an existed benchmark or
> >>>> add a
> >>>>>>> new
> >>>>>>>>> one then he/she needs to follow existed Yardstick guidance. So,
> no
> >>>>>> need
> >>>>>>> to
> >>>>>>>>> release benchmarks’s sources as a part of Ignite release.
> >>>>>>>>>
> >>>>>>>>> —
> >>>>>>>>> Denis
> >>>>>>>>>
> >>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> >>>>>> dsetrakyan@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> >>>> oostanin@gridgain.com
> >>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
> >>>>>>> Yardstick
> >>>>>>>>>>> benchmarks, which can be used to check Ignite performance.
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> In that case, why would it be under the "libs" folder at all? Do
> >> we
> >>>>>>>>> really
> >>>>>>>>>> need to include benchmarks into Ignite? If yes, then I would
> >>>> create a
> >>>>>>>>>> benchmarks folder under "examples" and add all the benchmarks
> >>>> there.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> >>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> >>>>>>> oostanin@gridgain.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hello Igniters!
> >>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
> >>>>>>> Simplification
> >>>>>>>>>>> and
> >>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
> >>>>>> ignite-yardstick:
> >>>>>>>>>>>> where
> >>>>>>>>>>>>> do you think is the most appropriate place to put a compiled
> >>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary assembly?
> >> We
> >>>>>> can
> >>>>>>>>>>> put
> >>>>>>>>>>>> it
> >>>>>>>>>>>>> in the libs/optional along with an others optional libraries,
> >> or
> >>>>>> we
> >>>>>>>>> can
> >>>>>>>>>>>>> create a new directory named "tools" in the root directory
> and
> >>>> put
> >>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Best regards
> >>>>>>>>>>>>> Oleg
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Sergey Kozlov
> >>>>>>>> GridGain Systems
> >>>>>>>> www.gridgain.com
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Sergey Kozlov
> >>>>> GridGain Systems
> >>>>> www.gridgain.com
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Sergey Kozlov
> >>> GridGain Systems
> >>> www.gridgain.com
> >>>
> >>
> >>
> >>
> >> --
> >> Sergey Kozlov
> >> GridGain Systems
> >> www.gridgain.com
> >>
>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Summarized the discussion updating the ticket
https://issues.apache.org/jira/browse/IGNITE-4212# <https://issues.apache.org/jira/browse/IGNITE-4212#>

—
Denis

> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <ds...@apache.org> wrote:
> 
> Sergey,
> 
> I am not sure I like "extras". I am voting for "benchmarks" folder right
> under the root folder.
> 
> D.
> 
> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <sk...@gridgain.com>
> wrote:
> 
>> Formatting has cut lines:
>> 
>> — apache_ignite_root_folder
>>  — bin
>>  — examples
>>  — extras
>>   — benchmarks
>>     — bin
>>     — src (benchmarks sources with pom.xml)
>>     — config
>>     — libs (compiled benchmarks)
>> 
>> 
>> 
>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <sk...@gridgain.com>
>> wrote:
>> 
>>> Denis,
>>> 
>>> Mostly yes. But I look ahead and think that we may include more things in
>>> future than yardstick only. It's why I suggest something like that:
>>> — apache_ignite_root_folder
>>>    — bin
>>>    — examples
>>>    — extras
>>>        — benchmarks
>>>            — bin
>>>            — src (benchmarks sources with pom.xml)
>>>            — config
>>>            — libs (compiled benchmarks)
>>> 
>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dm...@apache.org> wrote:
>>> 
>>>> Well, if to refer to Dmitriy suggestion we can have the following
>>>> structure
>>>> 
>>>> — apache_ignite_root_folder
>>>>    — examples
>>>>    — bin
>>>>    — benchmarks
>>>>        — bin
>>>>        — src (benchmarks sources with pom.xml)
>>>>        — config
>>>>        — libs (compiled benchmarks)
>>>> 
>>>> Sergey, will it cover all the use case you’ve met previously?
>>>> 
>>>> —
>>>> Denis
>>>> 
>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <sk...@gridgain.com>
>>>> wrote:
>>>>> 
>>>>> Yardstick requires own scripts/configurations (/bin, /config, /libs)
>> and
>>>>> creates work/logs directory under yardstick root. "libs/optional" is
>> for
>>>>> optional modules but in general we can't say that for Yardstick. Also
>> it
>>>>> may break the current user understanding of "libs/optional" directory
>> as
>>>>> place for additonal functionality activated by copying in "libs".
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>>>> dsetrakyan@apache.org>
>>>>> wrote:
>>>>> 
>>>>>> I would be against using libs/optional or libs/ folder for anything
>>>>>> benchmark related. I am also against adding any yardstick libraries
>>>> without
>>>>>> providing code.
>>>>>> 
>>>>>> In my view, if the community wants to include benchmarks in releases,
>>>> then
>>>>>> we should add a "benchmarks" folder, which provides everything
>>>> benchmark
>>>>>> related, from code to all the dependent libraries, and documentation
>>>>>> instructions.
>>>>>> 
>>>>>> D.
>>>>>> 
>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org>
>>>> wrote:
>>>>>> 
>>>>>>> Actually, “libs/optional” is already a kind of extra for me. Why do
>> we
>>>>>>> need this new folder if “libs/optional” semantic works well?
>>>>>>> 
>>>>>>> Is there anyone else who is concerned about “libs/optional”? If
>>>> there’re
>>>>>>> not, I would agree on this and get down to the implementation.
>>>>>>> 
>>>>>>> —
>>>>>>> Denis
>>>>>>> 
>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com>
>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hi
>>>>>>>> 
>>>>>>>> What's about to introduce the new root folder called 'extras' with
>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org>
>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Dmitriy,
>>>>>>>>> 
>>>>>>>>> Please have a look at IGNITE-4212 description (
>>>>>>> https://issues.apache.org/
>>>>>>>>> jira/browse/IGNITE-4212).
>>>>>>>>> 
>>>>>>>>> The whole purpose of the ticket is to automate benchmarks
>> execution
>>>>>> for
>>>>>>>>> the end user for a specific Ignite release. Now he/she needs to go
>>>>>>> through
>>>>>>>>> a number of steps like build, configure, run strictly following
>>>>>> lengthy
>>>>>>>>> Yardstick guidance.
>>>>>>>>> 
>>>>>>>>> Ideally, once a specific release is downloaded it should be
>> possible
>>>>>> to
>>>>>>>>> run a concrete benchmark with a ready-to-use script. The script
>>>> needs
>>>>>>>>> benchmarks' lib which makes sense to put under “libs/optional”
>>>> folder.
>>>>>>>>> 
>>>>>>>>> If someone wants to modify the source of an existed benchmark or
>>>> add a
>>>>>>> new
>>>>>>>>> one then he/she needs to follow existed Yardstick guidance. So, no
>>>>>> need
>>>>>>> to
>>>>>>>>> release benchmarks’s sources as a part of Ignite release.
>>>>>>>>> 
>>>>>>>>> —
>>>>>>>>> Denis
>>>>>>>>> 
>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>>>>>> dsetrakyan@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>>>> oostanin@gridgain.com
>>>>>>> 
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
>>>>>>> Yardstick
>>>>>>>>>>> benchmarks, which can be used to check Ignite performance.
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> In that case, why would it be under the "libs" folder at all? Do
>> we
>>>>>>>>> really
>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I would
>>>> create a
>>>>>>>>>> benchmarks folder under "examples" and add all the benchmarks
>>>> there.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>> 
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>>>>>> oostanin@gridgain.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
>>>>>>> Simplification
>>>>>>>>>>> and
>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about
>>>>>> ignite-yardstick:
>>>>>>>>>>>> where
>>>>>>>>>>>>> do you think is the most appropriate place to put a compiled
>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary assembly?
>> We
>>>>>> can
>>>>>>>>>>> put
>>>>>>>>>>>> it
>>>>>>>>>>>>> in the libs/optional along with an others optional libraries,
>> or
>>>>>> we
>>>>>>>>> can
>>>>>>>>>>>>> create a new directory named "tools" in the root directory and
>>>> put
>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>> Oleg
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Sergey Kozlov
>>>>>>>> GridGain Systems
>>>>>>>> www.gridgain.com
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Sergey Kozlov
>>>>> GridGain Systems
>>>>> www.gridgain.com
>>>> 
>>>> 
>>> 
>>> 
>>> --
>>> Sergey Kozlov
>>> GridGain Systems
>>> www.gridgain.com
>>> 
>> 
>> 
>> 
>> --
>> Sergey Kozlov
>> GridGain Systems
>> www.gridgain.com
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Dmitriy Setrakyan <ds...@apache.org>.

Sergey,

I am not sure I like "extras". I am voting for "benchmarks" folder right
under the root folder.

D.

On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <sk...@gridgain.com>
wrote:

> Formatting has cut lines:
>
> — apache_ignite_root_folder
>   — bin
>   — examples
>   — extras
>    — benchmarks
>      — bin
>      — src (benchmarks sources with pom.xml)
>      — config
>      — libs (compiled benchmarks)
>
>
>
> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <sk...@gridgain.com>
> wrote:
>
> > Denis,
> >
> > Mostly yes. But I look ahead and think that we may include more things in
> > future than yardstick only. It's why I suggest something like that:
> > — apache_ignite_root_folder
> >     — bin
> >     — examples
> >     — extras
> >         — benchmarks
> >             — bin
> >             — src (benchmarks sources with pom.xml)
> >             — config
> >             — libs (compiled benchmarks)
> >
> > On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dm...@apache.org> wrote:
> >
> >> Well, if to refer to Dmitriy suggestion we can have the following
> >> structure
> >>
> >> — apache_ignite_root_folder
> >>     — examples
> >>     — bin
> >>     — benchmarks
> >>         — bin
> >>         — src (benchmarks sources with pom.xml)
> >>         — config
> >>         — libs (compiled benchmarks)
> >>
> >> Sergey, will it cover all the use case you’ve met previously?
> >>
> >> —
> >> Denis
> >>
> >> > On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <sk...@gridgain.com>
> >> wrote:
> >> >
> >> > Yardstick requires own scripts/configurations (/bin, /config, /libs)
> and
> >> > creates work/logs directory under yardstick root. "libs/optional" is
> for
> >> > optional modules but in general we can't say that for Yardstick. Also
> it
> >> > may break the current user understanding of "libs/optional" directory
> as
> >> > place for additonal functionality activated by copying in "libs".
> >> >
> >> >
> >> >
> >> >
> >> > On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> >> dsetrakyan@apache.org>
> >> > wrote:
> >> >
> >> >> I would be against using libs/optional or libs/ folder for anything
> >> >> benchmark related. I am also against adding any yardstick libraries
> >> without
> >> >> providing code.
> >> >>
> >> >> In my view, if the community wants to include benchmarks in releases,
> >> then
> >> >> we should add a "benchmarks" folder, which provides everything
> >> benchmark
> >> >> related, from code to all the dependent libraries, and documentation
> >> >> instructions.
> >> >>
> >> >> D.
> >> >>
> >> >> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org>
> >> wrote:
> >> >>
> >> >>> Actually, “libs/optional” is already a kind of extra for me. Why do
> we
> >> >>> need this new folder if “libs/optional” semantic works well?
> >> >>>
> >> >>> Is there anyone else who is concerned about “libs/optional”? If
> >> there’re
> >> >>> not, I would agree on this and get down to the implementation.
> >> >>>
> >> >>> —
> >> >>> Denis
> >> >>>
> >> >>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com>
> >> >> wrote:
> >> >>>>
> >> >>>> Hi
> >> >>>>
> >> >>>> What's about to introduce the new root folder called 'extras' with
> >> >>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
> >> >>>>
> >> >>>>
> >> >>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org>
> >> >> wrote:
> >> >>>>
> >> >>>>> Dmitriy,
> >> >>>>>
> >> >>>>> Please have a look at IGNITE-4212 description (
> >> >>> https://issues.apache.org/
> >> >>>>> jira/browse/IGNITE-4212).
> >> >>>>>
> >> >>>>> The whole purpose of the ticket is to automate benchmarks
> execution
> >> >> for
> >> >>>>> the end user for a specific Ignite release. Now he/she needs to go
> >> >>> through
> >> >>>>> a number of steps like build, configure, run strictly following
> >> >> lengthy
> >> >>>>> Yardstick guidance.
> >> >>>>>
> >> >>>>> Ideally, once a specific release is downloaded it should be
> possible
> >> >> to
> >> >>>>> run a concrete benchmark with a ready-to-use script. The script
> >> needs
> >> >>>>> benchmarks' lib which makes sense to put under “libs/optional”
> >> folder.
> >> >>>>>
> >> >>>>> If someone wants to modify the source of an existed benchmark or
> >> add a
> >> >>> new
> >> >>>>> one then he/she needs to follow existed Yardstick guidance. So, no
> >> >> need
> >> >>> to
> >> >>>>> release benchmarks’s sources as a part of Ignite release.
> >> >>>>>
> >> >>>>> —
> >> >>>>> Denis
> >> >>>>>
> >> >>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> >> >> dsetrakyan@apache.org>
> >> >>>>> wrote:
> >> >>>>>>
> >> >>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> >> oostanin@gridgain.com
> >> >>>
> >> >>>>> wrote:
> >> >>>>>>
> >> >>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
> >> >>> Yardstick
> >> >>>>>>> benchmarks, which can be used to check Ignite performance.
> >> >>>>>>>
> >> >>>>>>
> >> >>>>>> In that case, why would it be under the "libs" folder at all? Do
> we
> >> >>>>> really
> >> >>>>>> need to include benchmarks into Ignite? If yes, then I would
> >> create a
> >> >>>>>> benchmarks folder under "examples" and add all the benchmarks
> >> there.
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>>
> >> >>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> >> >>>>> dsetrakyan@apache.org
> >> >>>>>>>>
> >> >>>>>>> wrote:
> >> >>>>>>>
> >> >>>>>>>> Oleg, what does ignite-yardstick module do?
> >> >>>>>>>>
> >> >>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> >> >>> oostanin@gridgain.com>
> >> >>>>>>>> wrote:
> >> >>>>>>>>
> >> >>>>>>>>> Hello Igniters!
> >> >>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
> >> >>> Simplification
> >> >>>>>>> and
> >> >>>>>>>>> Automation" and I'd like to ask your opinion about
> >> >> ignite-yardstick:
> >> >>>>>>>> where
> >> >>>>>>>>> do you think is the most appropriate place to put a compiled
> >> >>>>>>>>> ignite-yardstick module in the apache-ignite binary assembly?
> We
> >> >> can
> >> >>>>>>> put
> >> >>>>>>>> it
> >> >>>>>>>>> in the libs/optional along with an others optional libraries,
> or
> >> >> we
> >> >>>>> can
> >> >>>>>>>>> create a new directory named "tools" in the root directory and
> >> put
> >> >>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
> >> >>>>>>>>>
> >> >>>>>>>>> Best regards
> >> >>>>>>>>> Oleg
> >> >>>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>
> >> >>>>
> >> >>>> --
> >> >>>> Sergey Kozlov
> >> >>>> GridGain Systems
> >> >>>> www.gridgain.com
> >> >>>
> >> >>>
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Sergey Kozlov
> >> > GridGain Systems
> >> > www.gridgain.com
> >>
> >>
> >
> >
> > --
> > Sergey Kozlov
> > GridGain Systems
> > www.gridgain.com
> >
>
>
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Sergey Kozlov <sk...@gridgain.com>.

Formatting has cut lines:

— apache_ignite_root_folder
  — bin
  — examples
  — extras
   — benchmarks
     — bin
     — src (benchmarks sources with pom.xml)
     — config
     — libs (compiled benchmarks)



On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <sk...@gridgain.com>
wrote:

> Denis,
>
> Mostly yes. But I look ahead and think that we may include more things in
> future than yardstick only. It's why I suggest something like that:
> — apache_ignite_root_folder
>     — bin
>     — examples
>     — extras
>         — benchmarks
>             — bin
>             — src (benchmarks sources with pom.xml)
>             — config
>             — libs (compiled benchmarks)
>
> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dm...@apache.org> wrote:
>
>> Well, if to refer to Dmitriy suggestion we can have the following
>> structure
>>
>> — apache_ignite_root_folder
>>     — examples
>>     — bin
>>     — benchmarks
>>         — bin
>>         — src (benchmarks sources with pom.xml)
>>         — config
>>         — libs (compiled benchmarks)
>>
>> Sergey, will it cover all the use case you’ve met previously?
>>
>> —
>> Denis
>>
>> > On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <sk...@gridgain.com>
>> wrote:
>> >
>> > Yardstick requires own scripts/configurations (/bin, /config, /libs) and
>> > creates work/logs directory under yardstick root. "libs/optional" is for
>> > optional modules but in general we can't say that for Yardstick. Also it
>> > may break the current user understanding of "libs/optional" directory as
>> > place for additonal functionality activated by copying in "libs".
>> >
>> >
>> >
>> >
>> > On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>> > wrote:
>> >
>> >> I would be against using libs/optional or libs/ folder for anything
>> >> benchmark related. I am also against adding any yardstick libraries
>> without
>> >> providing code.
>> >>
>> >> In my view, if the community wants to include benchmarks in releases,
>> then
>> >> we should add a "benchmarks" folder, which provides everything
>> benchmark
>> >> related, from code to all the dependent libraries, and documentation
>> >> instructions.
>> >>
>> >> D.
>> >>
>> >> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org>
>> wrote:
>> >>
>> >>> Actually, “libs/optional” is already a kind of extra for me. Why do we
>> >>> need this new folder if “libs/optional” semantic works well?
>> >>>
>> >>> Is there anyone else who is concerned about “libs/optional”? If
>> there’re
>> >>> not, I would agree on this and get down to the implementation.
>> >>>
>> >>> —
>> >>> Denis
>> >>>
>> >>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com>
>> >> wrote:
>> >>>>
>> >>>> Hi
>> >>>>
>> >>>> What's about to introduce the new root folder called 'extras' with
>> >>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
>> >>>>
>> >>>>
>> >>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org>
>> >> wrote:
>> >>>>
>> >>>>> Dmitriy,
>> >>>>>
>> >>>>> Please have a look at IGNITE-4212 description (
>> >>> https://issues.apache.org/
>> >>>>> jira/browse/IGNITE-4212).
>> >>>>>
>> >>>>> The whole purpose of the ticket is to automate benchmarks execution
>> >> for
>> >>>>> the end user for a specific Ignite release. Now he/she needs to go
>> >>> through
>> >>>>> a number of steps like build, configure, run strictly following
>> >> lengthy
>> >>>>> Yardstick guidance.
>> >>>>>
>> >>>>> Ideally, once a specific release is downloaded it should be possible
>> >> to
>> >>>>> run a concrete benchmark with a ready-to-use script. The script
>> needs
>> >>>>> benchmarks' lib which makes sense to put under “libs/optional”
>> folder.
>> >>>>>
>> >>>>> If someone wants to modify the source of an existed benchmark or
>> add a
>> >>> new
>> >>>>> one then he/she needs to follow existed Yardstick guidance. So, no
>> >> need
>> >>> to
>> >>>>> release benchmarks’s sources as a part of Ignite release.
>> >>>>>
>> >>>>> —
>> >>>>> Denis
>> >>>>>
>> >>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>> >> dsetrakyan@apache.org>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
>> oostanin@gridgain.com
>> >>>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
>> >>> Yardstick
>> >>>>>>> benchmarks, which can be used to check Ignite performance.
>> >>>>>>>
>> >>>>>>
>> >>>>>> In that case, why would it be under the "libs" folder at all? Do we
>> >>>>> really
>> >>>>>> need to include benchmarks into Ignite? If yes, then I would
>> create a
>> >>>>>> benchmarks folder under "examples" and add all the benchmarks
>> there.
>> >>>>>>
>> >>>>>>
>> >>>>>>>
>> >>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>> >>>>> dsetrakyan@apache.org
>> >>>>>>>>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Oleg, what does ignite-yardstick module do?
>> >>>>>>>>
>> >>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>> >>> oostanin@gridgain.com>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hello Igniters!
>> >>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
>> >>> Simplification
>> >>>>>>> and
>> >>>>>>>>> Automation" and I'd like to ask your opinion about
>> >> ignite-yardstick:
>> >>>>>>>> where
>> >>>>>>>>> do you think is the most appropriate place to put a compiled
>> >>>>>>>>> ignite-yardstick module in the apache-ignite binary assembly? We
>> >> can
>> >>>>>>> put
>> >>>>>>>> it
>> >>>>>>>>> in the libs/optional along with an others optional libraries, or
>> >> we
>> >>>>> can
>> >>>>>>>>> create a new directory named "tools" in the root directory and
>> put
>> >>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
>> >>>>>>>>>
>> >>>>>>>>> Best regards
>> >>>>>>>>> Oleg
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Sergey Kozlov
>> >>>> GridGain Systems
>> >>>> www.gridgain.com
>> >>>
>> >>>
>> >>
>> >
>> >
>> >
>> > --
>> > Sergey Kozlov
>> > GridGain Systems
>> > www.gridgain.com
>>
>>
>
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Sergey Kozlov <sk...@gridgain.com>.

Denis,

Mostly yes. But I look ahead and think that we may include more things in
future than yardstick only. It's why I suggest something like that:
— apache_ignite_root_folder
    — bin
    — examples
    — extras
        — benchmarks
            — bin
            — src (benchmarks sources with pom.xml)
            — config
            — libs (compiled benchmarks)

On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dm...@apache.org> wrote:

> Well, if to refer to Dmitriy suggestion we can have the following structure
>
> — apache_ignite_root_folder
>     — examples
>     — bin
>     — benchmarks
>         — bin
>         — src (benchmarks sources with pom.xml)
>         — config
>         — libs (compiled benchmarks)
>
> Sergey, will it cover all the use case you’ve met previously?
>
> —
> Denis
>
> > On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
> >
> > Yardstick requires own scripts/configurations (/bin, /config, /libs) and
> > creates work/logs directory under yardstick root. "libs/optional" is for
> > optional modules but in general we can't say that for Yardstick. Also it
> > may break the current user understanding of "libs/optional" directory as
> > place for additonal functionality activated by copying in "libs".
> >
> >
> >
> >
> > On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> >> I would be against using libs/optional or libs/ folder for anything
> >> benchmark related. I am also against adding any yardstick libraries
> without
> >> providing code.
> >>
> >> In my view, if the community wants to include benchmarks in releases,
> then
> >> we should add a "benchmarks" folder, which provides everything benchmark
> >> related, from code to all the dependent libraries, and documentation
> >> instructions.
> >>
> >> D.
> >>
> >> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org> wrote:
> >>
> >>> Actually, “libs/optional” is already a kind of extra for me. Why do we
> >>> need this new folder if “libs/optional” semantic works well?
> >>>
> >>> Is there anyone else who is concerned about “libs/optional”? If
> there’re
> >>> not, I would agree on this and get down to the implementation.
> >>>
> >>> —
> >>> Denis
> >>>
> >>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com>
> >> wrote:
> >>>>
> >>>> Hi
> >>>>
> >>>> What's about to introduce the new root folder called 'extras' with
> >>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
> >>>>
> >>>>
> >>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org>
> >> wrote:
> >>>>
> >>>>> Dmitriy,
> >>>>>
> >>>>> Please have a look at IGNITE-4212 description (
> >>> https://issues.apache.org/
> >>>>> jira/browse/IGNITE-4212).
> >>>>>
> >>>>> The whole purpose of the ticket is to automate benchmarks execution
> >> for
> >>>>> the end user for a specific Ignite release. Now he/she needs to go
> >>> through
> >>>>> a number of steps like build, configure, run strictly following
> >> lengthy
> >>>>> Yardstick guidance.
> >>>>>
> >>>>> Ideally, once a specific release is downloaded it should be possible
> >> to
> >>>>> run a concrete benchmark with a ready-to-use script. The script needs
> >>>>> benchmarks' lib which makes sense to put under “libs/optional”
> folder.
> >>>>>
> >>>>> If someone wants to modify the source of an existed benchmark or add
> a
> >>> new
> >>>>> one then he/she needs to follow existed Yardstick guidance. So, no
> >> need
> >>> to
> >>>>> release benchmarks’s sources as a part of Ignite release.
> >>>>>
> >>>>> —
> >>>>> Denis
> >>>>>
> >>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> >> dsetrakyan@apache.org>
> >>>>> wrote:
> >>>>>>
> >>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <
> oostanin@gridgain.com
> >>>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
> >>> Yardstick
> >>>>>>> benchmarks, which can be used to check Ignite performance.
> >>>>>>>
> >>>>>>
> >>>>>> In that case, why would it be under the "libs" folder at all? Do we
> >>>>> really
> >>>>>> need to include benchmarks into Ignite? If yes, then I would create
> a
> >>>>>> benchmarks folder under "examples" and add all the benchmarks there.
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> >>>>> dsetrakyan@apache.org
> >>>>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Oleg, what does ignite-yardstick module do?
> >>>>>>>>
> >>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> >>> oostanin@gridgain.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hello Igniters!
> >>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
> >>> Simplification
> >>>>>>> and
> >>>>>>>>> Automation" and I'd like to ask your opinion about
> >> ignite-yardstick:
> >>>>>>>> where
> >>>>>>>>> do you think is the most appropriate place to put a compiled
> >>>>>>>>> ignite-yardstick module in the apache-ignite binary assembly? We
> >> can
> >>>>>>> put
> >>>>>>>> it
> >>>>>>>>> in the libs/optional along with an others optional libraries, or
> >> we
> >>>>> can
> >>>>>>>>> create a new directory named "tools" in the root directory and
> put
> >>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
> >>>>>>>>>
> >>>>>>>>> Best regards
> >>>>>>>>> Oleg
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Sergey Kozlov
> >>>> GridGain Systems
> >>>> www.gridgain.com
> >>>
> >>>
> >>
> >
> >
> >
> > --
> > Sergey Kozlov
> > GridGain Systems
> > www.gridgain.com
>
>


-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Well, if to refer to Dmitriy suggestion we can have the following structure

— apache_ignite_root_folder
    — examples
    — bin
    — benchmarks
        — bin
        — src (benchmarks sources with pom.xml)
        — config
        — libs (compiled benchmarks)

Sergey, will it cover all the use case you’ve met previously?

—
Denis

> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
> 
> Yardstick requires own scripts/configurations (/bin, /config, /libs) and
> creates work/logs directory under yardstick root. "libs/optional" is for
> optional modules but in general we can't say that for Yardstick. Also it
> may break the current user understanding of "libs/optional" directory as
> place for additonal functionality activated by copying in "libs".
> 
> 
> 
> 
> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> 
>> I would be against using libs/optional or libs/ folder for anything
>> benchmark related. I am also against adding any yardstick libraries without
>> providing code.
>> 
>> In my view, if the community wants to include benchmarks in releases, then
>> we should add a "benchmarks" folder, which provides everything benchmark
>> related, from code to all the dependent libraries, and documentation
>> instructions.
>> 
>> D.
>> 
>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org> wrote:
>> 
>>> Actually, “libs/optional” is already a kind of extra for me. Why do we
>>> need this new folder if “libs/optional” semantic works well?
>>> 
>>> Is there anyone else who is concerned about “libs/optional”? If there’re
>>> not, I would agree on this and get down to the implementation.
>>> 
>>> —
>>> Denis
>>> 
>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com>
>> wrote:
>>>> 
>>>> Hi
>>>> 
>>>> What's about to introduce the new root folder called 'extras' with
>>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
>>>> 
>>>> 
>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org>
>> wrote:
>>>> 
>>>>> Dmitriy,
>>>>> 
>>>>> Please have a look at IGNITE-4212 description (
>>> https://issues.apache.org/
>>>>> jira/browse/IGNITE-4212).
>>>>> 
>>>>> The whole purpose of the ticket is to automate benchmarks execution
>> for
>>>>> the end user for a specific Ignite release. Now he/she needs to go
>>> through
>>>>> a number of steps like build, configure, run strictly following
>> lengthy
>>>>> Yardstick guidance.
>>>>> 
>>>>> Ideally, once a specific release is downloaded it should be possible
>> to
>>>>> run a concrete benchmark with a ready-to-use script. The script needs
>>>>> benchmarks' lib which makes sense to put under “libs/optional” folder.
>>>>> 
>>>>> If someone wants to modify the source of an existed benchmark or add a
>>> new
>>>>> one then he/she needs to follow existed Yardstick guidance. So, no
>> need
>>> to
>>>>> release benchmarks’s sources as a part of Ignite release.
>>>>> 
>>>>> —
>>>>> Denis
>>>>> 
>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>>>>> wrote:
>>>>>> 
>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oostanin@gridgain.com
>>> 
>>>>> wrote:
>>>>>> 
>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
>>> Yardstick
>>>>>>> benchmarks, which can be used to check Ignite performance.
>>>>>>> 
>>>>>> 
>>>>>> In that case, why would it be under the "libs" folder at all? Do we
>>>>> really
>>>>>> need to include benchmarks into Ignite? If yes, then I would create a
>>>>>> benchmarks folder under "examples" and add all the benchmarks there.
>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>>>>> dsetrakyan@apache.org
>>>>>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>>> 
>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>> oostanin@gridgain.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hello Igniters!
>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
>>> Simplification
>>>>>>> and
>>>>>>>>> Automation" and I'd like to ask your opinion about
>> ignite-yardstick:
>>>>>>>> where
>>>>>>>>> do you think is the most appropriate place to put a compiled
>>>>>>>>> ignite-yardstick module in the apache-ignite binary assembly? We
>> can
>>>>>>> put
>>>>>>>> it
>>>>>>>>> in the libs/optional along with an others optional libraries, or
>> we
>>>>> can
>>>>>>>>> create a new directory named "tools" in the root directory and put
>>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
>>>>>>>>> 
>>>>>>>>> Best regards
>>>>>>>>> Oleg
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> Sergey Kozlov
>>>> GridGain Systems
>>>> www.gridgain.com
>>> 
>>> 
>> 
> 
> 
> 
> -- 
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Sergey Kozlov <sk...@gridgain.com>.

Yardstick requires own scripts/configurations (/bin, /config, /libs) and
creates work/logs directory under yardstick root. "libs/optional" is for
optional modules but in general we can't say that for Yardstick. Also it
may break the current user understanding of "libs/optional" directory as
place for additonal functionality activated by copying in "libs".




On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> I would be against using libs/optional or libs/ folder for anything
> benchmark related. I am also against adding any yardstick libraries without
> providing code.
>
> In my view, if the community wants to include benchmarks in releases, then
> we should add a "benchmarks" folder, which provides everything benchmark
> related, from code to all the dependent libraries, and documentation
> instructions.
>
> D.
>
> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org> wrote:
>
> > Actually, “libs/optional” is already a kind of extra for me. Why do we
> > need this new folder if “libs/optional” semantic works well?
> >
> > Is there anyone else who is concerned about “libs/optional”? If there’re
> > not, I would agree on this and get down to the implementation.
> >
> > —
> > Denis
> >
> > > On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com>
> wrote:
> > >
> > > Hi
> > >
> > > What's about to introduce the new root folder called 'extras' with
> > > subfolder 'ignite-yardstick' and put there yardstick binaries?
> > >
> > >
> > > On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org>
> wrote:
> > >
> > >> Dmitriy,
> > >>
> > >> Please have a look at IGNITE-4212 description (
> > https://issues.apache.org/
> > >> jira/browse/IGNITE-4212).
> > >>
> > >> The whole purpose of the ticket is to automate benchmarks execution
> for
> > >> the end user for a specific Ignite release. Now he/she needs to go
> > through
> > >> a number of steps like build, configure, run strictly following
> lengthy
> > >> Yardstick guidance.
> > >>
> > >> Ideally, once a specific release is downloaded it should be possible
> to
> > >> run a concrete benchmark with a ready-to-use script. The script needs
> > >> benchmarks' lib which makes sense to put under “libs/optional” folder.
> > >>
> > >> If someone wants to modify the source of an existed benchmark or add a
> > new
> > >> one then he/she needs to follow existed Yardstick guidance. So, no
> need
> > to
> > >> release benchmarks’s sources as a part of Ignite release.
> > >>
> > >> —
> > >> Denis
> > >>
> > >>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > >> wrote:
> > >>>
> > >>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oostanin@gridgain.com
> >
> > >> wrote:
> > >>>
> > >>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
> > Yardstick
> > >>>> benchmarks, which can be used to check Ignite performance.
> > >>>>
> > >>>
> > >>> In that case, why would it be under the "libs" folder at all? Do we
> > >> really
> > >>> need to include benchmarks into Ignite? If yes, then I would create a
> > >>> benchmarks folder under "examples" and add all the benchmarks there.
> > >>>
> > >>>
> > >>>>
> > >>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> > >> dsetrakyan@apache.org
> > >>>>>
> > >>>> wrote:
> > >>>>
> > >>>>> Oleg, what does ignite-yardstick module do?
> > >>>>>
> > >>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> > oostanin@gridgain.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hello Igniters!
> > >>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
> > Simplification
> > >>>> and
> > >>>>>> Automation" and I'd like to ask your opinion about
> ignite-yardstick:
> > >>>>> where
> > >>>>>> do you think is the most appropriate place to put a compiled
> > >>>>>> ignite-yardstick module in the apache-ignite binary assembly? We
> can
> > >>>> put
> > >>>>> it
> > >>>>>> in the libs/optional along with an others optional libraries, or
> we
> > >> can
> > >>>>>> create a new directory named "tools" in the root directory and put
> > >>>>>> "ignite-yardstick" in it, or we can find another solution.
> > >>>>>>
> > >>>>>> Best regards
> > >>>>>> Oleg
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> > >>
> > >
> > >
> > > --
> > > Sergey Kozlov
> > > GridGain Systems
> > > www.gridgain.com
> >
> >
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

> Not necessarily. Why not provide both, a prebuilt benchmark and an ability
> to run from code?
> 

Sure, we can do that. My point was to cover the most frequently asked use case which is the ability to run compiled benchmarks.

> Also, please keep in mind that any binary we include into release, must
> either come form maven or be built as part of the release itself.

Is there any reason why we don’t host ignite-yardstick on maven at the moment?

—
Denis

> On Dec 19, 2016, at 9:55 AM, Dmitriy Setrakyan <ds...@apache.org> wrote:
> 
> On Mon, Dec 19, 2016 at 9:46 AM, Denis Magda <dm...@apache.org> wrote:
> 
>> Then it will not much differ from the flow the users have to follow
>> presently. The only difference is that the sources and instructions will be
>> available in an Apache Ignite release as well. But the users will still
>> need to build the benchmarks manually, right? They already can do the same
>> referring to Yardstick.
>> 
> 
> Not necessarily. Why not provide both, a prebuilt benchmark and an ability
> to run from code?
> 
> Also, please keep in mind that any binary we include into release, must
> either come form maven or be built as part of the release itself.
> 
> 
>> 
>> —
>> Denis
>> 
>>> On Dec 19, 2016, at 8:53 AM, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>>> 
>>> I would be against using libs/optional or libs/ folder for anything
>>> benchmark related. I am also against adding any yardstick libraries
>> without
>>> providing code.
>>> 
>>> In my view, if the community wants to include benchmarks in releases,
>> then
>>> we should add a "benchmarks" folder, which provides everything benchmark
>>> related, from code to all the dependent libraries, and documentation
>>> instructions.
>>> 
>>> D.
>>> 
>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org> wrote:
>>> 
>>>> Actually, “libs/optional” is already a kind of extra for me. Why do we
>>>> need this new folder if “libs/optional” semantic works well?
>>>> 
>>>> Is there anyone else who is concerned about “libs/optional”? If there’re
>>>> not, I would agree on this and get down to the implementation.
>>>> 
>>>> —
>>>> Denis
>>>> 
>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com>
>> wrote:
>>>>> 
>>>>> Hi
>>>>> 
>>>>> What's about to introduce the new root folder called 'extras' with
>>>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
>>>>> 
>>>>> 
>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org>
>> wrote:
>>>>> 
>>>>>> Dmitriy,
>>>>>> 
>>>>>> Please have a look at IGNITE-4212 description (
>>>> https://issues.apache.org/
>>>>>> jira/browse/IGNITE-4212).
>>>>>> 
>>>>>> The whole purpose of the ticket is to automate benchmarks execution
>> for
>>>>>> the end user for a specific Ignite release. Now he/she needs to go
>>>> through
>>>>>> a number of steps like build, configure, run strictly following
>> lengthy
>>>>>> Yardstick guidance.
>>>>>> 
>>>>>> Ideally, once a specific release is downloaded it should be possible
>> to
>>>>>> run a concrete benchmark with a ready-to-use script. The script needs
>>>>>> benchmarks' lib which makes sense to put under “libs/optional” folder.
>>>>>> 
>>>>>> If someone wants to modify the source of an existed benchmark or add a
>>>> new
>>>>>> one then he/she needs to follow existed Yardstick guidance. So, no
>> need
>>>> to
>>>>>> release benchmarks’s sources as a part of Ignite release.
>>>>>> 
>>>>>> —
>>>>>> Denis
>>>>>> 
>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>>>>>> wrote:
>>>>>>> 
>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oostanin@gridgain.com
>>> 
>>>>>> wrote:
>>>>>>> 
>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
>>>> Yardstick
>>>>>>>> benchmarks, which can be used to check Ignite performance.
>>>>>>>> 
>>>>>>> 
>>>>>>> In that case, why would it be under the "libs" folder at all? Do we
>>>>>> really
>>>>>>> need to include benchmarks into Ignite? If yes, then I would create a
>>>>>>> benchmarks folder under "examples" and add all the benchmarks there.
>>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>>>>>> dsetrakyan@apache.org
>>>>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>>>> 
>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>>>> oostanin@gridgain.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hello Igniters!
>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
>>>> Simplification
>>>>>>>> and
>>>>>>>>>> Automation" and I'd like to ask your opinion about
>> ignite-yardstick:
>>>>>>>>> where
>>>>>>>>>> do you think is the most appropriate place to put a compiled
>>>>>>>>>> ignite-yardstick module in the apache-ignite binary assembly? We
>> can
>>>>>>>> put
>>>>>>>>> it
>>>>>>>>>> in the libs/optional along with an others optional libraries, or
>> we
>>>>>> can
>>>>>>>>>> create a new directory named "tools" in the root directory and put
>>>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
>>>>>>>>>> 
>>>>>>>>>> Best regards
>>>>>>>>>> Oleg
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Sergey Kozlov
>>>>> GridGain Systems
>>>>> www.gridgain.com
>>>> 
>>>> 
>> 
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Dmitriy Setrakyan <ds...@apache.org>.

On Mon, Dec 19, 2016 at 9:46 AM, Denis Magda <dm...@apache.org> wrote:

> Then it will not much differ from the flow the users have to follow
> presently. The only difference is that the sources and instructions will be
> available in an Apache Ignite release as well. But the users will still
> need to build the benchmarks manually, right? They already can do the same
> referring to Yardstick.
>

Not necessarily. Why not provide both, a prebuilt benchmark and an ability
to run from code?

Also, please keep in mind that any binary we include into release, must
either come form maven or be built as part of the release itself.


>
> —
> Denis
>
> > On Dec 19, 2016, at 8:53 AM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >
> > I would be against using libs/optional or libs/ folder for anything
> > benchmark related. I am also against adding any yardstick libraries
> without
> > providing code.
> >
> > In my view, if the community wants to include benchmarks in releases,
> then
> > we should add a "benchmarks" folder, which provides everything benchmark
> > related, from code to all the dependent libraries, and documentation
> > instructions.
> >
> > D.
> >
> > On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org> wrote:
> >
> >> Actually, “libs/optional” is already a kind of extra for me. Why do we
> >> need this new folder if “libs/optional” semantic works well?
> >>
> >> Is there anyone else who is concerned about “libs/optional”? If there’re
> >> not, I would agree on this and get down to the implementation.
> >>
> >> —
> >> Denis
> >>
> >>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com>
> wrote:
> >>>
> >>> Hi
> >>>
> >>> What's about to introduce the new root folder called 'extras' with
> >>> subfolder 'ignite-yardstick' and put there yardstick binaries?
> >>>
> >>>
> >>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org>
> wrote:
> >>>
> >>>> Dmitriy,
> >>>>
> >>>> Please have a look at IGNITE-4212 description (
> >> https://issues.apache.org/
> >>>> jira/browse/IGNITE-4212).
> >>>>
> >>>> The whole purpose of the ticket is to automate benchmarks execution
> for
> >>>> the end user for a specific Ignite release. Now he/she needs to go
> >> through
> >>>> a number of steps like build, configure, run strictly following
> lengthy
> >>>> Yardstick guidance.
> >>>>
> >>>> Ideally, once a specific release is downloaded it should be possible
> to
> >>>> run a concrete benchmark with a ready-to-use script. The script needs
> >>>> benchmarks' lib which makes sense to put under “libs/optional” folder.
> >>>>
> >>>> If someone wants to modify the source of an existed benchmark or add a
> >> new
> >>>> one then he/she needs to follow existed Yardstick guidance. So, no
> need
> >> to
> >>>> release benchmarks’s sources as a part of Ignite release.
> >>>>
> >>>> —
> >>>> Denis
> >>>>
> >>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> >>>> wrote:
> >>>>>
> >>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oostanin@gridgain.com
> >
> >>>> wrote:
> >>>>>
> >>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
> >> Yardstick
> >>>>>> benchmarks, which can be used to check Ignite performance.
> >>>>>>
> >>>>>
> >>>>> In that case, why would it be under the "libs" folder at all? Do we
> >>>> really
> >>>>> need to include benchmarks into Ignite? If yes, then I would create a
> >>>>> benchmarks folder under "examples" and add all the benchmarks there.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> >>>> dsetrakyan@apache.org
> >>>>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Oleg, what does ignite-yardstick module do?
> >>>>>>>
> >>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> >> oostanin@gridgain.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hello Igniters!
> >>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
> >> Simplification
> >>>>>> and
> >>>>>>>> Automation" and I'd like to ask your opinion about
> ignite-yardstick:
> >>>>>>> where
> >>>>>>>> do you think is the most appropriate place to put a compiled
> >>>>>>>> ignite-yardstick module in the apache-ignite binary assembly? We
> can
> >>>>>> put
> >>>>>>> it
> >>>>>>>> in the libs/optional along with an others optional libraries, or
> we
> >>>> can
> >>>>>>>> create a new directory named "tools" in the root directory and put
> >>>>>>>> "ignite-yardstick" in it, or we can find another solution.
> >>>>>>>>
> >>>>>>>> Best regards
> >>>>>>>> Oleg
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Sergey Kozlov
> >>> GridGain Systems
> >>> www.gridgain.com
> >>
> >>
>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Then it will not much differ from the flow the users have to follow presently. The only difference is that the sources and instructions will be available in an Apache Ignite release as well. But the users will still need to build the benchmarks manually, right? They already can do the same referring to Yardstick.

—
Denis

> On Dec 19, 2016, at 8:53 AM, Dmitriy Setrakyan <ds...@apache.org> wrote:
> 
> I would be against using libs/optional or libs/ folder for anything
> benchmark related. I am also against adding any yardstick libraries without
> providing code.
> 
> In my view, if the community wants to include benchmarks in releases, then
> we should add a "benchmarks" folder, which provides everything benchmark
> related, from code to all the dependent libraries, and documentation
> instructions.
> 
> D.
> 
> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org> wrote:
> 
>> Actually, “libs/optional” is already a kind of extra for me. Why do we
>> need this new folder if “libs/optional” semantic works well?
>> 
>> Is there anyone else who is concerned about “libs/optional”? If there’re
>> not, I would agree on this and get down to the implementation.
>> 
>> —
>> Denis
>> 
>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
>>> 
>>> Hi
>>> 
>>> What's about to introduce the new root folder called 'extras' with
>>> subfolder 'ignite-yardstick' and put there yardstick binaries?
>>> 
>>> 
>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org> wrote:
>>> 
>>>> Dmitriy,
>>>> 
>>>> Please have a look at IGNITE-4212 description (
>> https://issues.apache.org/
>>>> jira/browse/IGNITE-4212).
>>>> 
>>>> The whole purpose of the ticket is to automate benchmarks execution for
>>>> the end user for a specific Ignite release. Now he/she needs to go
>> through
>>>> a number of steps like build, configure, run strictly following lengthy
>>>> Yardstick guidance.
>>>> 
>>>> Ideally, once a specific release is downloaded it should be possible to
>>>> run a concrete benchmark with a ready-to-use script. The script needs
>>>> benchmarks' lib which makes sense to put under “libs/optional” folder.
>>>> 
>>>> If someone wants to modify the source of an existed benchmark or add a
>> new
>>>> one then he/she needs to follow existed Yardstick guidance. So, no need
>> to
>>>> release benchmarks’s sources as a part of Ignite release.
>>>> 
>>>> —
>>>> Denis
>>>> 
>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <ds...@apache.org>
>>>> wrote:
>>>>> 
>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oo...@gridgain.com>
>>>> wrote:
>>>>> 
>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
>> Yardstick
>>>>>> benchmarks, which can be used to check Ignite performance.
>>>>>> 
>>>>> 
>>>>> In that case, why would it be under the "libs" folder at all? Do we
>>>> really
>>>>> need to include benchmarks into Ignite? If yes, then I would create a
>>>>> benchmarks folder under "examples" and add all the benchmarks there.
>>>>> 
>>>>> 
>>>>>> 
>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>>>> dsetrakyan@apache.org
>>>>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> Oleg, what does ignite-yardstick module do?
>>>>>>> 
>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
>> oostanin@gridgain.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hello Igniters!
>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
>> Simplification
>>>>>> and
>>>>>>>> Automation" and I'd like to ask your opinion about ignite-yardstick:
>>>>>>> where
>>>>>>>> do you think is the most appropriate place to put a compiled
>>>>>>>> ignite-yardstick module in the apache-ignite binary assembly? We can
>>>>>> put
>>>>>>> it
>>>>>>>> in the libs/optional along with an others optional libraries, or we
>>>> can
>>>>>>>> create a new directory named "tools" in the root directory and put
>>>>>>>> "ignite-yardstick" in it, or we can find another solution.
>>>>>>>> 
>>>>>>>> Best regards
>>>>>>>> Oleg
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> --
>>> Sergey Kozlov
>>> GridGain Systems
>>> www.gridgain.com
>> 
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Dmitriy Setrakyan <ds...@apache.org>.

I would be against using libs/optional or libs/ folder for anything
benchmark related. I am also against adding any yardstick libraries without
providing code.

In my view, if the community wants to include benchmarks in releases, then
we should add a "benchmarks" folder, which provides everything benchmark
related, from code to all the dependent libraries, and documentation
instructions.

D.

On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda <dm...@apache.org> wrote:

> Actually, “libs/optional” is already a kind of extra for me. Why do we
> need this new folder if “libs/optional” semantic works well?
>
> Is there anyone else who is concerned about “libs/optional”? If there’re
> not, I would agree on this and get down to the implementation.
>
> —
> Denis
>
> > On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
> >
> > Hi
> >
> > What's about to introduce the new root folder called 'extras' with
> > subfolder 'ignite-yardstick' and put there yardstick binaries?
> >
> >
> > On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org> wrote:
> >
> >> Dmitriy,
> >>
> >> Please have a look at IGNITE-4212 description (
> https://issues.apache.org/
> >> jira/browse/IGNITE-4212).
> >>
> >> The whole purpose of the ticket is to automate benchmarks execution for
> >> the end user for a specific Ignite release. Now he/she needs to go
> through
> >> a number of steps like build, configure, run strictly following lengthy
> >> Yardstick guidance.
> >>
> >> Ideally, once a specific release is downloaded it should be possible to
> >> run a concrete benchmark with a ready-to-use script. The script needs
> >> benchmarks' lib which makes sense to put under “libs/optional” folder.
> >>
> >> If someone wants to modify the source of an existed benchmark or add a
> new
> >> one then he/she needs to follow existed Yardstick guidance. So, no need
> to
> >> release benchmarks’s sources as a part of Ignite release.
> >>
> >> —
> >> Denis
> >>
> >>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <ds...@apache.org>
> >> wrote:
> >>>
> >>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oo...@gridgain.com>
> >> wrote:
> >>>
> >>>> Dmitriy, ignite-yardstick allows user to run plenty of useful
> Yardstick
> >>>> benchmarks, which can be used to check Ignite performance.
> >>>>
> >>>
> >>> In that case, why would it be under the "libs" folder at all? Do we
> >> really
> >>> need to include benchmarks into Ignite? If yes, then I would create a
> >>> benchmarks folder under "examples" and add all the benchmarks there.
> >>>
> >>>
> >>>>
> >>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> >> dsetrakyan@apache.org
> >>>>>
> >>>> wrote:
> >>>>
> >>>>> Oleg, what does ignite-yardstick module do?
> >>>>>
> >>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <
> oostanin@gridgain.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hello Igniters!
> >>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking
> Simplification
> >>>> and
> >>>>>> Automation" and I'd like to ask your opinion about ignite-yardstick:
> >>>>> where
> >>>>>> do you think is the most appropriate place to put a compiled
> >>>>>> ignite-yardstick module in the apache-ignite binary assembly? We can
> >>>> put
> >>>>> it
> >>>>>> in the libs/optional along with an others optional libraries, or we
> >> can
> >>>>>> create a new directory named "tools" in the root directory and put
> >>>>>> "ignite-yardstick" in it, or we can find another solution.
> >>>>>>
> >>>>>> Best regards
> >>>>>> Oleg
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
> >
> >
> > --
> > Sergey Kozlov
> > GridGain Systems
> > www.gridgain.com
>
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Actually, “libs/optional” is already a kind of extra for me. Why do we need this new folder if “libs/optional” semantic works well?

Is there anyone else who is concerned about “libs/optional”? If there’re not, I would agree on this and get down to the implementation.

—
Denis

> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
> 
> Hi
> 
> What's about to introduce the new root folder called 'extras' with
> subfolder 'ignite-yardstick' and put there yardstick binaries?
> 
> 
> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org> wrote:
> 
>> Dmitriy,
>> 
>> Please have a look at IGNITE-4212 description (https://issues.apache.org/
>> jira/browse/IGNITE-4212).
>> 
>> The whole purpose of the ticket is to automate benchmarks execution for
>> the end user for a specific Ignite release. Now he/she needs to go through
>> a number of steps like build, configure, run strictly following lengthy
>> Yardstick guidance.
>> 
>> Ideally, once a specific release is downloaded it should be possible to
>> run a concrete benchmark with a ready-to-use script. The script needs
>> benchmarks' lib which makes sense to put under “libs/optional” folder.
>> 
>> If someone wants to modify the source of an existed benchmark or add a new
>> one then he/she needs to follow existed Yardstick guidance. So, no need to
>> release benchmarks’s sources as a part of Ignite release.
>> 
>> —
>> Denis
>> 
>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>>> 
>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oo...@gridgain.com>
>> wrote:
>>> 
>>>> Dmitriy, ignite-yardstick allows user to run plenty of useful Yardstick
>>>> benchmarks, which can be used to check Ignite performance.
>>>> 
>>> 
>>> In that case, why would it be under the "libs" folder at all? Do we
>> really
>>> need to include benchmarks into Ignite? If yes, then I would create a
>>> benchmarks folder under "examples" and add all the benchmarks there.
>>> 
>>> 
>>>> 
>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
>> dsetrakyan@apache.org
>>>>> 
>>>> wrote:
>>>> 
>>>>> Oleg, what does ignite-yardstick module do?
>>>>> 
>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <oo...@gridgain.com>
>>>>> wrote:
>>>>> 
>>>>>> Hello Igniters!
>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking Simplification
>>>> and
>>>>>> Automation" and I'd like to ask your opinion about ignite-yardstick:
>>>>> where
>>>>>> do you think is the most appropriate place to put a compiled
>>>>>> ignite-yardstick module in the apache-ignite binary assembly? We can
>>>> put
>>>>> it
>>>>>> in the libs/optional along with an others optional libraries, or we
>> can
>>>>>> create a new directory named "tools" in the root directory and put
>>>>>> "ignite-yardstick" in it, or we can find another solution.
>>>>>> 
>>>>>> Best regards
>>>>>> Oleg
>>>>>> 
>>>>> 
>>>> 
>> 
>> 
> 
> 
> -- 
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Sergey Kozlov <sk...@gridgain.com>.

Hi

What's about to introduce the new root folder called 'extras' with
subfolder 'ignite-yardstick' and put there yardstick binaries?


On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda <dm...@apache.org> wrote:

> Dmitriy,
>
> Please have a look at IGNITE-4212 description (https://issues.apache.org/
> jira/browse/IGNITE-4212).
>
> The whole purpose of the ticket is to automate benchmarks execution for
> the end user for a specific Ignite release. Now he/she needs to go through
> a number of steps like build, configure, run strictly following lengthy
> Yardstick guidance.
>
> Ideally, once a specific release is downloaded it should be possible to
> run a concrete benchmark with a ready-to-use script. The script needs
> benchmarks' lib which makes sense to put under “libs/optional” folder.
>
> If someone wants to modify the source of an existed benchmark or add a new
> one then he/she needs to follow existed Yardstick guidance. So, no need to
> release benchmarks’s sources as a part of Ignite release.
>
> —
> Denis
>
> > On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >
> > On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
> >
> >> Dmitriy, ignite-yardstick allows user to run plenty of useful Yardstick
> >> benchmarks, which can be used to check Ignite performance.
> >>
> >
> > In that case, why would it be under the "libs" folder at all? Do we
> really
> > need to include benchmarks into Ignite? If yes, then I would create a
> > benchmarks folder under "examples" and add all the benchmarks there.
> >
> >
> >>
> >> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org
> >>>
> >> wrote:
> >>
> >>> Oleg, what does ignite-yardstick module do?
> >>>
> >>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <oo...@gridgain.com>
> >>> wrote:
> >>>
> >>>> Hello Igniters!
> >>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking Simplification
> >> and
> >>>> Automation" and I'd like to ask your opinion about ignite-yardstick:
> >>> where
> >>>> do you think is the most appropriate place to put a compiled
> >>>> ignite-yardstick module in the apache-ignite binary assembly? We can
> >> put
> >>> it
> >>>> in the libs/optional along with an others optional libraries, or we
> can
> >>>> create a new directory named "tools" in the root directory and put
> >>>> "ignite-yardstick" in it, or we can find another solution.
> >>>>
> >>>> Best regards
> >>>> Oleg
> >>>>
> >>>
> >>
>
>


-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Denis Magda <dm...@apache.org>.

Dmitriy,

Please have a look at IGNITE-4212 description (https://issues.apache.org/jira/browse/IGNITE-4212).

The whole purpose of the ticket is to automate benchmarks execution for the end user for a specific Ignite release. Now he/she needs to go through a number of steps like build, configure, run strictly following lengthy Yardstick guidance.

Ideally, once a specific release is downloaded it should be possible to run a concrete benchmark with a ready-to-use script. The script needs benchmarks' lib which makes sense to put under “libs/optional” folder.

If someone wants to modify the source of an existed benchmark or add a new one then he/she needs to follow existed Yardstick guidance. So, no need to release benchmarks’s sources as a part of Ignite release.

—
Denis

> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan <ds...@apache.org> wrote:
> 
> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oo...@gridgain.com> wrote:
> 
>> Dmitriy, ignite-yardstick allows user to run plenty of useful Yardstick
>> benchmarks, which can be used to check Ignite performance.
>> 
> 
> In that case, why would it be under the "libs" folder at all? Do we really
> need to include benchmarks into Ignite? If yes, then I would create a
> benchmarks folder under "examples" and add all the benchmarks there.
> 
> 
>> 
>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <dsetrakyan@apache.org
>>> 
>> wrote:
>> 
>>> Oleg, what does ignite-yardstick module do?
>>> 
>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <oo...@gridgain.com>
>>> wrote:
>>> 
>>>> Hello Igniters!
>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking Simplification
>> and
>>>> Automation" and I'd like to ask your opinion about ignite-yardstick:
>>> where
>>>> do you think is the most appropriate place to put a compiled
>>>> ignite-yardstick module in the apache-ignite binary assembly? We can
>> put
>>> it
>>>> in the libs/optional along with an others optional libraries, or we can
>>>> create a new directory named "tools" in the root directory and put
>>>> "ignite-yardstick" in it, or we can find another solution.
>>>> 
>>>> Best regards
>>>> Oleg
>>>> 
>>> 
>>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Sergey Kozlov <sk...@gridgain.com>.

Dmitriy

Examples folder is not good location for yardstick binaries (but good  for
yardstick sources). I suppose we should look on how to other products
include additional tools.

On Sun, Dec 18, 2016 at 6:08 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
>
> > Dmitriy, ignite-yardstick allows user to run plenty of useful Yardstick
> > benchmarks, which can be used to check Ignite performance.
> >
>
> In that case, why would it be under the "libs" folder at all? Do we really
> need to include benchmarks into Ignite? If yes, then I would create a
> benchmarks folder under "examples" and add all the benchmarks there.
>
>
> >
> > On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org
> > >
> > wrote:
> >
> > > Oleg, what does ignite-yardstick module do?
> > >
> > > On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <oo...@gridgain.com>
> > > wrote:
> > >
> > > > Hello Igniters!
> > > > I'm working on ticket IGNITE-4212 "Ignite Benchmarking Simplification
> > and
> > > > Automation" and I'd like to ask your opinion about ignite-yardstick:
> > > where
> > > > do you think is the most appropriate place to put a compiled
> > > > ignite-yardstick module in the apache-ignite binary assembly? We can
> > put
> > > it
> > > > in the libs/optional along with an others optional libraries, or we
> can
> > > > create a new directory named "tools" in the root directory and put
> > > > "ignite-yardstick" in it, or we can find another solution.
> > > >
> > > > Best regards
> > > > Oleg
> > > >
> > >
> >
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Dmitriy Setrakyan <ds...@apache.org>.

On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin <oo...@gridgain.com> wrote:

> Dmitriy, ignite-yardstick allows user to run plenty of useful Yardstick
> benchmarks, which can be used to check Ignite performance.
>

In that case, why would it be under the "libs" folder at all? Do we really
need to include benchmarks into Ignite? If yes, then I would create a
benchmarks folder under "examples" and add all the benchmarks there.


>
> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <dsetrakyan@apache.org
> >
> wrote:
>
> > Oleg, what does ignite-yardstick module do?
> >
> > On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <oo...@gridgain.com>
> > wrote:
> >
> > > Hello Igniters!
> > > I'm working on ticket IGNITE-4212 "Ignite Benchmarking Simplification
> and
> > > Automation" and I'd like to ask your opinion about ignite-yardstick:
> > where
> > > do you think is the most appropriate place to put a compiled
> > > ignite-yardstick module in the apache-ignite binary assembly? We can
> put
> > it
> > > in the libs/optional along with an others optional libraries, or we can
> > > create a new directory named "tools" in the root directory and put
> > > "ignite-yardstick" in it, or we can find another solution.
> > >
> > > Best regards
> > > Oleg
> > >
> >
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Oleg Ostanin <oo...@gridgain.com>.

Dmitriy, ignite-yardstick allows user to run plenty of useful Yardstick
benchmarks, which can be used to check Ignite performance.

On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> Oleg, what does ignite-yardstick module do?
>
> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <oo...@gridgain.com>
> wrote:
>
> > Hello Igniters!
> > I'm working on ticket IGNITE-4212 "Ignite Benchmarking Simplification and
> > Automation" and I'd like to ask your opinion about ignite-yardstick:
> where
> > do you think is the most appropriate place to put a compiled
> > ignite-yardstick module in the apache-ignite binary assembly? We can put
> it
> > in the libs/optional along with an others optional libraries, or we can
> > create a new directory named "tools" in the root directory and put
> > "ignite-yardstick" in it, or we can find another solution.
> >
> > Best regards
> > Oleg
> >
>

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

Posted by Dmitriy Setrakyan <ds...@apache.org>.

Oleg, what does ignite-yardstick module do?

On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin <oo...@gridgain.com>
wrote:

> Hello Igniters!
> I'm working on ticket IGNITE-4212 "Ignite Benchmarking Simplification and
> Automation" and I'd like to ask your opinion about ignite-yardstick: where
> do you think is the most appropriate place to put a compiled
> ignite-yardstick module in the apache-ignite binary assembly? We can put it
> in the libs/optional along with an others optional libraries, or we can
> create a new directory named "tools" in the root directory and put
> "ignite-yardstick" in it, or we can find another solution.
>
> Best regards
> Oleg
>