Configuring ignite

[Ognen Duzlevski <og...@gmail.com>]

Hello all,

Is there some kind of a "guide" available to configuring/tweaking ignite?
So far I have been going with the default example-ignite.xml file that
comes with the ignite distribution, however, I am ready to move on and
tweak, add etc.

For example, I have 5 machines each with 30+GB of RAM that I can dedicate
to an ignite cache. However, when I start ignite using the
example-ignite.xml configuration file, I get something like this:

[08:02:16] Ignite node started OK (id=e4be78cf)
[08:02:16] Topology snapshot [ver=1, nodes=1, CPUs=8, heap=1.0GB]

I am not even sure what the last line means when referring to the
heap=1.0GB line - does it mean I only get 1GB for cache even though I have
30GB available? Does it mean I get to start with 1GB but it will
auto-expand to max available heap? So on and so on.

I am happy to write things up somewhere and contribute to documentation if
anyone on here is willing to answer questions in private or public as I go
along! :-)

Thanks,
Ognen

Re: Configuring ignite

[Ognen Duzlevski <og...@gmail.com>]

Dmitriy,

On Thu, Apr 30, 2015 at 8:49 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> On Thu, Apr 30, 2015 at 8:35 PM, Ognen Duzlevski <
> ognen.duzlevski@gmail.com>
> wrote:
>
> I have updated the javadoc for the withAsync() method in ignite-sprint-4
> branch: Gets instance of this component with asynchronous mode enabled.
>
> I have also updated the documentation:
> http://apacheignite.gridgain.org/v1.0/docs/async-support#igniteasyncsupport


Thanks!


> This will be greatly appreciated. Is there a particular area where you
> would like to start?
>
> Also, it would be great for the community and users if you could create a
> few blogs about your experiences with Ignite as well.
>

I was thinking about writing a blog with the experience so far.

I can start where I started first time I saw ignite. There will always be
people who are very proficient (more than me) who don't need to be coddled.
However, there are plenty of people who need the guidance, some more, some
less :-). Let me write something and I will submit it for review here.

Ognen

Re: Configuring ignite

[Dmitriy Setrakyan <ds...@apache.org>]

On Thu, Apr 30, 2015 at 8:35 PM, Ognen Duzlevski <og...@gmail.com>
wrote:

> On Thu, Apr 30, 2015 at 7:59 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > I think the best way is to write it up here, and one of us will publish
> it
> > on the official documentation site.
> >
>
> I remember when I was starting out with Spark, they had a page (still do)
> addressing tuning, configuring, tweaking a Spark install. Overall, I feel
> that there is plenty of knowledge in the current Ignite docs, it's just
> written with the assumption that people who are reading the docs are also
> the Ignite developers ;)
>
> For example, I asked the question about which ports need to be open on a
> network in EC2 and someone was nice enough to answer. I later found the
> answer myself here: http://apacheignite.readme.io/v1.0/docs/network-config
>
> However, the information in the above-referenced page did not "connect with
> me" until I got the reply on the list.
>
> Similarly - I just realized that if you start a cache with .withAsync() and
> later call containsKey() on it with a key that I know exists in the cache,
> the containsKey() will return false. This, in turn, has caused me to
> overwrite the value associated with the key instead of appending to the
> value. To make matters more confusing, if a cache is started with
> withAsync() and then you obtain a reference to the same, existing cache by
> not using withAsync() the containsKey() will work as expected (return true
> in my case). This is because the withAsync() applies to the current "user"
> of the cache, not to the way the cache was created by the original creator.
> VERY confusing! ;)
>

I have updated the javadoc for the withAsync() method in ignite-sprint-4
branch: Gets instance of this component with asynchronous mode enabled.

I have also updated the documentation:
http://apacheignite.gridgain.org/v1.0/docs/async-support#igniteasyncsupport



>
> I am willing to spend the time and write as much documentation as
> necessary. I have many years of experience under my belt, mostly in C but I
> made the leap to Scala a while ago. I am willing to learn the Java
> necessary to get involved by writing code but for starters I am happy to
> just write docs too.
>

This will be greatly appreciated. Is there a particular area where you
would like to start?

Also, it would be great for the community and users if you could create a
few blogs about your experiences with Ignite as well.


> Cheers,
> Ognen
>

Re: Configuring ignite

[Ognen Duzlevski <og...@gmail.com>]

On Thu, Apr 30, 2015 at 8:55 PM, Konstantin Boudnik <co...@apache.org> wrote:

> BTW, if you see some javadocs issues - just open a JIRA and file a patch.
> You
> will earn karma for that as well ;)
>
> heh - sure, will do :)

Re: Configuring ignite

[Konstantin Boudnik <co...@apache.org>]

On Thu, Apr 30, 2015 at 08:35PM, Ognen Duzlevski wrote:
> On Thu, Apr 30, 2015 at 7:59 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> 
> > I think the best way is to write it up here, and one of us will publish it
> > on the official documentation site.
> >
> 
> I remember when I was starting out with Spark, they had a page (still do)
> addressing tuning, configuring, tweaking a Spark install. Overall, I feel
> that there is plenty of knowledge in the current Ignite docs, it's just
> written with the assumption that people who are reading the docs are also
> the Ignite developers ;)
> 
> For example, I asked the question about which ports need to be open on a
> network in EC2 and someone was nice enough to answer. I later found the
> answer myself here: http://apacheignite.readme.io/v1.0/docs/network-config
> 
> However, the information in the above-referenced page did not "connect with
> me" until I got the reply on the list.
> 
> Similarly - I just realized that if you start a cache with .withAsync() and
> later call containsKey() on it with a key that I know exists in the cache,
> the containsKey() will return false. This, in turn, has caused me to
> overwrite the value associated with the key instead of appending to the
> value. To make matters more confusing, if a cache is started with
> withAsync() and then you obtain a reference to the same, existing cache by
> not using withAsync() the containsKey() will work as expected (return true
> in my case). This is because the withAsync() applies to the current "user"
> of the cache, not to the way the cache was created by the original creator.
> VERY confusing! ;)

BTW, if you see some javadocs issues - just open a JIRA and file a patch. You
will earn karma for that as well ;)

> I am willing to spend the time and write as much documentation as
> necessary. I have many years of experience under my belt, mostly in C but I
> made the leap to Scala a while ago. I am willing to learn the Java
> necessary to get involved by writing code but for starters I am happy to
> just write docs too.
> 
> Cheers,
> Ognen

Re: Configuring ignite

[Konstantin Boudnik <co...@apache.org>]

On Thu, Apr 30, 2015 at 08:35PM, Ognen Duzlevski wrote:
> On Thu, Apr 30, 2015 at 7:59 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> 
> > I think the best way is to write it up here, and one of us will publish it
> > on the official documentation site.
> >
> 
> I remember when I was starting out with Spark, they had a page (still do)
> addressing tuning, configuring, tweaking a Spark install. Overall, I feel
> that there is plenty of knowledge in the current Ignite docs, it's just
> written with the assumption that people who are reading the docs are also
> the Ignite developers ;)
> 
> For example, I asked the question about which ports need to be open on a
> network in EC2 and someone was nice enough to answer. I later found the
> answer myself here: http://apacheignite.readme.io/v1.0/docs/network-config
> 
> However, the information in the above-referenced page did not "connect with
> me" until I got the reply on the list.
> 
> Similarly - I just realized that if you start a cache with .withAsync() and
> later call containsKey() on it with a key that I know exists in the cache,
> the containsKey() will return false. This, in turn, has caused me to
> overwrite the value associated with the key instead of appending to the
> value. To make matters more confusing, if a cache is started with
> withAsync() and then you obtain a reference to the same, existing cache by
> not using withAsync() the containsKey() will work as expected (return true
> in my case). This is because the withAsync() applies to the current "user"
> of the cache, not to the way the cache was created by the original creator.
> VERY confusing! ;)
> 
> I am willing to spend the time and write as much documentation as
> necessary. I have many years of experience under my belt, mostly in C but I
> made the leap to Scala a while ago. I am willing to learn the Java
> necessary to get involved by writing code but for starters I am happy to
> just write docs too.

Thank you very much Ognen - that'd be a great contribution to the project and
will help a lot of ppl in the future!

Dmitriy, do you think we can create some staging on readme.io where such
projects could be performed? Something like wiki, essentially? 

Cos

Re: Configuring ignite

[Ognen Duzlevski <og...@gmail.com>]

On Thu, Apr 30, 2015 at 7:59 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> I think the best way is to write it up here, and one of us will publish it
> on the official documentation site.
>

I remember when I was starting out with Spark, they had a page (still do)
addressing tuning, configuring, tweaking a Spark install. Overall, I feel
that there is plenty of knowledge in the current Ignite docs, it's just
written with the assumption that people who are reading the docs are also
the Ignite developers ;)

For example, I asked the question about which ports need to be open on a
network in EC2 and someone was nice enough to answer. I later found the
answer myself here: http://apacheignite.readme.io/v1.0/docs/network-config

However, the information in the above-referenced page did not "connect with
me" until I got the reply on the list.

Similarly - I just realized that if you start a cache with .withAsync() and
later call containsKey() on it with a key that I know exists in the cache,
the containsKey() will return false. This, in turn, has caused me to
overwrite the value associated with the key instead of appending to the
value. To make matters more confusing, if a cache is started with
withAsync() and then you obtain a reference to the same, existing cache by
not using withAsync() the containsKey() will work as expected (return true
in my case). This is because the withAsync() applies to the current "user"
of the cache, not to the way the cache was created by the original creator.
VERY confusing! ;)

I am willing to spend the time and write as much documentation as
necessary. I have many years of experience under my belt, mostly in C but I
made the leap to Scala a while ago. I am willing to learn the Java
necessary to get involved by writing code but for starters I am happy to
just write docs too.

Cheers,
Ognen

Re: Configuring ignite

[Dmitriy Setrakyan <ds...@apache.org>]

On Thu, Apr 30, 2015 at 6:32 PM, Konstantin Boudnik <co...@apache.org> wrote:

> On Wed, Apr 29, 2015 at 08:09AM, Ognen Duzlevski wrote:
> > Hello all,
> >
> > Is there some kind of a "guide" available to configuring/tweaking ignite?
> > So far I have been going with the default example-ignite.xml file that
> > comes with the ignite distribution, however, I am ready to move on and
> > tweak, add etc.
> >
> > For example, I have 5 machines each with 30+GB of RAM that I can dedicate
> > to an ignite cache. However, when I start ignite using the
> > example-ignite.xml configuration file, I get something like this:
> >
> > [08:02:16] Ignite node started OK (id=e4be78cf)
> > [08:02:16] Topology snapshot [ver=1, nodes=1, CPUs=8, heap=1.0GB]
> >
> > I am not even sure what the last line means when referring to the
> > heap=1.0GB line - does it mean I only get 1GB for cache even though I
> have
> > 30GB available? Does it mean I get to start with 1GB but it will
> > auto-expand to max available heap? So on and so on.
> >
> > I am happy to write things up somewhere and contribute to documentation
> if
> > anyone on here is willing to answer questions in private or public as I
> go
> > along! :-)
>
> Please please do!
>
> Guys, is there a description on how to contribute to the documentation,
> btw?
>

I think the best way is to write it up here, and one of us will publish it
on the official documentation site.


>
> Cos
>

Re: Configuring ignite

[Konstantin Boudnik <co...@apache.org>]

On Wed, Apr 29, 2015 at 08:09AM, Ognen Duzlevski wrote:
> Hello all,
> 
> Is there some kind of a "guide" available to configuring/tweaking ignite?
> So far I have been going with the default example-ignite.xml file that
> comes with the ignite distribution, however, I am ready to move on and
> tweak, add etc.
> 
> For example, I have 5 machines each with 30+GB of RAM that I can dedicate
> to an ignite cache. However, when I start ignite using the
> example-ignite.xml configuration file, I get something like this:
> 
> [08:02:16] Ignite node started OK (id=e4be78cf)
> [08:02:16] Topology snapshot [ver=1, nodes=1, CPUs=8, heap=1.0GB]
> 
> I am not even sure what the last line means when referring to the
> heap=1.0GB line - does it mean I only get 1GB for cache even though I have
> 30GB available? Does it mean I get to start with 1GB but it will
> auto-expand to max available heap? So on and so on.
> 
> I am happy to write things up somewhere and contribute to documentation if
> anyone on here is willing to answer questions in private or public as I go
> along! :-)

Please please do!

Guys, is there a description on how to contribute to the documentation, btw?

Cos

Re: Configuring ignite

[Alexey Kuznetsov <ak...@gridgain.com>]

Ognen, good catch. Configuring / tweaking of Ignite should be documented.

In order to use more heap memory by your nodes you can use JVM_OPTS system
variable
 or -J argument when starting from ignite.sh

For example: ignite.sh -J-Xmx8g

You will get node with 8g heap max.

Could you open an issue in Ignite JIRA about lack of documentation?


On Wed, Apr 29, 2015 at 8:09 PM, Ognen Duzlevski <og...@gmail.com>
wrote:

> Hello all,
>
> Is there some kind of a "guide" available to configuring/tweaking ignite?
> So far I have been going with the default example-ignite.xml file that
> comes with the ignite distribution, however, I am ready to move on and
> tweak, add etc.
>
> For example, I have 5 machines each with 30+GB of RAM that I can dedicate
> to an ignite cache. However, when I start ignite using the
> example-ignite.xml configuration file, I get something like this:
>
> [08:02:16] Ignite node started OK (id=e4be78cf)
> [08:02:16] Topology snapshot [ver=1, nodes=1, CPUs=8, heap=1.0GB]
>
> I am not even sure what the last line means when referring to the
> heap=1.0GB line - does it mean I only get 1GB for cache even though I have
> 30GB available? Does it mean I get to start with 1GB but it will
> auto-expand to max available heap? So on and so on.
>
> I am happy to write things up somewhere and contribute to documentation if
> anyone on here is willing to answer questions in private or public as I go
> along! :-)
>
> Thanks,
> Ognen
>



-- 
Alexey Kuznetsov
GridGain Systems
www.gridgain.com