You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Lokesh Sharma <lo...@gmail.com> on 2018/09/19 05:36:53 UTC
Problems are enabling Ignite Persistence
I enabled Ignite Persistence. Following is my configuration:
// New Data Region
> DataStorageConfiguration storageCfgPersistence = new DataStorageConfiguration();
> DataRegionConfiguration regionCfg = new DataRegionConfiguration();
> regionCfg.setName("Persistence");
> regionCfg.setInitialSize(100L * 1024 * 1024);
> regionCfg.setMaxSize(500L * 1024 * 1024);
> regionCfg.setPersistenceEnabled(true);
> storageCfgPersistence.setDataRegionConfigurations(regionCfg);
> cfg.setDataStorageConfiguration(storageCfgPersistence);
>
>
I am facing 2 problems after enabling Ignite persistence:
1 The applications boots up fine with no error but its stuck at that point.
It doesn't perform any action that its supposed to perform. I know this
because as soon the Ignite completes boot up my application logs starts to
print on the console but with persistence on, nothing is printed. Following
is the output when the node boots up:
2018-09-19 10:53:58.957 WARN 6032 --- [-worker-#42%cm%]
>> o.a.i.i.p.c.p.file.FilePageStoreManager : Persistence store directory is
>> in the temp directory and may be cleaned.To avoid this set "IGNITE_HOME"
>> environment variable properly or change location of persistence directories
>> in data storage configuration (see DataStorageConfiguration#walPath,
>> DataStorageConfiguration#walArchivePath,
>> DataStorageConfiguration#storagePath properties). Current persistence store
>> directory is: [/tmp]
>
> [10:53:59] Performance suggestions for grid 'cm' (fix if possible)
>
> [10:53:59] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
>
> [10:53:59] ^-- Enable G1 Garbage Collector (add '-XX:+UseG1GC' to JVM
>> options)
>
> [10:53:59] ^-- Specify JVM heap max size (add '-Xmx<size>[g|G|m|M|k|K]'
>> to JVM options)
>
> [10:53:59] ^-- Set max direct memory size if getting 'OOME: Direct
>> buffer memory' (add '-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM
>> options)
>
> [10:53:59] ^-- Disable processing of calls to System.gc() (add
>> '-XX:+DisableExplicitGC' to JVM options)
>
> [10:53:59] Refer to this page for more performance suggestions:
>> https://apacheignite.readme.io/docs/jvm-and-system-tuning
>
> [10:53:59]
>
> [10:53:59] To start Console Management & Monitoring run
>> ignitevisorcmd.{sh|bat}
>
> [10:53:59]
>
> [10:53:59] Ignite node started OK (id=085939f2, instance name=cm)
>
> [10:53:59] Topology snapshot [ver=1, servers=1, clients=0, CPUs=8,
>> offheap=2.8GB, heap=2.6GB]
>
> [10:53:59] ^-- Node [id=085939F2-56B0-42C7-BC17-A7BBC8F000F0,
>> clusterState=INACTIVE]
>
> [10:53:59] ^-- Baseline [id=0, size=1, online=1, offline=0]
>
> [10:53:59] ^-- All baseline nodes are online, will start auto-activation
>
> [10:53:59] Data Regions Configured:
>
>
>
As you can see the baseline topology is configured correctly and cluster is
activated.
2. When I try to kill the application using Ctrl+C, it doesn't work. It
used to work before I enabled Ignite Persistence.
Any clue what might be wrong here?
Re: Problems are enabling Ignite Persistence
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
My only idea is that maybe you have PDS directory left over from 2.3 or
earlier so it does not contain baseline.
Otherwise, can you try adding your node to baseline?
Regards,
--
Ilya Kasnacheev
ср, 3 окт. 2018 г. в 9:33, Lokesh Sharma <lo...@gmail.com>:
> I activated the cluster using "./control.sh --activate" and it
> successfully activated the cluster. But when I try to see the baseline it
> says "no nodes found". The docs
> <https://apacheignite.readme.io/docs/baseline-topology#section-setting-the-topology-from-code>
> says "*As explained above, baseline topology is automatically initialized
> when you activate the cluster manually.*" How can this be so?
>
> Following is the output
>
> ➜ bin ./control.sh --activate
>> Control utility [ver. 2.6.0#20180710-sha1:669feacc]
>> 2018 Copyright(C) Apache Software Foundation
>> User: lokesh
>>
>> --------------------------------------------------------------------------------
>> Cluster activated
>> ➜ bin ./control.sh --baseline
>> Control utility [ver. 2.6.0#20180710-sha1:669feacc]
>> 2018 Copyright(C) Apache Software Foundation
>> User: lokesh
>>
>> --------------------------------------------------------------------------------
>> Cluster state: active
>> Current topology version: 1
>> Baseline nodes not found.
>
>
>
>
> On Mon, Oct 1, 2018 at 12:49 PM Mikael <mi...@telia.com> wrote:
>
>> Yes it works fine with just one node, I do it all the time for testing.
>>
>> Mikael
>>
>> Den 2018-10-01 kl. 09:06, skrev Ilya Kasnacheev:
>>
>> Hello!
>>
>> Due to https://apacheignite.readme.io/docs/baseline-topology your
>> cluster should indeed auto-activate on a second run provided that all nodes
>> which were in the original cluster are still available.
>> Note that it will only work if you keep persistence directory intact
>> between runs.
>> I'm also not completely sure whether it will work with one node only.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 28 сент. 2018 г. в 17:33, Lokesh Sharma <lo...@gmail.com>:
>>
>>> Could you also set correct path for IGNITE_HOME or set work directory in
>>>> Ignite configuration to exclude chances for your /tmp directory to be
>>>> wiped.
>>>
>>>
>>> Yes, tried but had no luck.
>>>
>>> On Fri, Sep 28, 2018 at 7:59 PM Lokesh Sharma <lo...@gmail.com>
>>> wrote:
>>>
>>>> Hi
>>>>
>>>> Thanks for looking into this.
>>>>
>>>>
>>>>> First, I had to change h2 version dependency from 1.4.197 to 1.4.195 -
>>>>> maybe
>>>>> this is because I have later snapshot of Apache Ignite compiled.
>>>>
>>>>
>>>> The version is already 1.4.195.
>>>>
>>>> Second, I had to activate the cluster! You have to activate persistent
>>>>> cluster after you bring all nodes up. Just add ignite.active(true); on
>>>>> IgniteConfig.java:92
>>>>
>>>>
>>>> After, I added "ignite.active(true)" it is working for me. Although,
>>>> when I remove this line on second run of the app, the original problem
>>>> persists. Shouldn't the cluster be automatically activated in the second
>>>> run as said in the documentation?
>>>>
>>>>
>>>>
>>>> On Fri, Sep 28, 2018 at 6:55 PM ilya.kasnacheev <
>>>> ilya.kasnacheev@gmail.com> wrote:
>>>>
>>>>> Hello!
>>>>>
>>>>> Seems to be working for me after two changes.
>>>>>
>>>>> First, I had to change h2 version dependency from 1.4.197 to 1.4.195 -
>>>>> maybe
>>>>> this is because I have later snapshot of Apache Ignite compiled.
>>>>>
>>>>> Second, I had to activate the cluster! You have to activate persistent
>>>>> cluster after you bring all nodes up. Just add ignite.active(true); on
>>>>> IgniteConfig.java:92
>>>>>
>>>>> The error message about activation requirement was clear and I still
>>>>> could
>>>>> exist app with Ctrl-C. Maybe you have extra problems if you are on some
>>>>> unstable commit of master branch.
>>>>>
>>>>> Regards,
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>
>>>>
>>
Re: Problems are enabling Ignite Persistence
Posted by Lokesh Sharma <lo...@gmail.com>.
I activated the cluster using "./control.sh --activate" and it successfully
activated the cluster. But when I try to see the baseline it says "no nodes
found". The docs
<https://apacheignite.readme.io/docs/baseline-topology#section-setting-the-topology-from-code>
says "*As explained above, baseline topology is automatically initialized
when you activate the cluster manually.*" How can this be so?
Following is the output
➜ bin ./control.sh --activate
> Control utility [ver. 2.6.0#20180710-sha1:669feacc]
> 2018 Copyright(C) Apache Software Foundation
> User: lokesh
>
> --------------------------------------------------------------------------------
> Cluster activated
> ➜ bin ./control.sh --baseline
> Control utility [ver. 2.6.0#20180710-sha1:669feacc]
> 2018 Copyright(C) Apache Software Foundation
> User: lokesh
>
> --------------------------------------------------------------------------------
> Cluster state: active
> Current topology version: 1
> Baseline nodes not found.
On Mon, Oct 1, 2018 at 12:49 PM Mikael <mi...@telia.com> wrote:
> Yes it works fine with just one node, I do it all the time for testing.
>
> Mikael
>
> Den 2018-10-01 kl. 09:06, skrev Ilya Kasnacheev:
>
> Hello!
>
> Due to https://apacheignite.readme.io/docs/baseline-topology your cluster
> should indeed auto-activate on a second run provided that all nodes which
> were in the original cluster are still available.
> Note that it will only work if you keep persistence directory intact
> between runs.
> I'm also not completely sure whether it will work with one node only.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 28 сент. 2018 г. в 17:33, Lokesh Sharma <lo...@gmail.com>:
>
>> Could you also set correct path for IGNITE_HOME or set work directory in
>>> Ignite configuration to exclude chances for your /tmp directory to be
>>> wiped.
>>
>>
>> Yes, tried but had no luck.
>>
>> On Fri, Sep 28, 2018 at 7:59 PM Lokesh Sharma <lo...@gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> Thanks for looking into this.
>>>
>>>
>>>> First, I had to change h2 version dependency from 1.4.197 to 1.4.195 -
>>>> maybe
>>>> this is because I have later snapshot of Apache Ignite compiled.
>>>
>>>
>>> The version is already 1.4.195.
>>>
>>> Second, I had to activate the cluster! You have to activate persistent
>>>> cluster after you bring all nodes up. Just add ignite.active(true); on
>>>> IgniteConfig.java:92
>>>
>>>
>>> After, I added "ignite.active(true)" it is working for me. Although,
>>> when I remove this line on second run of the app, the original problem
>>> persists. Shouldn't the cluster be automatically activated in the second
>>> run as said in the documentation?
>>>
>>>
>>>
>>> On Fri, Sep 28, 2018 at 6:55 PM ilya.kasnacheev <
>>> ilya.kasnacheev@gmail.com> wrote:
>>>
>>>> Hello!
>>>>
>>>> Seems to be working for me after two changes.
>>>>
>>>> First, I had to change h2 version dependency from 1.4.197 to 1.4.195 -
>>>> maybe
>>>> this is because I have later snapshot of Apache Ignite compiled.
>>>>
>>>> Second, I had to activate the cluster! You have to activate persistent
>>>> cluster after you bring all nodes up. Just add ignite.active(true); on
>>>> IgniteConfig.java:92
>>>>
>>>> The error message about activation requirement was clear and I still
>>>> could
>>>> exist app with Ctrl-C. Maybe you have extra problems if you are on some
>>>> unstable commit of master branch.
>>>>
>>>> Regards,
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>
>>>
>
Re: Problems are enabling Ignite Persistence
Posted by Mikael <mi...@telia.com>.
Yes it works fine with just one node, I do it all the time for testing.
Mikael
Den 2018-10-01 kl. 09:06, skrev Ilya Kasnacheev:
> Hello!
>
> Due to https://apacheignite.readme.io/docs/baseline-topology your
> cluster should indeed auto-activate on a second run provided that all
> nodes which were in the original cluster are still available.
> Note that it will only work if you keep persistence directory intact
> between runs.
> I'm also not completely sure whether it will work with one node only.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 28 сент. 2018 г. в 17:33, Lokesh Sharma <lokeshh.sharma@gmail.com
> <ma...@gmail.com>>:
>
> Could you also set correct path for IGNITE_HOME or set work
> directory in
> Ignite configuration to exclude chances for your /tmp
> directory to be wiped.
>
>
> Yes, tried but had no luck.
>
> On Fri, Sep 28, 2018 at 7:59 PM Lokesh Sharma
> <lokeshh.sharma@gmail.com <ma...@gmail.com>> wrote:
>
> Hi
>
> Thanks for looking into this.
>
> First, I had to change h2 version dependency from 1.4.197
> to 1.4.195 - maybe
> this is because I have later snapshot of Apache Ignite
> compiled.
>
>
> The version is already 1.4.195.
>
> Second, I had to activate the cluster! You have to
> activate persistent
> cluster after you bring all nodes up. Just add
> ignite.active(true); on
> IgniteConfig.java:92
>
> After, I added "ignite.active(true)" it is working for me.
> Although, when I remove this line on second run of the app,
> the original problem persists. Shouldn't the cluster be
> automatically activated in the second run as said in the
> documentation?
>
>
>
> On Fri, Sep 28, 2018 at 6:55 PM ilya.kasnacheev
> <ilya.kasnacheev@gmail.com <ma...@gmail.com>>
> wrote:
>
> Hello!
>
> Seems to be working for me after two changes.
>
> First, I had to change h2 version dependency from 1.4.197
> to 1.4.195 - maybe
> this is because I have later snapshot of Apache Ignite
> compiled.
>
> Second, I had to activate the cluster! You have to
> activate persistent
> cluster after you bring all nodes up. Just add
> ignite.active(true); on
> IgniteConfig.java:92
>
> The error message about activation requirement was clear
> and I still could
> exist app with Ctrl-C. Maybe you have extra problems if
> you are on some
> unstable commit of master branch.
>
> Regards,
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Problems are enabling Ignite Persistence
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Due to https://apacheignite.readme.io/docs/baseline-topology your cluster
should indeed auto-activate on a second run provided that all nodes which
were in the original cluster are still available.
Note that it will only work if you keep persistence directory intact
between runs.
I'm also not completely sure whether it will work with one node only.
Regards,
--
Ilya Kasnacheev
пт, 28 сент. 2018 г. в 17:33, Lokesh Sharma <lo...@gmail.com>:
> Could you also set correct path for IGNITE_HOME or set work directory in
>> Ignite configuration to exclude chances for your /tmp directory to be
>> wiped.
>
>
> Yes, tried but had no luck.
>
> On Fri, Sep 28, 2018 at 7:59 PM Lokesh Sharma <lo...@gmail.com>
> wrote:
>
>> Hi
>>
>> Thanks for looking into this.
>>
>>
>>> First, I had to change h2 version dependency from 1.4.197 to 1.4.195 -
>>> maybe
>>> this is because I have later snapshot of Apache Ignite compiled.
>>
>>
>> The version is already 1.4.195.
>>
>> Second, I had to activate the cluster! You have to activate persistent
>>> cluster after you bring all nodes up. Just add ignite.active(true); on
>>> IgniteConfig.java:92
>>
>>
>> After, I added "ignite.active(true)" it is working for me. Although, when
>> I remove this line on second run of the app, the original problem persists.
>> Shouldn't the cluster be automatically activated in the second run as said
>> in the documentation?
>>
>>
>>
>> On Fri, Sep 28, 2018 at 6:55 PM ilya.kasnacheev <
>> ilya.kasnacheev@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> Seems to be working for me after two changes.
>>>
>>> First, I had to change h2 version dependency from 1.4.197 to 1.4.195 -
>>> maybe
>>> this is because I have later snapshot of Apache Ignite compiled.
>>>
>>> Second, I had to activate the cluster! You have to activate persistent
>>> cluster after you bring all nodes up. Just add ignite.active(true); on
>>> IgniteConfig.java:92
>>>
>>> The error message about activation requirement was clear and I still
>>> could
>>> exist app with Ctrl-C. Maybe you have extra problems if you are on some
>>> unstable commit of master branch.
>>>
>>> Regards,
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>
Re: Problems are enabling Ignite Persistence
Posted by Lokesh Sharma <lo...@gmail.com>.
>
> Could you also set correct path for IGNITE_HOME or set work directory in
> Ignite configuration to exclude chances for your /tmp directory to be
> wiped.
Yes, tried but had no luck.
On Fri, Sep 28, 2018 at 7:59 PM Lokesh Sharma <lo...@gmail.com>
wrote:
> Hi
>
> Thanks for looking into this.
>
>
>> First, I had to change h2 version dependency from 1.4.197 to 1.4.195 -
>> maybe
>> this is because I have later snapshot of Apache Ignite compiled.
>
>
> The version is already 1.4.195.
>
> Second, I had to activate the cluster! You have to activate persistent
>> cluster after you bring all nodes up. Just add ignite.active(true); on
>> IgniteConfig.java:92
>
>
> After, I added "ignite.active(true)" it is working for me. Although, when
> I remove this line on second run of the app, the original problem persists.
> Shouldn't the cluster be automatically activated in the second run as said
> in the documentation?
>
>
>
> On Fri, Sep 28, 2018 at 6:55 PM ilya.kasnacheev <il...@gmail.com>
> wrote:
>
>> Hello!
>>
>> Seems to be working for me after two changes.
>>
>> First, I had to change h2 version dependency from 1.4.197 to 1.4.195 -
>> maybe
>> this is because I have later snapshot of Apache Ignite compiled.
>>
>> Second, I had to activate the cluster! You have to activate persistent
>> cluster after you bring all nodes up. Just add ignite.active(true); on
>> IgniteConfig.java:92
>>
>> The error message about activation requirement was clear and I still could
>> exist app with Ctrl-C. Maybe you have extra problems if you are on some
>> unstable commit of master branch.
>>
>> Regards,
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
Re: Problems are enabling Ignite Persistence
Posted by Lokesh Sharma <lo...@gmail.com>.
Hi
Thanks for looking into this.
> First, I had to change h2 version dependency from 1.4.197 to 1.4.195 -
> maybe
> this is because I have later snapshot of Apache Ignite compiled.
The version is already 1.4.195.
Second, I had to activate the cluster! You have to activate persistent
> cluster after you bring all nodes up. Just add ignite.active(true); on
> IgniteConfig.java:92
After, I added "ignite.active(true)" it is working for me. Although, when I
remove this line on second run of the app, the original problem persists.
Shouldn't the cluster be automatically activated in the second run as said
in the documentation?
On Fri, Sep 28, 2018 at 6:55 PM ilya.kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> Seems to be working for me after two changes.
>
> First, I had to change h2 version dependency from 1.4.197 to 1.4.195 -
> maybe
> this is because I have later snapshot of Apache Ignite compiled.
>
> Second, I had to activate the cluster! You have to activate persistent
> cluster after you bring all nodes up. Just add ignite.active(true); on
> IgniteConfig.java:92
>
> The error message about activation requirement was clear and I still could
> exist app with Ctrl-C. Maybe you have extra problems if you are on some
> unstable commit of master branch.
>
> Regards,
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Problems are enabling Ignite Persistence
Posted by "ilya.kasnacheev" <il...@gmail.com>.
Hello!
Seems to be working for me after two changes.
First, I had to change h2 version dependency from 1.4.197 to 1.4.195 - maybe
this is because I have later snapshot of Apache Ignite compiled.
Second, I had to activate the cluster! You have to activate persistent
cluster after you bring all nodes up. Just add ignite.active(true); on
IgniteConfig.java:92
The error message about activation requirement was clear and I still could
exist app with Ctrl-C. Maybe you have extra problems if you are on some
unstable commit of master branch.
Regards,
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Problems are enabling Ignite Persistence
Posted by Lokesh Sharma <lo...@gmail.com>.
>
> Could you try to send minimalistic reproducer that
> shows issue on clean setup?
I have reproduced a minimal project that shows this behaviour. Here it is
https://bitbucket.org/lokesh_blue/bugdemo/src/master/
It's a simple application. It should execute function "test" when ran. See (
https://bitbucket.org/lokesh_blue/bugdemo/src/56fdf18154d08fd0d5330cf36a78c01cadcb8044/src/main/java/com/bluecoin/common/igniteserver/config/IgniteConfig.java#lines-97
)
It works fine when you run it with "mvn clean spring-boot:run".
But when you enable persistent by uncommenting this line (
https://bitbucket.org/lokesh_blue/bugdemo/src/56fdf18154d08fd0d5330cf36a78c01cadcb8044/src/main/java/com/bluecoin/common/igniteserver/config/IgniteConfig.java#lines-66),
it stops working as expected. The function "test" is executed no more on
application startrup and also the application doesn't exit on Ctrl + C.
On Sat, Sep 22, 2018 at 6:48 AM Denis Magda <dm...@apache.org> wrote:
> As a general recommendation, whenever you see a hang like that make a
> thread dump to see why the threads are blocked.
>
> --
> Denis
>
> On Wed, Sep 19, 2018 at 12:59 PM akurbanov <an...@gmail.com> wrote:
>
>> Hello,
>>
>> Could you also set correct path for IGNITE_HOME or set work directory in
>> Ignite configuration to exclude chances for your /tmp directory to be
>> wiped.
>>
>> Regards,
>>
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
Re: Problems are enabling Ignite Persistence
Posted by Denis Magda <dm...@apache.org>.
As a general recommendation, whenever you see a hang like that make a
thread dump to see why the threads are blocked.
--
Denis
On Wed, Sep 19, 2018 at 12:59 PM akurbanov <an...@gmail.com> wrote:
> Hello,
>
> Could you also set correct path for IGNITE_HOME or set work directory in
> Ignite configuration to exclude chances for your /tmp directory to be
> wiped.
>
> Regards,
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Problems are enabling Ignite Persistence
Posted by akurbanov <an...@gmail.com>.
Hello,
Could you also set correct path for IGNITE_HOME or set work directory in
Ignite configuration to exclude chances for your /tmp directory to be wiped.
Regards,
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Problems are enabling Ignite Persistence
Posted by akurbanov <an...@gmail.com>.
Hello,
There are no issues with starting nodes with enabled persistence, your
DataStorageConfiguration is fine, are there any other preconditions to be
met to face same issue? Could you try to send minimalistic reproducer that
shows issue on clean setup?
Regards,
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/