You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "Ray Liu (rayliu)" <ra...@cisco.com> on 2017/09/15 08:49:35 UTC

Ignite cluster with persistent store enabled did not load from wal after restarting.

Hi All,

I’m using Ignite version 2.1.
I opened persistent sotre by adding these configuration in the xml file according to https://apacheignite.readme.io/docs/distributed-persistent-store

  <property name="persistentStoreConfiguration">
    <bean class="org.apache.ignite.configuration.PersistentStoreConfiguration"/>
  </property>

Here’s the code I used to ingest data to ignite.


public class IgniteTest {
    public static void main(String[] args) throws Exception {

        try (Ignite ignite = Ignition.start("example-ignite.xml")) {

            CacheConfiguration<Integer, Person> personCacheCfg = new CacheConfiguration<>("person");

            personCacheCfg.setIndexedTypes(Integer.class, Person.class);

            IgniteCache<Integer, Person> personCache = ignite.getOrCreateCache(personCacheCfg);

            Person p1 = new Person(1, "John", "Doe", 2000);
            Person p2 = new Person(1, "Jane", "Doe", 1000);
            Person p3 = new Person(2, "John", "Smith", 1000);
            Person p4 = new Person(2, "Jane", "Smith", 2000);

            personCache.put(p1.getId(), p1);
            personCache.put(p2.getId(), p2);
            personCache.put(p3.getId(), p3);
            personCache.put(p4.getId(), p4);
        }
    }
     public static class Person implements Serializable {
        private static int PERSON_ID = 0;

        @QuerySqlField(index = true)
        private Integer id;

        @QuerySqlField(index = true)
        private Integer orgId;

        @QuerySqlField
        private String firstName;

        @QuerySqlField
        private String lastName;

        @QuerySqlField(index = true)
        private double salary;

        Person(Integer orgId, String firstName, String lastName, double salary) {
            id = PERSON_ID++;

            this.orgId = orgId;
            this.firstName = firstName;
            this.lastName = lastName;
            this.salary = salary;
        }

        public Integer getOrganizationId() {
            return orgId;
        }

        public String getFirstName() {
            return firstName;
        }

        public String getLastName() {
            return lastName;
        }

        public double getSalary() {

            return salary;
        }

        public Integer getId() {
            return id;
        }
    }

}

And I can see the wal log files generated under /$IgniteHome/work/db folder.
After restarting the cluster, the “person” cache should be re-created automatically, right?
But it didn’t.
Here’s some debug log may help locate the problem

[2017-09-15T16:34:13,090][INFO ][main][CacheObjectBinaryProcessorImpl] Resolved directory for serialized binary metadata:  /apache-ignite-fabric-2.1.0-bin/work/binary_meta/0_0_0_0_0_0_0_1_10_140_48_140_127_0_0_1_2001_420_589a_1250_516a_c5d4_fe7d_863c_47500
[2017-09-15T16:34:13,225][DEBUG][main][GridCacheMvccManager] Cache manager started.
[2017-09-15T16:34:13,225][DEBUG][main][GridCacheVersionManager] Cache manager started.
[2017-09-15T16:34:13,243][DEBUG][main][IgniteTxManager] Cache manager started.
[2017-09-15T16:34:13,243][INFO ][main][FilePageStoreManager] Resolved page store work directory: //apache-ignite-fabric-2.1.0-bin/work/db/0_0_0_0_0_0_0_1_10_140_48_140_127_0_0_1_2001_420_589a_1250_516a_c5d4_fe7d_863c_47500
[2017-09-15T16:34:13,243][DEBUG][main][FilePageStoreManager] Cache manager started.
[2017-09-15T16:34:13,243][INFO ][main][FileWriteAheadLogManager] Resolved write ahead log work directory: /apache-ignite-fabric-2.1.0-bin/work/db/wal/0_0_0_0_0_0_0_1_10_140_48_140_127_0_0_1_2001_420_589a_1250_516a_c5d4_fe7d_863c_47500
[2017-09-15T16:34:13,244][INFO ][main][FileWriteAheadLogManager] Resolved write ahead log archive directory: //apache-ignite-fabric-2.1.0-bin/work/db/wal/archive/0_0_0_0_0_0_0_1_10_140_48_140_127_0_0_1_2001_420_589a_1250_516a_c5d4_fe7d_863c_47500
[2017-09-15T16:34:13,253][DEBUG][main][FileWriteAheadLogManager] Cache manager started.

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Ray <ra...@cisco.com>.
This problem magically fixed itself after a few restart.

Don't know what happened, but delete all files under work/db might help.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Ray <ra...@cisco.com>.
After a few try, now the cache can be re-created after the cluster restart.
Looks like it is the IP issue.

But a new problem come up.
After cluster restarting, the "person" cache is re-created and all the data
is there.
But my other cache's metadata information is also re-created, but the data
is not reloaded.
I can confirm that the data of that cache is in
work/db/0_0_0_0_0_0_0_1_10_140_48_140_127_0_0_1_2001_420_589a_1250_516a_c5d4_fe7d_863c_47500/cache-cacheName
folder.
And here's some debug log may be relevant.
From the above log, looks like the there's some "injecting resources"
operations done for person cache but there's no "injecting resources"
operations done for my cache.

[2017-09-18T11:50:27,491][DEBUG][exchange-worker-#34%null%][GridCacheEventManager]
<person> Cache manager started: person
[2017-09-18T11:50:27,491][DEBUG][exchange-worker-#34%null%][CacheOsStoreManager]
<person> Cache manager started: person
[2017-09-18T11:50:27,491][DEBUG][exchange-worker-#34%null%][CacheOffheapEvictionManager]
<person> Cache manager started: person
[2017-09-18T11:50:27,491][DEBUG][exchange-worker-#34%null%][GridCacheDistributedQueryManager]
<person> Cache manager started: person
[2017-09-18T11:50:27,491][DEBUG][exchange-worker-#34%null%][CacheContinuousQueryManager]
<person> Cache manager started: person
[2017-09-18T11:50:27,491][DEBUG][exchange-worker-#34%null%][CacheDataStructuresManager]
<person> Cache manager started: person
[2017-09-18T11:50:27,492][DEBUG][exchange-worker-#34%null%][GridCacheTtlManager]
<person> Cache manager started: person
[2017-09-18T11:50:27,492][DEBUG][exchange-worker-#34%null%][CachePluginManager]
<person> Cache manager started: person
[2017-09-18T11:50:27,492][DEBUG][exchange-worker-#34%null%][GridCacheAffinityManager]
<person> Cache manager started: person
[2017-09-18T11:50:27,492][DEBUG][ttl-cleanup-worker-#44%null%][GridCacheSharedTtlCleanupManager]
Grid runnable started: ttl-cleanup-worker
[2017-09-18T11:50:27,499][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting resources
[obj=org.apache.ignite.internal.processors.cache.CacheDefaultBinaryAffinityKeyMapper@4cfbdbf9]
[2017-09-18T11:50:27,499][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting resources
[obj=org.apache.ignite.internal.processors.cache.CacheDefaultBinaryAffinityKeyMapper@5563654c]
[2017-09-18T11:50:27,502][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Created H2 schema for index database: person
[2017-09-18T11:50:27,503][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Set schema: person
[2017-09-18T11:50:27,511][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Creating DB table with SQL: CREATE TABLE "person"."PERSON" (_KEY INT
INVISIBLE NOT NULL,_VAL OTHER INVISIBLE,_VER OTHER INVISIBLE,"ID"
INT,"ORGID" INT,"FIRSTNAME" VARCHAR,"LASTNAME" VARCHAR,"SALARY" DOUBLE)
[2017-09-18T11:50:27,517][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Creating cache index [cacheId=-991716523, idxName=_key_PK]
[2017-09-18T11:50:27,525][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Creating cache index [cacheId=-991716523, idxName=PERSON_SALARY_IDX]
[2017-09-18T11:50:27,527][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Creating cache index [cacheId=-991716523, idxName=PERSON_ORGID_IDX]
[2017-09-18T11:50:27,527][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Creating cache index [cacheId=-991716523, idxName=PERSON_ID_IDX]
[2017-09-18T11:50:27,531][INFO
][exchange-worker-#34%null%][GridCacheProcessor] Started cache [name=person,
memoryPolicyName=default, mode=PARTITIONED, atomicity=ATOMIC]
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][GridCacheEventManager]
<person> Cache manager received onKernalStart() callback: person
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][CacheOsStoreManager]
<person> Cache manager received onKernalStart() callback: person
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][CacheOffheapEvictionManager]
<person> Cache manager received onKernalStart() callback: person
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][GridCacheDistributedQueryManager]
<person> Cache manager received onKernalStart() callback: person
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][CacheContinuousQueryManager]
<person> Cache manager received onKernalStart() callback: person
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][CacheDataStructuresManager]
<person> Cache manager received onKernalStart() callback: person
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][GridCacheTtlManager]
<person> Cache manager received onKernalStart() callback: person
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][CachePluginManager]
<person> Cache manager received onKernalStart() callback: person
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][GridCacheAffinityManager]
<person> Cache manager received onKernalStart() callback: person
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][GridCacheProcessor]
Executed onKernalStart() callback for cache [name=person, mode=PARTITIONED]
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting resources
[obj=org.apache.ignite.internal.processors.cache.CacheDefaultBinaryAffinityKeyMapper@3877603c]
[2017-09-18T11:50:27,533][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting resources
[obj=org.apache.ignite.internal.processors.cache.CacheDefaultBinaryAffinityKeyMapper@1418f2a3]
[2017-09-18T11:50:27,534][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting resources
[obj=org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction@1291b2a6]
[2017-09-18T11:50:27,534][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting cache name:
org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction@1291b2a6
[2017-09-18T11:50:27,534][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting resources [obj=IgniteAllNodesPredicate []]
[2017-09-18T11:50:27,534][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting cache name: IgniteAllNodesPredicate []
[2017-09-18T11:50:27,534][DEBUG][exchange-worker-#34%null%][GridDhtPreloader]
Starting DHT rebalancer...
[2017-09-18T11:50:27,535][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting resources
[obj=org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction@1291b2a6]
[2017-09-18T11:50:27,535][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting cache name:
org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction@1291b2a6
[2017-09-18T11:50:27,535][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting resources
[obj=org.apache.ignite.internal.processors.cache.CacheDefaultBinaryAffinityKeyMapper@3877603c]
[2017-09-18T11:50:27,535][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting cache name:
org.apache.ignite.internal.processors.cache.CacheDefaultBinaryAffinityKeyMapper@3877603c
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][GridCacheEventManager]
<DailyAggData> Cache manager started: DailyAggData
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][CacheOsStoreManager]
<DailyAggData> Cache manager started: DailyAggData
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][CacheOffheapEvictionManager]
<DailyAggData> Cache manager started: DailyAggData
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][GridCacheDistributedQueryManager]
<DailyAggData> Cache manager started: DailyAggData
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][CacheContinuousQueryManager]
<DailyAggData> Cache manager started: DailyAggData
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][CacheDataStructuresManager]
<DailyAggData> Cache manager started: DailyAggData
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][GridCacheTtlManager]
<DailyAggData> Cache manager started: DailyAggData
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][CachePluginManager]
<DailyAggData> Cache manager started: DailyAggData
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][GridCacheAffinityManager]
<DailyAggData> Cache manager started: DailyAggData
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting resources
[obj=org.apache.ignite.internal.processors.cache.CacheDefaultBinaryAffinityKeyMapper@3877603c]
[2017-09-18T11:50:27,536][DEBUG][exchange-worker-#34%null%][GridResourceProcessor]
Injecting resources
[obj=org.apache.ignite.internal.processors.cache.CacheDefaultBinaryAffinityKeyMapper@65419213]
[2017-09-18T11:50:27,538][DEBUG][grid-timeout-worker-#15%null%][GridTimeoutProcessor]
Timeout has occurred: CancelableTask
[id=f558b139e51-7c159f4b-d1c4-410b-96c9-6a71327bf3f9, endTime=1505706627526,
period=3000, cancel=false,
task=org.apache.ignite.internal.processors.query.GridQueryProcessor$2@4acb667]
[2017-09-18T11:50:27,539][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Set schema: PUBLIC
[2017-09-18T11:50:27,539][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Creating DB table with SQL: CREATE TABLE "PUBLIC"."MEETINGAGGVALUE" (_KEY
OTHER INVISIBLE NOT NULL,_VAL OTHER INVISIBLE,_VER OTHER INVISIBLE,SOME_KEY)
[2017-09-18T11:50:27,539][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Creating cache index [cacheId=1917330450, idxName=_key_PK]
[2017-09-18T11:50:27,540][DEBUG][exchange-worker-#34%null%][IgniteH2Indexing]
Creating cache index [cacheId=1917330450, idxName=SOME_IDX]
[2017-09-18T11:50:27,543][INFO
][exchange-worker-#34%null%][GridCacheProcessor] Started cache
[name=DailyAggData, memoryPolicyName=default, mode=PARTITIONED,
atomicity=ATOMIC]
[2017-09-18T11:50:27,543][DEBUG][exchange-worker-#34%null%][GridCacheEventManager]
<DailyAggData> Cache manager received onKernalStart() callback: DailyAggData
[2017-09-18T11:50:27,543][DEBUG][exchange-worker-#34%null%][CacheOsStoreManager]
<DailyAggData> Cache manager received onKernalStart() callback: DailyAggData
[2017-09-18T11:50:27,543][DEBUG][exchange-worker-#34%null%][CacheOffheapEvictionManager]
<DailyAggData> Cache manager received onKernalStart() callback: DailyAggData
[2017-09-18T11:50:27,543][DEBUG][exchange-worker-#34%null%][GridCacheDistributedQueryManager]
<DailyAggData> Cache manager received onKernalStart() callback: DailyAggData
[2017-09-18T11:50:27,543][DEBUG][exchange-worker-#34%null%][CacheContinuousQueryManager]
<DailyAggData> Cache manager received onKernalStart() callback: DailyAggData
[2017-09-18T11:50:27,543][DEBUG][exchange-worker-#34%null%][CacheDataStructuresManager]
<DailyAggData> Cache manager received onKernalStart() callback: DailyAggData
[2017-09-18T11:50:27,543][DEBUG][exchange-worker-#34%null%][GridCacheTtlManager]
<DailyAggData> Cache manager received onKernalStart() callback: DailyAggData
[2017-09-18T11:50:27,543][DEBUG][exchange-worker-#34%null%][CachePluginManager]
<DailyAggData> Cache manager received onKernalStart() callback: DailyAggData
[2017-09-18T11:50:27,543][DEBUG][exchange-worker-#34%null%][GridCacheAffinityManager]
<DailyAggData> Cache manager received onKernalStart() callback: DailyAggData
[2017-09-18T11:50:27,543][DEBUG][exchange-worker-#34%null%][GridCacheProcessor]
Executed onKernalStart() callback for cache [name=DailyAggData,
mode=PARTITIONED]
[2017-09-18T11:50:27,544][DEBUG][exchange-worker-#34%null%][GridAffinityAssignmentCache]
Calculating affinity [topVer=AffinityTopologyVersion [topVer=1,
minorTopVer=1], locNodeId=ca11c749-ddef-42e6-9204-26e16631fa9b,
discoEvt=DiscoveryCustomEvent [customMsg=ChangeGlobalStateMessage
[id=a758b139e51-7c159f4b-d1c4-410b-96c9-6a71327bf3f9,
reqId=03f047f1-df59-4569-83e3-08fffa9fc399,
initiatingNodeId=ca11c749-ddef-42e6-9204-26e16631fa9b, activate=true],
affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1],
super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=ca11c749-ddef-42e6-9204-26e16631fa9b, addrs=[0:0:0:0:0:0:0:1,
10.140.48.140, 127.0.0.1, 2001:420:589a:1250:516a:c5d4:fe7d:863c],
sockAddrs=[/10.140.48.140:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
/2001:420:589a:1250:516a:c5d4:fe7d:863c:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1505706609739, loc=true,
ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false], topVer=1,
nodeId8=ca11c749, msg=null, type=DISCOVERY_CUSTOM_EVT,
tstamp=1505706626737]]]
[2017-09-18T11:50:27,545][DEBUG][exchange-worker-#34%null%][GridAffinityAssignmentCache]
Calculating affinity [topVer=AffinityTopologyVersion [topVer=1,
minorTopVer=1], locNodeId=ca11c749-ddef-42e6-9204-26e16631fa9b,
discoEvt=DiscoveryCustomEvent [customMsg=ChangeGlobalStateMessage
[id=a758b139e51-7c159f4b-d1c4-410b-96c9-6a71327bf3f9,
reqId=03f047f1-df59-4569-83e3-08fffa9fc399,
initiatingNodeId=ca11c749-ddef-42e6-9204-26e16631fa9b, activate=true],
affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1],
super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=ca11c749-ddef-42e6-9204-26e16631fa9b, addrs=[0:0:0:0:0:0:0:1,
10.140.48.140, 127.0.0.1, 2001:420:589a:1250:516a:c5d4:fe7d:863c],
sockAddrs=[/10.140.48.140:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
/2001:420:589a:1250:516a:c5d4:fe7d:863c:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1505706609739, loc=true,
ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false], topVer=1,
nodeId8=ca11c749, msg=null, type=DISCOVERY_CUSTOM_EVT,
tstamp=1505706626737]]]
[2017-09-18T11:50:27,546][DEBUG][exchange-worker-#34%null%][GridAffinityAssignmentCache]
Calculating affinity [topVer=AffinityTopologyVersion [topVer=1,
minorTopVer=1], locNodeId=ca11c749-ddef-42e6-9204-26e16631fa9b,
discoEvt=DiscoveryCustomEvent [customMsg=ChangeGlobalStateMessage
[id=a758b139e51-7c159f4b-d1c4-410b-96c9-6a71327bf3f9,
reqId=03f047f1-df59-4569-83e3-08fffa9fc399,
initiatingNodeId=ca11c749-ddef-42e6-9204-26e16631fa9b, activate=true],
affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1],
super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=ca11c749-ddef-42e6-9204-26e16631fa9b, addrs=[0:0:0:0:0:0:0:1,
10.140.48.140, 127.0.0.1, 2001:420:589a:1250:516a:c5d4:fe7d:863c],
sockAddrs=[/10.140.48.140:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
/2001:420:589a:1250:516a:c5d4:fe7d:863c:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1505706609739, loc=true,
ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false], topVer=1,
nodeId8=ca11c749, msg=null, type=DISCOVERY_CUSTOM_EVT,
tstamp=1505706626737]]]
[2017-09-18T11:50:27,547][INFO
][exchange-worker-#34%null%][GridDhtPartitionsExchangeFuture] Successfully
activated caches [nodeId=ca11c749-ddef-42e6-9204-26e16631fa9b, client=false,
topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
[2017-09-18T11:50:27,551][DEBUG][exchange-worker-#34%null%][GridDhtPartitionsExchangeFuture]
Before waiting for partition release future: GridDhtPartitionsExchangeFuture
[dummy=false, forcePreload=false, reassign=false,
discoEvt=DiscoveryCustomEvent [customMsg=ChangeGlobalStateMessage
[id=a758b139e51-7c159f4b-d1c4-410b-96c9-6a71327bf3f9,
reqId=03f047f1-df59-4569-83e3-08fffa9fc399,
initiatingNodeId=ca11c749-ddef-42e6-9204-26e16631fa9b, activate=true],
affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1],
super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=ca11c749-ddef-42e6-9204-26e16631fa9b, addrs=[0:0:0:0:0:0:0:1,
10.140.48.140, 127.0.0.1, 2001:420:589a:1250:516a:c5d4:fe7d:863c],
sockAddrs=[/10.140.48.140:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
/2001:420:589a:1250:516a:c5d4:fe7d:863c:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1505706609739, loc=true,
ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false], topVer=1,
nodeId8=ca11c749, msg=null, type=DISCOVERY_CUSTOM_EVT,
tstamp=1505706626737]], crd=TcpDiscoveryNode
[id=ca11c749-ddef-42e6-9204-26e16631fa9b, addrs=[0:0:0:0:0:0:0:1,
10.140.48.140, 127.0.0.1, 2001:420:589a:1250:516a:c5d4:fe7d:863c],
sockAddrs=[/10.140.48.140:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
/2001:420:589a:1250:516a:c5d4:fe7d:863c:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1505706609739, loc=true,
ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false],
exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1,
minorTopVer=1], nodeId=ca11c749, evt=DISCOVERY_CUSTOM_EVT], added=true,
initFut=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1302695064], init=false, lastVer=null,
partReleaseFut=GridCompoundFuture [rdc=null, initFlag=1, lsnrCalls=4,
done=true, cancelled=false, err=null, futs=[true, true, true, true]],
exchActions=ExchangeActions [startCaches=[ignite-sys-cache, person,
DailyAggData], stopCaches=null, startGrps=[ignite-sys-cache, person,
DailyAggData], stopGrps=[], resetParts=null,
stateChangeRequest=StateChangeRequest [msg=ChangeGlobalStateMessage
[id=a758b139e51-7c159f4b-d1c4-410b-96c9-6a71327bf3f9,
reqId=03f047f1-df59-4569-83e3-08fffa9fc399,
initiatingNodeId=ca11c749-ddef-42e6-9204-26e16631fa9b, activate=true],
topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]],
affChangeMsg=null, skipPreload=false, clientOnlyExchange=false,
initTs=1505706626749, centralizedAff=false, changeGlobalStateE=null,
forcedRebFut=null, done=false, evtLatch=0, remaining=[],
super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=2123048598]]
[2017-09-18T11:50:27,551][INFO
][exchange-worker-#34%null%][GridDhtPartitionsExchangeFuture] Finished
waiting for partition release future [topVer=AffinityTopologyVersion
[topVer=1, minorTopVer=1], waitTime=0ms]
[2017-09-18T11:50:27,552][DEBUG][exchange-worker-#34%null%][exchange]
Pending lock set [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1],
locks={}]
[2017-09-18T11:50:27,552][DEBUG][exchange-worker-#34%null%][GridCacheMvccManager]
After rechecking finished future: FinishLockFuture
[topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], pendingLocks={},
super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=219314823]]
[2017-09-18T11:50:27,552][DEBUG][exchange-worker-#34%null%][exchange] Finish
lock future is done: FinishLockFuture [topVer=AffinityTopologyVersion
[topVer=1, minorTopVer=1], pendingLocks={}, super=GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=219314823]]
[2017-09-18T11:50:27,553][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Partition map beforeExchange [exchId=GridDhtPartitionExchangeId
[topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], nodeId=ca11c749,
evt=DISCOVERY_CUSTOM_EVT], fullMap=null]
[2017-09-18T11:50:27,553][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Created brand new full topology map on oldest node
[exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion
[topVer=1, minorTopVer=1], nodeId=ca11c749, evt=DISCOVERY_CUSTOM_EVT],
fullMap=GridDhtPartitionFullMap
[nodeId=ca11c749-ddef-42e6-9204-26e16631fa9b, nodeOrder=1, updateSeq=2,
size=0]]
[2017-09-18T11:50:27,560][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=0, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,560][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=0, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,561][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=1, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,561][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=1, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,561][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=2, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,561][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=2, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,561][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=3, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,561][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=3, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,561][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=4, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,562][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=4, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,562][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=5, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,562][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=5, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,562][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=6, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,562][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=6, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,562][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=7, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,562][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=7, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,562][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=8, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,563][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=8, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,563][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=9, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,563][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=9, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,563][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=10, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,563][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=10, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,563][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=11, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,563][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=11, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,563][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=12, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,563][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=12, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,563][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=13, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,564][DEBUG][exchange-worker-#34%null%][GridDhtPartitionTopologyImpl]
Owned partition for oldest node: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=13, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]
[2017-09-18T11:50:27,564][DEBUG][exchange-worker-#34%null%][GridDhtLocalPartition]
Owned partition: GridDhtLocalPartition [rmvQueueMaxSize=128,
rmvdEntryTtl=10000, id=14, shouldBeRenting=false, reload=false,
grp=ignite-sys-cache, state=OWNING, reservations=0, empty=true,
createTime=09/18/2017 11:50:27]



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Denis Magda <dm...@gridgain.com>.
Ray,

This looks odd. Sure, data has to be available for you after the restart,
it's a basic capability. Then I tend to agree with the rest of the folks
that you might haven encountered the issue mentioned by Yakov. To to be
100% sure about that please do the following:

* Clean "work/db" directory completely to avoid side effects.
* Run the app that preloads data. Copy the generated logs somewhere.
* Restart the cluster and start the second app.
* Compare the first and second logs checking that the "page store" and WAL
directories are identical in both files.
* If the directories vary then you hit issue [1] and let us know if this
the case. We have to come up how to solve this automatically. As a
workaround try to send IgniteConfiguration.setConsistentId() to a
predefined value.

[1] https://issues.apache.org/jira/browse/IGNITE-6285

--
Denis

On Fri, Sep 15, 2017 at 6:52 PM, Ray <ra...@cisco.com> wrote:

> Yes, I tried to work with the data after restarting.
> As I stated in the original post, I can't see any cache after I restart the
> cluster.
> So I called
> https://apacheignite.readme.io/docs/rest-api#section-get-or-create-cache
> API
> with the exact cache name before restarting as parameter.
> Then I try to run a sql query with that cache name and I got an empty
> cache.
> If the data loading from disk to RAM is transparent, I should see the data
> before restarting, right?
>
> I agree with you Dmitriy, according to the debug log, the WAL folder is
> resolved.
> And I can confirm there're several WAL logs under that folder.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Denis Magda <dm...@gridgain.com>.
Ray,

This looks odd. Sure, data has to be available for you after the restart,
it's a basic capability. Then I tend to agree with the rest of the folks
that you might haven encountered the issue mentioned by Yakov. To to be
100% sure about that please do the following:

* Clean "work/db" directory completely to avoid side effects.
* Run the app that preloads data. Copy the generated logs somewhere.
* Restart the cluster and start the second app.
* Compare the first and second logs checking that the "page store" and WAL
directories are identical in both files.
* If the directories vary then you hit issue [1] and let us know if this
the case. We have to come up how to solve this automatically. As a
workaround try to send IgniteConfiguration.setConsistentId() to a
predefined value.

[1] https://issues.apache.org/jira/browse/IGNITE-6285

--
Denis

On Fri, Sep 15, 2017 at 6:52 PM, Ray <ra...@cisco.com> wrote:

> Yes, I tried to work with the data after restarting.
> As I stated in the original post, I can't see any cache after I restart the
> cluster.
> So I called
> https://apacheignite.readme.io/docs/rest-api#section-get-or-create-cache
> API
> with the exact cache name before restarting as parameter.
> Then I try to run a sql query with that cache name and I got an empty
> cache.
> If the data loading from disk to RAM is transparent, I should see the data
> before restarting, right?
>
> I agree with you Dmitriy, according to the debug log, the WAL folder is
> resolved.
> And I can confirm there're several WAL logs under that folder.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Ray <ra...@cisco.com>.
Yes, I tried to work with the data after restarting.
As I stated in the original post, I can't see any cache after I restart the
cluster.
So I called
https://apacheignite.readme.io/docs/rest-api#section-get-or-create-cache API
with the exact cache name before restarting as parameter.
Then I try to run a sql query with that cache name and I got an empty cache.
If the data loading from disk to RAM is transparent, I should see the data
before restarting, right?

I agree with you Dmitriy, according to the debug log, the WAL folder is
resolved.
And I can confirm there're several WAL logs under that folder.
 




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Denis Magda <dm...@apache.org>.
Dmitriy,

Check up Ray’s initial message. There is no single cache.get command or SQL query that can trigger data movement from disk to RAM. Plus, the output generated after the restart suggests that the node found all the persistence directories. Thus, not sure how this issue is related to consistent ID mess discovered by Yakov before.

Ray, please chime in and clarify if you tried to work with the data after the restart. 

—
Denis

> On Sep 15, 2017, at 4:02 PM, Dmitry Pavlov <dp...@gmail.com> wrote:
> 
> Hi Denis,
> 
> I agree that data presence should be double-checked. In case check will show that 1) cache doesn't exist at all after restart and 2) we can find two folders with binary_meta and wal that means this question is similar with recent question http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html <http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html> 
> 
> It is completely transparent to user that data is in RAM or on disk.
> 
> 
> Hi Yakov,
> 
> To my mind ticket is reasonable. I can suggest to generate such warning only in case consistent ID is not overriden by IgniteConfiguration. Parameter override may indicate that several nodes running at the same machine.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> сб, 16 сент. 2017 г. в 1:47, Denis Magda <dmagda@apache.org <ma...@apache.org>>:
> Alexey,
> 
> The data preloading from disk to RAM doesn’t occur automatically after the restart. It will be moved to RAM on demand while your application executes its logic.
> 
> Ray,
> 
> Have you executed any query (key-value, SQL) to confirm that the data is missing after the shutdown?
> 
> —
> Denis
> 
> > On Sep 15, 2017, at 4:09 AM, Alexey Kukushkin <kukushkinalexey@gmail.com <ma...@gmail.com>> wrote:
> >
> > Ray,
> >
> > Yes, enabling persistence must automatically bring all your data and metadata upon the cluster startup. Please give us more details why you think the cache was not created. BTW, do you activate the cluster after startup? Enabling persistence requires explicit cluster activation:
> >
> > $IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate
> >
> >
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Denis Magda <dm...@apache.org>.
Dmitriy,

Check up Ray’s initial message. There is no single cache.get command or SQL query that can trigger data movement from disk to RAM. Plus, the output generated after the restart suggests that the node found all the persistence directories. Thus, not sure how this issue is related to consistent ID mess discovered by Yakov before.

Ray, please chime in and clarify if you tried to work with the data after the restart. 

—
Denis

> On Sep 15, 2017, at 4:02 PM, Dmitry Pavlov <dp...@gmail.com> wrote:
> 
> Hi Denis,
> 
> I agree that data presence should be double-checked. In case check will show that 1) cache doesn't exist at all after restart and 2) we can find two folders with binary_meta and wal that means this question is similar with recent question http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html <http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html> 
> 
> It is completely transparent to user that data is in RAM or on disk.
> 
> 
> Hi Yakov,
> 
> To my mind ticket is reasonable. I can suggest to generate such warning only in case consistent ID is not overriden by IgniteConfiguration. Parameter override may indicate that several nodes running at the same machine.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> сб, 16 сент. 2017 г. в 1:47, Denis Magda <dmagda@apache.org <ma...@apache.org>>:
> Alexey,
> 
> The data preloading from disk to RAM doesn’t occur automatically after the restart. It will be moved to RAM on demand while your application executes its logic.
> 
> Ray,
> 
> Have you executed any query (key-value, SQL) to confirm that the data is missing after the shutdown?
> 
> —
> Denis
> 
> > On Sep 15, 2017, at 4:09 AM, Alexey Kukushkin <kukushkinalexey@gmail.com <ma...@gmail.com>> wrote:
> >
> > Ray,
> >
> > Yes, enabling persistence must automatically bring all your data and metadata upon the cluster startup. Please give us more details why you think the cache was not created. BTW, do you activate the cluster after startup? Enabling persistence requires explicit cluster activation:
> >
> > $IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate
> >
> >
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Dmitry Pavlov <dp...@gmail.com>.
Hi Denis,

I agree that data presence should be double-checked. In case check will
show that 1) cache doesn't exist at all after restart and 2) we can find
two folders with binary_meta and wal that means this question is similar
with recent question
http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html


It is completely transparent to user that data is in RAM or on disk.


Hi Yakov,

To my mind ticket is reasonable. I can suggest to generate such warning
only in case consistent ID is not overriden by IgniteConfiguration.
Parameter override may indicate that several nodes running at the same
machine.

Sincerely,
Dmitriy Pavlov

сб, 16 сент. 2017 г. в 1:47, Denis Magda <dm...@apache.org>:

> Alexey,
>
> The data preloading from disk to RAM doesn’t occur automatically after the
> restart. It will be moved to RAM on demand while your application executes
> its logic.
>
> Ray,
>
> Have you executed any query (key-value, SQL) to confirm that the data is
> missing after the shutdown?
>
> —
> Denis
>
> > On Sep 15, 2017, at 4:09 AM, Alexey Kukushkin <ku...@gmail.com>
> wrote:
> >
> > Ray,
> >
> > Yes, enabling persistence must automatically bring all your data and
> metadata upon the cluster startup. Please give us more details why you
> think the cache was not created. BTW, do you activate the cluster after
> startup? Enabling persistence requires explicit cluster activation:
> >
> > $IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate
> >
> >
>
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Dmitry Pavlov <dp...@gmail.com>.
Hi Denis,

I agree that data presence should be double-checked. In case check will
show that 1) cache doesn't exist at all after restart and 2) we can find
two folders with binary_meta and wal that means this question is similar
with recent question
http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html


It is completely transparent to user that data is in RAM or on disk.


Hi Yakov,

To my mind ticket is reasonable. I can suggest to generate such warning
only in case consistent ID is not overriden by IgniteConfiguration.
Parameter override may indicate that several nodes running at the same
machine.

Sincerely,
Dmitriy Pavlov

сб, 16 сент. 2017 г. в 1:47, Denis Magda <dm...@apache.org>:

> Alexey,
>
> The data preloading from disk to RAM doesn’t occur automatically after the
> restart. It will be moved to RAM on demand while your application executes
> its logic.
>
> Ray,
>
> Have you executed any query (key-value, SQL) to confirm that the data is
> missing after the shutdown?
>
> —
> Denis
>
> > On Sep 15, 2017, at 4:09 AM, Alexey Kukushkin <ku...@gmail.com>
> wrote:
> >
> > Ray,
> >
> > Yes, enabling persistence must automatically bring all your data and
> metadata upon the cluster startup. Please give us more details why you
> think the cache was not created. BTW, do you activate the cluster after
> startup? Enabling persistence requires explicit cluster activation:
> >
> > $IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate
> >
> >
>
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Denis Magda <dm...@apache.org>.
Alexey,

The data preloading from disk to RAM doesn’t occur automatically after the restart. It will be moved to RAM on demand while your application executes its logic.

Ray, 

Have you executed any query (key-value, SQL) to confirm that the data is missing after the shutdown?

—
Denis

> On Sep 15, 2017, at 4:09 AM, Alexey Kukushkin <ku...@gmail.com> wrote:
> 
> Ray,
> 
> Yes, enabling persistence must automatically bring all your data and metadata upon the cluster startup. Please give us more details why you think the cache was not created. BTW, do you activate the cluster after startup? Enabling persistence requires explicit cluster activation:
> 
> $IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate
> 
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Alexey Kukushkin <ku...@gmail.com>.
Ray,

Yakov just pointed me to this thread
<http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html>
where
user had a problem with IP address changing between the cluster restarts.
The persistent data directories inside work/db have names built as a
concatenation of local IP addresses and ports. Thus, if your node receives
a different IP address between the restarts, you will not have your
persistent data since the node will initialise from a different persistent
data file. You would need to bind your discovery SPI to specific address to
work around that. There is an issue opened to properly fix it.

Please also consider that as a potential cause.

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Alexey Kukushkin <ku...@gmail.com>.
Ray,

Yakov just pointed me to this thread
<http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html>
where
user had a problem with IP address changing between the cluster restarts.
The persistent data directories inside work/db have names built as a
concatenation of local IP addresses and ports. Thus, if your node receives
a different IP address between the restarts, you will not have your
persistent data since the node will initialise from a different persistent
data file. You would need to bind your discovery SPI to specific address to
work around that. There is an issue opened to properly fix it.

Please also consider that as a potential cause.

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Yakov Zhdanov <yz...@apache.org>.
Guys, it seems that we have the same issue that was described in thread -
Specifying location of persistent storage location

I filed a ticket some time ago -
https://issues.apache.org/jira/browse/IGNITE-6285, Alex G, can you take a
look if this ticket makes sense?

--Yakov

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Yakov Zhdanov <yz...@apache.org>.
Guys, it seems that we have the same issue that was described in thread -
Specifying location of persistent storage location

I filed a ticket some time ago -
https://issues.apache.org/jira/browse/IGNITE-6285, Alex G, can you take a
look if this ticket makes sense?

--Yakov

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Alexey Kukushkin <ku...@gmail.com>.
Ray,

You will not see anything in WebConsole until you activate the cluster.
Could you please confirm you did step #7:

   1. You started the cluster with persistence enabled
   2. Activated the cluster
   3. Ran your IgniteTest app
   4. Checked the person cache in WebConsole - you should see it.
   5. Stopped the cluster
   6. Started the cluster. At this point you will NOT see the cache in
   WebConsole
   7. Activated the cluster
   8. Now you should see the person cache in WebConsole.



On Fri, Sep 15, 2017 at 2:29 PM, Ray <ra...@cisco.com> wrote:

> The cluster is activated using ignite.activate(true) command in a Java
> application.
> I checked the web-console and h2 debug console and found out there's no
> cache after restarting.
>
> I also tried call ignite?cmd=getorcreate&cacheName= API with the exact
> cache
> name before restarting and I got an empty cache.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Alexey

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Ray <ra...@cisco.com>.
The cluster is activated using ignite.activate(true) command in a Java
application.
I checked the web-console and h2 debug console and found out there's no
cache after restarting.

I also tried call ignite?cmd=getorcreate&cacheName= API with the exact cache
name before restarting and I got an empty cache.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Denis Magda <dm...@apache.org>.
Alexey,

The data preloading from disk to RAM doesn’t occur automatically after the restart. It will be moved to RAM on demand while your application executes its logic.

Ray, 

Have you executed any query (key-value, SQL) to confirm that the data is missing after the shutdown?

—
Denis

> On Sep 15, 2017, at 4:09 AM, Alexey Kukushkin <ku...@gmail.com> wrote:
> 
> Ray,
> 
> Yes, enabling persistence must automatically bring all your data and metadata upon the cluster startup. Please give us more details why you think the cache was not created. BTW, do you activate the cluster after startup? Enabling persistence requires explicit cluster activation:
> 
> $IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate
> 
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

Posted by Alexey Kukushkin <ku...@gmail.com>.
Ray,

Yes, enabling persistence must automatically bring all your data and
metadata upon the cluster startup. Please give us more details why you
think the cache was not created. BTW, do you activate the cluster after
startup? Enabling persistence requires explicit cluster activation:

$IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate