You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by nabarun nag <nn...@apache.org> on 2021/09/27 18:48:55 UTC
[DISCUSS] Upgrading to Lucene 7.1.0
Hi dev team,
Recently, a commit was pushed to develop which upgraded the Lucene
version used in Apache Geode to 7.1.0. These new Lucene indexes are
not compatible with the previous versions hence it breaks the rolling
upgrade contract. We are no longer able to execute Lucene queries when
there are severs of mixed versions in the cluster. One solution that
was provided was to not allow Lucene queries to be executed when there
are mixed versions of servers present in the cluster.
After a discussion, it was put forward that this was not an optimal
user experience. Also, that this change of default behavior was not
discussed in the dev list.
Alternative solutions that were put forward were that Geode allowed
the user to pick the version of Lucene to be used. Another that since
this is a major upgrade to Lucene with some breaking API changes,
should we sync this upgrade with the release of Apache Geode 2.0
We would love to hear your thoughts or alternative solutions to this issue.
Regards
Nabarun.
Odg: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Mario Kevo <ma...@est.tech>.
Hi,
When the servers are in a mixed version state in our tests, we blocked queries until the servers are on the versions with the same Lucene version.
If we allow queries while the servers are in a mixed state we got errors like IndexFormatTooNewException if the query is executed on the server with an older version of Lucene(It cannot read the new Lucene index format).
In the previous mail thread, I got the following:
"The pr against your branch just prevents the repo from being constructed until all old members are upgraded. This requires test changes to not try to validate using queries (since we prevent draining and repo creation, the query will just wait)"
So we changed tests to not do queries until all members are on the same version.
BR,
Mario
________________________________
Šalje: Dan Smith <da...@vmware.com>
Poslano: 28. rujna 2021. 19:41
Prima: dev@geode.apache.org <de...@geode.apache.org>
Predmet: Re: [DISCUSS] Upgrading to Lucene 7.1.0
My understanding from our previous discussion about upgrading lucene was that we talked about pausing the asynchronous indexing process during the rolling upgrade. I don't remember a discussion that it was ok to not allow queries during the upgrade. But this is what we added to the docs:
"All cluster members must be running the same major Lucene version in order to execute Lucene queries."
What happens if a user runs a query during the rolling upgrade and why do we need to have this restriction? It seems to me like at a minimum we need to allow queries during the upgrade.
We also should consider what will happen to users with server-side query or indexing code - will they be able to upgrade or are they likely to hit breaking changes in the Lucene API?
-Dan
________________________________
From: Nabarun Nag <nn...@vmware.com>
Sent: Tuesday, September 28, 2021 7:13 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
But Mario, just for my clarification, if we re-enable the queries in the tests in the mixed version servers mode, it goes into a stackoverflow situation. That what i saw when i set hasLuceneVersionMismatch(host) to false in the test so that it does the query.
Regards
Naba
________________________________
From: Mario Kevo <ma...@est.tech>
Sent: Tuesday, September 28, 2021 4:49 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Odg: [DISCUSS] Upgrading to Lucene 7.1.0
Hi all,
Just a small clarification of the reverted PR.
There were a lot of changes between Lucene versions 6.x and 7.x. There is an article for that Upgrading+to+Lucene+7.1.0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FGEODE%2FUpgrading%2Bto%2BLucene%2B7.1.0&data=04%7C01%7Cdasmith%40vmware.com%7C1d4830d3975e4380893508d9828a4707%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637684352682888690%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=QMiJUG1HtmOKNCXzn6KrBVM4gYdLoJeV8FjDFMkUL8I%3D&reserved=0>.
The first larger change was in the scoring mechanism. We adapt it to one that is correct for us. (verified by DistributedScoringJUnitTest)
The main change was in Lucene index format. There we come into a problem with our tests.
Lucene 6.x cannot read the index format of Lucene 7.x.
Through PRs we decided to include Lucene uplift in Geode 1.15.0 and add check if all members are on 1.15.0 version or higher (after uplift Lucene to a newer version with index format changes this should be changed). If a check is passed it will allow doing Lucene query, if not there will be a printed log that not all members on 1.15.0 or higher version.
Also, you can found a discussion on dev list from 2 years ago about Lucene upgrade: Lucene Upgrade<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmarkmail.org%2Fmessage%2Fqwooctuz7ekaezor%3Fq%3Dlist%3Aorg.apache.geode.dev%2Border%3Adate-backward%2BLucene%2Bupgrade%26page%3D4%23query%3Alist%253Aorg.apache.geode.dev%2520order%253Adate-backward%2520Lucene%2520upgrade%2Bpage%3A4%2Bmid%3Aygjhsuikdrbuihap%2Bstate%3Aresults&data=04%7C01%7Cdasmith%40vmware.com%7C1d4830d3975e4380893508d9828a4707%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637684352682898695%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ZyKGoWh6nhpWPTiNIJHzLicfjCmW0yoq1fZa9aLngbQ%3D&reserved=0>
BR,
Mario
________________________________
Šalje: Udo Kohlmeyer <ud...@vmware.com>
Poslano: 28. rujna 2021. 1:44
Prima: dev@geode.apache.org <de...@geode.apache.org>
Predmet: Re: [DISCUSS] Upgrading to Lucene 7.1.0
Might I propose something here.
There is currently a significant amount of work going into completing Geode-8705, which is the Classloader isolation. We are currently targeting to getting this release in Geode 1.16.
My proposal is, that we use the capability that Patrick demo’d at the Community meeting (on this topic) where one, at runtime, can unload / load extensions (like our integration with Lucene). This means that one could possibly do a rolling upgrade on the existing system, and keep the versions of the Lucene integration stable.
Once the whole system has been upgraded, the existing Lucene extension component is then unloaded, and the newer version of the extension component is then loaded. What this means, is that at runtime, there will be a period of time where Lucene queries will not be available and as part of the “load” lifecycle of the extension, there needs to be an initialization step, which will initialize the extension component safely.
Once initialized, Lucene queries can then become available again, etc.
This if course requires some work around the lifecycles of extension components and making sure that I can add the extension on at runtime and safely initialize it.
I think this approach allows for a more seamless (lower downtime) upgrading of system and extension components.
Thoughts?
--Udo
From: Nabarun Nag <nn...@vmware.com>
Date: Tuesday, September 28, 2021 at 7:33 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
The solution for preventing the query executions to occur in the mixed version mode also caused some problems where the query function executions get repeatedly executed and that results in stack overflow.
________________________________
From: Nabarun Nag <nn...@vmware.com>
Sent: Monday, September 27, 2021 2:30 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
In simple words, if Lucene indexes were created by a new version (7.1.0), then replicated to others that are still in the older version, they won't understand the index, and the event processors start throwing exceptions.
This can be simply seen by just re-enabling the query execution in the DUnit tests and commenting out the check blocks: [develop SHA: 68629356f561a932f5dfbace70b01d9971a42473]
In LuceneEventListener
if (cache.hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return false;
}
In IndexRepositoryFactory:
if (userRegion.getCache() != null
&& userRegion.getCache().hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return null;
}
This is the exception that will be encountered:
[Exception]
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.251 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_5> tid=102] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
Also:
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.134 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_7> tid=106] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
________________________________
From: Jacob Barrett <ja...@vmware.com>
Sent: Monday, September 27, 2021 2:08 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
> On Sep 27, 2021, at 11:48 AM, nabarun nag <nn...@apache.org> wrote:
>
> Recently, a commit was pushed to develop which upgraded the Lucene
> version used in Apache Geode to 7.1.0. These new Lucene indexes are
> not compatible with the previous versions hence it breaks the rolling
> upgrade contract. We are no longer able to execute Lucene queries when
> there are severs of mixed versions in the cluster.
Can you describe the problem with a little more detail? Does this mean that while there is a mix the execution throws an exception on all servers or is there a subset for which it works? If there is a subset for which it works, are those instances sufficient to provide accurate results if the instances that fail are ignored?
-Jake
Re: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Dan Smith <da...@vmware.com>.
My understanding from our previous discussion about upgrading lucene was that we talked about pausing the asynchronous indexing process during the rolling upgrade. I don't remember a discussion that it was ok to not allow queries during the upgrade. But this is what we added to the docs:
"All cluster members must be running the same major Lucene version in order to execute Lucene queries."
What happens if a user runs a query during the rolling upgrade and why do we need to have this restriction? It seems to me like at a minimum we need to allow queries during the upgrade.
We also should consider what will happen to users with server-side query or indexing code - will they be able to upgrade or are they likely to hit breaking changes in the Lucene API?
-Dan
________________________________
From: Nabarun Nag <nn...@vmware.com>
Sent: Tuesday, September 28, 2021 7:13 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
But Mario, just for my clarification, if we re-enable the queries in the tests in the mixed version servers mode, it goes into a stackoverflow situation. That what i saw when i set hasLuceneVersionMismatch(host) to false in the test so that it does the query.
Regards
Naba
________________________________
From: Mario Kevo <ma...@est.tech>
Sent: Tuesday, September 28, 2021 4:49 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Odg: [DISCUSS] Upgrading to Lucene 7.1.0
Hi all,
Just a small clarification of the reverted PR.
There were a lot of changes between Lucene versions 6.x and 7.x. There is an article for that Upgrading+to+Lucene+7.1.0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FGEODE%2FUpgrading%2Bto%2BLucene%2B7.1.0&data=04%7C01%7Cdasmith%40vmware.com%7C1d4830d3975e4380893508d9828a4707%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637684352682888690%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=QMiJUG1HtmOKNCXzn6KrBVM4gYdLoJeV8FjDFMkUL8I%3D&reserved=0>.
The first larger change was in the scoring mechanism. We adapt it to one that is correct for us. (verified by DistributedScoringJUnitTest)
The main change was in Lucene index format. There we come into a problem with our tests.
Lucene 6.x cannot read the index format of Lucene 7.x.
Through PRs we decided to include Lucene uplift in Geode 1.15.0 and add check if all members are on 1.15.0 version or higher (after uplift Lucene to a newer version with index format changes this should be changed). If a check is passed it will allow doing Lucene query, if not there will be a printed log that not all members on 1.15.0 or higher version.
Also, you can found a discussion on dev list from 2 years ago about Lucene upgrade: Lucene Upgrade<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmarkmail.org%2Fmessage%2Fqwooctuz7ekaezor%3Fq%3Dlist%3Aorg.apache.geode.dev%2Border%3Adate-backward%2BLucene%2Bupgrade%26page%3D4%23query%3Alist%253Aorg.apache.geode.dev%2520order%253Adate-backward%2520Lucene%2520upgrade%2Bpage%3A4%2Bmid%3Aygjhsuikdrbuihap%2Bstate%3Aresults&data=04%7C01%7Cdasmith%40vmware.com%7C1d4830d3975e4380893508d9828a4707%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637684352682898695%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ZyKGoWh6nhpWPTiNIJHzLicfjCmW0yoq1fZa9aLngbQ%3D&reserved=0>
BR,
Mario
________________________________
Šalje: Udo Kohlmeyer <ud...@vmware.com>
Poslano: 28. rujna 2021. 1:44
Prima: dev@geode.apache.org <de...@geode.apache.org>
Predmet: Re: [DISCUSS] Upgrading to Lucene 7.1.0
Might I propose something here.
There is currently a significant amount of work going into completing Geode-8705, which is the Classloader isolation. We are currently targeting to getting this release in Geode 1.16.
My proposal is, that we use the capability that Patrick demo’d at the Community meeting (on this topic) where one, at runtime, can unload / load extensions (like our integration with Lucene). This means that one could possibly do a rolling upgrade on the existing system, and keep the versions of the Lucene integration stable.
Once the whole system has been upgraded, the existing Lucene extension component is then unloaded, and the newer version of the extension component is then loaded. What this means, is that at runtime, there will be a period of time where Lucene queries will not be available and as part of the “load” lifecycle of the extension, there needs to be an initialization step, which will initialize the extension component safely.
Once initialized, Lucene queries can then become available again, etc.
This if course requires some work around the lifecycles of extension components and making sure that I can add the extension on at runtime and safely initialize it.
I think this approach allows for a more seamless (lower downtime) upgrading of system and extension components.
Thoughts?
--Udo
From: Nabarun Nag <nn...@vmware.com>
Date: Tuesday, September 28, 2021 at 7:33 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
The solution for preventing the query executions to occur in the mixed version mode also caused some problems where the query function executions get repeatedly executed and that results in stack overflow.
________________________________
From: Nabarun Nag <nn...@vmware.com>
Sent: Monday, September 27, 2021 2:30 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
In simple words, if Lucene indexes were created by a new version (7.1.0), then replicated to others that are still in the older version, they won't understand the index, and the event processors start throwing exceptions.
This can be simply seen by just re-enabling the query execution in the DUnit tests and commenting out the check blocks: [develop SHA: 68629356f561a932f5dfbace70b01d9971a42473]
In LuceneEventListener
if (cache.hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return false;
}
In IndexRepositoryFactory:
if (userRegion.getCache() != null
&& userRegion.getCache().hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return null;
}
This is the exception that will be encountered:
[Exception]
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.251 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_5> tid=102] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
Also:
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.134 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_7> tid=106] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
________________________________
From: Jacob Barrett <ja...@vmware.com>
Sent: Monday, September 27, 2021 2:08 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
> On Sep 27, 2021, at 11:48 AM, nabarun nag <nn...@apache.org> wrote:
>
> Recently, a commit was pushed to develop which upgraded the Lucene
> version used in Apache Geode to 7.1.0. These new Lucene indexes are
> not compatible with the previous versions hence it breaks the rolling
> upgrade contract. We are no longer able to execute Lucene queries when
> there are severs of mixed versions in the cluster.
Can you describe the problem with a little more detail? Does this mean that while there is a mix the execution throws an exception on all servers or is there a subset for which it works? If there is a subset for which it works, are those instances sufficient to provide accurate results if the instances that fail are ignored?
-Jake
Re: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Nabarun Nag <nn...@vmware.com>.
But Mario, just for my clarification, if we re-enable the queries in the tests in the mixed version servers mode, it goes into a stackoverflow situation. That what i saw when i set hasLuceneVersionMismatch(host) to false in the test so that it does the query.
Regards
Naba
________________________________
From: Mario Kevo <ma...@est.tech>
Sent: Tuesday, September 28, 2021 4:49 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Odg: [DISCUSS] Upgrading to Lucene 7.1.0
Hi all,
Just a small clarification of the reverted PR.
There were a lot of changes between Lucene versions 6.x and 7.x. There is an article for that Upgrading+to+Lucene+7.1.0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FGEODE%2FUpgrading%2Bto%2BLucene%2B7.1.0&data=04%7C01%7Cnnag%40vmware.com%7C1e6e4524534549d419cf08d982760f65%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637684265872534660%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jTD3ZIUk69DR77NWRMaS%2BG2ajpPa1Spgg0q1iYxc34I%3D&reserved=0>.
The first larger change was in the scoring mechanism. We adapt it to one that is correct for us. (verified by DistributedScoringJUnitTest)
The main change was in Lucene index format. There we come into a problem with our tests.
Lucene 6.x cannot read the index format of Lucene 7.x.
Through PRs we decided to include Lucene uplift in Geode 1.15.0 and add check if all members are on 1.15.0 version or higher (after uplift Lucene to a newer version with index format changes this should be changed). If a check is passed it will allow doing Lucene query, if not there will be a printed log that not all members on 1.15.0 or higher version.
Also, you can found a discussion on dev list from 2 years ago about Lucene upgrade: Lucene Upgrade<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmarkmail.org%2Fmessage%2Fqwooctuz7ekaezor%3Fq%3Dlist%3Aorg.apache.geode.dev%2Border%3Adate-backward%2BLucene%2Bupgrade%26page%3D4%23query%3Alist%253Aorg.apache.geode.dev%2520order%253Adate-backward%2520Lucene%2520upgrade%2Bpage%3A4%2Bmid%3Aygjhsuikdrbuihap%2Bstate%3Aresults&data=04%7C01%7Cnnag%40vmware.com%7C1e6e4524534549d419cf08d982760f65%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637684265872544660%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=E%2FIR9IdWMBWOrZl%2FYYnRHOOuLccmbvet6fyOgx%2FFeBk%3D&reserved=0>
BR,
Mario
________________________________
Šalje: Udo Kohlmeyer <ud...@vmware.com>
Poslano: 28. rujna 2021. 1:44
Prima: dev@geode.apache.org <de...@geode.apache.org>
Predmet: Re: [DISCUSS] Upgrading to Lucene 7.1.0
Might I propose something here.
There is currently a significant amount of work going into completing Geode-8705, which is the Classloader isolation. We are currently targeting to getting this release in Geode 1.16.
My proposal is, that we use the capability that Patrick demo’d at the Community meeting (on this topic) where one, at runtime, can unload / load extensions (like our integration with Lucene). This means that one could possibly do a rolling upgrade on the existing system, and keep the versions of the Lucene integration stable.
Once the whole system has been upgraded, the existing Lucene extension component is then unloaded, and the newer version of the extension component is then loaded. What this means, is that at runtime, there will be a period of time where Lucene queries will not be available and as part of the “load” lifecycle of the extension, there needs to be an initialization step, which will initialize the extension component safely.
Once initialized, Lucene queries can then become available again, etc.
This if course requires some work around the lifecycles of extension components and making sure that I can add the extension on at runtime and safely initialize it.
I think this approach allows for a more seamless (lower downtime) upgrading of system and extension components.
Thoughts?
--Udo
From: Nabarun Nag <nn...@vmware.com>
Date: Tuesday, September 28, 2021 at 7:33 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
The solution for preventing the query executions to occur in the mixed version mode also caused some problems where the query function executions get repeatedly executed and that results in stack overflow.
________________________________
From: Nabarun Nag <nn...@vmware.com>
Sent: Monday, September 27, 2021 2:30 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
In simple words, if Lucene indexes were created by a new version (7.1.0), then replicated to others that are still in the older version, they won't understand the index, and the event processors start throwing exceptions.
This can be simply seen by just re-enabling the query execution in the DUnit tests and commenting out the check blocks: [develop SHA: 68629356f561a932f5dfbace70b01d9971a42473]
In LuceneEventListener
if (cache.hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return false;
}
In IndexRepositoryFactory:
if (userRegion.getCache() != null
&& userRegion.getCache().hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return null;
}
This is the exception that will be encountered:
[Exception]
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.251 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_5> tid=102] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
Also:
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.134 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_7> tid=106] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
________________________________
From: Jacob Barrett <ja...@vmware.com>
Sent: Monday, September 27, 2021 2:08 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
> On Sep 27, 2021, at 11:48 AM, nabarun nag <nn...@apache.org> wrote:
>
> Recently, a commit was pushed to develop which upgraded the Lucene
> version used in Apache Geode to 7.1.0. These new Lucene indexes are
> not compatible with the previous versions hence it breaks the rolling
> upgrade contract. We are no longer able to execute Lucene queries when
> there are severs of mixed versions in the cluster.
Can you describe the problem with a little more detail? Does this mean that while there is a mix the execution throws an exception on all servers or is there a subset for which it works? If there is a subset for which it works, are those instances sufficient to provide accurate results if the instances that fail are ignored?
-Jake
Odg: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Mario Kevo <ma...@est.tech>.
Hi all,
Just a small clarification of the reverted PR.
There were a lot of changes between Lucene versions 6.x and 7.x. There is an article for that Upgrading+to+Lucene+7.1.0<https://cwiki.apache.org/confluence/display/GEODE/Upgrading+to+Lucene+7.1.0>.
The first larger change was in the scoring mechanism. We adapt it to one that is correct for us. (verified by DistributedScoringJUnitTest)
The main change was in Lucene index format. There we come into a problem with our tests.
Lucene 6.x cannot read the index format of Lucene 7.x.
Through PRs we decided to include Lucene uplift in Geode 1.15.0 and add check if all members are on 1.15.0 version or higher (after uplift Lucene to a newer version with index format changes this should be changed). If a check is passed it will allow doing Lucene query, if not there will be a printed log that not all members on 1.15.0 or higher version.
Also, you can found a discussion on dev list from 2 years ago about Lucene upgrade: Lucene Upgrade<https://markmail.org/message/qwooctuz7ekaezor?q=list:org%2Eapache%2Egeode%2Edev+order:date-backward+Lucene+upgrade&page=4#query:list%3Aorg.apache.geode.dev%20order%3Adate-backward%20Lucene%20upgrade+page:4+mid:ygjhsuikdrbuihap+state:results>
BR,
Mario
________________________________
Šalje: Udo Kohlmeyer <ud...@vmware.com>
Poslano: 28. rujna 2021. 1:44
Prima: dev@geode.apache.org <de...@geode.apache.org>
Predmet: Re: [DISCUSS] Upgrading to Lucene 7.1.0
Might I propose something here.
There is currently a significant amount of work going into completing Geode-8705, which is the Classloader isolation. We are currently targeting to getting this release in Geode 1.16.
My proposal is, that we use the capability that Patrick demo’d at the Community meeting (on this topic) where one, at runtime, can unload / load extensions (like our integration with Lucene). This means that one could possibly do a rolling upgrade on the existing system, and keep the versions of the Lucene integration stable.
Once the whole system has been upgraded, the existing Lucene extension component is then unloaded, and the newer version of the extension component is then loaded. What this means, is that at runtime, there will be a period of time where Lucene queries will not be available and as part of the “load” lifecycle of the extension, there needs to be an initialization step, which will initialize the extension component safely.
Once initialized, Lucene queries can then become available again, etc.
This if course requires some work around the lifecycles of extension components and making sure that I can add the extension on at runtime and safely initialize it.
I think this approach allows for a more seamless (lower downtime) upgrading of system and extension components.
Thoughts?
--Udo
From: Nabarun Nag <nn...@vmware.com>
Date: Tuesday, September 28, 2021 at 7:33 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
The solution for preventing the query executions to occur in the mixed version mode also caused some problems where the query function executions get repeatedly executed and that results in stack overflow.
________________________________
From: Nabarun Nag <nn...@vmware.com>
Sent: Monday, September 27, 2021 2:30 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
In simple words, if Lucene indexes were created by a new version (7.1.0), then replicated to others that are still in the older version, they won't understand the index, and the event processors start throwing exceptions.
This can be simply seen by just re-enabling the query execution in the DUnit tests and commenting out the check blocks: [develop SHA: 68629356f561a932f5dfbace70b01d9971a42473]
In LuceneEventListener
if (cache.hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return false;
}
In IndexRepositoryFactory:
if (userRegion.getCache() != null
&& userRegion.getCache().hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return null;
}
This is the exception that will be encountered:
[Exception]
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.251 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_5> tid=102] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
Also:
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.134 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_7> tid=106] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
________________________________
From: Jacob Barrett <ja...@vmware.com>
Sent: Monday, September 27, 2021 2:08 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
> On Sep 27, 2021, at 11:48 AM, nabarun nag <nn...@apache.org> wrote:
>
> Recently, a commit was pushed to develop which upgraded the Lucene
> version used in Apache Geode to 7.1.0. These new Lucene indexes are
> not compatible with the previous versions hence it breaks the rolling
> upgrade contract. We are no longer able to execute Lucene queries when
> there are severs of mixed versions in the cluster.
Can you describe the problem with a little more detail? Does this mean that while there is a mix the execution throws an exception on all servers or is there a subset for which it works? If there is a subset for which it works, are those instances sufficient to provide accurate results if the instances that fail are ignored?
-Jake
Re: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Udo Kohlmeyer <ud...@vmware.com>.
Might I propose something here.
There is currently a significant amount of work going into completing Geode-8705, which is the Classloader isolation. We are currently targeting to getting this release in Geode 1.16.
My proposal is, that we use the capability that Patrick demo’d at the Community meeting (on this topic) where one, at runtime, can unload / load extensions (like our integration with Lucene). This means that one could possibly do a rolling upgrade on the existing system, and keep the versions of the Lucene integration stable.
Once the whole system has been upgraded, the existing Lucene extension component is then unloaded, and the newer version of the extension component is then loaded. What this means, is that at runtime, there will be a period of time where Lucene queries will not be available and as part of the “load” lifecycle of the extension, there needs to be an initialization step, which will initialize the extension component safely.
Once initialized, Lucene queries can then become available again, etc.
This if course requires some work around the lifecycles of extension components and making sure that I can add the extension on at runtime and safely initialize it.
I think this approach allows for a more seamless (lower downtime) upgrading of system and extension components.
Thoughts?
--Udo
From: Nabarun Nag <nn...@vmware.com>
Date: Tuesday, September 28, 2021 at 7:33 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
The solution for preventing the query executions to occur in the mixed version mode also caused some problems where the query function executions get repeatedly executed and that results in stack overflow.
________________________________
From: Nabarun Nag <nn...@vmware.com>
Sent: Monday, September 27, 2021 2:30 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
In simple words, if Lucene indexes were created by a new version (7.1.0), then replicated to others that are still in the older version, they won't understand the index, and the event processors start throwing exceptions.
This can be simply seen by just re-enabling the query execution in the DUnit tests and commenting out the check blocks: [develop SHA: 68629356f561a932f5dfbace70b01d9971a42473]
In LuceneEventListener
if (cache.hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return false;
}
In IndexRepositoryFactory:
if (userRegion.getCache() != null
&& userRegion.getCache().hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return null;
}
This is the exception that will be encountered:
[Exception]
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.251 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_5> tid=102] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
Also:
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.134 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_7> tid=106] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
________________________________
From: Jacob Barrett <ja...@vmware.com>
Sent: Monday, September 27, 2021 2:08 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
> On Sep 27, 2021, at 11:48 AM, nabarun nag <nn...@apache.org> wrote:
>
> Recently, a commit was pushed to develop which upgraded the Lucene
> version used in Apache Geode to 7.1.0. These new Lucene indexes are
> not compatible with the previous versions hence it breaks the rolling
> upgrade contract. We are no longer able to execute Lucene queries when
> there are severs of mixed versions in the cluster.
Can you describe the problem with a little more detail? Does this mean that while there is a mix the execution throws an exception on all servers or is there a subset for which it works? If there is a subset for which it works, are those instances sufficient to provide accurate results if the instances that fail are ignored?
-Jake
Re: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Jacob Barrett <ja...@vmware.com>.
I asked this previously but didn’t see a response in the follow ups. If were were to improve upon the code so that it didn’t stack overflow when it can’t read the index version on a specific member/bucket, ignore the members/buckets that don’t return a result because they can’t read the index, could we not aggregate the results of the members/buckets that can read the index and still produce the expected results?
It seems like there should be sufficient coverage across all the nodes that if the buckets are redundant, given they are replicating, that one node should be able to read the index contained in that bucket and the other nodes can be ignored for that bucket.
-Jake
> On Sep 27, 2021, at 2:33 PM, Nabarun Nag <nn...@vmware.com> wrote:
>
> The solution for preventing the query executions to occur in the mixed version mode also caused some problems where the query function executions get repeatedly executed and that results in stack overflow.
>
>
> ________________________________
> From: Nabarun Nag <nn...@vmware.com>
> Sent: Monday, September 27, 2021 2:30 PM
> To: dev@geode.apache.org <de...@geode.apache.org>
> Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
>
> In simple words, if Lucene indexes were created by a new version (7.1.0), then replicated to others that are still in the older version, they won't understand the index, and the event processors start throwing exceptions.
>
> This can be simply seen by just re-enabling the query execution in the DUnit tests and commenting out the check blocks: [develop SHA: 68629356f561a932f5dfbace70b01d9971a42473]
>
> In LuceneEventListener
> if (cache.hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
> logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
> return false;
> }
>
> In IndexRepositoryFactory:
> if (userRegion.getCache() != null
> && userRegion.getCache().hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
> logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
> return null;
> }
>
>
> This is the exception that will be encountered:
>
> [Exception]
>
> [vm2_v1.2.0] [warn 2021/09/27 14:24:42.251 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_5> tid=102] An Exception occurred. The dispatcher will continue.
> [vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
> [vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
> [vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
> [vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
> [vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
> [vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
> [vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
> [vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
> [vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
> [vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
> [vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
> [vm2_v1.2.0] ... 10 more
>
>
>
> Also:
> [vm2_v1.2.0] [warn 2021/09/27 14:24:42.134 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_7> tid=106] An Exception occurred. The dispatcher will continue.
> [vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
> [vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
> [vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
> [vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
> [vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
> [vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
> [vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
> [vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
> [vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
> [vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
> [vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
> [vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
> [vm2_v1.2.0] ... 10 more
>
>
> ________________________________
> From: Jacob Barrett <ja...@vmware.com>
> Sent: Monday, September 27, 2021 2:08 PM
> To: dev@geode.apache.org <de...@geode.apache.org>
> Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
>
>
>
>> On Sep 27, 2021, at 11:48 AM, nabarun nag <nn...@apache.org> wrote:
>>
>> Recently, a commit was pushed to develop which upgraded the Lucene
>> version used in Apache Geode to 7.1.0. These new Lucene indexes are
>> not compatible with the previous versions hence it breaks the rolling
>> upgrade contract. We are no longer able to execute Lucene queries when
>> there are severs of mixed versions in the cluster.
>
>
> Can you describe the problem with a little more detail? Does this mean that while there is a mix the execution throws an exception on all servers or is there a subset for which it works? If there is a subset for which it works, are those instances sufficient to provide accurate results if the instances that fail are ignored?
>
> -Jake
>
Re: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Nabarun Nag <nn...@vmware.com>.
The solution for preventing the query executions to occur in the mixed version mode also caused some problems where the query function executions get repeatedly executed and that results in stack overflow.
________________________________
From: Nabarun Nag <nn...@vmware.com>
Sent: Monday, September 27, 2021 2:30 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
In simple words, if Lucene indexes were created by a new version (7.1.0), then replicated to others that are still in the older version, they won't understand the index, and the event processors start throwing exceptions.
This can be simply seen by just re-enabling the query execution in the DUnit tests and commenting out the check blocks: [develop SHA: 68629356f561a932f5dfbace70b01d9971a42473]
In LuceneEventListener
if (cache.hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return false;
}
In IndexRepositoryFactory:
if (userRegion.getCache() != null
&& userRegion.getCache().hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return null;
}
This is the exception that will be encountered:
[Exception]
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.251 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_5> tid=102] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
Also:
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.134 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_7> tid=106] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
________________________________
From: Jacob Barrett <ja...@vmware.com>
Sent: Monday, September 27, 2021 2:08 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
> On Sep 27, 2021, at 11:48 AM, nabarun nag <nn...@apache.org> wrote:
>
> Recently, a commit was pushed to develop which upgraded the Lucene
> version used in Apache Geode to 7.1.0. These new Lucene indexes are
> not compatible with the previous versions hence it breaks the rolling
> upgrade contract. We are no longer able to execute Lucene queries when
> there are severs of mixed versions in the cluster.
Can you describe the problem with a little more detail? Does this mean that while there is a mix the execution throws an exception on all servers or is there a subset for which it works? If there is a subset for which it works, are those instances sufficient to provide accurate results if the instances that fail are ignored?
-Jake
Re: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Nabarun Nag <nn...@vmware.com>.
In simple words, if Lucene indexes were created by a new version (7.1.0), then replicated to others that are still in the older version, they won't understand the index, and the event processors start throwing exceptions.
This can be simply seen by just re-enabling the query execution in the DUnit tests and commenting out the check blocks: [develop SHA: 68629356f561a932f5dfbace70b01d9971a42473]
In LuceneEventListener
if (cache.hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return false;
}
In IndexRepositoryFactory:
if (userRegion.getCache() != null
&& userRegion.getCache().hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " + KnownVersion.GEODE_1_15_0.getName());
return null;
}
This is the exception that will be encountered:
[Exception]
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.251 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_5> tid=102] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
Also:
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.134 PDT <Event Processor for GatewaySender_AsyncEventQueue_index#_aRegion_7> tid=106] An Exception occurred. The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index repository
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
________________________________
From: Jacob Barrett <ja...@vmware.com>
Sent: Monday, September 27, 2021 2:08 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
> On Sep 27, 2021, at 11:48 AM, nabarun nag <nn...@apache.org> wrote:
>
> Recently, a commit was pushed to develop which upgraded the Lucene
> version used in Apache Geode to 7.1.0. These new Lucene indexes are
> not compatible with the previous versions hence it breaks the rolling
> upgrade contract. We are no longer able to execute Lucene queries when
> there are severs of mixed versions in the cluster.
Can you describe the problem with a little more detail? Does this mean that while there is a mix the execution throws an exception on all servers or is there a subset for which it works? If there is a subset for which it works, are those instances sufficient to provide accurate results if the instances that fail are ignored?
-Jake
Re: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Jacob Barrett <ja...@vmware.com>.
> On Sep 27, 2021, at 11:48 AM, nabarun nag <nn...@apache.org> wrote:
>
> Recently, a commit was pushed to develop which upgraded the Lucene
> version used in Apache Geode to 7.1.0. These new Lucene indexes are
> not compatible with the previous versions hence it breaks the rolling
> upgrade contract. We are no longer able to execute Lucene queries when
> there are severs of mixed versions in the cluster.
Can you describe the problem with a little more detail? Does this mean that while there is a mix the execution throws an exception on all servers or is there a subset for which it works? If there is a subset for which it works, are those instances sufficient to provide accurate results if the instances that fail are ignored?
-Jake
Re: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Dan Smith <da...@vmware.com>.
Does anyone have more context on why lucene queries won't work during the rolling upgrade? I can see what added a line to the documentation and changed the tests not to do queries, but I'm not sure why we needed to do that.
-Dan
________________________________
From: nabarun nag <nn...@apache.org>
Sent: Monday, September 27, 2021 11:48 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: [DISCUSS] Upgrading to Lucene 7.1.0
Hi dev team,
Recently, a commit was pushed to develop which upgraded the Lucene
version used in Apache Geode to 7.1.0. These new Lucene indexes are
not compatible with the previous versions hence it breaks the rolling
upgrade contract. We are no longer able to execute Lucene queries when
there are severs of mixed versions in the cluster. One solution that
was provided was to not allow Lucene queries to be executed when there
are mixed versions of servers present in the cluster.
After a discussion, it was put forward that this was not an optimal
user experience. Also, that this change of default behavior was not
discussed in the dev list.
Alternative solutions that were put forward were that Geode allowed
the user to pick the version of Lucene to be used. Another that since
this is a major upgrade to Lucene with some breaking API changes,
should we sync this upgrade with the release of Apache Geode 2.0
We would love to hear your thoughts or alternative solutions to this issue.
Regards
Nabarun.
Re: [DISCUSS] Upgrading to Lucene 7.1.0
Posted by Jens Deppe <jd...@vmware.com>.
Would it be possible to allow an operator to choose which version of Lucene to use? Such that if they are prepared for the issues you describe, they could still go ahead and upgrade to 7.10. Or are there breaking API changes which would make that hard/impossible to accommodate in our code base?
--Jens
From: nabarun nag <nn...@apache.org>
Date: Monday, September 27, 2021 at 11:49 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: [DISCUSS] Upgrading to Lucene 7.1.0
Hi dev team,
Recently, a commit was pushed to develop which upgraded the Lucene
version used in Apache Geode to 7.1.0. These new Lucene indexes are
not compatible with the previous versions hence it breaks the rolling
upgrade contract. We are no longer able to execute Lucene queries when
there are severs of mixed versions in the cluster. One solution that
was provided was to not allow Lucene queries to be executed when there
are mixed versions of servers present in the cluster.
After a discussion, it was put forward that this was not an optimal
user experience. Also, that this change of default behavior was not
discussed in the dev list.
Alternative solutions that were put forward were that Geode allowed
the user to pick the version of Lucene to be used. Another that since
this is a major upgrade to Lucene with some breaking API changes,
should we sync this upgrade with the release of Apache Geode 2.0
We would love to hear your thoughts or alternative solutions to this issue.
Regards
Nabarun.