You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by "Barry Oglesby (JIRA)" <ji...@apache.org> on 2017/05/31 19:02:04 UTC
[jira] [Comment Edited] (GEODE-2979) Adding server after defining
Lucene index results in unusable cluster
[ https://issues.apache.org/jira/browse/GEODE-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030461#comment-16030461 ]
Barry Oglesby edited comment on GEODE-2979 at 5/31/17 7:01 PM:
---------------------------------------------------------------
Creating an index currently just creates an entry in each existing member's LuceneServiceImpl definedIndexMap. This is an in-memory map. There is no entry in cluster config. The lucene index element has to be created in the context of a region element. Currently, there isn't any mechanism for creating an unattached element which is basically what an index without a region is.
If the lucene indexes are listed, the existing members show the Status as Defined. That means the region hasn't been created yet.
{noformat}
gfsh>list lucene indexes
Index Name | Region Path | Server Name | Indexed Fields | Field Analyzer | Status
---------- | ----------- | ----------- | ---------------------- | --------------------------------------- | -------
testIndex | /testRegion | server1 | [__REGION_VALUE_FIELD] | {__REGION_VALUE_FIELD=StandardAnalyzer} | Defined
{noformat}
In this case, if another member is started, it will not get the index definition since it is not saved in cluster config nor is there a message between the members exchanging defined indexes.
If the region is created in this scenario, then only one of the servers will successfully create it.
{noformat}
gfsh>create region --name=testRegion --type=PARTITION_REDUNDANT_PERSISTENT
Member | Status
------- | --------------------------------------------------------------------------------------------------------
server2 | ERROR: Must create Lucene index testIndex on region /testRegion because it is defined in another member.
server1 | Region "/testRegion" created on "server1"
{noformat}
When the region is created, the defined indexes for it are then created and an element like this is added to cluster config:
{noformat}
<region name="testRegion">
<region-attributes async-event-queue-ids="testIndex#_testRegion" data-policy="persistent-partition">
<partition-attributes redundant-copies="1"/>
</region-attributes>
<lucene:index xmlns:lucene="http://geode.apache.org/schema/lucene" name="testIndex">
<lucene:field analyzer="org.apache.lucene.analysis.standard.StandardAnalyzer" name="__REGION_VALUE_FIELD"/>
</lucene:index>
</region>
{noformat}
Any member started after the region element is in cluster config will get the region and index. The Initialized Status means that the region has been created.
{noformat}
gfsh>list lucene indexes
Index Name | Region Path | Server Name | Indexed Fields | Field Analyzer | Status
---------- | ----------- | ----------- | ---------------------- | --------------------------------------- | -----------
testIndex | /testRegion | server1 | [__REGION_VALUE_FIELD] | {__REGION_VALUE_FIELD=StandardAnalyzer} | Initialized
testIndex | /testRegion | server2 | [__REGION_VALUE_FIELD] | {__REGION_VALUE_FIELD=StandardAnalyzer} | Initialized
{noformat}
So, this sequence fails:
- start server
- create index
- start other server
- create region
This sequence succeeds:
- start server
- create index
- create region
- start other server
In order to make the first scenario successful, we would have to either:
- persist the index definition in the cluster config so that when other members start, they get all the defined indexes and create them when the region is created
- pass LuceneService defined indexes between members at startup. Maybe the CacheServiceProfile should be exchanged in its own message instead of as part of the CacheProfile (which is for a region) as it is now.
was (Author: barry.oglesby):
Creating an index currently just creates an entry in each existing member's LuceneServiceImpl definedIndexMap. This is an in-memory map. There is no entry in cluster config. The lucene index element has to be created in the context of a region element. Currently, there isn't any mechanism for creating an unattached element which is basically what an index without a region is.
If the lucene indexes are listed, the existing members show the Status as Defined. That means the region hasn't been created yet.
{noformat}
gfsh>list lucene indexes
Index Name | Region Path | Server Name | Indexed Fields | Field Analyzer | Status
---------- | ----------- | ----------- | ---------------------- | --------------------------------------- | -------
testIndex | /testRegion | server1 | [__REGION_VALUE_FIELD] | {__REGION_VALUE_FIELD=StandardAnalyzer} | Defined
{noformat}
In this case, if another member is started, it will not get the index definition since it is not saved in cluster config nor is there a message between the members exchanging defined indexes.
If the region is created in this scenario, then only one of the servers will successfully create it.
{noformat}
gfsh>create region --name=testRegion --type=PARTITION_REDUNDANT_PERSISTENT
Member | Status
------- | --------------------------------------------------------------------------------------------------------
server2 | ERROR: Must create Lucene index testIndex on region /testRegion because it is defined in another member.
server1 | Region "/testRegion" created on "server1"
{noformat}
When the region is created, the defined indexes for it are then created and an element like this is added to cluster config:
{noformat}
<region name="testRegion">
<region-attributes async-event-queue-ids="testIndex#_testRegion" data-policy="persistent-partition">
<partition-attributes redundant-copies="1"/>
</region-attributes>
<lucene:index xmlns:lucene="http://geode.apache.org/schema/lucene" name="testIndex">
<lucene:field analyzer="org.apache.lucene.analysis.standard.StandardAnalyzer" name="__REGION_VALUE_FIELD"/>
</lucene:index>
</region>
{noformat}
Any member started after the region element is in cluster config will get the region and index. The Initialized Status means that the region has been created.
{noformat}
gfsh>list lucene indexes
Index Name | Region Path | Server Name | Indexed Fields | Field Analyzer | Status
---------- | ----------- | ----------- | ---------------------- | --------------------------------------- | -----------
testIndex | /testRegion | server1 | [__REGION_VALUE_FIELD] | {__REGION_VALUE_FIELD=StandardAnalyzer} | Initialized
testIndex | /testRegion | server2 | [__REGION_VALUE_FIELD] | {__REGION_VALUE_FIELD=StandardAnalyzer} | Initialized
{noformat}
So, this sequence fails:
- start server
- create index
- start other server
- create region
This sequence succeeds:
- start server
- create index
- create region
- start other server
In order to make the first scenario successful, we would have to either:
- persist the index definition in the cluster config so that when other members start, they get all the defined indexes and create them when the region is created
- pass LuceneService defined indexes between members at startup (maybe the StartupMessage can be used for this?)
> Adding server after defining Lucene index results in unusable cluster
> ---------------------------------------------------------------------
>
> Key: GEODE-2979
> URL: https://issues.apache.org/jira/browse/GEODE-2979
> Project: Geode
> Issue Type: Bug
> Components: lucene
> Reporter: Diane Hardman
> Fix For: 1.2.0
>
>
> Here are the gfsh commands I used:
> {noformat}
> ## start locator
> start locator --name=locator1 --port=12345
> ## start first server
> start server --name=server50505 --server-port=50505 --locators=localhost[12345] --start-rest-api --http-service-port=8080 --http-service-bind-address=localhost
> ## create lucene index on region testRegion
> create lucene index --name=testIndex --region=testRegion --field=__REGION_VALUE_FIELD
> ## start second server
> start server --name=server50506 --server-port=50506 --locators=localhost[12345] --start-rest-api --http-service-port=8080 --http-service-bind-address=localhost
> ## list indexes - NOTE lucene index only listed on first server
> gfsh>list members
> Name | Id
> ----------- | -------------------------------------------------
> locator1 | 192.168.1.57(locator1:60525:locator)<ec><v0>:1024
> server50505 | 192.168.1.57(server50505:60533)<v1>:1025
> server50506 | 192.168.1.57(server50506:60587)<v2>:1026
> gfsh>list lucene indexes --with-stats
> Index Name | Region Path | Server Name | Inde.. | Field Anal.. | Status | Query Executions | Updates | Commits | Documents
> ---------- | ----------- | ----------- | ------ | ------------ | ------- | ---------------- | ------- | ------- | ---------
> testIndex | /testRegion | server50505 | [__R.. | {__REGION_.. | Defined | NA | NA | NA | NA
> ## Create region testRegion
> gfsh>create region --name=testRegion --type=PARTITION_REDUNDANT_PERSISTENT
> Member | Status
> ----------- | --------------------------------------------------------------------------------------------------------
> server50506 | ERROR: Must create Lucene index testIndex on region /testRegion because it is defined in another member.
> server50505 | Region "/testRegion" created on "server50505"
> ## Add data to region - NOTE this causes a crash with an NPE
> gfsh>put --key=1 --value=value1 --region=testRegion
> Exception in thread "Gfsh Launcher" java.lang.NoClassDefFoundError: org/apache/commons/collections/CollectionUtils
> at org.apache.geode.management.internal.cli.commands.DataCommands.put(DataCommands.java:895)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:216)
> at org.apache.geode.management.internal.cli.remote.RemoteExecutionStrategy.execute(RemoteExecutionStrategy.java:91)
> at org.apache.geode.management.internal.cli.remote.CommandProcessor.executeCommand(CommandProcessor.java:113)
> at org.apache.geode.management.internal.cli.remote.CommandStatementImpl.process(CommandStatementImpl.java:71)
> at org.apache.geode.management.internal.cli.remote.MemberCommandService.processCommand(MemberCommandService.java:52)
> at org.apache.geode.management.internal.beans.MemberMBeanBridge.processCommand(MemberMBeanBridge.java:1597)
> at org.apache.geode.management.internal.beans.MemberMBean.processCommand(MemberMBean.java:404)
> at org.apache.geode.management.internal.beans.MemberMBean.processCommand(MemberMBean.java:397)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
> at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:193)
> at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:175)
> at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:117)
> at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:54)
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:324)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: org.apache.commons.collections.CollectionUtils
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 53 more
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.geode.management.internal.cli.Launcher.parseOptions(Launcher.java:249)
> at org.apache.geode.management.internal.cli.Launcher.parseCommandLine(Launcher.java:257)
> at org.apache.geode.management.internal.cli.Launcher.main(Launcher.java:135)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)