You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Varun Thacker (JIRA)" <ji...@apache.org> on 2016/06/27 13:40:52 UTC
[jira] [Updated] (SOLR-9088) solr.schema.TestManagedSchemaAPI.test
failures ([doc=2] unknown field 'myNewField1')
[ https://issues.apache.org/jira/browse/SOLR-9088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Varun Thacker updated SOLR-9088:
--------------------------------
Attachment: SOLR-9088.patch
Sample log excerpt from a failure
{code}
[junit4] 2> 1995579 INFO (zkCallback-22745-thread-2-processing-n:127.0.0.1:39653_solr) [n:127.0.0.1:39653_solr ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
[junit4] 2> 1995580 INFO (zkCallback-22746-thread-3-processing-n:127.0.0.1:38103_solr) [n:127.0.0.1:38103_solr ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
[junit4] 2> 1995580 INFO (qtp694202271-87996) [n:127.0.0.1:38103_solr c:testschemaapi s:shard1 r:core_node1 x:testschemaapi_shard1_replica1] o.a.s.s.ManagedIndexSchema Waiting up to 599 secs for 1 replicas to apply schema update version 1 for collection testschemaapi
[junit4] 2> 1995580 INFO (zkCallback-22745-thread-2-processing-n:127.0.0.1:39653_solr) [n:127.0.0.1:39653_solr ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 2 from ZooKeeper
[junit4] 2> 1995582 INFO (qtp178914546-87997) [n:127.0.0.1:39653_solr c:testschemaapi s:shard1 r:core_node2 x:testschemaapi_shard1_replica2] o.a.s.c.S.Request [testschemaapi_shard1_replica2] webapp=/solr path=/schema/zkversion params={refreshIfBelowVersion=1&version=2&wt=javabin} status=0 QTime=0
[junit4] 2> 1995583 INFO (qtp694202271-87996) [n:127.0.0.1:38103_solr c:testschemaapi s:shard1 r:core_node1 x:testschemaapi_shard1_replica1] o.a.s.s.ManagedIndexSchema Took 2.0ms for 1 replicas to apply schema update version 1 for collection testschemaapi
[junit4] 2> 1995583 INFO (qtp694202271-87996) [n:127.0.0.1:38103_solr c:testschemaapi s:shard1 r:core_node1 x:testschemaapi_shard1_replica1] o.a.s.c.S.Request [testschemaapi_shard1_replica1] webapp=/solr path=/schema params={version=2&wt=javabin} status=0 QTime=16
[junit4] 2> 1995583 INFO (zkCallback-22745-thread-2-processing-n:127.0.0.1:39653_solr) [n:127.0.0.1:39653_solr ] o.a.s.s.IndexSchema [testschemaapi_shard1_replica2] Schema name=minimal
[junit4] 2> 1995622 INFO (TEST-TestManagedSchemaAPI.test-seed#[79C11866C0E16F74]) [ ] o.a.s.s.TestManagedSchemaAPI added new field=myNewField1
[junit4] 2> 1995622 INFO (zkCallback-22746-thread-3-processing-n:127.0.0.1:38103_solr) [n:127.0.0.1:38103_solr ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 2 from ZooKeeper
[junit4] 2> 1995622 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.ZkController Running listeners for /configs/conf1
[junit4] 2> 1995622 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.SolrCore config update listener called for core testschemaapi_shard1_replica1
[junit4] 2> 1995623 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.SolrConfig current version of requestparams : -1
[junit4] 2> 1995624 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.SolrCore /configs/conf1/managed-schema is stale will need an update from 1 to 2
[junit4] 2> 1995624 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.SolrCore core reload testschemaapi_shard1_replica1
[junit4] 2> 1995624 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.ZkController Check for collection zkNode:testschemaapi
[junit4] 2> 1995624 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.ZkController Collection zkNode exists
[junit4] 2> 1995624 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/testschemaapi]
[junit4] 2> 1995624 INFO (Thread-5663) [n:127.0.0.1:39653_solr ] o.a.s.c.ZkController Running listeners for /configs/conf1
[junit4] 2> 1995624 INFO (Thread-5663) [n:127.0.0.1:39653_solr ] o.a.s.c.SolrCore config update listener called for core testschemaapi_shard1_replica2
[junit4] 2> 1995625 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.c.ZkStateReader path=[/collections/testschemaapi] [configName]=[conf1] specified config exists in ZooKeeper
[junit4] 2> 1995625 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.5-Linux/solr/build/solr-core/test/J2/temp/solr.schema.TestManagedSchemaAPI_79C11866C0E16F74-001/tempDir-001/node1/testschemaapi_shard1_replica1'
[junit4] 2> 1995626 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1995626 INFO (Thread-5662) [n:127.0.0.1:38103_solr ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 1995626 INFO (qtp178914546-87995) [n:127.0.0.1:39653_solr c:testschemaapi s:shard1 r:core_node2 x:testschemaapi_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [testschemaapi_shard1_replica2] webapp=/solr path=/update params={version=2&wt=javabin}{} 0 0
[junit4] 2> 1995626 ERROR (qtp178914546-87995) [n:127.0.0.1:39653_solr c:testschemaapi s:shard1 r:core_node2 x:testschemaapi_shard1_replica2] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: ERROR: [doc=2] unknown field 'myNewField1'
[junit4] 2> at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:182)
[junit4] 2> at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:82)
...
{code}
Here's my analysis on the current implementation and my attempted fix
- The collection has 1 shard and two replicas ( replica1 on JVM 1 and replica2 on JVM 2)
- A new add field request comes in to JVM1 for the collection.
- {{SchemaManager.java}} adds the field and persists the schema in ZK.
- Since ZK has a new schema and both replicas are watching for changes , a core reload is triggered.
- {{SchemaManager.java}} had ensured that on the local core ( replica1 ) has updated the schema before the reload by doing {{core.setLatestSchema(managedIndexSchema);}}
- If however a new document with the new field comes in against replica2 , it still has the old schema ( the reload will fetch the new schema ) . Since the reload hasn't completed we get the unknown field error
So all this patch does is makes sure to set the latest schema on a core before the reload starts.
[~noble.paul] Do you think this solution is good enough or are there still races that we need to take care of?
> solr.schema.TestManagedSchemaAPI.test failures ([doc=2] unknown field 'myNewField1')
> ------------------------------------------------------------------------------------
>
> Key: SOLR-9088
> URL: https://issues.apache.org/jira/browse/SOLR-9088
> Project: Solr
> Issue Type: Test
> Reporter: Christine Poerschke
> Priority: Minor
> Attachments: SOLR-9088.patch
>
>
> e.g.
> http://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3256/
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/588/
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.5-Linux/266/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org