You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2020/03/05 00:00:07 UTC

[GitHub] [lucene-solr] andyvuong opened a new pull request #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls

andyvuong opened a new pull request #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls
URL: https://github.com/apache/lucene-solr/pull/1317
 
 
   Shared collections make use of something called the metadataSuffix znode per shard in ZooKeeper to understand what core.metadata file to pull from blob store and hydrate a replica with the most up-to-date and correct index data. Shared collections would create the node on-demand with a default value (e.g. indexing) but this is not correct behavior when considering restore operations in the future. We'll want to create the node with a non-default value down the line. 
   
   This change initiates the node when we create the shard in common collection API commands and includes CREATE, CREATESHARD, SPLITSHARD. Other operations such as RESTORE are not currently supported by solr shared storage so are not in scope of this change. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org

[GitHub] [lucene-solr] andyvuong commented on issue #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls

Posted by GitBox <gi...@apache.org>.
andyvuong commented on issue #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls
URL: https://github.com/apache/lucene-solr/pull/1317#issuecomment-599655907
 
 
   cc @yonik 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org

[GitHub] [lucene-solr] mbwaheed commented on a change in pull request #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls

Posted by GitBox <gi...@apache.org>.
mbwaheed commented on a change in pull request #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls
URL: https://github.com/apache/lucene-solr/pull/1317#discussion_r391124376
 
 

 ##########
 File path: solr/core/src/java/org/apache/solr/store/shared/metadata/SharedShardMetadataController.java
 ##########
 @@ -53,12 +53,12 @@ public SharedShardMetadataController(SolrCloudManager cloudManager) {
   
   /**
    * Creates a new metadata node if it doesn't exist for shared shared index whose correctness metadata 
-   * is managed by ZooKeeper
+   * is managed by ZooKeeper. This node should only be created during Shard creation or Recovery events
    *  
    * @param collectionName name of the collection that needs a metadata node
    * @param shardName name of the shard that needs a metadata node
    */
-  public void ensureMetadataNodeExists(String collectionName, String shardName) throws IOException {
+  public void initiateMetadataNode(String collectionName, String shardName) throws IOException {
 
 Review comment:
   Wouldn't it be more appropriate to call it createMetadataNode and fail in case the node already exists? 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org

[GitHub] [lucene-solr] yonik merged pull request #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls

Posted by GitBox <gi...@apache.org>.
yonik merged pull request #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls
URL: https://github.com/apache/lucene-solr/pull/1317
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org

[GitHub] [lucene-solr] mbwaheed commented on a change in pull request #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls

Posted by GitBox <gi...@apache.org>.
mbwaheed commented on a change in pull request #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls
URL: https://github.com/apache/lucene-solr/pull/1317#discussion_r392545214
 
 

 ##########
 File path: solr/core/src/test/org/apache/solr/store/shared/metadata/SharedShardMetadataControllerTest.java
 ##########
 @@ -77,10 +77,26 @@ public void cleanup() throws Exception {
    */
   @Test
   public void testSetupMetadataNode() throws Exception {    
-    shardMetadataController.ensureMetadataNodeExists(TEST_COLLECTION_NAME, TEST_SHARD_NAME);
+    shardMetadataController.createMetadataNode(TEST_COLLECTION_NAME, TEST_SHARD_NAME);
     assertTrue(cluster.getZkClient().exists(metadataNodePath, false));
   }
   
+  /**
+   * Test if we try to create the same metadat anode twice, we faile
 
 Review comment:
   few typos.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org

[GitHub] [lucene-solr] andyvuong commented on a change in pull request #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls

Posted by GitBox <gi...@apache.org>.
andyvuong commented on a change in pull request #1317: SOLR-13101: Create metadataSuffix znode only at common shard creating api calls
URL: https://github.com/apache/lucene-solr/pull/1317#discussion_r392544473
 
 

 ##########
 File path: solr/core/src/java/org/apache/solr/store/shared/metadata/SharedShardMetadataController.java
 ##########
 @@ -146,20 +148,18 @@ public void cleanUpMetadataNodes(String collectionName, String shardName) {
     }
   }
 
-  private void createPersistentNodeIfNonExistent(String path, byte[] data) {
+  private void createPersistentNode(String path, byte[] data) {
     try {
       if (!stateManager.hasData(path)) {
-        try {
-          stateManager.makePath(path, data, CreateMode.PERSISTENT, /* failOnExists */ false);
-        } catch (AlreadyExistsException e) {
-          // it's okay if another beats us creating the node
-        }
+        stateManager.makePath(path, data, CreateMode.PERSISTENT, /* failOnExists */ true);
 
 Review comment:
   #makePath will also throw an AlreadyExistsException if the node exists and failOnExists=true

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org