You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-commits@jackrabbit.apache.org by al...@apache.org on 2013/06/14 10:22:54 UTC

svn commit: r1492995 - /jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexEditor.java

Author: alexparvulescu
Date: Fri Jun 14 08:22:54 2013
New Revision: 1492995

URL: http://svn.apache.org/r1492995
Log:
OAK-763 Asynchronous indexing
 - added some progress logs for the lucene index 

Modified:
    jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexEditor.java

Modified: jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexEditor.java
URL: http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexEditor.java?rev=1492995&r1=1492994&r2=1492995&view=diff
==============================================================================
--- jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexEditor.java (original)
+++ jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexEditor.java Fri Jun 14 08:22:54 2013
@@ -33,6 +33,7 @@ import static org.apache.lucene.store.No
 import java.io.File;
 import java.io.IOException;
 import java.io.InputStream;
+import java.util.concurrent.atomic.AtomicLong;
 
 import javax.jcr.PropertyType;
 
@@ -80,6 +81,8 @@ public class LuceneIndexEditor implement
 
     private static final Parser parser = new AutoDetectParser();
 
+    private AtomicLong indexedNodes;
+
     private static IndexWriterConfig getIndexWriterConfig() {
         // FIXME: Hack needed to make Lucene work in an OSGi environment
         Thread thread = Thread.currentThread();
@@ -157,6 +160,7 @@ public class LuceneIndexEditor implement
         } else {
             this.propertyTypes = -1;
         }
+        this.indexedNodes = new AtomicLong(0);
     }
 
     private LuceneIndexEditor(LuceneIndexEditor parent, String name) {
@@ -166,6 +170,7 @@ public class LuceneIndexEditor implement
         this.definition = parent.definition;
         this.writer = parent.writer;
         this.propertyTypes = parent.propertyTypes;
+        this.indexedNodes = parent.indexedNodes;
     }
 
     public String getPath() {
@@ -200,14 +205,22 @@ public class LuceneIndexEditor implement
                 throw new CommitFailedException(
                         "Lucene", 3, "Failed to index the node " + path, e);
             }
+            long indexed = indexedNodes.incrementAndGet();
+            if (indexed % 1000 == 0) {
+                log.debug("Indexed {} nodes...", indexed);
+            }
         }
 
         if (parent == null) {
             try {
                 writer.close();
             } catch (IOException e) {
-                throw new CommitFailedException(
-                        "Lucene", 4, "Failed to close the Lucene index", e);
+                throw new CommitFailedException("Lucene", 4,
+                        "Failed to close the Lucene index", e);
+            }
+            long indexed = indexedNodes.get();
+            if (indexed > 0) {
+                log.debug("Indexed {} nodes, done.", indexed);
             }
         }
     }

Re: svn commit: r1492995 - /jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexEditor.java

Posted by Alex Parvulescu <al...@gmail.com>.
As it turns out, using a normal Long doesn't work (because of primitive
unwrapping we lose the original reference).

As per Jukka's suggestions I'm looking into introducing a context class to
wrap all the baggage we pass from the 'parent' and this counter as well.


On Fri, Jun 14, 2013 at 10:41 AM, Alex Parvulescu <alex.parvulescu@gmail.com
> wrote:

> good point, I was thinking about concurrent commits, but that's not how it
> works.
> I'll change to a normal long then
>
> thanks,
> alex
>
>
> On Fri, Jun 14, 2013 at 10:28 AM, Jukka Zitting <ju...@gmail.com>wrote:
>
>> Hi,
>>
>> On Fri, Jun 14, 2013 at 11:22 AM,  <al...@apache.org> wrote:
>> > +    private AtomicLong indexedNodes;
>>
>> A normal long should be fine here, as the editor instance isn't being
>> accessed concurrently from multiple threads.
>>
>> BR,
>>
>> Jukka Zitting
>>
>
>

Re: svn commit: r1492995 - /jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexEditor.java

Posted by Alex Parvulescu <al...@gmail.com>.
good point, I was thinking about concurrent commits, but that's not how it
works.
I'll change to a normal long then

thanks,
alex


On Fri, Jun 14, 2013 at 10:28 AM, Jukka Zitting <ju...@gmail.com>wrote:

> Hi,
>
> On Fri, Jun 14, 2013 at 11:22 AM,  <al...@apache.org> wrote:
> > +    private AtomicLong indexedNodes;
>
> A normal long should be fine here, as the editor instance isn't being
> accessed concurrently from multiple threads.
>
> BR,
>
> Jukka Zitting
>

Re: svn commit: r1492995 - /jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneIndexEditor.java

Posted by Jukka Zitting <ju...@gmail.com>.
Hi,

On Fri, Jun 14, 2013 at 11:22 AM,  <al...@apache.org> wrote:
> +    private AtomicLong indexedNodes;

A normal long should be fine here, as the editor instance isn't being
accessed concurrently from multiple threads.

BR,

Jukka Zitting