You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by bbende <gi...@git.apache.org> on 2018/10/31 20:24:23 UTC
[GitHub] nifi pull request #3119: NIFI-5673 Support auto loading of new NARs
GitHub user bbende opened a pull request:
https://github.com/apache/nifi/pull/3119
NIFI-5673 Support auto loading of new NARs
This PR introduces a new property in nifi.properties where you can specify a directory that will be watched for new NARs (defaults ./extensions), and upon finding them will make any extensions in those NARs available to the running application without requiring a restart.
In addition, this PR includes a refactoring of the ExtensionManager to remove it's static use and make it an instance that is injected everywhere. This will help make the framework code more testable going forward.
NOTE: Currently the UI retrieves the available types during initial load of the UI and never retrieves them again because they couldn't change, so this will have to be changed in a follow on PR.
Some scenarios to consider when testing...
- Dependencies across NARs - handled by keeping track of skipped NARs until the a new NAR is available matching the required dependency
- NARs still being written - handled by only consider the NAR once the last modified time is at least 5 seconds old
- Custom UIs - make sure the custom UI is available for a component that was auto-loaded
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bbende/nifi NIFI-5673
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/3119.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3119
----
commit d7561ffa5601e4afd708fdbf8198f696bf3b2bbd
Author: Bryan Bende <bb...@...>
Date: 2018-10-12T15:15:30Z
NIFI-5673 Set up property/assembly for new auto-load directory
- Set up NarAutoLoader to watch directory for new files
- Move NarAutoLoader to JettyServer since it will need access to ExtensionManager
- Created NarLoader to shared between NarAutoLoader and the framework
- Created nifi-framework-nar-loading-utils so we can use nifi-documentation to call DocGenerator
- Add additional bundles to overall map in NarClassLoaders as they are loaded
- Added handling of skipped NARs to include them in next iteration
- Added check of last modified timestamp on NARs
- Refactored JettyServer so we can load additional web contexts while the application is running
- Setting up unit tests
commit 81ed267f68c69566851d75b4a99b15e1a05f503c
Author: Bryan Bende <bb...@...>
Date: 2018-10-29T20:19:57Z
NIFI-5673 Remove static use of ExtensionManager
commit e126bfca87c79272836c7c281d6d80f09d9721e4
Author: Bryan Bende <bb...@...>
Date: 2018-10-30T19:51:37Z
NIFI-5673 Adding unit tests for NarLoader
commit 01a7feb9bdd02ef3407a3b3acd5f05c0938e2e49
Author: Bryan Bende <bb...@...>
Date: 2018-10-31T15:36:49Z
NIFI-5673 Extracting interface for ExtensionManager and splitting discovery into it's own interface
----
---
[GitHub] nifi pull request #3119: NIFI-5673 Support auto loading of new NARs
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3119#discussion_r230120699
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-loading-utils/src/main/java/org/apache/nifi/nar/NarAutoLoader.java ---
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.nar;
+
+import org.apache.nifi.util.FileUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileSystems;
+import java.nio.file.Path;
+import java.nio.file.StandardWatchEventKinds;
+import java.nio.file.WatchService;
+import java.util.Objects;
+
+/**
+ * Starts a thread to monitor the auto-load directory for new NARs.
+ */
+public class NarAutoLoader {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(NarAutoLoader.class);
+
+ private static final long POLL_INTERVAL_MS = 5000;
+
+ private final File autoLoadDir;
+ private final NarLoader narLoader;
+
+ private volatile NarAutoLoaderTask narAutoLoaderTask;
+ private volatile boolean started = false;
+
+ public NarAutoLoader(final File autoLoadDir, final NarLoader narLoader) {
+ this.autoLoadDir = autoLoadDir;
--- End diff --
This should probably just be written as `this.autoLoadDir = Objects.requireNonNull(autoLoadDir);` and `this.narLoader = Objects.requireNonNull(narLoader);`
---
[GitHub] nifi pull request #3119: NIFI-5673 Support auto loading of new NARs
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3119#discussion_r230120979
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-loading-utils/src/main/java/org/apache/nifi/nar/NarAutoLoader.java ---
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.nar;
+
+import org.apache.nifi.util.FileUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileSystems;
+import java.nio.file.Path;
+import java.nio.file.StandardWatchEventKinds;
+import java.nio.file.WatchService;
+import java.util.Objects;
+
+/**
+ * Starts a thread to monitor the auto-load directory for new NARs.
+ */
+public class NarAutoLoader {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(NarAutoLoader.class);
+
+ private static final long POLL_INTERVAL_MS = 5000;
+
+ private final File autoLoadDir;
+ private final NarLoader narLoader;
+
+ private volatile NarAutoLoaderTask narAutoLoaderTask;
+ private volatile boolean started = false;
+
+ public NarAutoLoader(final File autoLoadDir, final NarLoader narLoader) {
+ this.autoLoadDir = autoLoadDir;
+ this.narLoader = narLoader;
+ Objects.requireNonNull(this.autoLoadDir);
+ Objects.requireNonNull(this.narLoader);
+ }
+
+ public synchronized void start() throws IOException {
+ if (started) {
+ return;
+ }
+
+ FileUtils.ensureDirectoryExistAndCanReadAndWrite(autoLoadDir);
+
+ final WatchService watcher = FileSystems.getDefault().newWatchService();
+
+ final Path autoLoadPath = autoLoadDir.toPath();
+ autoLoadPath.register(watcher, StandardWatchEventKinds.ENTRY_CREATE);
+
+ narAutoLoaderTask = new NarAutoLoaderTask.Builder()
+ .autoLoadPath(autoLoadPath)
+ .watchService(watcher)
+ .pollIntervalMillis(POLL_INTERVAL_MS)
+ .narLoader(narLoader)
+ .build();
+
+ LOGGER.info("Starting NAR Auto-Loader for directory {} ...", new Object[]{autoLoadPath});
+
+ final Thread thread = new Thread(narAutoLoaderTask);
+ thread.setName("NAR Auto-Loader");
+ thread.setDaemon(true);
+ thread.start();
+
+ LOGGER.info("NAR Auto-Loader started");
+ started = true;
+ }
+
+ public synchronized void stop() {
+ started = false;
+ narAutoLoaderTask.stop();;
--- End diff --
Extra semi-colon at the end :)
---
[GitHub] nifi issue #3119: NIFI-5673 Support auto loading of new NARs
Posted by bbende <gi...@git.apache.org>.
Github user bbende commented on the issue:
https://github.com/apache/nifi/pull/3119
@markap14 additional commit pushed that addresses your initial comments
---
[GitHub] nifi pull request #3119: NIFI-5673 Support auto loading of new NARs
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3119#discussion_r230116442
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-loading-utils/src/main/java/org/apache/nifi/nar/NarLoader.java ---
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.nar;
+
+import org.apache.nifi.bundle.Bundle;
+import org.apache.nifi.bundle.BundleCoordinate;
+import org.apache.nifi.bundle.BundleDetails;
+import org.apache.nifi.documentation.DocGenerator;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Set;
+import java.util.jar.Attributes;
+import java.util.jar.JarFile;
+import java.util.jar.Manifest;
+
+/**
+ * Loads a set of NARs from the file system into the running application.
+ *
+ * NOTE: Initially this will only be used from the NarAutoLoader which is watching a directory for new files, but eventually
+ * this may also be used for loading a NAR that was downloaded from the extension registry, and thus the load method
+ * is synchronized to ensure only one set of NARs can be in process of loading at a given time.
+ */
+public class NarLoader {
--- End diff --
I feel like this should probably be represented as an interface
---
[GitHub] nifi pull request #3119: NIFI-5673 Support auto loading of new NARs
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3119#discussion_r230121837
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-loading-utils/src/main/java/org/apache/nifi/nar/NarAutoLoaderTask.java ---
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.nar;
+
+import org.apache.nifi.bundle.BundleDetails;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.nio.file.Path;
+import java.nio.file.StandardWatchEventKinds;
+import java.nio.file.WatchEvent;
+import java.nio.file.WatchKey;
+import java.nio.file.WatchService;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * The runnable task that polls the WatchService for new NAR files found in the auto-load directory.
+ *
+ * Each new NAR file found will be passed to the NarLoader to be unpacked and loaded into the ExtensionManager.
+ *
+ */
+public class NarAutoLoaderTask implements Runnable {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(NarAutoLoaderTask.class);
+
+ private static final long MIN_FILE_AGE = 5000;
+
+ private final Path autoLoadPath;
+ private final WatchService watchService;
+ private final long pollIntervalMillis;
+ private final NarLoader narLoader;
+ private final List<File> candidateNars;
+
+ private NarLoadResult prevLoadResult;
+ private volatile boolean stopped = false;
+
+ private NarAutoLoaderTask(final Builder builder) {
+ this.autoLoadPath = builder.autoLoadPath;
+ this.watchService = builder.watchService;
+ this.pollIntervalMillis = builder.pollIntervalMillis;
+ this.narLoader = builder.narLoader;
+ this.candidateNars = new ArrayList<>();
+ this.prevLoadResult = null;
+ }
+
+ @Override
+ public void run() {
+ while (!stopped) {
+ WatchKey key;
--- End diff --
If any unexpected Exception gets thrown from the task, the Thread will silently die. Should probably catch `Exception` within the `while` and continue running. Or at least log an ERROR with a Stack Trace.
---
[GitHub] nifi issue #3119: NIFI-5673 Support auto loading of new NARs
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/3119
Thanks @bbende all is looking good. Was able to test and everything works as expected. +1 merged to master.
---
[GitHub] nifi pull request #3119: NIFI-5673 Support auto loading of new NARs
Posted by bbende <gi...@git.apache.org>.
Github user bbende commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3119#discussion_r230134587
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-loading-utils/src/main/java/org/apache/nifi/nar/NarAutoLoaderTask.java ---
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.nar;
+
+import org.apache.nifi.bundle.BundleDetails;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.nio.file.Path;
+import java.nio.file.StandardWatchEventKinds;
+import java.nio.file.WatchEvent;
+import java.nio.file.WatchKey;
+import java.nio.file.WatchService;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * The runnable task that polls the WatchService for new NAR files found in the auto-load directory.
+ *
+ * Each new NAR file found will be passed to the NarLoader to be unpacked and loaded into the ExtensionManager.
+ *
+ */
+public class NarAutoLoaderTask implements Runnable {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(NarAutoLoaderTask.class);
+
+ private static final long MIN_FILE_AGE = 5000;
+
+ private final Path autoLoadPath;
+ private final WatchService watchService;
+ private final long pollIntervalMillis;
+ private final NarLoader narLoader;
+ private final List<File> candidateNars;
+
+ private NarLoadResult prevLoadResult;
+ private volatile boolean stopped = false;
+
+ private NarAutoLoaderTask(final Builder builder) {
+ this.autoLoadPath = builder.autoLoadPath;
+ this.watchService = builder.watchService;
+ this.pollIntervalMillis = builder.pollIntervalMillis;
+ this.narLoader = builder.narLoader;
+ this.candidateNars = new ArrayList<>();
+ this.prevLoadResult = null;
+ }
+
+ @Override
+ public void run() {
+ while (!stopped) {
+ WatchKey key;
+ try {
+ LOGGER.debug("Polling for new NARs at {}", new Object[]{autoLoadPath});
+ key = watchService.poll(pollIntervalMillis, TimeUnit.MILLISECONDS);
+ } catch (InterruptedException x) {
+ LOGGER.info("WatchService interrupted, returning...");
+ return;
+ }
+
+ // Key comes back as null when there are no new create events, but we still want to continue processing
+ // so we can consider files added to the candidateNars list in previous iterations
+
+ if (key != null) {
+ for (WatchEvent<?> event : key.pollEvents()) {
+ final WatchEvent.Kind<?> kind = event.kind();
+ if (kind == StandardWatchEventKinds.OVERFLOW) {
--- End diff --
This was the only comment I didn't address because the WatchService is registering for only ENTRY_CREATE events, so we should be ok there.
---
[GitHub] nifi issue #3119: NIFI-5673 Support auto loading of new NARs
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/3119
Will review...
---
[GitHub] nifi pull request #3119: NIFI-5673 Support auto loading of new NARs
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/3119
---
[GitHub] nifi pull request #3119: NIFI-5673 Support auto loading of new NARs
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3119#discussion_r230117182
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-loading-utils/src/main/java/org/apache/nifi/nar/NarLoader.java ---
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.nar;
+
+import org.apache.nifi.bundle.Bundle;
+import org.apache.nifi.bundle.BundleCoordinate;
+import org.apache.nifi.bundle.BundleDetails;
+import org.apache.nifi.documentation.DocGenerator;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Set;
+import java.util.jar.Attributes;
+import java.util.jar.JarFile;
+import java.util.jar.Manifest;
+
+/**
+ * Loads a set of NARs from the file system into the running application.
+ *
+ * NOTE: Initially this will only be used from the NarAutoLoader which is watching a directory for new files, but eventually
+ * this may also be used for loading a NAR that was downloaded from the extension registry, and thus the load method
+ * is synchronized to ensure only one set of NARs can be in process of loading at a given time.
+ */
+public class NarLoader {
+
+ private static Logger LOGGER = LoggerFactory.getLogger(NarLoader.class);
+
+ private File extensionsWorkingDir;
+ private File docsWorkingDir;
+ private NarClassLoaders narClassLoaders;
+ private ExtensionDiscoveringManager extensionManager;
+ private ExtensionMapping extensionMapping;
+ private ExtensionUiLoader extensionUiLoader;
+
+ public NarLoader(final File extensionsWorkingDir,
+ final File docsWorkingDir,
+ final NarClassLoaders narClassLoaders,
+ final ExtensionDiscoveringManager extensionManager,
+ final ExtensionMapping extensionMapping,
+ final ExtensionUiLoader extensionUiLoader) {
+ this.extensionsWorkingDir = extensionsWorkingDir;
+ this.docsWorkingDir = docsWorkingDir;
+ this.narClassLoaders = narClassLoaders;
+ this.extensionManager = extensionManager;
+ this.extensionMapping = extensionMapping;
+ this.extensionUiLoader = extensionUiLoader;
+ }
+
+ public synchronized NarLoadResult load(final List<File> narFiles, final Set<BundleDetails> previouslySkippedBundles) {
--- End diff --
The `Set<BundleDetails>`, I feel, is really internal state that should be maintained by this class, rather than having another Object responsible for maintaining the previously skipped bundles and then having it tell this component which ones it has already skipped.
I would also recommend accepting a `Collection<File>` instead of `List<File>` because the ordering of the given Files does not matter, and `List` implies (or at least hints that) the order is relevant
---
[GitHub] nifi pull request #3119: NIFI-5673 Support auto loading of new NARs
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3119#discussion_r230119940
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-nar-loading-utils/src/main/java/org/apache/nifi/nar/NarAutoLoaderTask.java ---
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.nar;
+
+import org.apache.nifi.bundle.BundleDetails;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.nio.file.Path;
+import java.nio.file.StandardWatchEventKinds;
+import java.nio.file.WatchEvent;
+import java.nio.file.WatchKey;
+import java.nio.file.WatchService;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * The runnable task that polls the WatchService for new NAR files found in the auto-load directory.
+ *
+ * Each new NAR file found will be passed to the NarLoader to be unpacked and loaded into the ExtensionManager.
+ *
+ */
+public class NarAutoLoaderTask implements Runnable {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(NarAutoLoaderTask.class);
+
+ private static final long MIN_FILE_AGE = 5000;
+
+ private final Path autoLoadPath;
+ private final WatchService watchService;
+ private final long pollIntervalMillis;
+ private final NarLoader narLoader;
+ private final List<File> candidateNars;
+
+ private NarLoadResult prevLoadResult;
+ private volatile boolean stopped = false;
+
+ private NarAutoLoaderTask(final Builder builder) {
+ this.autoLoadPath = builder.autoLoadPath;
+ this.watchService = builder.watchService;
+ this.pollIntervalMillis = builder.pollIntervalMillis;
+ this.narLoader = builder.narLoader;
+ this.candidateNars = new ArrayList<>();
+ this.prevLoadResult = null;
+ }
+
+ @Override
+ public void run() {
+ while (!stopped) {
+ WatchKey key;
+ try {
+ LOGGER.debug("Polling for new NARs at {}", new Object[]{autoLoadPath});
+ key = watchService.poll(pollIntervalMillis, TimeUnit.MILLISECONDS);
+ } catch (InterruptedException x) {
+ LOGGER.info("WatchService interrupted, returning...");
+ return;
+ }
+
+ // Key comes back as null when there are no new create events, but we still want to continue processing
+ // so we can consider files added to the candidateNars list in previous iterations
+
+ if (key != null) {
+ for (WatchEvent<?> event : key.pollEvents()) {
+ final WatchEvent.Kind<?> kind = event.kind();
+ if (kind == StandardWatchEventKinds.OVERFLOW) {
--- End diff --
StandardWatchEventKinds has 4 different Kinds: OVERFLOW, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY. It looks like here we process ENTRY_CREATE, ENTRY_DELETE, and ENTRY_MODIFY the same way, trying to load the file. I think we should `continue` here for all events unless `kind == StandardWatchEventKinds.ENTRY_CREATE`, no?
---