You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by JPercivall <gi...@git.apache.org> on 2016/04/13 22:24:00 UTC
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
GitHub user JPercivall opened a pull request:
https://github.com/apache/nifi-minifi/pull/8
MINIFI-15 Created a config file format with documentation, and a Util…
… class to transform prospective config.yml into flow.xml and nifi.properties
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JPercivall/nifi-minifi MINIFI-15
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi-minifi/pull/8.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8
----
commit 345576f8d52f970cf9d74d0824879c3a90ffb0aa
Author: Joseph Percivall <jo...@yahoo.com>
Date: 2016-03-30T18:14:15Z
MINIFI-15 Created a config file format with documentation, and a Util class to transform prospective config.yml into flow.xml and nifi.properties
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59922487
--- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java ---
@@ -0,0 +1,566 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.util;
+
+
+import org.apache.nifi.controller.FlowSerializationException;
+import org.w3c.dom.DOMException;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.yaml.snakeyaml.Yaml;
+
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.TransformerFactoryConfigurationError;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.io.PrintWriter;
+import java.io.UnsupportedEncodingException;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.zip.GZIPOutputStream;
+
+public final class ConfigTransformer {
+ // Underlying version NIFI POC will be using
+ public static final String NIFI_VERSION = "0.6.0";
+
+ public static final String NAME_KEY = "name";
+ public static final String COMMENT_KEY = "comment";
+ public static final String ALWAYS_SYNC_KEY = "always sync";
+ public static final String YIELD_PERIOD_KEY = "yield period";
+ public static final String MAX_CONCURRENT_TASKS_KEY = "max concurrent tasks";
+ public static final String ID_KEY = "id";
+
+ public static final String FLOW_CONTROLLER_PROPS_KEY = "Flow Controller";
+
+ public static final String CORE_PROPS_KEY = "Core Properties";
+ public static final String FLOW_CONTROLLER_SHUTDOWN_PERIOD_KEY = "flow controller graceful shutdown period";
+ public static final String FLOW_SERVICE_WRITE_DELAY_INTERVAL_KEY = "flow service write delay interval";
+ public static final String ADMINISTRATIVE_YIELD_DURATION_KEY = "administrative yield duration";
+ public static final String BORED_YIELD_DURATION_KEY = "bored yield duration";
+
+ public static final String FLOWFILE_REPO_KEY = "FlowFile Repository";
+ public static final String PARTITIONS_KEY = "partitions";
+ public static final String CHECKPOINT_INTERVAL_KEY = "checkpoint interval";
+ public static final String THRESHOLD_KEY = "queue swap threshold";
+ public static final String SWAP_PROPS_KEY = "Swap";
+ public static final String IN_PERIOD_KEY = "in period";
+ public static final String IN_THREADS_KEY = "in threads";
+ public static final String OUT_PERIOD_KEY = "out period";
+ public static final String OUT_THREADS_KEY = "out threads";
+
+
+ public static final String CONTENT_REPO_KEY = "Content Repository";
+ public static final String CONTENT_CLAIM_MAX_APPENDABLE_SIZE_KEY = "content claim max appendable size";
+ public static final String CONTENT_CLAIM_MAX_FLOW_FILES_KEY = "content claim max flow files";
+
+ public static final String COMPONENT_STATUS_REPO_KEY = "Component Status Repository";
+ public static final String BUFFER_SIZE_KEY = "buffer size";
+ public static final String SNAPSHOT_FREQUENCY_KEY = "snapshot frequency";
+
+ public static final String SECURITY_PROPS_KEY = "Security Properties";
+ public static final String KEYSTORE_KEY = "keystore";
+ public static final String KEYSTORE_TYPE_KEY = "keystore type";
+ public static final String KEYSTORE_PASSWORD_KEY = "keystore password";
+ public static final String KEY_PASSWORD_KEY = "key password";
+ public static final String TRUSTSTORE_KEY = "truststore";
+ public static final String TRUSTSTORE_TYPE_KEY = "truststore type";
+ public static final String TRUSTSTORE_PASSWORD_KEY = "truststore password";
+ public static final String SENSITIVE_PROPS_KEY = "Sensitive Props";
+ public static final String SENSITIVE_PROPS_KEY__KEY = "key";
+ public static final String SENSITIVE_PROPS_ALGORITHM_KEY = "algorithm";
+ public static final String SENSITIVE_PROPS_PROVIDER_KEY = "provider";
+
+ public static final String PROCESSOR_CONFIG_KEY = "Processor Configuration";
+ public static final String CLASS_KEY = "class";
+ public static final String SCHEDULING_PERIOD_KEY = "scheduling period";
+ public static final String PENALIZATION_PERIOD_KEY = "penalization period";
+ public static final String SCHEDULING_STRATEGY_KEY = "scheduling strategy";
+ public static final String RUN_DURATION_NANOS_KEY = "run duration nanos";
+ public static final String AUTO_TERMINATED_RELATIONSHIPS_LIST_KEY = "auto-terminated relationships list";
+
+ public static final String PROCESSOR_PROPS_KEY = "Properties";
+
+ public static final String CONNECTION_PROPS_KEY = "Connection Properties";
+ public static final String MAX_WORK_QUEUE_SIZE_KEY = "max work queue size";
+ public static final String MAX_WORK_QUEUE_DATA_SIZE_KEY = "max work queue data size";
+ public static final String FLOWFILE_EXPIRATION__KEY = "flowfile expiration";
+ public static final String QUEUE_PRIORITIZER_CLASS_KEY = "queue prioritizer class";
+
+ public static final String REMOTE_PROCESSING_GROUP_KEY = "Remote Processing Group";
+ public static final String URL_KEY = "url";
+ public static final String TIMEOUT_KEY = "timeout";
+
+ public static final String INPUT_PORT_KEY = "Input Port";
+ public static final String USE_COMPRESSION_KEY = "use compression";
+
+ public static final String PROVENANCE_REPORTING_KEY = "Provenance Reporting";
+ public static final String DESTINATION_URL_KEY = "destination url";
+ public static final String PORT_NAME_KEY = "port name";
+ public static final String ORIGINATING_URL_KEY = "originating url";
+ public static final String BATCH_SIZE_KEY = "batch size";
+
+ public static final String SSL_PROTOCOL_KEY = "ssl protocol";
+
+ // Final util classes should have private constructor
+ private ConfigTransformer() {}
+
+ public static void transformConfigFile(String sourceFile, String destPath) throws Exception {
--- End diff --
this method could likely feed into the other transformConfigFile(InputStream, String).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59920842
--- Diff: minifi-bootstrap/pom.xml ---
@@ -45,5 +45,16 @@ limitations under the License.
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-expression-language</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.yaml</groupId>
--- End diff --
Will need to include this in the appropriate LICENSE/NOTICE files.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by JPercivall <gi...@git.apache.org>.
Github user JPercivall commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59950579
--- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java ---
@@ -0,0 +1,566 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.util;
+
+
+import org.apache.nifi.controller.FlowSerializationException;
+import org.w3c.dom.DOMException;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.yaml.snakeyaml.Yaml;
+
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.TransformerFactoryConfigurationError;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.io.PrintWriter;
+import java.io.UnsupportedEncodingException;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.zip.GZIPOutputStream;
+
+public final class ConfigTransformer {
+ // Underlying version NIFI POC will be using
+ public static final String NIFI_VERSION = "0.6.0";
+
+ public static final String NAME_KEY = "name";
+ public static final String COMMENT_KEY = "comment";
+ public static final String ALWAYS_SYNC_KEY = "always sync";
+ public static final String YIELD_PERIOD_KEY = "yield period";
+ public static final String MAX_CONCURRENT_TASKS_KEY = "max concurrent tasks";
+ public static final String ID_KEY = "id";
+
+ public static final String FLOW_CONTROLLER_PROPS_KEY = "Flow Controller";
+
+ public static final String CORE_PROPS_KEY = "Core Properties";
+ public static final String FLOW_CONTROLLER_SHUTDOWN_PERIOD_KEY = "flow controller graceful shutdown period";
+ public static final String FLOW_SERVICE_WRITE_DELAY_INTERVAL_KEY = "flow service write delay interval";
+ public static final String ADMINISTRATIVE_YIELD_DURATION_KEY = "administrative yield duration";
+ public static final String BORED_YIELD_DURATION_KEY = "bored yield duration";
+
+ public static final String FLOWFILE_REPO_KEY = "FlowFile Repository";
+ public static final String PARTITIONS_KEY = "partitions";
+ public static final String CHECKPOINT_INTERVAL_KEY = "checkpoint interval";
+ public static final String THRESHOLD_KEY = "queue swap threshold";
+ public static final String SWAP_PROPS_KEY = "Swap";
+ public static final String IN_PERIOD_KEY = "in period";
+ public static final String IN_THREADS_KEY = "in threads";
+ public static final String OUT_PERIOD_KEY = "out period";
+ public static final String OUT_THREADS_KEY = "out threads";
+
+
+ public static final String CONTENT_REPO_KEY = "Content Repository";
+ public static final String CONTENT_CLAIM_MAX_APPENDABLE_SIZE_KEY = "content claim max appendable size";
+ public static final String CONTENT_CLAIM_MAX_FLOW_FILES_KEY = "content claim max flow files";
+
+ public static final String COMPONENT_STATUS_REPO_KEY = "Component Status Repository";
+ public static final String BUFFER_SIZE_KEY = "buffer size";
+ public static final String SNAPSHOT_FREQUENCY_KEY = "snapshot frequency";
+
+ public static final String SECURITY_PROPS_KEY = "Security Properties";
+ public static final String KEYSTORE_KEY = "keystore";
+ public static final String KEYSTORE_TYPE_KEY = "keystore type";
+ public static final String KEYSTORE_PASSWORD_KEY = "keystore password";
+ public static final String KEY_PASSWORD_KEY = "key password";
+ public static final String TRUSTSTORE_KEY = "truststore";
+ public static final String TRUSTSTORE_TYPE_KEY = "truststore type";
+ public static final String TRUSTSTORE_PASSWORD_KEY = "truststore password";
+ public static final String SENSITIVE_PROPS_KEY = "Sensitive Props";
+ public static final String SENSITIVE_PROPS_KEY__KEY = "key";
+ public static final String SENSITIVE_PROPS_ALGORITHM_KEY = "algorithm";
+ public static final String SENSITIVE_PROPS_PROVIDER_KEY = "provider";
+
+ public static final String PROCESSOR_CONFIG_KEY = "Processor Configuration";
+ public static final String CLASS_KEY = "class";
+ public static final String SCHEDULING_PERIOD_KEY = "scheduling period";
+ public static final String PENALIZATION_PERIOD_KEY = "penalization period";
+ public static final String SCHEDULING_STRATEGY_KEY = "scheduling strategy";
+ public static final String RUN_DURATION_NANOS_KEY = "run duration nanos";
+ public static final String AUTO_TERMINATED_RELATIONSHIPS_LIST_KEY = "auto-terminated relationships list";
+
+ public static final String PROCESSOR_PROPS_KEY = "Properties";
+
+ public static final String CONNECTION_PROPS_KEY = "Connection Properties";
+ public static final String MAX_WORK_QUEUE_SIZE_KEY = "max work queue size";
+ public static final String MAX_WORK_QUEUE_DATA_SIZE_KEY = "max work queue data size";
+ public static final String FLOWFILE_EXPIRATION__KEY = "flowfile expiration";
+ public static final String QUEUE_PRIORITIZER_CLASS_KEY = "queue prioritizer class";
+
+ public static final String REMOTE_PROCESSING_GROUP_KEY = "Remote Processing Group";
+ public static final String URL_KEY = "url";
+ public static final String TIMEOUT_KEY = "timeout";
+
+ public static final String INPUT_PORT_KEY = "Input Port";
+ public static final String USE_COMPRESSION_KEY = "use compression";
+
+ public static final String PROVENANCE_REPORTING_KEY = "Provenance Reporting";
+ public static final String DESTINATION_URL_KEY = "destination url";
+ public static final String PORT_NAME_KEY = "port name";
+ public static final String ORIGINATING_URL_KEY = "originating url";
+ public static final String BATCH_SIZE_KEY = "batch size";
+
+ public static final String SSL_PROTOCOL_KEY = "ssl protocol";
+
+ // Final util classes should have private constructor
+ private ConfigTransformer() {}
+
+ public static void transformConfigFile(String sourceFile, String destPath) throws Exception {
+ Yaml yaml = new Yaml();
+ File ymlConfigFile = new File(sourceFile);
+ InputStream ios = new FileInputStream(ymlConfigFile);
--- End diff --
Good catch, I always forget to close my streams
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by JPercivall <gi...@git.apache.org>.
Github user JPercivall commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59926348
--- Diff: minifi-bootstrap/pom.xml ---
@@ -45,5 +45,16 @@ limitations under the License.
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-expression-language</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.yaml</groupId>
--- End diff --
http://i.imgur.com/BbgL7x3.gif
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by JPercivall <gi...@git.apache.org>.
Github user JPercivall commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59952289
--- Diff: minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-resources/src/main/resources/conf/config.yml ---
@@ -0,0 +1,101 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the \"License\"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an \"AS IS\" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+Flow Controller:
--- End diff --
Yeah I wasn't sure what to do with them. I figured I'd expose them so that way later on users could maybe tell what each part was meant to do.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59924347
--- Diff: minifi-bootstrap/pom.xml ---
@@ -45,5 +45,16 @@ limitations under the License.
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-expression-language</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.yaml</groupId>
--- End diff --
Yep, agreed. Didn't notice there wasn't a... NOTICE. :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by JPercivall <gi...@git.apache.org>.
Github user JPercivall commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59950322
--- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java ---
@@ -0,0 +1,566 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.util;
+
+
+import org.apache.nifi.controller.FlowSerializationException;
+import org.w3c.dom.DOMException;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.yaml.snakeyaml.Yaml;
+
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.TransformerFactoryConfigurationError;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.io.PrintWriter;
+import java.io.UnsupportedEncodingException;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.zip.GZIPOutputStream;
+
+public final class ConfigTransformer {
+ // Underlying version NIFI POC will be using
+ public static final String NIFI_VERSION = "0.6.0";
+
+ public static final String NAME_KEY = "name";
+ public static final String COMMENT_KEY = "comment";
+ public static final String ALWAYS_SYNC_KEY = "always sync";
+ public static final String YIELD_PERIOD_KEY = "yield period";
+ public static final String MAX_CONCURRENT_TASKS_KEY = "max concurrent tasks";
+ public static final String ID_KEY = "id";
+
+ public static final String FLOW_CONTROLLER_PROPS_KEY = "Flow Controller";
+
+ public static final String CORE_PROPS_KEY = "Core Properties";
+ public static final String FLOW_CONTROLLER_SHUTDOWN_PERIOD_KEY = "flow controller graceful shutdown period";
+ public static final String FLOW_SERVICE_WRITE_DELAY_INTERVAL_KEY = "flow service write delay interval";
+ public static final String ADMINISTRATIVE_YIELD_DURATION_KEY = "administrative yield duration";
+ public static final String BORED_YIELD_DURATION_KEY = "bored yield duration";
+
+ public static final String FLOWFILE_REPO_KEY = "FlowFile Repository";
+ public static final String PARTITIONS_KEY = "partitions";
+ public static final String CHECKPOINT_INTERVAL_KEY = "checkpoint interval";
+ public static final String THRESHOLD_KEY = "queue swap threshold";
+ public static final String SWAP_PROPS_KEY = "Swap";
+ public static final String IN_PERIOD_KEY = "in period";
+ public static final String IN_THREADS_KEY = "in threads";
+ public static final String OUT_PERIOD_KEY = "out period";
+ public static final String OUT_THREADS_KEY = "out threads";
+
+
+ public static final String CONTENT_REPO_KEY = "Content Repository";
+ public static final String CONTENT_CLAIM_MAX_APPENDABLE_SIZE_KEY = "content claim max appendable size";
+ public static final String CONTENT_CLAIM_MAX_FLOW_FILES_KEY = "content claim max flow files";
+
+ public static final String COMPONENT_STATUS_REPO_KEY = "Component Status Repository";
+ public static final String BUFFER_SIZE_KEY = "buffer size";
+ public static final String SNAPSHOT_FREQUENCY_KEY = "snapshot frequency";
+
+ public static final String SECURITY_PROPS_KEY = "Security Properties";
+ public static final String KEYSTORE_KEY = "keystore";
+ public static final String KEYSTORE_TYPE_KEY = "keystore type";
+ public static final String KEYSTORE_PASSWORD_KEY = "keystore password";
+ public static final String KEY_PASSWORD_KEY = "key password";
+ public static final String TRUSTSTORE_KEY = "truststore";
+ public static final String TRUSTSTORE_TYPE_KEY = "truststore type";
+ public static final String TRUSTSTORE_PASSWORD_KEY = "truststore password";
+ public static final String SENSITIVE_PROPS_KEY = "Sensitive Props";
+ public static final String SENSITIVE_PROPS_KEY__KEY = "key";
+ public static final String SENSITIVE_PROPS_ALGORITHM_KEY = "algorithm";
+ public static final String SENSITIVE_PROPS_PROVIDER_KEY = "provider";
+
+ public static final String PROCESSOR_CONFIG_KEY = "Processor Configuration";
+ public static final String CLASS_KEY = "class";
+ public static final String SCHEDULING_PERIOD_KEY = "scheduling period";
+ public static final String PENALIZATION_PERIOD_KEY = "penalization period";
+ public static final String SCHEDULING_STRATEGY_KEY = "scheduling strategy";
+ public static final String RUN_DURATION_NANOS_KEY = "run duration nanos";
+ public static final String AUTO_TERMINATED_RELATIONSHIPS_LIST_KEY = "auto-terminated relationships list";
+
+ public static final String PROCESSOR_PROPS_KEY = "Properties";
+
+ public static final String CONNECTION_PROPS_KEY = "Connection Properties";
+ public static final String MAX_WORK_QUEUE_SIZE_KEY = "max work queue size";
+ public static final String MAX_WORK_QUEUE_DATA_SIZE_KEY = "max work queue data size";
+ public static final String FLOWFILE_EXPIRATION__KEY = "flowfile expiration";
+ public static final String QUEUE_PRIORITIZER_CLASS_KEY = "queue prioritizer class";
+
+ public static final String REMOTE_PROCESSING_GROUP_KEY = "Remote Processing Group";
+ public static final String URL_KEY = "url";
+ public static final String TIMEOUT_KEY = "timeout";
+
+ public static final String INPUT_PORT_KEY = "Input Port";
+ public static final String USE_COMPRESSION_KEY = "use compression";
+
+ public static final String PROVENANCE_REPORTING_KEY = "Provenance Reporting";
+ public static final String DESTINATION_URL_KEY = "destination url";
+ public static final String PORT_NAME_KEY = "port name";
+ public static final String ORIGINATING_URL_KEY = "originating url";
+ public static final String BATCH_SIZE_KEY = "batch size";
+
+ public static final String SSL_PROTOCOL_KEY = "ssl protocol";
+
+ // Final util classes should have private constructor
+ private ConfigTransformer() {}
+
+ public static void transformConfigFile(String sourceFile, String destPath) throws Exception {
--- End diff --
Ah good point, changing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59925817
--- Diff: minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-resources/src/main/resources/conf/config.yml ---
@@ -0,0 +1,101 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the \"License\"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an \"AS IS\" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+Flow Controller:
--- End diff --
Overall this looks good. I think a few of the items that are more tailored toward the UI may be on the chopping block moving forward. name/comment and their ilk could probably be hidden from this config. Not saying a change is needed now, just wanted to make a note.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi-minifi/pull/8
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59759228
--- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java ---
@@ -0,0 +1,571 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.util;
+
+
+import org.apache.nifi.controller.FlowSerializationException;
+import org.w3c.dom.DOMException;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.yaml.snakeyaml.Yaml;
+
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.TransformerFactoryConfigurationError;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.io.PrintWriter;
+import java.io.UnsupportedEncodingException;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.zip.GZIPOutputStream;
+
+public final class ConfigTransformer {
+ // Underlying version NIFI POC will be using
+ public static final String NIFI_VERSION = "0.6.0";
+
+ public static final String NAME_KEY = "name";
+ public static final String COMMENT_KEY = "comment";
+ public static final String ALWAYS_SYNC_KEY = "always sync";
+ public static final String YIELD_PERIOD_KEY = "yield period";
+ public static final String MAX_CONCURRENT_TASKS_KEY = "max concurrent tasks";
+ public static final String ID_KEY = "id";
+
+ public static final String FLOW_CONTROLLER_PROPS_KEY = "Flow Controller";
+
+ public static final String CORE_PROPS_KEY = "Core Properties";
+ public static final String FLOW_CONTROLLER_SHUTDOWN_PERIOD_KEY = "flow controller graceful shutdown period";
+ public static final String FLOW_SERVICE_WRITE_DELAY_INTERVAL_KEY = "flow service write delay interval";
+ public static final String ADMINISTRATIVE_YIELD_DURATION_KEY = "administrative yield duration";
+ public static final String BORED_YIELD_DURATION_KEY = "bored yield duration";
+
+ public static final String FLOWFILE_REPO_KEY = "FlowFile Repository";
+ public static final String PARTITIONS_KEY = "partitions";
+ public static final String CHECKPOINT_INTERVAL_KEY = "checkpoint interval";
+ public static final String THRESHOLD_KEY = "queue swap threshold";
+ public static final String SWAP_PROPS_KEY = "Swap";
+ public static final String IN_PERIOD_KEY = "in period";
+ public static final String IN_THREADS_KEY = "in threads";
+ public static final String OUT_PERIOD_KEY = "out period";
+ public static final String OUT_THREADS_KEY = "out threads";
+
+
+ public static final String CONTENT_REPO_KEY = "Content Repository";
+ public static final String CONTENT_CLAIM_MAX_APPENDABLE_SIZE_KEY = "content claim max appendable size";
+ public static final String CONTENT_CLAIM_MAX_FLOW_FILES_KEY = "content claim max flow files";
+
+ public static final String COMPONENT_STATUS_REPO_KEY = "Component Status Repository";
+ public static final String BUFFER_SIZE_KEY = "buffer size";
+ public static final String SNAPSHOT_FREQUENCY_KEY = "snapshot frequency";
+
+ public static final String SECURITY_PROPS_KEY = "Security Properties";
+ public static final String KEYSTORE_KEY = "keystore";
+ public static final String KEYSTORE_TYPE_KEY = "keystore type";
+ public static final String KEYSTORE_PASSWORD_KEY = "keystore password";
+ public static final String KEY_PASSWORD_KEY = "key password";
+ public static final String TRUSTSTORE_KEY = "truststore";
+ public static final String TRUSTSTORE_TYPE_KEY = "truststore type";
+ public static final String TRUSTSTORE_PASSWORD_KEY = "truststore password";
+ public static final String SENSITIVE_PROPS_KEY = "Sensitive Props";
+ public static final String SENSITIVE_PROPS_KEY__KEY = "key";
+ public static final String SENSITIVE_PROPS_ALGORITHM_KEY = "algorithm";
+ public static final String SENSITIVE_PROPS_PROVIDER_KEY = "provider";
+
+ public static final String PROCESSOR_CONFIG_KEY = "Processor Configuration";
+ public static final String CLASS_KEY = "class";
+ public static final String SCHEDULING_PERIOD_KEY = "scheduling period";
+ public static final String PENALIZATION_PERIOD_KEY = "penalization period";
+ public static final String SCHEDULING_STRATEGY_KEY = "scheduling strategy";
+ public static final String RUN_DURATION_NANOS_KEY = "run duration nanos";
+ public static final String AUTO_TERMINATED_RELATIONSHIPS_LIST_KEY = "auto-terminated relationships list";
+
+ public static final String PROCESSOR_PROPS_KEY = "Properties";
+
+ public static final String CONNECTION_PROPS_KEY = "Connection Properties";
+ public static final String MAX_WORK_QUEUE_SIZE_KEY = "max work queue size";
+ public static final String MAX_WORK_QUEUE_DATA_SIZE_KEY = "max work queue data size";
+ public static final String FLOWFILE_EXPIRATION__KEY = "flowfile expiration";
+ public static final String QUEUE_PRIORITIZER_CLASS_KEY = "queue prioritizer class";
+
+ public static final String REMOTE_PROCESSING_GROUP_KEY = "Remote Processing Group";
+ public static final String URL_KEY = "url";
+ public static final String TIMEOUT_KEY = "timeout";
+
+ public static final String INPUT_PORT_KEY = "Input Port";
+ public static final String USE_COMPRESSION_KEY = "use compression";
+
+ public static final String PROVENANCE_REPORTING_KEY = "Provenance Reporting";
+ public static final String DESTINATION_URL_KEY = "destination url";
+ public static final String PORT_NAME_KEY = "port name";
+ public static final String ORIGINATING_URL_KEY = "originating url";
+ public static final String BATCH_SIZE_KEY = "batch size";
+
+ public static final String SSL_PROTOCOL_KEY = "ssl protocol";
+
+ // Final util classes should have private constructor
+ private ConfigTransformer() {}
+
+ public static void main(String[] args) throws Exception {
+ transformConfigFile("./config.yml",
--- End diff --
This looks like a remnant from development
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by JPercivall <gi...@git.apache.org>.
Github user JPercivall commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59806518
--- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java ---
@@ -0,0 +1,571 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.util;
+
+
+import org.apache.nifi.controller.FlowSerializationException;
+import org.w3c.dom.DOMException;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.yaml.snakeyaml.Yaml;
+
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.TransformerFactoryConfigurationError;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.io.PrintWriter;
+import java.io.UnsupportedEncodingException;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.zip.GZIPOutputStream;
+
+public final class ConfigTransformer {
+ // Underlying version NIFI POC will be using
+ public static final String NIFI_VERSION = "0.6.0";
+
+ public static final String NAME_KEY = "name";
+ public static final String COMMENT_KEY = "comment";
+ public static final String ALWAYS_SYNC_KEY = "always sync";
+ public static final String YIELD_PERIOD_KEY = "yield period";
+ public static final String MAX_CONCURRENT_TASKS_KEY = "max concurrent tasks";
+ public static final String ID_KEY = "id";
+
+ public static final String FLOW_CONTROLLER_PROPS_KEY = "Flow Controller";
+
+ public static final String CORE_PROPS_KEY = "Core Properties";
+ public static final String FLOW_CONTROLLER_SHUTDOWN_PERIOD_KEY = "flow controller graceful shutdown period";
+ public static final String FLOW_SERVICE_WRITE_DELAY_INTERVAL_KEY = "flow service write delay interval";
+ public static final String ADMINISTRATIVE_YIELD_DURATION_KEY = "administrative yield duration";
+ public static final String BORED_YIELD_DURATION_KEY = "bored yield duration";
+
+ public static final String FLOWFILE_REPO_KEY = "FlowFile Repository";
+ public static final String PARTITIONS_KEY = "partitions";
+ public static final String CHECKPOINT_INTERVAL_KEY = "checkpoint interval";
+ public static final String THRESHOLD_KEY = "queue swap threshold";
+ public static final String SWAP_PROPS_KEY = "Swap";
+ public static final String IN_PERIOD_KEY = "in period";
+ public static final String IN_THREADS_KEY = "in threads";
+ public static final String OUT_PERIOD_KEY = "out period";
+ public static final String OUT_THREADS_KEY = "out threads";
+
+
+ public static final String CONTENT_REPO_KEY = "Content Repository";
+ public static final String CONTENT_CLAIM_MAX_APPENDABLE_SIZE_KEY = "content claim max appendable size";
+ public static final String CONTENT_CLAIM_MAX_FLOW_FILES_KEY = "content claim max flow files";
+
+ public static final String COMPONENT_STATUS_REPO_KEY = "Component Status Repository";
+ public static final String BUFFER_SIZE_KEY = "buffer size";
+ public static final String SNAPSHOT_FREQUENCY_KEY = "snapshot frequency";
+
+ public static final String SECURITY_PROPS_KEY = "Security Properties";
+ public static final String KEYSTORE_KEY = "keystore";
+ public static final String KEYSTORE_TYPE_KEY = "keystore type";
+ public static final String KEYSTORE_PASSWORD_KEY = "keystore password";
+ public static final String KEY_PASSWORD_KEY = "key password";
+ public static final String TRUSTSTORE_KEY = "truststore";
+ public static final String TRUSTSTORE_TYPE_KEY = "truststore type";
+ public static final String TRUSTSTORE_PASSWORD_KEY = "truststore password";
+ public static final String SENSITIVE_PROPS_KEY = "Sensitive Props";
+ public static final String SENSITIVE_PROPS_KEY__KEY = "key";
+ public static final String SENSITIVE_PROPS_ALGORITHM_KEY = "algorithm";
+ public static final String SENSITIVE_PROPS_PROVIDER_KEY = "provider";
+
+ public static final String PROCESSOR_CONFIG_KEY = "Processor Configuration";
+ public static final String CLASS_KEY = "class";
+ public static final String SCHEDULING_PERIOD_KEY = "scheduling period";
+ public static final String PENALIZATION_PERIOD_KEY = "penalization period";
+ public static final String SCHEDULING_STRATEGY_KEY = "scheduling strategy";
+ public static final String RUN_DURATION_NANOS_KEY = "run duration nanos";
+ public static final String AUTO_TERMINATED_RELATIONSHIPS_LIST_KEY = "auto-terminated relationships list";
+
+ public static final String PROCESSOR_PROPS_KEY = "Properties";
+
+ public static final String CONNECTION_PROPS_KEY = "Connection Properties";
+ public static final String MAX_WORK_QUEUE_SIZE_KEY = "max work queue size";
+ public static final String MAX_WORK_QUEUE_DATA_SIZE_KEY = "max work queue data size";
+ public static final String FLOWFILE_EXPIRATION__KEY = "flowfile expiration";
+ public static final String QUEUE_PRIORITIZER_CLASS_KEY = "queue prioritizer class";
+
+ public static final String REMOTE_PROCESSING_GROUP_KEY = "Remote Processing Group";
+ public static final String URL_KEY = "url";
+ public static final String TIMEOUT_KEY = "timeout";
+
+ public static final String INPUT_PORT_KEY = "Input Port";
+ public static final String USE_COMPRESSION_KEY = "use compression";
+
+ public static final String PROVENANCE_REPORTING_KEY = "Provenance Reporting";
+ public static final String DESTINATION_URL_KEY = "destination url";
+ public static final String PORT_NAME_KEY = "port name";
+ public static final String ORIGINATING_URL_KEY = "originating url";
+ public static final String BATCH_SIZE_KEY = "batch size";
+
+ public static final String SSL_PROTOCOL_KEY = "ssl protocol";
+
+ // Final util classes should have private constructor
+ private ConfigTransformer() {}
+
+ public static void main(String[] args) throws Exception {
+ transformConfigFile("./config.yml",
--- End diff --
Yup, that is correct. Removing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59923164
--- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java ---
@@ -0,0 +1,566 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.util;
+
+
+import org.apache.nifi.controller.FlowSerializationException;
+import org.w3c.dom.DOMException;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.yaml.snakeyaml.Yaml;
+
+import javax.xml.parsers.DocumentBuilder;
+import javax.xml.parsers.DocumentBuilderFactory;
+import javax.xml.parsers.ParserConfigurationException;
+import javax.xml.transform.OutputKeys;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.TransformerFactory;
+import javax.xml.transform.TransformerFactoryConfigurationError;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.stream.StreamResult;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.io.PrintWriter;
+import java.io.UnsupportedEncodingException;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.zip.GZIPOutputStream;
+
+public final class ConfigTransformer {
+ // Underlying version NIFI POC will be using
+ public static final String NIFI_VERSION = "0.6.0";
+
+ public static final String NAME_KEY = "name";
+ public static final String COMMENT_KEY = "comment";
+ public static final String ALWAYS_SYNC_KEY = "always sync";
+ public static final String YIELD_PERIOD_KEY = "yield period";
+ public static final String MAX_CONCURRENT_TASKS_KEY = "max concurrent tasks";
+ public static final String ID_KEY = "id";
+
+ public static final String FLOW_CONTROLLER_PROPS_KEY = "Flow Controller";
+
+ public static final String CORE_PROPS_KEY = "Core Properties";
+ public static final String FLOW_CONTROLLER_SHUTDOWN_PERIOD_KEY = "flow controller graceful shutdown period";
+ public static final String FLOW_SERVICE_WRITE_DELAY_INTERVAL_KEY = "flow service write delay interval";
+ public static final String ADMINISTRATIVE_YIELD_DURATION_KEY = "administrative yield duration";
+ public static final String BORED_YIELD_DURATION_KEY = "bored yield duration";
+
+ public static final String FLOWFILE_REPO_KEY = "FlowFile Repository";
+ public static final String PARTITIONS_KEY = "partitions";
+ public static final String CHECKPOINT_INTERVAL_KEY = "checkpoint interval";
+ public static final String THRESHOLD_KEY = "queue swap threshold";
+ public static final String SWAP_PROPS_KEY = "Swap";
+ public static final String IN_PERIOD_KEY = "in period";
+ public static final String IN_THREADS_KEY = "in threads";
+ public static final String OUT_PERIOD_KEY = "out period";
+ public static final String OUT_THREADS_KEY = "out threads";
+
+
+ public static final String CONTENT_REPO_KEY = "Content Repository";
+ public static final String CONTENT_CLAIM_MAX_APPENDABLE_SIZE_KEY = "content claim max appendable size";
+ public static final String CONTENT_CLAIM_MAX_FLOW_FILES_KEY = "content claim max flow files";
+
+ public static final String COMPONENT_STATUS_REPO_KEY = "Component Status Repository";
+ public static final String BUFFER_SIZE_KEY = "buffer size";
+ public static final String SNAPSHOT_FREQUENCY_KEY = "snapshot frequency";
+
+ public static final String SECURITY_PROPS_KEY = "Security Properties";
+ public static final String KEYSTORE_KEY = "keystore";
+ public static final String KEYSTORE_TYPE_KEY = "keystore type";
+ public static final String KEYSTORE_PASSWORD_KEY = "keystore password";
+ public static final String KEY_PASSWORD_KEY = "key password";
+ public static final String TRUSTSTORE_KEY = "truststore";
+ public static final String TRUSTSTORE_TYPE_KEY = "truststore type";
+ public static final String TRUSTSTORE_PASSWORD_KEY = "truststore password";
+ public static final String SENSITIVE_PROPS_KEY = "Sensitive Props";
+ public static final String SENSITIVE_PROPS_KEY__KEY = "key";
+ public static final String SENSITIVE_PROPS_ALGORITHM_KEY = "algorithm";
+ public static final String SENSITIVE_PROPS_PROVIDER_KEY = "provider";
+
+ public static final String PROCESSOR_CONFIG_KEY = "Processor Configuration";
+ public static final String CLASS_KEY = "class";
+ public static final String SCHEDULING_PERIOD_KEY = "scheduling period";
+ public static final String PENALIZATION_PERIOD_KEY = "penalization period";
+ public static final String SCHEDULING_STRATEGY_KEY = "scheduling strategy";
+ public static final String RUN_DURATION_NANOS_KEY = "run duration nanos";
+ public static final String AUTO_TERMINATED_RELATIONSHIPS_LIST_KEY = "auto-terminated relationships list";
+
+ public static final String PROCESSOR_PROPS_KEY = "Properties";
+
+ public static final String CONNECTION_PROPS_KEY = "Connection Properties";
+ public static final String MAX_WORK_QUEUE_SIZE_KEY = "max work queue size";
+ public static final String MAX_WORK_QUEUE_DATA_SIZE_KEY = "max work queue data size";
+ public static final String FLOWFILE_EXPIRATION__KEY = "flowfile expiration";
+ public static final String QUEUE_PRIORITIZER_CLASS_KEY = "queue prioritizer class";
+
+ public static final String REMOTE_PROCESSING_GROUP_KEY = "Remote Processing Group";
+ public static final String URL_KEY = "url";
+ public static final String TIMEOUT_KEY = "timeout";
+
+ public static final String INPUT_PORT_KEY = "Input Port";
+ public static final String USE_COMPRESSION_KEY = "use compression";
+
+ public static final String PROVENANCE_REPORTING_KEY = "Provenance Reporting";
+ public static final String DESTINATION_URL_KEY = "destination url";
+ public static final String PORT_NAME_KEY = "port name";
+ public static final String ORIGINATING_URL_KEY = "originating url";
+ public static final String BATCH_SIZE_KEY = "batch size";
+
+ public static final String SSL_PROTOCOL_KEY = "ssl protocol";
+
+ // Final util classes should have private constructor
+ private ConfigTransformer() {}
+
+ public static void transformConfigFile(String sourceFile, String destPath) throws Exception {
+ Yaml yaml = new Yaml();
+ File ymlConfigFile = new File(sourceFile);
+ InputStream ios = new FileInputStream(ymlConfigFile);
--- End diff --
doesn't look like this is getting closed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request: MINIFI-15 Created a config file format w...
Posted by JPercivall <gi...@git.apache.org>.
Github user JPercivall commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/8#discussion_r59921795
--- Diff: minifi-bootstrap/pom.xml ---
@@ -45,5 +45,16 @@ limitations under the License.
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-expression-language</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.yaml</groupId>
--- End diff --
Snakeyaml is licensed using an apache 2.0 license and does not include a notice[1]. It is my understanding that you don't need to do anything if both those are true.
[1] https://bitbucket.org/asomov/snakeyaml/src
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---