You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by brosander <gi...@git.apache.org> on 2016/09/07 22:07:42 UTC
[GitHub] nifi-minifi pull request #32: MINIFI-104 - Making connection ids filesystem ...
GitHub user brosander opened a pull request:
https://github.com/apache/nifi-minifi/pull/32
MINIFI-104 - Making connection ids filesystem friendly, unique
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/brosander/nifi-minifi MINIFI-104
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi-minifi/pull/32.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #32
----
commit 1143b336df143c28051c558e68f20dd50d824d00
Author: Bryan Rosander <br...@gmail.com>
Date: 2016-09-07T21:36:32Z
MINIFI-104 - Making connection ids filesystem friendly, unique
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request #32: MINIFI-104 - Making connection ids filesystem ...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/32#discussion_r78058194
--- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java ---
@@ -546,6 +550,37 @@ protected static void addConnection(final Element parentElement, ConnectionSchem
}
}
+ protected static String getConnectionName(ConnectionSchema connectionProperties) {
+ String connectionName = connectionProperties.getName();
+ if (StringUtil.isNullOrEmpty(connectionName)) {
+ return EMPTY_NAME;
+ }
+ return connectionName;
+ }
+
+ /**
+ * Will replace all characters not in [A-Za-z0-9_] with _
+ *
+ * This has potential for collisions so it will also append numbers as necessary to prevent that
+ *
+ * @param ids id map of already incremented numbers
+ * @param name the name
+ * @return a unique filesystem-friendly id
+ */
+ protected static String getUniqueId(Map<String, Integer> ids, String name) {
+ String baseId = name.replaceAll("[^A-Za-z0-9_]", "_");
+ String id = baseId;
+ Integer idNum = ids.get(baseId);
+ while (ids.containsKey(id)) {
+ id = baseId + "_" + idNum++;
+ }
+ if (id != baseId) {
+ ids.put(baseId, idNum);
+ }
+ ids.put(id, 2);
--- End diff --
@apiri so I may have overthought this a little bit.
The initial approach I just tracked used names and kept incrementing a suffix. It bothered me a little that that would be n^2 with respect to duplicate names, the above should avoid that pitfall by tracking the next suffix available for every prefix. That way the following wouldn't collide:
test -> test
test -> test_2
test_2 -> test_2_2
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request #32: MINIFI-104 - Making connection ids filesystem ...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/32#discussion_r78056820
--- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java ---
@@ -546,6 +550,37 @@ protected static void addConnection(final Element parentElement, ConnectionSchem
}
}
+ protected static String getConnectionName(ConnectionSchema connectionProperties) {
+ String connectionName = connectionProperties.getName();
+ if (StringUtil.isNullOrEmpty(connectionName)) {
+ return EMPTY_NAME;
+ }
+ return connectionName;
+ }
+
+ /**
+ * Will replace all characters not in [A-Za-z0-9_] with _
+ *
+ * This has potential for collisions so it will also append numbers as necessary to prevent that
+ *
+ * @param ids id map of already incremented numbers
+ * @param name the name
+ * @return a unique filesystem-friendly id
+ */
+ protected static String getUniqueId(Map<String, Integer> ids, String name) {
--- End diff --
I see what is happening here but wonder if hashing might be a better alternative. While this does avoid collisions that could result, could keep things simpler given the requirement for unique names currently.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request #32: MINIFI-104 - Making connection ids filesystem ...
Posted by brosander <gi...@git.apache.org>.
Github user brosander closed the pull request at:
https://github.com/apache/nifi-minifi/pull/32
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request #32: MINIFI-104 - Making connection ids filesystem ...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/32#discussion_r78056972
--- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java ---
@@ -546,6 +550,37 @@ protected static void addConnection(final Element parentElement, ConnectionSchem
}
}
+ protected static String getConnectionName(ConnectionSchema connectionProperties) {
+ String connectionName = connectionProperties.getName();
+ if (StringUtil.isNullOrEmpty(connectionName)) {
+ return EMPTY_NAME;
+ }
+ return connectionName;
+ }
+
+ /**
+ * Will replace all characters not in [A-Za-z0-9_] with _
+ *
+ * This has potential for collisions so it will also append numbers as necessary to prevent that
+ *
+ * @param ids id map of already incremented numbers
+ * @param name the name
+ * @return a unique filesystem-friendly id
+ */
+ protected static String getUniqueId(Map<String, Integer> ids, String name) {
+ String baseId = name.replaceAll("[^A-Za-z0-9_]", "_");
+ String id = baseId;
+ Integer idNum = ids.get(baseId);
+ while (ids.containsKey(id)) {
+ id = baseId + "_" + idNum++;
+ }
+ if (id != baseId) {
+ ids.put(baseId, idNum);
+ }
+ ids.put(id, 2);
--- End diff --
if we opt to keep the current approach, not sure I follow this. Was this a remnant of testing?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi pull request #32: MINIFI-104 - Making connection ids filesystem ...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi/pull/32#discussion_r78068727
--- Diff: minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java ---
@@ -546,6 +550,37 @@ protected static void addConnection(final Element parentElement, ConnectionSchem
}
}
+ protected static String getConnectionName(ConnectionSchema connectionProperties) {
+ String connectionName = connectionProperties.getName();
+ if (StringUtil.isNullOrEmpty(connectionName)) {
+ return EMPTY_NAME;
+ }
+ return connectionName;
+ }
+
+ /**
+ * Will replace all characters not in [A-Za-z0-9_] with _
+ *
+ * This has potential for collisions so it will also append numbers as necessary to prevent that
+ *
+ * @param ids id map of already incremented numbers
+ * @param name the name
+ * @return a unique filesystem-friendly id
+ */
+ protected static String getUniqueId(Map<String, Integer> ids, String name) {
+ String baseId = name.replaceAll("[^A-Za-z0-9_]", "_");
+ String id = baseId;
+ Integer idNum = ids.get(baseId);
+ while (ids.containsKey(id)) {
+ id = baseId + "_" + idNum++;
+ }
+ if (id != baseId) {
+ ids.put(baseId, idNum);
+ }
+ ids.put(id, 2);
--- End diff --
Ah, I see. Wasn't tracking and certainly didn't have that pop into mind.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---