You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by brosander <gi...@git.apache.org> on 2017/06/22 21:12:50 UTC
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
GitHub user brosander opened a pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115
MINIFI-324 - More flexible uid generation
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with MINIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [X] Is your initial contribution a single, squashed commit?
### For code changes:
- [X] - N/A - If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [X] - N/A - If applicable, have you updated the LICENSE file?
- [X] - N/A - If applicable, have you updated the NOTICE file?
### For documentation related changes:
- [X] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/brosander/nifi-minifi-cpp MINIFI-324
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi-minifi-cpp/pull/115.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #115
----
commit 2ce366c5547aa6608653ce71d98e335ff3a0a9ff
Author: Bryan Rosander <br...@apache.org>
Date: 2017-06-21T18:29:03Z
MINIFI-324 - More flexible uid generation
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115#discussion_r123642563
--- Diff: libminifi/include/utils/Id.h ---
@@ -0,0 +1,77 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef LIBMINIFI_INCLUDE_UTILS_ID_H_
+#define LIBMINIFI_INCLUDE_UTILS_ID_H_
+
+#include <atomic>
+#include <memory>
+#include <string>
+#include <uuid/uuid.h>
+
+#include "core/logging/Logger.h"
+#include "properties/Properties.h"
+
+#define UNSIGNED_CHAR_MAX 255
+#define UUID_TIME_IMPL 0
+#define UUID_RANDOM_IMPL 1
+#define UUID_DEFAULT_IMPL 2
+#define MINIFI_UID_IMPL 3
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+class IdGenerator {
+ public:
+ void generate(uuid_t output);
+ void initialize(const std::shared_ptr<Properties> & properties);
+
+ static std::shared_ptr<IdGenerator> getIdGenerator() {
+ static std::shared_ptr<IdGenerator> generator = std::shared_ptr<IdGenerator>(new IdGenerator());
--- End diff --
why not use make_shared?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115#discussion_r123744302
--- Diff: README.md ---
@@ -326,26 +326,44 @@ Additionally, users can utilize the MiNiFi Toolkit Converter (version 0.0.1 - sc
### Http Configuration Listener
- Http Configuration Listener will pull flow file configuration from the remote command control server,
+ Http Configuration Listener will pull flow file configuration from the remote command control server,
validate and apply the new flow configuration
- in minifi.properties
+ in minifi.properties
nifi.configuration.listener.type=http
nifi.configuration.listener.http.url=https://localhost:8080
- nifi.configuration.listener.pull.interval=1 sec
+ nifi.configuration.listener.pull.interval=1 sec
nifi.configuration.listener.client.ca.certificate=./conf/nifi-cert.pem
if you want to enable client certificate
nifi.configuration.listener.need.ClientAuth=true
nifi.configuration.listener.client.certificate=./conf/client.pem
nifi.configuration.listener.client.private.key=./conf/client.key
nifi.configuration.listener.client.pass.phrase=./conf/password
-
+
+### UID generation
+
+MiNiFi need to generate many unique identifiers in the course of operations. There are a few different uid implementations available that can be configured in minifi-uid.properties.
+
+Implementation for uid generation can be selected using the uid.implementation property values:
+1. time - use uuid_generate_time
+2. random - use uuid_generate_random
+3. uuid_default - use uuid_generate (will attempt to use uuid_generate_random and fall back to uuid_generate_time if no high quality randomness is available)
--- End diff --
Well, uuid_generate_time is a lot faster than the random version but it leaks information about which node and what time the uuid originate from, I thought that might be relevant info if there are privacy-conscious applications of MiNiFi.
https://linux.die.net/man/3/uuid_generate_time
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115#discussion_r123644079
--- Diff: libminifi/include/provenance/Provenance.h ---
@@ -160,21 +161,9 @@ class ProvenanceEventRecord : protected org::apache::nifi::minifi::io::Serializa
/*!
* Create a new provenance event record
*/
- ProvenanceEventRecord(ProvenanceEventType event, std::string componentId, std::string componentType)
- : logger_(logging::LoggerFactory<ProvenanceEventRecord>::getLogger()) {
- _eventType = event;
- _componentId = componentId;
- _componentType = componentType;
- _eventTime = getTimeMillis();
- char eventIdStr[37];
- // Generate the global UUID for th event
- uuid_generate(_eventId);
- uuid_unparse_lower(_eventId, eventIdStr);
- _eventIdStr = eventIdStr;
- }
+ ProvenanceEventRecord(ProvenanceEventType event, std::string componentId, std::string componentType);
--- End diff --
Thanks. I'm guilty of doing this too and it's so much cleaner to move it out.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #115: MINIFI-324 - More flexible uid generation
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/115
Looks good will merge shortly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115#discussion_r123746263
--- Diff: README.md ---
@@ -326,26 +326,44 @@ Additionally, users can utilize the MiNiFi Toolkit Converter (version 0.0.1 - sc
### Http Configuration Listener
- Http Configuration Listener will pull flow file configuration from the remote command control server,
+ Http Configuration Listener will pull flow file configuration from the remote command control server,
validate and apply the new flow configuration
- in minifi.properties
+ in minifi.properties
nifi.configuration.listener.type=http
nifi.configuration.listener.http.url=https://localhost:8080
- nifi.configuration.listener.pull.interval=1 sec
+ nifi.configuration.listener.pull.interval=1 sec
nifi.configuration.listener.client.ca.certificate=./conf/nifi-cert.pem
if you want to enable client certificate
nifi.configuration.listener.need.ClientAuth=true
nifi.configuration.listener.client.certificate=./conf/client.pem
nifi.configuration.listener.client.private.key=./conf/client.key
nifi.configuration.listener.client.pass.phrase=./conf/password
-
+
+### UID generation
+
+MiNiFi need to generate many unique identifiers in the course of operations. There are a few different uid implementations available that can be configured in minifi-uid.properties.
+
+Implementation for uid generation can be selected using the uid.implementation property values:
+1. time - use uuid_generate_time
+2. random - use uuid_generate_random
+3. uuid_default - use uuid_generate (will attempt to use uuid_generate_random and fall back to uuid_generate_time if no high quality randomness is available)
--- End diff --
I added a comment in Id.cpp to clarify the location in code I was referring to. Sorry for the confusion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115#discussion_r123743347
--- Diff: README.md ---
@@ -326,26 +326,44 @@ Additionally, users can utilize the MiNiFi Toolkit Converter (version 0.0.1 - sc
### Http Configuration Listener
- Http Configuration Listener will pull flow file configuration from the remote command control server,
+ Http Configuration Listener will pull flow file configuration from the remote command control server,
validate and apply the new flow configuration
- in minifi.properties
+ in minifi.properties
nifi.configuration.listener.type=http
nifi.configuration.listener.http.url=https://localhost:8080
- nifi.configuration.listener.pull.interval=1 sec
+ nifi.configuration.listener.pull.interval=1 sec
nifi.configuration.listener.client.ca.certificate=./conf/nifi-cert.pem
if you want to enable client certificate
nifi.configuration.listener.need.ClientAuth=true
nifi.configuration.listener.client.certificate=./conf/client.pem
nifi.configuration.listener.client.private.key=./conf/client.key
nifi.configuration.listener.client.pass.phrase=./conf/password
-
+
+### UID generation
+
+MiNiFi need to generate many unique identifiers in the course of operations. There are a few different uid implementations available that can be configured in minifi-uid.properties.
+
+Implementation for uid generation can be selected using the uid.implementation property values:
+1. time - use uuid_generate_time
+2. random - use uuid_generate_random
+3. uuid_default - use uuid_generate (will attempt to use uuid_generate_random and fall back to uuid_generate_time if no high quality randomness is available)
--- End diff --
This statement confused me, but I'm guessing that it's in the context of uid.implementation.
The following is logged to debug if uid.implementation isn't defined ( let's say there is no uid properties file )
"Using uuid_generate_time implementation for uids."
Does it make sense to make clear that uuid_generate_time is used as the fallback if the file does not exist?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi-minifi-cpp/pull/115
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115#discussion_r123745514
--- Diff: libminifi/src/utils/Id.cpp ---
@@ -0,0 +1,154 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "utils/Id.h"
+
+#include <inttypes.h>
+#include <uuid/uuid.h>
+
+#include <algorithm>
+#include <chrono>
+#include <cmath>
+#include <memory>
+#include <string>
+
+#include "core/logging/LoggerConfiguration.h"
+#include "utils/StringUtils.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+uint64_t timestamp = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
+
+NonRepeatingStringGenerator::NonRepeatingStringGenerator() : prefix_((std::to_string(timestamp) + "-")) {}
+
+IdGenerator::IdGenerator() : implementation_(UUID_TIME_IMPL), logger_(logging::LoggerFactory<IdGenerator>::getLogger()), incrementor_(0) {
+}
+
+uint64_t IdGenerator::getDeviceSegmentFromString(const std::string& str, int numBits) {
+ uint64_t deviceSegment = 0;
+ for (int i = 0; i < str.length(); i++) {
+ unsigned char c = toupper(str[i]);
+ if (c >= '0' && c <= '9') {
+ deviceSegment = deviceSegment + (c - '0');
+ } else if (c >= 'A' && c <= 'F') {
+ deviceSegment = deviceSegment + (c - 'A' + 10);
+ } else {
+ logger_->log_error("Expected hex char (0-9, A-F). Got %c.", c);
+ }
+ deviceSegment = deviceSegment << 4;
+ }
+ deviceSegment <<= 64 - (4 * (str.length() + 1));
+ deviceSegment >>= 64 - numBits;
+ logger_->log_debug("Using user defined device segment: %" PRIx64, deviceSegment);
+ deviceSegment <<= 64 - numBits;
+ return deviceSegment;
+}
+
+uint64_t IdGenerator::getRandomDeviceSegment(int numBits) {
+ uint64_t deviceSegment = 0;
+ uuid_t random_uuid;
+ for (int word = 0; word < 2; word++) {
+ uuid_generate_random(random_uuid);
+ for (int i = 0; i < 4; i++) {
+ deviceSegment += random_uuid[i];
+ deviceSegment <<= 8;
+ }
+ }
+ deviceSegment >>= 64 - numBits;
+ logger_->log_debug("Using random device segment: %" PRIx64, deviceSegment);
+ deviceSegment <<= 64 - numBits;
+ return deviceSegment;
+}
+
+void IdGenerator::initialize(const std::shared_ptr<Properties> & properties) {
+ std::string implementation_str;
+ implementation_ = UUID_TIME_IMPL;
+ if (properties->get("uid.implementation", implementation_str)) {
+ std::transform(implementation_str.begin(), implementation_str.end(), implementation_str.begin(), ::tolower);
+ if ("random" == implementation_str) {
+ logger_->log_debug("Using uuid_generate_random for uids.");
+ implementation_ = UUID_RANDOM_IMPL;
+ } else if ("uuid_default" == implementation_str) {
+ logger_->log_debug("Using uuid_generate for uids.");
+ implementation_ = UUID_DEFAULT_IMPL;
+ } else if ("minifi_uid" == implementation_str) {
+ logger_->log_debug("Using minifi uid implementation for uids");
+ implementation_ = MINIFI_UID_IMPL;
+
+ uint64_t timestamp = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
+ int device_bits = properties->getInt("uid.minifi.device.segment.bits", 16);
+ std::string device_segment;
+ uint64_t prefix = timestamp;
+ if (device_bits > 0) {
+ if (properties->get("uid.minifi.device.segment", device_segment)) {
+ prefix = getDeviceSegmentFromString(device_segment, device_bits);
+ } else {
+ logger_->log_warn("uid.minifi.device.segment not specified, generating random device segment");
+ prefix = getRandomDeviceSegment(device_bits);
+ }
+ timestamp <<= device_bits;
+ timestamp >>= device_bits;
+ prefix = prefix + timestamp;
+ logger_->log_debug("Using minifi uid prefix: %16" PRIx64, prefix);
+ }
+ for (int i = 0; i < 8; i++) {
+ unsigned char prefix_element = (prefix >> ((7 - i) * 8)) & UNSIGNED_CHAR_MAX;
+ deterministic_prefix_[i] = prefix_element;
+ }
+ incrementor_ = 0;
+ } else if ("time" == implementation_str) {
+ logger_->log_debug("Using uuid_generate_time implementation for uids.");
+ } else {
+ logger_->log_debug("Invalid value for uid.implementation (%s). Using uuid_generate_time implementation for uids.", implementation_str);
+ }
+ } else {
+ logger_->log_debug("Using uuid_generate_time implementation for uids.");
--- End diff --
That documentation question was regarding this log statement and how it seems without the first conditional being true, we simply default to time irrespective of random entry.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115#discussion_r123743755
--- Diff: libminifi/src/provenance/Provenance.cpp ---
@@ -34,9 +36,24 @@ namespace nifi {
namespace minifi {
namespace provenance {
+std::shared_ptr<utils::IdGenerator> ProvenanceEventRecord::id_generator_ = utils::IdGenerator::getIdGenerator();
+std::shared_ptr<logging::Logger> ProvenanceEventRecord::logger_ = logging::LoggerFactory<ProvenanceEventRecord>::getLogger();
+
const char *ProvenanceEventRecord::ProvenanceEventTypeStr[REPLAY + 1] = { "CREATE", "RECEIVE", "FETCH", "SEND", "DOWNLOAD", "DROP", "EXPIRE", "FORK", "JOIN", "CLONE", "CONTENT_MODIFIED",
"ATTRIBUTES_MODIFIED", "ROUTE", "ADDINFO", "REPLAY" };
+ProvenanceEventRecord::ProvenanceEventRecord(ProvenanceEventRecord::ProvenanceEventType event, std::__cxx11::string componentId, std::__cxx11::string componentType) {
--- End diff --
Autocomplete fail, will fix
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115#discussion_r123643744
--- Diff: libminifi/src/provenance/Provenance.cpp ---
@@ -34,9 +36,24 @@ namespace nifi {
namespace minifi {
namespace provenance {
+std::shared_ptr<utils::IdGenerator> ProvenanceEventRecord::id_generator_ = utils::IdGenerator::getIdGenerator();
+std::shared_ptr<logging::Logger> ProvenanceEventRecord::logger_ = logging::LoggerFactory<ProvenanceEventRecord>::getLogger();
+
const char *ProvenanceEventRecord::ProvenanceEventTypeStr[REPLAY + 1] = { "CREATE", "RECEIVE", "FETCH", "SEND", "DOWNLOAD", "DROP", "EXPIRE", "FORK", "JOIN", "CLONE", "CONTENT_MODIFIED",
"ATTRIBUTES_MODIFIED", "ROUTE", "ADDINFO", "REPLAY" };
+ProvenanceEventRecord::ProvenanceEventRecord(ProvenanceEventRecord::ProvenanceEventType event, std::__cxx11::string componentId, std::__cxx11::string componentType) {
--- End diff --
was the use of std::__cxx11 intentional?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115#discussion_r123745136
--- Diff: libminifi/include/utils/Id.h ---
@@ -0,0 +1,77 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef LIBMINIFI_INCLUDE_UTILS_ID_H_
+#define LIBMINIFI_INCLUDE_UTILS_ID_H_
+
+#include <atomic>
+#include <memory>
+#include <string>
+#include <uuid/uuid.h>
+
+#include "core/logging/Logger.h"
+#include "properties/Properties.h"
+
+#define UNSIGNED_CHAR_MAX 255
+#define UUID_TIME_IMPL 0
+#define UUID_RANDOM_IMPL 1
+#define UUID_DEFAULT_IMPL 2
+#define MINIFI_UID_IMPL 3
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+class IdGenerator {
+ public:
+ void generate(uuid_t output);
+ void initialize(const std::shared_ptr<Properties> & properties);
+
+ static std::shared_ptr<IdGenerator> getIdGenerator() {
+ static std::shared_ptr<IdGenerator> generator = std::shared_ptr<IdGenerator>(new IdGenerator());
--- End diff --
I wanted to make the constructor private that seems to require manual invocation of new
```
error: ‘org::apache::nifi::minifi::utils::IdGenerator::IdGenerator()’ is private
IdGenerator();
^
In file included from /usr/include/x86_64-linux-gnu/c++/5/bits/c++allocator.h:33:0,
from /usr/include/c++/5/bits/allocator.h:46,
from /usr/include/c++/5/bits/stl_tree.h:64,
from /usr/include/c++/5/map:60,
from /home/bryan/Github/nifi-minifi-cpp/libminifi/include/core/logging/LoggerConfiguration.h:24,
from /home/bryan/Github/nifi-minifi-cpp/libminifi/src/core/logging/LoggerConfiguration.cpp:21:
/usr/include/c++/5/ext/new_allocator.h:120:4: error: within this context
{ ::new((void *)__p) _Up(std::forward<_Args>(__args)...); }
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #115: MINIFI-324 - More flexible uid generation
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/115#discussion_r123745177
--- Diff: README.md ---
@@ -326,26 +326,44 @@ Additionally, users can utilize the MiNiFi Toolkit Converter (version 0.0.1 - sc
### Http Configuration Listener
- Http Configuration Listener will pull flow file configuration from the remote command control server,
+ Http Configuration Listener will pull flow file configuration from the remote command control server,
validate and apply the new flow configuration
- in minifi.properties
+ in minifi.properties
nifi.configuration.listener.type=http
nifi.configuration.listener.http.url=https://localhost:8080
- nifi.configuration.listener.pull.interval=1 sec
+ nifi.configuration.listener.pull.interval=1 sec
nifi.configuration.listener.client.ca.certificate=./conf/nifi-cert.pem
if you want to enable client certificate
nifi.configuration.listener.need.ClientAuth=true
nifi.configuration.listener.client.certificate=./conf/client.pem
nifi.configuration.listener.client.private.key=./conf/client.key
nifi.configuration.listener.client.pass.phrase=./conf/password
-
+
+### UID generation
+
+MiNiFi need to generate many unique identifiers in the course of operations. There are a few different uid implementations available that can be configured in minifi-uid.properties.
+
+Implementation for uid generation can be selected using the uid.implementation property values:
+1. time - use uuid_generate_time
+2. random - use uuid_generate_random
+3. uuid_default - use uuid_generate (will attempt to use uuid_generate_random and fall back to uuid_generate_time if no high quality randomness is available)
--- End diff --
Sorry, I think my question was poorly worded. You make clear what happens when the file exists, which is what I meant by "in the context of uid.implementation [existing]".
My question was more about documenting what will be used if the uid configuration file and the options don't exist. It seemed from the log message, "Using uuid_generate_time implementation for uids." that we were simply relying on time and not random first and then time.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---