You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2021/05/06 21:32:52 UTC
[GitHub] [nifi] mattyb149 opened a new pull request #5062: MINIFI-554: Add OSUtils class to MiNiFi to get PID correctly
mattyb149 opened a new pull request #5062:
URL: https://github.com/apache/nifi/pull/5062
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
#### Description of PR
Duplicates the function of OSUtils to get the processor ID (PID) correctly (avoiding reflective access warnings in Java 11). I considered moving OSUtils to nifi-utils but didn't want to increase the footprint 1+ MB (due to JNA dependencies) in all places that use nifi-utils. If we ever move nifi-utils to lib in NiFi we should do the same to MiNiFi and use the same OSUtils.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder?
- [x] Have you written or updated unit tests to verify your changes?
- [x] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5062: MINIFI-554: Add OSUtils class to MiNiFi to get PID correctly
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on a change in pull request #5062:
URL: https://github.com/apache/nifi/pull/5062#discussion_r628364975
##########
File path: minifi/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/OSUtils.java
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.util;
+
+import com.sun.jna.Pointer;
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.WinNT;
+import org.slf4j.Logger;
+
+import java.lang.reflect.Field;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * OS specific utilities with generic method interfaces
+ */
+public final class OSUtils {
+ /**
+ * @param process MiNiFi Process Reference
+ * @param logger Logger Reference for Debug
+ * @return Returns pid or null in-case pid could not be determined
+ * This method takes {@link Process} and {@link Logger} and returns
+ * the platform specific ProcessId for Unix like systems or Handle for Win32 Systems, a.k.a <b>pid</b>
+ * In-case it fails to determine the pid, it will return Null.
+ * Purpose for the Logger is to log any interaction for debugging.
+ */
+ public static Long getProcessId(final Process process, final Logger logger) {
+ /*
+ * NIFI-5175: NiFi built with Java 1.8 and running on Java 9. Reflectively invoke Process.pid() on
+ * the given process instance to get the PID of this Java process. Reflection is required in this scenario
+ * due to NiFi being compiled on Java 1.8, which does not have the Process API improvements available in
+ * Java 9.
+ *
+ * Otherwise, if NiFi is running on Java 1.8, attempt to get PID using capabilities available on Java 1.8.
+ *
+ * TODO: When minimum Java version updated to Java 9+, this class should be removed with the addition
+ * of the pid method to the Process API.
+ */
+ Long pid = null;
+ try {
+ // Get Process.pid() interface method to avoid illegal reflective access
+ final Method pidMethod = Process.class.getDeclaredMethod("pid");
+ final Object pidNumber = pidMethod.invoke(process);
+ if (pidNumber instanceof Long) {
+ pid = (Long) pidNumber;
+ }
+ } catch (final NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
+ final String processClassName = process.getClass().getName();
+ if (processClassName.equals("java.lang.UNIXProcess")) {
+ pid = getUnixPid(process, logger);
+ } else if (processClassName.equals("java.lang.Win32Process")
+ || processClassName.equals("java.lang.ProcessImpl")) {
+ pid = getWindowsProcessId(process, logger);
+ } else {
+ logger.debug("Failed to determine Process ID from [{}]: {}", processClassName, e.getMessage());
+ }
+ }
+
+ return pid;
+ }
+
+ /**
+ * Returns the major version parsed from the provided Java version string (e.g. {@code "1.8.0.231"} -> {@code 8}).
+ *
+ * @param version the Java version string
+ * @return the major version as an int
+ */
+ public static int parseJavaVersion(final String version) {
+ String majorVersion;
+ if (version.startsWith("1.")) {
+ majorVersion = version.substring(2, 3);
+ } else {
+ Pattern majorVersion9PlusPattern = Pattern.compile("(\\d+).*");
+ Matcher m = majorVersion9PlusPattern.matcher(version);
+ if (m.find()) {
+ majorVersion = m.group(1);
+ } else {
+ throw new IllegalArgumentException("Could not detect major version of " + version);
+ }
+ }
+ return Integer.parseInt(majorVersion);
+ }
Review comment:
Thanks, that sounds like a good way forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a change in pull request #5062: MINIFI-554: Add OSUtils class to MiNiFi to get PID correctly
Posted by GitBox <gi...@apache.org>.
mattyb149 commented on a change in pull request #5062:
URL: https://github.com/apache/nifi/pull/5062#discussion_r628281726
##########
File path: minifi/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/OSUtils.java
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.util;
+
+import com.sun.jna.Pointer;
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.WinNT;
+import org.slf4j.Logger;
+
+import java.lang.reflect.Field;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * OS specific utilities with generic method interfaces
+ */
+public final class OSUtils {
+ /**
+ * @param process MiNiFi Process Reference
+ * @param logger Logger Reference for Debug
+ * @return Returns pid or null in-case pid could not be determined
+ * This method takes {@link Process} and {@link Logger} and returns
+ * the platform specific ProcessId for Unix like systems or Handle for Win32 Systems, a.k.a <b>pid</b>
+ * In-case it fails to determine the pid, it will return Null.
+ * Purpose for the Logger is to log any interaction for debugging.
+ */
+ public static Long getProcessId(final Process process, final Logger logger) {
+ /*
+ * NIFI-5175: NiFi built with Java 1.8 and running on Java 9. Reflectively invoke Process.pid() on
+ * the given process instance to get the PID of this Java process. Reflection is required in this scenario
+ * due to NiFi being compiled on Java 1.8, which does not have the Process API improvements available in
+ * Java 9.
+ *
+ * Otherwise, if NiFi is running on Java 1.8, attempt to get PID using capabilities available on Java 1.8.
+ *
+ * TODO: When minimum Java version updated to Java 9+, this class should be removed with the addition
+ * of the pid method to the Process API.
+ */
+ Long pid = null;
+ try {
+ // Get Process.pid() interface method to avoid illegal reflective access
+ final Method pidMethod = Process.class.getDeclaredMethod("pid");
+ final Object pidNumber = pidMethod.invoke(process);
+ if (pidNumber instanceof Long) {
+ pid = (Long) pidNumber;
+ }
+ } catch (final NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
+ final String processClassName = process.getClass().getName();
+ if (processClassName.equals("java.lang.UNIXProcess")) {
+ pid = getUnixPid(process, logger);
+ } else if (processClassName.equals("java.lang.Win32Process")
+ || processClassName.equals("java.lang.ProcessImpl")) {
+ pid = getWindowsProcessId(process, logger);
+ } else {
+ logger.debug("Failed to determine Process ID from [{}]: {}", processClassName, e.getMessage());
+ }
+ }
+
+ return pid;
+ }
+
+ /**
+ * Returns the major version parsed from the provided Java version string (e.g. {@code "1.8.0.231"} -> {@code 8}).
+ *
+ * @param version the Java version string
+ * @return the major version as an int
+ */
+ public static int parseJavaVersion(final String version) {
+ String majorVersion;
+ if (version.startsWith("1.")) {
+ majorVersion = version.substring(2, 3);
+ } else {
+ Pattern majorVersion9PlusPattern = Pattern.compile("(\\d+).*");
+ Matcher m = majorVersion9PlusPattern.matcher(version);
+ if (m.find()) {
+ majorVersion = m.group(1);
+ } else {
+ throw new IllegalArgumentException("Could not detect major version of " + version);
+ }
+ }
+ return Integer.parseInt(majorVersion);
+ }
Review comment:
I waffled on whether to make a new module, but I think you're right. I can create a "nifi-bootstrap-utils" module and just include this for now (and share it among nifi-bootstrap and minifi-bootstrap). That way the code is in one place to maintain, but doesn't add any footprint as it would if it were in nifi-utils.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5062: MINIFI-554: Add OSUtils class to MiNiFi to get PID correctly
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on a change in pull request #5062:
URL: https://github.com/apache/nifi/pull/5062#discussion_r628220809
##########
File path: minifi/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/OSUtils.java
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.util;
+
+import com.sun.jna.Pointer;
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.WinNT;
+import org.slf4j.Logger;
+
+import java.lang.reflect.Field;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * OS specific utilities with generic method interfaces
+ */
+public final class OSUtils {
+ /**
+ * @param process MiNiFi Process Reference
+ * @param logger Logger Reference for Debug
+ * @return Returns pid or null in-case pid could not be determined
+ * This method takes {@link Process} and {@link Logger} and returns
+ * the platform specific ProcessId for Unix like systems or Handle for Win32 Systems, a.k.a <b>pid</b>
+ * In-case it fails to determine the pid, it will return Null.
+ * Purpose for the Logger is to log any interaction for debugging.
+ */
+ public static Long getProcessId(final Process process, final Logger logger) {
+ /*
+ * NIFI-5175: NiFi built with Java 1.8 and running on Java 9. Reflectively invoke Process.pid() on
+ * the given process instance to get the PID of this Java process. Reflection is required in this scenario
+ * due to NiFi being compiled on Java 1.8, which does not have the Process API improvements available in
+ * Java 9.
+ *
+ * Otherwise, if NiFi is running on Java 1.8, attempt to get PID using capabilities available on Java 1.8.
+ *
+ * TODO: When minimum Java version updated to Java 9+, this class should be removed with the addition
+ * of the pid method to the Process API.
+ */
+ Long pid = null;
+ try {
+ // Get Process.pid() interface method to avoid illegal reflective access
+ final Method pidMethod = Process.class.getDeclaredMethod("pid");
+ final Object pidNumber = pidMethod.invoke(process);
+ if (pidNumber instanceof Long) {
+ pid = (Long) pidNumber;
+ }
+ } catch (final NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
+ final String processClassName = process.getClass().getName();
+ if (processClassName.equals("java.lang.UNIXProcess")) {
+ pid = getUnixPid(process, logger);
+ } else if (processClassName.equals("java.lang.Win32Process")
+ || processClassName.equals("java.lang.ProcessImpl")) {
+ pid = getWindowsProcessId(process, logger);
+ } else {
+ logger.debug("Failed to determine Process ID from [{}]: {}", processClassName, e.getMessage());
+ }
+ }
+
+ return pid;
+ }
+
+ /**
+ * Returns the major version parsed from the provided Java version string (e.g. {@code "1.8.0.231"} -> {@code 8}).
+ *
+ * @param version the Java version string
+ * @return the major version as an int
+ */
+ public static int parseJavaVersion(final String version) {
+ String majorVersion;
+ if (version.startsWith("1.")) {
+ majorVersion = version.substring(2, 3);
+ } else {
+ Pattern majorVersion9PlusPattern = Pattern.compile("(\\d+).*");
+ Matcher m = majorVersion9PlusPattern.matcher(version);
+ if (m.find()) {
+ majorVersion = m.group(1);
+ } else {
+ throw new IllegalArgumentException("Could not detect major version of " + version);
+ }
+ }
+ return Integer.parseInt(majorVersion);
+ }
Review comment:
This method can be removed since it is not used for Process ID determination.
##########
File path: minifi/minifi-bootstrap/src/test/java/org/apache/nifi/minifi/bootstrap/util/OSUtilsTest.java
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.minifi.bootstrap.util;
+
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+
+public class OSUtilsTest {
+
+ @Test
+ public void testGetPid() throws IOException {
+ final ProcessBuilder builder = new ProcessBuilder();
+ final Process process = builder.command("java").start();
+ final Logger logger = LoggerFactory.getLogger("testing");
+ final Long pid = OSUtils.getProcessId(process, logger);
+ process.destroy();
+ assertNotNull("Process ID not found", pid);
+ }
+
+ @Test
+ public void testParseJavaVersion8() {
+ final String[] versions = new String[] { "1.8", "1.8.0", "1.8.0_100" };
+ for (final String version : versions) {
+ assertEquals(8, OSUtils.parseJavaVersion(version));
+ }
+ }
+
+ @Test
+ public void testParseJavaVersion11() {
+ final String[] versions = new String[] { "11", "11.0", "11.0.11" };
+ for (final String version : versions) {
+ assertEquals(11, OSUtils.parseJavaVersion(version));
+ }
+ }
Review comment:
These two test methods can be removed along with `OSUtils.parseJavaVersion`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #5062: MINIFI-554: Move OSUtils to nifi-bootstrap-utils so other assemblies like MiNiFi can use it
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #5062:
URL: https://github.com/apache/nifi/pull/5062
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org