You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by mt...@apache.org on 2019/08/19 22:09:17 UTC
[nifi] branch master updated: NIFI-6502 RecordPath padding
functions: - PadLeft(label, desiredLength,
paddingChar) prepends the paddingChar (or the default value '_' if a
paddingChar is not provided) to the label string until desiredLength is
reached. - PadRight(label, desiredLength,
paddingChar) appends the paddingChar (or the default value '_' if a
paddingChar is not provided) to the label string until desiredLength is
reached. Added Apache license disclaimers checkstyle fixes Replaced
functional interface with abstr [...]
This is an automated email from the ASF dual-hosted git repository.
mthomsen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/master by this push:
new 81136bf NIFI-6502 RecordPath padding functions: - PadLeft(label, desiredLength, paddingChar) prepends the paddingChar (or the default value '_' if a paddingChar is not provided) to the label string until desiredLength is reached. - PadRight(label, desiredLength, paddingChar) appends the paddingChar (or the default value '_' if a paddingChar is not provided) to the label string until desiredLength is reached. Added Apache license disclaimers checkstyle fixes Replaced functional i [...]
81136bf is described below
commit 81136bf5502458521eb4841be7e45b3f6223241b
Author: Alessandro D'Armiento <al...@agilelab.it>
AuthorDate: Mon Jul 29 18:35:38 2019 +0200
NIFI-6502 RecordPath padding functions:
- PadLeft(label, desiredLength, paddingChar) prepends the paddingChar (or the default value '_' if a paddingChar is not provided) to the label string until desiredLength is reached.
- PadRight(label, desiredLength, paddingChar) appends the paddingChar (or the default value '_' if a paddingChar is not provided) to the label string until desiredLength is reached.
Added Apache license disclaimers
checkstyle fixes
Replaced functional interface with abstract method
Fixes and Further test cases:
- Returns null if the input string is null
- Returns a string full of padding if the input string is empty
wip support padding string
In order to be consistent with the feature introduced in #3615, RecordPath padLeft and padRight supports String padding.
Since nifi-record-path doesn't have the Apache Commons StringUtils dependency, the padding methods have been added to the available NiFi commons StringUtils class.
NIFI-6502 Updated top level NOTICE file to include citation for code borrowed from commons-lang3.
borrowed pad methods from lang3 StringUtils
Replaced `PadLeft` and `PadRight` record path functions with borrowed Apache Lang `StringUtils` padding methods and updated `nifi-assembly/NOTICE` accordingly
This closes #3613
Signed-off-by: Mike Thomsen <mt...@apache.org>
---
NOTICE | 5 +
nifi-assembly/NOTICE | 5 +
.../java/org/apache/nifi/util/StringUtils.java | 398 ++++++++++++++++++++-
.../java/org/apache/nifi/util/StringUtilsTest.java | 29 +-
.../apache/nifi/record/path/functions/PadLeft.java | 42 +++
.../nifi/record/path/functions/PadRight.java | 42 +++
.../apache/nifi/record/path/functions/Padding.java | 106 ++++++
.../nifi/record/path/paths/RecordPathCompiler.java | 24 ++
.../apache/nifi/record/path/TestRecordPath.java | 51 +++
nifi-docs/src/main/asciidoc/record-path-guide.adoc | 61 ++++
10 files changed, 739 insertions(+), 24 deletions(-)
diff --git a/NOTICE b/NOTICE
index 94a3b97..8ca7ff7 100644
--- a/NOTICE
+++ b/NOTICE
@@ -7,6 +7,11 @@ The Apache Software Foundation (http://www.apache.org/).
This product includes the following work from the Apache Hadoop project under Apache Software License V2:
BoundedByteArrayOutputStream.java adapted to SoftLimitBoundedByteArrayOutputStream.java
+This product includes derived works from Apache Commons Lang (ASLv2 licensed)
+ Copyright 2019 The Apache Software Foundation
+ The derived work is adapted from the class StringUtils:
+ https://github.com/apache/commons-lang/blob/465e6becd8beda9569fbdc9ff44fac7dc3c5d1c6/src/main/java/org/apache/commons/lang3/StringUtils.java
+
This product includes derived works from the Apache Software License V2 library python-evtx (https://github.com/williballenthin/python-evtx)
Copyright 2012, 2013 Willi Ballenthin william.ballenthin@mandiant.com
while at Mandiant http://www.mandiant.com
diff --git a/nifi-assembly/NOTICE b/nifi-assembly/NOTICE
index 2dbad99..d62f0bf 100644
--- a/nifi-assembly/NOTICE
+++ b/nifi-assembly/NOTICE
@@ -7,6 +7,11 @@ The Apache Software Foundation (http://www.apache.org/).
This product includes the following work from the Apache Hadoop project under Apache Software License V2:
BoundedByteArrayOutputStream.java adapted to SoftLimitBoundedByteArrayOutputStream.java
+This product includes derived works from Apache Commons Lang (ASLv2 licensed)
+ Copyright 2019 The Apache Software Foundation
+ The derived work is adapted from the class StringUtils:
+ https://github.com/apache/commons-lang/blob/465e6becd8beda9569fbdc9ff44fac7dc3c5d1c6/src/main/java/org/apache/commons/lang3/StringUtils.java
+
This product includes derived works from the Apache Software License V2 library python-evtx (https://github.com/williballenthin/python-evtx)
Copyright 2012, 2013 Willi Ballenthin william.ballenthin@mandiant.com
while at Mandiant http://www.mandiant.com
diff --git a/nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/StringUtils.java b/nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/StringUtils.java
index 37408f7..87f12a2 100644
--- a/nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/StringUtils.java
+++ b/nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/StringUtils.java
@@ -78,25 +78,395 @@ public class StringUtils {
return sb.toString().substring(0, sb.lastIndexOf(delimiter));
}
- public static String padLeft(final String source, int length, char padding) {
- if (source != null) {
- StringBuilder sb = new StringBuilder(source).reverse();
- while (sb.length() < length) {
- sb.append(padding);
+ /**
+ * <p>The maximum size to which the padding constant(s) can expand.</p>
+ */
+ private static final int PAD_LIMIT = 8192;
+
+ /**
+ * A String for a space character.
+ *
+ * @since 3.2
+ */
+ public static final String SPACE = " ";
+
+ /**
+ * <p>Left pad a String with spaces (' ').</p>
+ *
+ * <p>The String is padded to the size of {@code size}.</p>
+ *
+ * <pre>
+ * StringUtils.leftPad(null, *) = null
+ * StringUtils.leftPad("", 3) = " "
+ * StringUtils.leftPad("bat", 3) = "bat"
+ * StringUtils.leftPad("bat", 5) = " bat"
+ * StringUtils.leftPad("bat", 1) = "bat"
+ * StringUtils.leftPad("bat", -1) = "bat"
+ * </pre>
+ *
+ * @param str the String to pad out, may be null
+ * @param size the size to pad to
+ * @return left padded String or original String if no padding is necessary,
+ * {@code null} if null String input
+ */
+ public static String leftPad(final String str, final int size) {
+ return leftPad(str, size, ' ');
+ }
+
+ /**
+ * <p>Left pad a String with a specified character.</p>
+ *
+ * <p>Pad to a size of {@code size}.</p>
+ *
+ * <pre>
+ * StringUtils.leftPad(null, *, *) = null
+ * StringUtils.leftPad("", 3, 'z') = "zzz"
+ * StringUtils.leftPad("bat", 3, 'z') = "bat"
+ * StringUtils.leftPad("bat", 5, 'z') = "zzbat"
+ * StringUtils.leftPad("bat", 1, 'z') = "bat"
+ * StringUtils.leftPad("bat", -1, 'z') = "bat"
+ * </pre>
+ *
+ * @param str the String to pad out, may be null
+ * @param size the size to pad to
+ * @param padChar the character to pad with
+ * @return left padded String or original String if no padding is necessary,
+ * {@code null} if null String input
+ * @since 2.0
+ */
+ public static String leftPad(final String str, final int size, final char padChar) {
+ if (str == null) {
+ return null;
+ }
+ final int pads = size - str.length();
+ if (pads <= 0) {
+ return str; // returns original String when possible
+ }
+ if (pads > PAD_LIMIT) {
+ return leftPad(str, size, String.valueOf(padChar));
+ }
+ return repeat(padChar, pads).concat(str);
+ }
+
+ /**
+ * <p>Left pad a String with a specified String.</p>
+ *
+ * <p>Pad to a size of {@code size}.</p>
+ *
+ * <pre>
+ * StringUtils.leftPad(null, *, *) = null
+ * StringUtils.leftPad("", 3, "z") = "zzz"
+ * StringUtils.leftPad("bat", 3, "yz") = "bat"
+ * StringUtils.leftPad("bat", 5, "yz") = "yzbat"
+ * StringUtils.leftPad("bat", 8, "yz") = "yzyzybat"
+ * StringUtils.leftPad("bat", 1, "yz") = "bat"
+ * StringUtils.leftPad("bat", -1, "yz") = "bat"
+ * StringUtils.leftPad("bat", 5, null) = " bat"
+ * StringUtils.leftPad("bat", 5, "") = " bat"
+ * </pre>
+ *
+ * @param str the String to pad out, may be null
+ * @param size the size to pad to
+ * @param padStr the String to pad with, null or empty treated as single space
+ * @return left padded String or original String if no padding is necessary,
+ * {@code null} if null String input
+ */
+ public static String leftPad(final String str, final int size, String padStr) {
+ if (str == null) {
+ return null;
+ }
+ if (isEmpty(padStr)) {
+ padStr = SPACE;
+ }
+ final int padLen = padStr.length();
+ final int strLen = str.length();
+ final int pads = size - strLen;
+ if (pads <= 0) {
+ return str; // returns original String when possible
+ }
+ if (padLen == 1 && pads <= PAD_LIMIT) {
+ return leftPad(str, size, padStr.charAt(0));
+ }
+
+ if (pads == padLen) {
+ return padStr.concat(str);
+ } else if (pads < padLen) {
+ return padStr.substring(0, pads).concat(str);
+ } else {
+ final char[] padding = new char[pads];
+ final char[] padChars = padStr.toCharArray();
+ for (int i = 0; i < pads; i++) {
+ padding[i] = padChars[i % padLen];
}
- return sb.reverse().toString();
+ return new String(padding).concat(str);
}
- return null;
}
- public static String padRight(final String source, int length, char padding) {
- if (source != null) {
- StringBuilder sb = new StringBuilder(source);
- while (sb.length() < length) {
- sb.append(padding);
+ /**
+ * <p>Right pad a String with spaces (' ').</p>
+ *
+ * <p>The String is padded to the size of {@code size}.</p>
+ *
+ * <pre>
+ * StringUtils.rightPad(null, *) = null
+ * StringUtils.rightPad("", 3) = " "
+ * StringUtils.rightPad("bat", 3) = "bat"
+ * StringUtils.rightPad("bat", 5) = "bat "
+ * StringUtils.rightPad("bat", 1) = "bat"
+ * StringUtils.rightPad("bat", -1) = "bat"
+ * </pre>
+ *
+ * @param str the String to pad out, may be null
+ * @param size the size to pad to
+ * @return right padded String or original String if no padding is necessary,
+ * {@code null} if null String input
+ */
+ public static String rightPad(final String str, final int size) {
+ return rightPad(str, size, ' ');
+ }
+
+ /**
+ * <p>Right pad a String with a specified character.</p>
+ *
+ * <p>The String is padded to the size of {@code size}.</p>
+ *
+ * <pre>
+ * StringUtils.rightPad(null, *, *) = null
+ * StringUtils.rightPad("", 3, 'z') = "zzz"
+ * StringUtils.rightPad("bat", 3, 'z') = "bat"
+ * StringUtils.rightPad("bat", 5, 'z') = "batzz"
+ * StringUtils.rightPad("bat", 1, 'z') = "bat"
+ * StringUtils.rightPad("bat", -1, 'z') = "bat"
+ * </pre>
+ *
+ * @param str the String to pad out, may be null
+ * @param size the size to pad to
+ * @param padChar the character to pad with
+ * @return right padded String or original String if no padding is necessary,
+ * {@code null} if null String input
+ * @since 2.0
+ */
+ public static String rightPad(final String str, final int size, final char padChar) {
+ if (str == null) {
+ return null;
+ }
+ final int pads = size - str.length();
+ if (pads <= 0) {
+ return str; // returns original String when possible
+ }
+ if (pads > PAD_LIMIT) {
+ return rightPad(str, size, String.valueOf(padChar));
+ }
+ return str.concat(repeat(padChar, pads));
+ }
+
+ /**
+ * <p>Right pad a String with a specified String.</p>
+ *
+ * <p>The String is padded to the size of {@code size}.</p>
+ *
+ * <pre>
+ * StringUtils.rightPad(null, *, *) = null
+ * StringUtils.rightPad("", 3, "z") = "zzz"
+ * StringUtils.rightPad("bat", 3, "yz") = "bat"
+ * StringUtils.rightPad("bat", 5, "yz") = "batyz"
+ * StringUtils.rightPad("bat", 8, "yz") = "batyzyzy"
+ * StringUtils.rightPad("bat", 1, "yz") = "bat"
+ * StringUtils.rightPad("bat", -1, "yz") = "bat"
+ * StringUtils.rightPad("bat", 5, null) = "bat "
+ * StringUtils.rightPad("bat", 5, "") = "bat "
+ * </pre>
+ *
+ * @param str the String to pad out, may be null
+ * @param size the size to pad to
+ * @param padStr the String to pad with, null or empty treated as single space
+ * @return right padded String or original String if no padding is necessary,
+ * {@code null} if null String input
+ */
+ public static String rightPad(final String str, final int size, String padStr) {
+ if (str == null) {
+ return null;
+ }
+ if (isEmpty(padStr)) {
+ padStr = SPACE;
+ }
+ final int padLen = padStr.length();
+ final int strLen = str.length();
+ final int pads = size - strLen;
+ if (pads <= 0) {
+ return str; // returns original String when possible
+ }
+ if (padLen == 1 && pads <= PAD_LIMIT) {
+ return rightPad(str, size, padStr.charAt(0));
+ }
+
+ if (pads == padLen) {
+ return str.concat(padStr);
+ } else if (pads < padLen) {
+ return str.concat(padStr.substring(0, pads));
+ } else {
+ final char[] padding = new char[pads];
+ final char[] padChars = padStr.toCharArray();
+ for (int i = 0; i < pads; i++) {
+ padding[i] = padChars[i % padLen];
}
- return sb.toString();
+ return str.concat(new String(padding));
}
- return null;
}
+
+ /**
+ * <p>Returns padding using the specified delimiter repeated
+ * to a given length.</p>
+ *
+ * <pre>
+ * StringUtils.repeat('e', 0) = ""
+ * StringUtils.repeat('e', 3) = "eee"
+ * StringUtils.repeat('e', -2) = ""
+ * </pre>
+ *
+ * <p>Note: this method does not support padding with
+ * <a href="http://www.unicode.org/glossary/#supplementary_character">Unicode Supplementary Characters</a>
+ * as they require a pair of {@code char}s to be represented.
+ * If you are needing to support full I18N of your applications
+ * consider using {@link #repeat(String, int)} instead.
+ * </p>
+ *
+ * @param ch character to repeat
+ * @param repeat number of times to repeat char, negative treated as zero
+ * @return String with repeated character
+ * @see #repeat(String, int)
+ */
+ public static String repeat(final char ch, final int repeat) {
+ if (repeat <= 0) {
+ return EMPTY;
+ }
+ final char[] buf = new char[repeat];
+ for (int i = repeat - 1; i >= 0; i--) {
+ buf[i] = ch;
+ }
+ return new String(buf);
+ }
+
+ // Padding
+ //-----------------------------------------------------------------------
+ /**
+ * <p>Repeat a String {@code repeat} times to form a
+ * new String.</p>
+ *
+ * <pre>
+ * StringUtils.repeat(null, 2) = null
+ * StringUtils.repeat("", 0) = ""
+ * StringUtils.repeat("", 2) = ""
+ * StringUtils.repeat("a", 3) = "aaa"
+ * StringUtils.repeat("ab", 2) = "abab"
+ * StringUtils.repeat("a", -2) = ""
+ * </pre>
+ *
+ * @param str the String to repeat, may be null
+ * @param repeat number of times to repeat str, negative treated as zero
+ * @return a new String consisting of the original String repeated,
+ * {@code null} if null String input
+ */
+ public static String repeat(final String str, final int repeat) {
+ // Performance tuned for 2.0 (JDK1.4)
+
+ if (str == null) {
+ return null;
+ }
+ if (repeat <= 0) {
+ return EMPTY;
+ }
+ final int inputLength = str.length();
+ if (repeat == 1 || inputLength == 0) {
+ return str;
+ }
+ if (inputLength == 1 && repeat <= PAD_LIMIT) {
+ return repeat(str.charAt(0), repeat);
+ }
+
+ final int outputLength = inputLength * repeat;
+ switch (inputLength) {
+ case 1 :
+ return repeat(str.charAt(0), repeat);
+ case 2 :
+ final char ch0 = str.charAt(0);
+ final char ch1 = str.charAt(1);
+ final char[] output2 = new char[outputLength];
+ for (int i = repeat * 2 - 2; i >= 0; i--, i--) {
+ output2[i] = ch0;
+ output2[i + 1] = ch1;
+ }
+ return new String(output2);
+ default :
+ final StringBuilder buf = new StringBuilder(outputLength);
+ for (int i = 0; i < repeat; i++) {
+ buf.append(str);
+ }
+ return buf.toString();
+ }
+ }
+
+ /**
+ * <p>Repeat a String {@code repeat} times to form a
+ * new String, with a String separator injected each time. </p>
+ *
+ * <pre>
+ * StringUtils.repeat(null, null, 2) = null
+ * StringUtils.repeat(null, "x", 2) = null
+ * StringUtils.repeat("", null, 0) = ""
+ * StringUtils.repeat("", "", 2) = ""
+ * StringUtils.repeat("", "x", 3) = "xxx"
+ * StringUtils.repeat("?", ", ", 3) = "?, ?, ?"
+ * </pre>
+ *
+ * @param str the String to repeat, may be null
+ * @param separator the String to inject, may be null
+ * @param repeat number of times to repeat str, negative treated as zero
+ * @return a new String consisting of the original String repeated,
+ * {@code null} if null String input
+ * @since 2.5
+ */
+ public static String repeat(final String str, final String separator, final int repeat) {
+ if (str == null || separator == null) {
+ return repeat(str, repeat);
+ }
+ // given that repeat(String, int) is quite optimized, better to rely on it than try and splice this into it
+ final String result = repeat(str + separator, repeat);
+ return removeEnd(result, separator);
+ }
+
+ /**
+ * <p>Removes a substring only if it is at the end of a source string,
+ * otherwise returns the source string.</p>
+ *
+ * <p>A {@code null} source string will return {@code null}.
+ * An empty ("") source string will return the empty string.
+ * A {@code null} search string will return the source string.</p>
+ *
+ * <pre>
+ * StringUtils.removeEnd(null, *) = null
+ * StringUtils.removeEnd("", *) = ""
+ * StringUtils.removeEnd(*, null) = *
+ * StringUtils.removeEnd("www.domain.com", ".com.") = "www.domain.com"
+ * StringUtils.removeEnd("www.domain.com", ".com") = "www.domain"
+ * StringUtils.removeEnd("www.domain.com", "domain") = "www.domain.com"
+ * StringUtils.removeEnd("abc", "") = "abc"
+ * </pre>
+ *
+ * @param str the source String to search, may be null
+ * @param remove the String to search for and remove, may be null
+ * @return the substring with the string removed if found,
+ * {@code null} if null String input
+ * @since 2.1
+ */
+ public static String removeEnd(final String str, final String remove) {
+ if (isEmpty(str) || isEmpty(remove)) {
+ return str;
+ }
+ if (str.endsWith(remove)) {
+ return str.substring(0, str.length() - remove.length());
+ }
+ return str;
+ }
+
}
diff --git a/nifi-commons/nifi-properties/src/test/java/org/apache/nifi/util/StringUtilsTest.java b/nifi-commons/nifi-properties/src/test/java/org/apache/nifi/util/StringUtilsTest.java
index 5c689c6..1690bab 100644
--- a/nifi-commons/nifi-properties/src/test/java/org/apache/nifi/util/StringUtilsTest.java
+++ b/nifi-commons/nifi-properties/src/test/java/org/apache/nifi/util/StringUtilsTest.java
@@ -48,20 +48,29 @@ public class StringUtilsTest {
@Test
public void testPadRight() {
- assertEquals("sample", StringUtils.padRight("sample", 0, '0'));
- assertEquals("sample0000", StringUtils.padRight("sample", 10, '0'));
- assertEquals("0000000000", StringUtils.padRight("", 10, '0'));
-
- assertNull(StringUtils.padRight(null, 0, '0'));
+ assertEquals("sample", StringUtils.rightPad("sample", 0, '0'));
+ assertEquals("sample", StringUtils.rightPad("sample", -5, '0'));
+ assertEquals("sample0000", StringUtils.rightPad("sample", 10, '0'));
+ assertEquals("0000000000", StringUtils.rightPad("", 10, '0'));
+ assertEquals("samplexyxy", StringUtils.rightPad("sample", 10, "xy")); // multiple pads
+ assertEquals("samplexy", StringUtils.rightPad("sample", 8, "xyz")); // less than 1 pad
+ assertEquals("samplexy", StringUtils.rightPad("sample", 8, "xy")); // exactly 1 pad
+ assertEquals("sample ", StringUtils.rightPad("sample", 10, null)); // null pad
+
+ assertNull(StringUtils.rightPad(null, 0, '0'));
}
@Test
public void testPadLeft() {
- assertEquals("sample", StringUtils.padLeft("sample", 0, '0'));
- assertEquals("0000sample", StringUtils.padLeft("sample", 10, '0'));
- assertEquals("0000000000", StringUtils.padLeft("", 10, '0'));
-
- assertNull(StringUtils.padLeft(null, 0, '0'));
+ assertEquals("sample", StringUtils.leftPad("sample", 0, '0'));
+ assertEquals("0000sample", StringUtils.leftPad("sample", 10, '0'));
+ assertEquals("0000000000", StringUtils.leftPad("", 10, '0'));
+ assertEquals("xyxysample", StringUtils.leftPad("sample", 10, "xy")); // multiple pads
+ assertEquals("xysample", StringUtils.leftPad("sample", 8, "xyz")); // less than 1 pad
+ assertEquals("xysample", StringUtils.leftPad("sample", 8, "xy")); // exactly 1 pad
+ assertEquals(" sample", StringUtils.leftPad("sample", 10, null)); // null pad
+
+ assertNull(StringUtils.leftPad(null, 0, '0'));
}
@Test
diff --git a/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/PadLeft.java b/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/PadLeft.java
new file mode 100644
index 0000000..67087a3
--- /dev/null
+++ b/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/PadLeft.java
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.record.path.functions;
+
+import org.apache.nifi.record.path.paths.RecordPathSegment;
+import org.apache.nifi.util.StringUtils;
+
+public class PadLeft extends Padding {
+
+ public PadLeft( final RecordPathSegment inputStringPath,
+ final RecordPathSegment desiredLengthPath,
+ final RecordPathSegment paddingStringPath,
+ final boolean absolute) {
+ super("padLeft", null, inputStringPath, desiredLengthPath, paddingStringPath, absolute);
+ }
+
+ public PadLeft( final RecordPathSegment inputStringPath,
+ final RecordPathSegment desiredLengthPath,
+ final boolean absolute) {
+ super("padLeft", null, inputStringPath, desiredLengthPath, null, absolute);
+ }
+
+ @Override
+ protected String doPad(String inputString, int desiredLength, String pad) {
+ return StringUtils.leftPad(inputString, desiredLength, pad);
+ }
+}
diff --git a/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/PadRight.java b/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/PadRight.java
new file mode 100644
index 0000000..f45d491
--- /dev/null
+++ b/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/PadRight.java
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.record.path.functions;
+
+import org.apache.nifi.record.path.paths.RecordPathSegment;
+import org.apache.nifi.util.StringUtils;
+
+public class PadRight extends Padding {
+
+ public PadRight( final RecordPathSegment inputStringPath,
+ final RecordPathSegment desiredLengthPath,
+ final RecordPathSegment paddingStringPath,
+ final boolean absolute) {
+ super("padRight", null, inputStringPath, desiredLengthPath, paddingStringPath, absolute);
+ }
+
+ public PadRight( final RecordPathSegment inputStringPath,
+ final RecordPathSegment desiredLengthPath,
+ final boolean absolute) {
+ super("padRight", null, inputStringPath, desiredLengthPath, null, absolute);
+ }
+
+ @Override
+ protected String doPad(String inputString, int desiredLength, String pad) {
+ return StringUtils.rightPad(inputString, desiredLength, pad);
+ }
+}
diff --git a/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/Padding.java b/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/Padding.java
new file mode 100644
index 0000000..f0d445b
--- /dev/null
+++ b/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/functions/Padding.java
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.record.path.functions;
+
+import org.apache.nifi.record.path.FieldValue;
+import org.apache.nifi.record.path.RecordPathEvaluationContext;
+import org.apache.nifi.record.path.StandardFieldValue;
+import org.apache.nifi.record.path.paths.RecordPathSegment;
+import org.apache.nifi.record.path.util.RecordPathUtils;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+
+import java.util.Optional;
+import java.util.OptionalInt;
+import java.util.stream.Stream;
+
+abstract class Padding extends RecordPathSegment {
+
+ public final static String DEFAULT_PADDING_STRING = "_";
+
+ private RecordPathSegment paddingStringPath;
+ private RecordPathSegment inputStringPath;
+ private RecordPathSegment desiredLengthPath;
+
+ Padding( final String path,
+ final RecordPathSegment parentPath,
+ final RecordPathSegment inputStringPath,
+ final RecordPathSegment desiredLengthPath,
+ final RecordPathSegment paddingStringPath,
+ final boolean absolute) {
+
+ super(path, parentPath, absolute);
+ this.paddingStringPath = paddingStringPath;
+ this.inputStringPath = inputStringPath;
+ this.desiredLengthPath = desiredLengthPath;
+ }
+
+ public Stream<FieldValue> evaluate(RecordPathEvaluationContext context) {
+ String pad = getPaddingString(context);
+
+ final Stream<FieldValue> evaluatedStr = inputStringPath.evaluate(context);
+ return evaluatedStr.map(fv -> {
+
+ final OptionalInt desiredLengthOpt = getDesiredLength(context);
+ if (!desiredLengthOpt.isPresent()) {
+ return new StandardFieldValue("", fv.getField(), fv.getParent().orElse(null));
+ }
+
+ int desiredLength = desiredLengthOpt.getAsInt();
+ final String value = DataTypeUtils.toString(fv.getValue(), (String) null);
+ return new StandardFieldValue(doPad(value, desiredLength, pad), fv.getField(), fv.getParent().orElse(null));
+ });
+ }
+
+ protected abstract String doPad(String inputString, int desiredLength, String pad);
+
+ private OptionalInt getDesiredLength(RecordPathEvaluationContext context) {
+
+ Optional<FieldValue> lengthOption = desiredLengthPath.evaluate(context).findFirst();
+
+ if (!lengthOption.isPresent()) {
+ return OptionalInt.empty();
+ }
+
+ final FieldValue fieldValue = lengthOption.get();
+ final Object length = fieldValue.getValue();
+ if (!DataTypeUtils.isIntegerTypeCompatible(length)) {
+ return OptionalInt.empty();
+ }
+
+ final String fieldName;
+ final RecordField field = fieldValue.getField();
+ fieldName = field == null ? "<Unknown Field>" : field.getFieldName();
+
+ return OptionalInt.of(DataTypeUtils.toInteger(length, fieldName));
+ }
+
+ private String getPaddingString(RecordPathEvaluationContext context){
+
+ if (null == paddingStringPath) {
+ return DEFAULT_PADDING_STRING;
+ }
+
+ String padStr = RecordPathUtils.getFirstStringValue(paddingStringPath, context);
+
+ if (null != padStr && !padStr.isEmpty()){
+ return padStr;
+ }
+ return DEFAULT_PADDING_STRING;
+ }
+}
diff --git a/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/paths/RecordPathCompiler.java b/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/paths/RecordPathCompiler.java
index 11ff55d..b3fea23 100644
--- a/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/paths/RecordPathCompiler.java
+++ b/nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/paths/RecordPathCompiler.java
@@ -69,6 +69,8 @@ import org.apache.nifi.record.path.functions.Base64Encode;
import org.apache.nifi.record.path.functions.Concat;
import org.apache.nifi.record.path.functions.Format;
import org.apache.nifi.record.path.functions.FieldName;
+import org.apache.nifi.record.path.functions.PadLeft;
+import org.apache.nifi.record.path.functions.PadRight;
import org.apache.nifi.record.path.functions.Replace;
import org.apache.nifi.record.path.functions.ReplaceNull;
import org.apache.nifi.record.path.functions.ReplaceRegex;
@@ -303,6 +305,28 @@ public class RecordPathCompiler {
final RecordPathSegment[] args = getArgPaths(argumentListTree, 1, functionName, absolute);
return new Base64Decode(args[0], absolute);
}
+ case "padLeft": {
+ final int numArgs = argumentListTree.getChildCount();
+
+ if (numArgs == 2) {
+ final RecordPathSegment[] args = getArgPaths(argumentListTree, 2, functionName, absolute);
+ return new PadLeft(args[0], args[1], absolute);
+ } else {
+ final RecordPathSegment[] args = getArgPaths(argumentListTree, 3, functionName, absolute);
+ return new PadLeft(args[0], args[1], args[2], absolute);
+ }
+ }
+ case "padRight": {
+ final int numArgs = argumentListTree.getChildCount();
+
+ if (numArgs == 2) {
+ final RecordPathSegment[] args = getArgPaths(argumentListTree, 2, functionName, absolute);
+ return new PadRight(args[0], args[1], absolute);
+ } else {
+ final RecordPathSegment[] args = getArgPaths(argumentListTree, 3, functionName, absolute);
+ return new PadRight(args[0], args[1], args[2], absolute);
+ }
+ }
default: {
throw new RecordPathException("Invalid function call: The '" + functionName + "' function does not exist or can only "
+ "be used within a predicate, not as a standalone function");
diff --git a/nifi-commons/nifi-record-path/src/test/java/org/apache/nifi/record/path/TestRecordPath.java b/nifi-commons/nifi-record-path/src/test/java/org/apache/nifi/record/path/TestRecordPath.java
index 2f4abd9..0369c7c 100644
--- a/nifi-commons/nifi-record-path/src/test/java/org/apache/nifi/record/path/TestRecordPath.java
+++ b/nifi-commons/nifi-record-path/src/test/java/org/apache/nifi/record/path/TestRecordPath.java
@@ -46,6 +46,7 @@ import java.util.stream.IntStream;
import static org.junit.Assert.assertArrayEquals;
import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertTrue;
public class TestRecordPath {
@@ -1580,6 +1581,56 @@ public class TestRecordPath {
});
}
+ @Test
+ public void testPadLeft() {
+ final List<RecordField> fields = new ArrayList<>();
+ fields.add(new RecordField("someString", RecordFieldType.STRING.getDataType()));
+ fields.add(new RecordField("emptyString", RecordFieldType.STRING.getDataType()));
+ fields.add(new RecordField("nullString", RecordFieldType.STRING.getDataType()));
+ final RecordSchema schema = new SimpleRecordSchema(fields);
+
+ final Map<String, Object> values = new HashMap<>();
+ values.put("someString", "MyString");
+ values.put("emptyString", "");
+ values.put("nullString", null);
+ final Record record = new MapRecord(schema, values);
+
+ assertEquals("##MyString", RecordPath.compile("padLeft(/someString, 10, '#')").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("__MyString", RecordPath.compile("padLeft(/someString, 10)").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("MyString", RecordPath.compile("padLeft(/someString, 3)").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("MyString", RecordPath.compile("padLeft(/someString, -10)").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("@@@@@@@@@@", RecordPath.compile("padLeft(/emptyString, 10, '@')").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertNull(RecordPath.compile("padLeft(/nullString, 10, '@')").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("xyMyString", RecordPath.compile("padLeft(/someString, 10, \"xy\")").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("aVMyString", RecordPath.compile("padLeft(/someString, 10, \"aVeryLongPadding\")").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("fewfewfewfewMyString", RecordPath.compile("padLeft(/someString, 20, \"few\")").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ }
+
+ @Test
+ public void testPadRight() {
+ final List<RecordField> fields = new ArrayList<>();
+ fields.add(new RecordField("someString", RecordFieldType.STRING.getDataType()));
+ fields.add(new RecordField("emptyString", RecordFieldType.STRING.getDataType()));
+ fields.add(new RecordField("nullString", RecordFieldType.STRING.getDataType()));
+ final RecordSchema schema = new SimpleRecordSchema(fields);
+
+ final Map<String, Object> values = new HashMap<>();
+ values.put("someString", "MyString");
+ values.put("emptyString", "");
+ values.put("nullString", null);
+ final Record record = new MapRecord(schema, values);
+
+ assertEquals("MyString##", RecordPath.compile("padRight(/someString, 10, \"#\")").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("MyString__", RecordPath.compile("padRight(/someString, 10)").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("MyString", RecordPath.compile("padRight(/someString, 3)").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("MyString", RecordPath.compile("padRight(/someString, -10)").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("@@@@@@@@@@", RecordPath.compile("padRight(/emptyString, 10, '@')").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertNull(null, RecordPath.compile("padRight(/nullString, 10, '@')").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("MyStringxy", RecordPath.compile("padRight(/someString, 10, \"xy\")").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("MyStringaV", RecordPath.compile("padRight(/someString, 10, \"aVeryLongPadding\")").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ assertEquals("MyStringfewfewfewfew", RecordPath.compile("padRight(/someString, 20, \"few\")").evaluate(record).getSelectedFields().findFirst().get().getValue());
+ }
+
private List<RecordField> getDefaultFields() {
final List<RecordField> fields = new ArrayList<>();
fields.add(new RecordField("id", RecordFieldType.INT.getDataType()));
diff --git a/nifi-docs/src/main/asciidoc/record-path-guide.adoc b/nifi-docs/src/main/asciidoc/record-path-guide.adoc
index 976a35e..3778f33 100644
--- a/nifi-docs/src/main/asciidoc/record-path-guide.adoc
+++ b/nifi-docs/src/main/asciidoc/record-path-guide.adoc
@@ -778,6 +778,67 @@ The following record path expression would decode the String using Base64:
| `base64Decode(/name)` | John
|==========================================================
+=== PadLeft
+
+Prepends characters to the input String until it reaches the desired length.
+
+----
+{
+ "type": "record",
+ "name": "events",
+ "fields": [
+ { "name": "name", "type": "string" }
+ ]
+}
+----
+
+and a record such as:
+
+----
+{
+ "name" : "john smith"
+}
+----
+
+The following record path expression would prepend '@' characters to the input String:
+
+|==========================================================
+| RecordPath | Return value
+| `padLeft(/name, 15, '@')` | @@@@@john smith
+|==========================================================
+
+
+
+=== PadRight
+
+Appends characters to the input String until it reaches the desired length.
+
+----
+{
+ "type": "record",
+ "name": "events",
+ "fields": [
+ { "name": "name", "type": "string" }
+ ]
+}
+----
+
+and a record such as:
+
+----
+{
+ "name" : "john smith"
+}
+----
+
+The following record path expression would append '@' characters to the input String:
+
+|==========================================================
+| RecordPath | Return value
+| `padRight(/name, 15, '@')` | john smith@@@@@
+|==========================================================
+
+
[[filter_functions]]
== Filter Functions