You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Vinz2168 (via GitHub)" <gi...@apache.org> on 2026/10/08 14:22:22 UTC
[PR] NIFI-16443 Added Ignore Cells Beyond Header property to ExcelReader [nifi]
Vinz2168 opened a new pull request, #11783:
URL: https://github.com/apache/nifi/pull/11783
# Summary
[NIFI-16443](https://issues.apache.org/jira/browse/NIFI-16443)
Adds an optional `Ignore Cells Beyond Header` property to `ExcelReader`, visible only with the `Use Starting Row` schema access strategy. It defaults to `false`, so the current behavior is unchanged.
When enabled, cells beyond the last header column are ignored during schema inference instead of failing with `Row N has X cells, more than the expected Y number of field names`. Blank cells, whitespace-only strings and formulas evaluating to an empty string are ignored silently. When ignored cells contain a value, a single warning per sheet is logged with the number of affected rows and the first position, so data is never dropped silently.
Only the rows evaluated according to the `Row Evaluation Strategy` are checked. `ExcelRecordReader` is unchanged, as it already handles cells beyond the schema fields.
# Tracking
### Issue Tracking
- [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
### Pull Request Tracking
- [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
- [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
- [x] Pull request contains [commits signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) with a registered key indicating `Verified` status
### Pull Request Formatting
- [x] Pull Request based on current revision of the `main` branch
- [x] Pull Request refers to a feature branch with one commit containing changes
# Verification
### Build
- [ ] Build completed using `./mvnw clean install -P contrib-check`
- [ ] JDK 21
- [ ] JDK 25
### Licensing
- [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
- [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
### Documentation
- [x] Documentation formatting appears as expected in rendered files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-16443 Added Ignore Cells Beyond Header property to ExcelReader [nifi]
Posted by "MikeThomsen (via GitHub)" <gi...@apache.org>.
MikeThomsen merged PR #11783:
URL: https://github.com/apache/nifi/pull/11783
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-16443 Added Ignore Cells Beyond Header property to ExcelReader [nifi]
Posted by "dan-s1 (via GitHub)" <gi...@apache.org>.
dan-s1 commented on code in PR #11783:
URL: https://github.com/apache/nifi/pull/11783#discussion_r4231762275
##########
nifi-extension-bundles/nifi-poi-bundle/nifi-poi-services/src/main/java/org/apache/nifi/excel/ExcelStartingRowSchemaInference.java:
##########
@@ -50,17 +54,27 @@ public class ExcelStartingRowSchemaInference implements SchemaInferenceEngine<Ro
private final int firstRow;
private final CellFieldTypeReader cellFieldTypeReader;
private final DataFormatter dataFormatter;
+ private final boolean ignoreCellsBeyondHeader;
+ private final ComponentLog logger;
public ExcelStartingRowSchemaInference(RowEvaluationStrategy rowEvaluationStrategy, int firstRow, TimeValueInference timeValueInference) {
+ this(rowEvaluationStrategy, firstRow, timeValueInference, false, null);
+ }
+
+ public ExcelStartingRowSchemaInference(RowEvaluationStrategy rowEvaluationStrategy, int firstRow, TimeValueInference timeValueInference,
+ boolean ignoreCellsBeyondHeader, ComponentLog logger) {
this.rowEvaluationStrategy = rowEvaluationStrategy;
this.firstRow = firstRow;
this.cellFieldTypeReader = new StandardCellFieldTypeReader(timeValueInference);
this.dataFormatter = new DataFormatter();
+ this.ignoreCellsBeyondHeader = ignoreCellsBeyondHeader;
+ this.logger = logger;
}
Review Comment:
Since all the instances of this class use the constructor collapse to one constructor and then you would not need to check later in the code whether the logger is null or not.
```suggestion
public ExcelStartingRowSchemaInference(RowEvaluationStrategy rowEvaluationStrategy, int firstRow, TimeValueInference timeValueInference,
boolean ignoreCellsBeyondHeader, ComponentLog logger) {
this.rowEvaluationStrategy = rowEvaluationStrategy;
this.firstRow = firstRow;
this.cellFieldTypeReader = new StandardCellFieldTypeReader(timeValueInference);
this.dataFormatter = new DataFormatter();
this.ignoreCellsBeyondHeader = ignoreCellsBeyondHeader;
this.logger = logger;
}
```
##########
nifi-extension-bundles/nifi-poi-bundle/nifi-poi-services/src/main/java/org/apache/nifi/excel/ExcelReader.java:
##########
@@ -108,6 +108,17 @@ public class ExcelReader extends SchemaRegistryService implements RecordReaderFa
.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
.build();
+ public static final PropertyDescriptor IGNORE_CELLS_BEYOND_HEADER = new PropertyDescriptor
+ .Builder().name("Ignore Cells Beyond Header")
+ .description("Whether to ignore cells located beyond the last column of the header row, instead of failing schema inference."
+ + " Such cells are often blank but formatted. A warning is logged when ignored cells contain a value."
+ + " Only the rows evaluated by the '" + ROW_EVALUATION_STRATEGY.getName() + "' property are checked.")
+ .required(true)
+ .allowableValues("true", "false")
+ .defaultValue("false")
+ .dependsOn(SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, ExcelStartingRowSchemaInference.USE_STARTING_ROW)
+ .build();
Review Comment:
Replace with text block
```suggestion
.description("""
Whether to ignore cells located beyond the last column of the header row, instead of failing schema inference. \
Such cells are often blank but formatted. A warning is logged when ignored cells contain a value. \
Only the rows evaluated by the '%s' property are checked.\
""".formatted(ROW_EVALUATION_STRATEGY.getName()))
.required(true)
.allowableValues("true", "false")
.defaultValue("false")
.dependsOn(SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, ExcelStartingRowSchemaInference.USE_STARTING_ROW)
.build();
```
##########
nifi-extension-bundles/nifi-poi-bundle/nifi-poi-services/src/main/java/org/apache/nifi/excel/ExcelStartingRowSchemaInference.java:
##########
@@ -79,16 +93,24 @@ public RecordSchema inferSchema(RecordSource<Row> recordSource) throws IOExcepti
} else {
if (RowEvaluationStrategy.STANDARD == rowEvaluationStrategy) {
if (index <= RowEvaluationStrategy.NUM_ROWS_TO_DETERMINE_TYPES) {
- inferSchema(row, fieldNames, typeMap);
+ inferSchema(row, fieldNames, typeMap, ignoredCellsBySheet);
} else {
break;
}
} else {
- inferSchema(row, fieldNames, typeMap);
+ inferSchema(row, fieldNames, typeMap, ignoredCellsBySheet);
}
}
index++;
}
+
+ if (logger != null) {
+ for (Map.Entry<String, IgnoredCells> entry : ignoredCellsBySheet.entrySet()) {
+ final IgnoredCells ignoredCells = entry.getValue();
+ logger.warn("Ignored non-empty cells beyond the {} header columns in {} row(s) of sheet [{}], first at row {} column {}",
+ fieldNames.size(), ignoredCells.rowCount(), entry.getKey(), ignoredCells.firstRowNumber(), ignoredCells.firstColumnName());
+ }
+ }
Review Comment:
Per earlier suggestion `logger` should never be null
```suggestion
for (Map.Entry<String, IgnoredCells> entry : ignoredCellsBySheet.entrySet()) {
final IgnoredCells ignoredCells = entry.getValue();
logger.warn("Ignored non-empty cells beyond the {} header columns in {} row(s) of sheet [{}], first at row {} column {}", fieldNames.size(), ignoredCells.rowCount(), entry.getKey(), ignoredCells.firstRowNumber(), ignoredCells.firstColumnName());
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-16443 Added Ignore Cells Beyond Header property to ExcelReader [nifi]
Posted by "Vinz2168 (via GitHub)" <gi...@apache.org>.
Vinz2168 commented on code in PR #11783:
URL: https://github.com/apache/nifi/pull/11783#discussion_r4236774033
##########
nifi-extension-bundles/nifi-poi-bundle/nifi-poi-services/src/main/java/org/apache/nifi/excel/ExcelStartingRowSchemaInference.java:
##########
@@ -79,16 +93,24 @@ public RecordSchema inferSchema(RecordSource<Row> recordSource) throws IOExcepti
} else {
if (RowEvaluationStrategy.STANDARD == rowEvaluationStrategy) {
if (index <= RowEvaluationStrategy.NUM_ROWS_TO_DETERMINE_TYPES) {
- inferSchema(row, fieldNames, typeMap);
+ inferSchema(row, fieldNames, typeMap, ignoredCellsBySheet);
} else {
break;
}
} else {
- inferSchema(row, fieldNames, typeMap);
+ inferSchema(row, fieldNames, typeMap, ignoredCellsBySheet);
}
}
index++;
}
+
+ if (logger != null) {
+ for (Map.Entry<String, IgnoredCells> entry : ignoredCellsBySheet.entrySet()) {
+ final IgnoredCells ignoredCells = entry.getValue();
+ logger.warn("Ignored non-empty cells beyond the {} header columns in {} row(s) of sheet [{}], first at row {} column {}",
+ fieldNames.size(), ignoredCells.rowCount(), entry.getKey(), ignoredCells.firstRowNumber(), ignoredCells.firstColumnName());
+ }
+ }
Review Comment:
Thanks @dan-s1, applied all three suggestions in a50d8a2."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org