You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "pvillard31 (via GitHub)" <gi...@apache.org> on 2025/12/08 12:05:51 UTC
[PR] NIFI-14233 - Fix Python Processor imported property dependencies [nifi]
pvillard31 opened a new pull request, #10616:
URL: https://github.com/apache/nifi/pull/10616
# Summary
NIFI-14233 - Fix Python Processor imported property dependencies
Fixes NIFI-14233 where Python Processors could not use imported `PropertyDescriptors` as `PropertyDependencies`. This regression was introduced in NIFI-13872 and prevented processors from using a common pattern of sharing property definitions across multiple processors via imports.
When a `PropertyDescriptor` was imported from another module and used in a `PropertyDependency`, NiFi failed to load the processor with a `KeyError`:
```python
from SharedModule import SHARED_PROPERTY
from nifiapi.properties import PropertyDescriptor, PropertyDependency
class MyProcessor:
MY_PROPERTY = PropertyDescriptor(
name="My Property",
dependencies=[PropertyDependency(SHARED_PROPERTY, "value")] # Failed with KeyError
)
```
The AST-based `ProcessorInspection` only discovered PropertyDescriptors defined within the current class, not imported ones.
- Added functions to parse import statements and resolve PropertyDescriptors from imported modules
- Modified `CollectPropertyDescriptorVisitors` to accept and check imported descriptors when resolving dependencies
- Changed direct dictionary access `[]` to `.get()` to handle missing properties gracefully
# Tracking
Please complete the following tracking steps prior to pull request creation.
### Issue Tracking
- [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
### Pull Request Tracking
- [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
- [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
### Pull Request Formatting
- [ ] Pull Request based on current revision of the `main` branch
- [ ] Pull Request refers to a feature branch with one commit containing changes
# Verification
Please indicate the verification steps performed prior to pull request creation.
### Build
- [ ] Build completed using `./mvnw clean install -P contrib-check`
- [ ] JDK 21
- [ ] JDK 25
### Licensing
- [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
- [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
### Documentation
- [ ] Documentation formatting appears as expected in rendered files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-14233 - Fix Python Processor imported property dependencies [nifi]
Posted by "pvillard31 (via GitHub)" <gi...@apache.org>.
pvillard31 commented on PR #10616:
URL: https://github.com/apache/nifi/pull/10616#issuecomment-3834865994
Good catch thanks @exceptionfactory
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-14233 - Fix Python Processor imported property dependencies [nifi]
Posted by "pvillard31 (via GitHub)" <gi...@apache.org>.
pvillard31 commented on code in PR #10616:
URL: https://github.com/apache/nifi/pull/10616#discussion_r2695803752
##########
nifi-framework-bundle/nifi-framework-extensions/nifi-py4j-framework-bundle/nifi-python-framework/src/main/python/framework/ProcessorInspection.py:
##########
@@ -121,6 +126,138 @@ def get_module_string_constants(module_file: str) -> dict:
return visitor.string_assignments
+def get_imports_from_module(module_file: str) -> dict:
+ """
+ Parse a module file and return a mapping of imported names to their source module files.
+
+ This function extracts import statements from the given module file and resolves
+ them to actual file paths. It handles:
+ - 'from module import name' style imports
+ - Relative imports within the same directory
+
+ :param module_file: Path to the Python module file to parse
+ :return: Dictionary mapping imported names to their source file paths
+ e.g., {'SHARED_PROPERTY': '/path/to/SharedModule.py'}
+ """
+ with open(module_file) as file:
+ root_node = ast.parse(file.read())
Review Comment:
Thanks for the review, I pushed a commit to address your comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-14233 - Fix Python Processor imported property dependencies [nifi]
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #10616:
URL: https://github.com/apache/nifi/pull/10616#discussion_r2695703090
##########
nifi-framework-bundle/nifi-framework-extensions/nifi-py4j-framework-bundle/nifi-python-framework/src/main/python/framework/ProcessorInspection.py:
##########
@@ -121,6 +126,138 @@ def get_module_string_constants(module_file: str) -> dict:
return visitor.string_assignments
+def get_imports_from_module(module_file: str) -> dict:
+ """
+ Parse a module file and return a mapping of imported names to their source module files.
+
+ This function extracts import statements from the given module file and resolves
+ them to actual file paths. It handles:
+ - 'from module import name' style imports
+ - Relative imports within the same directory
+
+ :param module_file: Path to the Python module file to parse
+ :return: Dictionary mapping imported names to their source file paths
+ e.g., {'SHARED_PROPERTY': '/path/to/SharedModule.py'}
+ """
+ with open(module_file) as file:
+ root_node = ast.parse(file.read())
Review Comment:
Instead of reading the file and parsing it multiple times, it would be helpful to refactor the approach to read and parse once, then pass the parsed tree around
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-14233 - Fix Python Processor imported property dependencies [nifi]
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory merged PR #10616:
URL: https://github.com/apache/nifi/pull/10616
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org