[GitHub] [streampipes] dominikriemer added a comment to the discussion: Difference Between Apache Nifi and Apache Streampipes

[GitBox <gi...@apache.org>]

GitHub user dominikriemer added a comment to the discussion: Difference Between Apache Nifi and Apache Streampipes

Hi @Praveenstein, good question! I totally agree with your findings. In general and from a use case perspective , I'd say Nifi is a generic dataflow processing system, while StreamPipes targets the Industrial IoT space in form of a toolbox with the goal to support non-technical users in analyzing IIoT data. E.g., the pipeline editor which is closest to the graphical modeling approach of Nifi (or also NodeRED) is supplemented with other tools such as the live dashboard, data explorer, or very new additions such as standalone functions and Python support targeted at data scientists.
From an architectural point of view, StreamPipes follows an event-driven way (pull-based sources can be connected, but internally everything is a stream). The exception is the data explorer, which relies on a time-series database for offline analytics. While Nifi and StreamPipes are both extensible in terms of data processors and sinks, I'm not 100% sure how the Nifi guys do it, but StreamPipes consists of microservices which contain one or more pipeline elements and which can be deployed in a geo-distributed way, so that also edge processing is supported (we are also working on further improving support for such use cases). Nifi has MiNifi which also aimes for such cases. In terms of supported pipeline elements, we try to add more and more analytics-oriented processors to StreamPipes, e.g., for easy integration of AI models or working with PLC data while we are not so much focusing on data transformation and processing, which is an area Nifi is great at.

In terms of architecture, there are probably quite a lot of differences between both tools (you have mentioned a few), e.g., the underlying data model allows to define requirements on input streams of pipeline elements, which aims at reducing modeling errors and hiding technical details of connected event streams.

Hope this explains it a little bit! I'll be happy to dive deeper in case you have any questions and maybe other community members have other views ;-)

GitHub link: https://github.com/apache/streampipes/discussions/1067#discussioncomment-4636447

----
This is an automatically sent email for dev@streampipes.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@streampipes.apache.org