Nifi UI Enhancements

[Rick Braddy <rb...@softnas.com>]

After using Nifi for some weeks now, I have an enhancement request to recommend for the UI that I believe will dramatically improve the usability.

Before I jump into the problems, let me say this about the UI.  It's very intuitive, easy to learn and consistent.  It's also very clean and attractive from an appearance standpoint.  As described below, there are some areas that are becoming tedious with dozens of processors today, and that will become acute from a usability perspective if untreated.

The Challenges

Issue #1 - Processors take too much time to select and configure initially

Overall, the Nifi UI is fantastic from an ease of use perspective; however, there is one area that's problematic and that will become increasingly challenging... adding new processor blocks and choosing the processor type.

The primary issue stems from the lengthy list of processors to scroll through and pick from.  The filter cloud is helpful, but with time even this mechanism is reaching its limits, as the number of processors continues to grow with the success of the project.

There needs to be an easier, more productive way to simply drag and drop processors to the canvass.

Issue #2 - Processor blocks all look the same instead of being more visually distinct and easily recognizable by function

Our brains are wired for rapid pattern recognition of images, text takes more cycle of higher-level reasoning to interpret.

Because processor blocks are shown, by default, with their I/O statistics and textual names, they all kind of look the same.  This "runtime view" is great for troubleshooting or monitoring, but not as useful when building complex data flows.  An "iconic view" would provide an easier way to visualize the structure and intent behind each graph and its flows, especially when developing the flows.

Additionally, an icon representing each type of processor block would make it much faster and easier to recognize what that processor block does, versus having to read and interpret each one individually (especially for complex graphs).  Processors that handle files could be represented by a "file" icon, Hadoop by a Hadoop icon, HTTP by a globe icon, etc.

#3  - When dropping a new processor, open its Properties dialog automatically (to avoid right-click, then "Configure" and choose tab steps)

Every time a new processor is dropped onto the canvass, we must go through the process to select its type.  Then, the dialog closes and we're usually left with an incomplete processor with errors.  We know that most processors require some initial Property configuration, so why not just proceed to that dialog after choosing the processor type, so can finish configuring it, then apply so we have a processor that's ready to integrate?

Potential Solutions

Visio has a great model for addressing #1 that I would propose as a starting point for resolving this issue - use of a "tool palette" that snaps into place on the left side of the canvass.  Each group of tools (e.g., file-related processors, Hadoop processors, HTTP processors, etc.) would be grouped together within a toolbox area, with an icon representing each tool/processor with a brief description of each tool.

As with Visio, the user would open up several commonly used toolboxes and then just drag and drop a tool from the toolbox directly onto the canvass, with no need to select the processor type (each processor is shown as a unique type in its toolbox).  This approach is very familiar to Visio users and other tools that operate in a similar manner with object drag and drop.  Scrolling through lengthy lists is time-consuming and becomes tedious when developing large graphs.

Once processors have icons associated with them, several things become much easier:


1.       The toolboxes are much easier to create, leveraging each processors inherent icon representation

2.       The runtime view (current view) could simply have each processor's icon shown (either in the white space to left of "5 mins" or in the border area)

3.       If a purely iconic view were added at some future point, then a clear "as built" drawing of the data flow would make the graphs even more self-documenting and obvious

Lastly, when the generic processor is dragged onto the canvass (as it is today), and a processor type is selected, it would be very easy to proceed next to the Property dialog (if there are any mandatory properties that must be configured before first use), reducing the number of clicks required to get a processor up and running.

I believe a usability study with target users would likely reveal the above (or similar) conclusions.  In order for Nifi to scale with dozens or even hundreds more processors, it's clear that something has to give, as the current method of choosing processor type has about run its course from a usability standpoint IMHO.

Hope that's helpful.

Rick

Re: Nifi UI Enhancements

[Jennifer Barnabee <je...@gmail.com>]

I think you are both right. Adobe products like Illustrator/Photoshop provide the ability to pull things onto your canvas from panels that you can expand or collapse, and you can typically get to what you want more than one way. I think NiFi could add that type of thing and not clutter up the UI too much.
-Jenn

Sent from my iPhone

> On Sep 4, 2015, at 5:48 PM, Ian Ragsdale <ia...@gmail.com> wrote:
> 
> Another way to look at silence is a lack of disagreement. :) I haven't yet used NiFi enough to have these things become annoyances, but I can't disagree with any of those general suggestions.
> 
> A 2x2 grid of processors down the side that would let you choose the processor type *before* dragging it onto the canvas sounds great to me, assuming that the processors have distinctive icons. I think there would likely still be a need for the processor picker, but the common use case where you know exactly what you want to add could definitely be sped up.
> 
> - Ian
> 
>> On Sep 4, 2015, at 4:37 PM, Rick Braddy <rb...@softnas.com> wrote:
>> 
>> Okay.  Looks like I’m the only one who thinks this is an issue…
>>  
>> Onward.
>>  
>> From: Rick Braddy [mailto:rbraddy@softnas.com] 
>> Sent: Wednesday, September 02, 2015 11:23 PM
>> To: users@nifi.apache.org
>> Subject: Nifi UI Enhancements
>>  
>> After using Nifi for some weeks now, I have an enhancement request to recommend for the UI that I believe will dramatically improve the usability.
>>  
>> Before I jump into the problems, let me say this about the UI.  It’s very intuitive, easy to learn and consistent.  It’s also very clean and attractive from an appearance standpoint.  As described below, there are some areas that are becoming tedious with dozens of processors today, and that will become acute from a usability perspective if untreated.
>>  
>> The Challenges
>>  
>> Issue #1 – Processors take too much time to select and configure initially
>>  
>> Overall, the Nifi UI is fantastic from an ease of use perspective; however, there is one area that’s problematic and that will become increasingly challenging… adding new processor blocks and choosing the processor type.
>>  
>> The primary issue stems from the lengthy list of processors to scroll through and pick from.  The filter cloud is helpful, but with time even this mechanism is reaching its limits, as the number of processors continues to grow with the success of the project.
>>  
>> There needs to be an easier, more productive way to simply drag and drop processors to the canvass.
>>  
>> Issue #2 – Processor blocks all look the same instead of being more visually distinct and easily recognizable by function
>>  
>> Our brains are wired for rapid pattern recognition of images, text takes more cycle of higher-level reasoning to interpret.
>>  
>> Because processor blocks are shown, by default, with their I/O statistics and textual names, they all kind of look the same.  This “runtime view” is great for troubleshooting or monitoring, but not as useful when building complex data flows.  An “iconic view” would provide an easier way to visualize the structure and intent behind each graph and its flows, especially when developing the flows.
>>  
>> Additionally, an icon representing each type of processor block would make it much faster and easier to recognize what that processor block does, versus having to read and interpret each one individually (especially for complex graphs).  Processors that handle files could be represented by a “file” icon, Hadoop by a Hadoop icon, HTTP by a globe icon, etc.
>>  
>> #3  - When dropping a new processor, open its Properties dialog automatically (to avoid right-click, then “Configure” and choose tab steps)
>>  
>> Every time a new processor is dropped onto the canvass, we must go through the process to select its type.  Then, the dialog closes and we’re usually left with an incomplete processor with errors.  We know that most processors require some initial Property configuration, so why not just proceed to that dialog after choosing the processor type, so can finish configuring it, then apply so we have a processor that’s ready to integrate?
>>  
>> Potential Solutions
>>  
>> Visio has a great model for addressing #1 that I would propose as a starting point for resolving this issue – use of a “tool palette” that snaps into place on the left side of the canvass.  Each group of tools (e.g., file-related processors, Hadoop processors, HTTP processors, etc.) would be grouped together within a toolbox area, with an icon representing each tool/processor with a brief description of each tool.
>>  
>> As with Visio, the user would open up several commonly used toolboxes and then just drag and drop a tool from the toolbox directly onto the canvass, with no need to select the processor type (each processor is shown as a unique type in its toolbox).  This approach is very familiar to Visio users and other tools that operate in a similar manner with object drag and drop.  Scrolling through lengthy lists is time-consuming and becomes tedious when developing large graphs.
>>  
>> Once processors have icons associated with them, several things become much easier:
>>  
>> 1.       The toolboxes are much easier to create, leveraging each processors inherent icon representation
>> 
>> 2.       The runtime view (current view) could simply have each processor’s icon shown (either in the white space to left of “5 mins” or in the border area)
>> 
>> 3.       If a purely iconic view were added at some future point, then a clear “as built” drawing of the data flow would make the graphs even more self-documenting and obvious
>> 
>>  
>> Lastly, when the generic processor is dragged onto the canvass (as it is today), and a processor type is selected, it would be very easy to proceed next to the Property dialog (if there are any mandatory properties that must be configured before first use), reducing the number of clicks required to get a processor up and running.
>>  
>> I believe a usability study with target users would likely reveal the above (or similar) conclusions.  In order for Nifi to scale with dozens or even hundreds more processors, it’s clear that something has to give, as the current method of choosing processor type has about run its course from a usability standpoint IMHO.
>>  
>> Hope that’s helpful.
>>  
>> Rick
> 

Re: Nifi UI Enhancements

[Rick Braddy <rb...@softnas.com>]

In thinking about this further, it would also be helpful to treat templates in a similar way to processor picking from the visual toolbox perspective, so they are as easy to drag, drop and connect up as processors.

Thanks for the feedback.

Rick



> On Sep 4, 2015, at 10:20 PM, Joe Witt <jo...@gmail.com> wrote:
> 
> Issue #1: Make it easier/more intuitive to find the desired processor
> 
> Yep.  Several things we can do here.  All of the suggestions so far on
> this thread I personally like.
> 
> Issue #2: Provide visual distinction to processors
> 
> Yep.  Wanted this as far back as 2010.  In fact, the plan was to use
> the Enterprise Integration Patterns as the basis for it.  By having
> developers annotate the processor with its 'type' which would be
> 'route, transform, mediate' or more specific classifications thereof
> we could provide an intuitive icon.  But then we got tripped up on how
> much real estate it would take up, what to do if multiple
> classifications fit, etc..  And the idea just stalled.  If we did this
> though it would offer additional ways to tackle #1 as well.
> 
> Issue #3:
> 
> This is a user preference thing.
> 
>    'I don't build flows often, but when I do I prefer breadth first'.
> 
> As Rob points out some users prefer to lay the foundation first when
> building flows and then go back in and configure the details of each
> component.  If one is more of a depth first thinker the approach of
> having the config dialogue popup would be preferred.  Now having said
> this it is humorous to note that when drawing connections it does
> immediately go to a configuration dialogue and that is quite nice.
> Hmmm I might be changing my mind about which I prefer as I write
> this...
> 
> --
> 
> So, ultimately this thread should probably turn into a feature
> proposal around 'Streamlining visual flow design' perhaps and then
> spawn off a series of JIRAs.  Anyone want to volunteer to do that?
> 
> Thanks
> Joe
> 
>> On Fri, Sep 4, 2015 at 7:02 PM, Rob Moran <rm...@gmail.com> wrote:
>> Rick - you bring up several very good usability issues.
>> 
>> Re: Issue #1:
>> 
>> I too believe the ‘generic processor picker’ is functional and should
>> remain. It’s useful for someone who doesn’t know what they need. However,
>> you’re right that there can be more efficient interaction for those users
>> who already know what they want.
>> 
>> I like where Alan was headed with his thoughts - recognition of last used,
>> popular, or set by preference. To expand on user preferences, perhaps the
>> ability to set a default type accessed by a single click, or access
>> favorites via keyboard shortcuts. Also as Alan and others mentioned, logical
>> groupings of processors could help as well - this could be available as a
>> traditional menu from the current processor icon.
>> 
>> Re: Issue #2:
>> 
>> I agree. Unique icons and/or styling in general are ways we could start to
>> differentiate parts of a flow.
>> 
>> I really like the idea of a more responsive interface that adapts to scaling
>> - only particular details of a component may be relevant at certain zoom
>> levels.
>> 
>> Re: Issue #3:
>> 
>> That’s a tough one for me. I understand your view and it makes sense - if
>> some standard configuration steps are necessary why not prompt them from the
>> start. However, I could also see someone building a flow and just want to
>> sort of visually create, or map out the entire flow without being
>> interrupted each time to configure. Again, maybe it comes down to a choice
>> (i.e., user preference).
>> 
>> You also brought up usability studies and Issue #3 would be a good one to
>> gauge user opinion on.
>> 
>> 
>>> On Fri, Sep 4, 2015 at 6:45 PM Alan Jackoway <al...@cloudera.com> wrote:
>>> 
>>> Hello,
>>> 
>>> I think that icons and opening the properties when you drop a processor
>>> (at least as an option) are both very good ideas. For the icons NiFi might
>>> have to deal with a little bit of licensing, but I'm sure we can figure that
>>> out. I think it should be somewhat easy for NiFi to provide some standard
>>> icons.
>>> 
>>> I personally like the big boxes because I'm often interested in the
>>> numbers they contain, but since NiFi already has a zoom tool, I think an
>>> all-icon view when you zoom out far enough or some other kind of condensed
>>> view would make a lot of sense. In our deployment, we use process groups
>>> pretty heavily to augment the zooming, so I definitely understand how the UI
>>> can get overwhelmed by processors.
>>> 
>>> Do you have specific ideas on how to improve the processor selector? The
>>> filtering usually works for me, but I can see what you're saying about it
>>> getting harder as the project expands. Here are two ideas off the top of my
>>> head:
>>> * Set favorite processors explicitly, and pin them to the top.
>>> * Sort processors based on things like "most commonly used", "last used",
>>> and "type/icon" (tying in with your previous idea).
>>> 
>>> Alan
>>> 
>>>> On Sat, Sep 5, 2015 at 12:03 AM, Rick Braddy <rb...@softnas.com> wrote:
>>>> 
>>>> Good point.
>>>> 
>>>> 
>>>> 
>>>> I was not suggesting the generic processor picker be done away with at
>>>> all, as it’s very functional, just adding a more convenient way to drag and
>>>> drop processors that are most commonly used onto the canvas and adding an
>>>> iconic representation to processors.
>>>> 
>>>> 
>>>> 
>>>> Rick
>>>> 
>>>> 
>>>> 
>>>> From: Ian Ragsdale [mailto:ian.ragsdale@gmail.com]
>>>> Sent: Friday, September 04, 2015 3:48 PM
>>>> To: users@nifi.apache.org
>>>> Subject: Re: Nifi UI Enhancements
>>>> 
>>>> 
>>>> 
>>>> Another way to look at silence is a lack of disagreement. :) I haven't
>>>> yet used NiFi enough to have these things become annoyances, but I can't
>>>> disagree with any of those general suggestions.
>>>> 
>>>> 
>>>> 
>>>> A 2x2 grid of processors down the side that would let you choose the
>>>> processor type *before* dragging it onto the canvas sounds great to me,
>>>> assuming that the processors have distinctive icons. I think there would
>>>> likely still be a need for the processor picker, but the common use case
>>>> where you know exactly what you want to add could definitely be sped up.
>>>> 
>>>> 
>>>> 
>>>> - Ian
>>>> 
>>>> 
>>>> 
>>>> On Sep 4, 2015, at 4:37 PM, Rick Braddy <rb...@softnas.com> wrote:
>>>> 
>>>> 
>>>> 
>>>> Okay.  Looks like I’m the only one who thinks this is an issue…
>>>> 
>>>> 
>>>> 
>>>> Onward.
>>>> 
>>>> 
>>>> 
>>>> From: Rick Braddy [mailto:rbraddy@softnas.com]
>>>> Sent: Wednesday, September 02, 2015 11:23 PM
>>>> To: users@nifi.apache.org
>>>> Subject: Nifi UI Enhancements
>>>> 
>>>> 
>>>> 
>>>> After using Nifi for some weeks now, I have an enhancement request to
>>>> recommend for the UI that I believe will dramatically improve the usability.
>>>> 
>>>> 
>>>> 
>>>> Before I jump into the problems, let me say this about the UI.  It’s very
>>>> intuitive, easy to learn and consistent.  It’s also very clean and
>>>> attractive from an appearance standpoint.  As described below, there are
>>>> some areas that are becoming tedious with dozens of processors today, and
>>>> that will become acute from a usability perspective if untreated.
>>>> 
>>>> 
>>>> 
>>>> The Challenges
>>>> 
>>>> 
>>>> 
>>>> Issue #1 – Processors take too much time to select and configure
>>>> initially
>>>> 
>>>> 
>>>> 
>>>> Overall, the Nifi UI is fantastic from an ease of use perspective;
>>>> however, there is one area that’s problematic and that will become
>>>> increasingly challenging… adding new processor blocks and choosing the
>>>> processor type.
>>>> 
>>>> 
>>>> 
>>>> The primary issue stems from the lengthy list of processors to scroll
>>>> through and pick from.  The filter cloud is helpful, but with time even this
>>>> mechanism is reaching its limits, as the number of processors continues to
>>>> grow with the success of the project.
>>>> 
>>>> 
>>>> 
>>>> There needs to be an easier, more productive way to simply drag and drop
>>>> processors to the canvass.
>>>> 
>>>> 
>>>> 
>>>> Issue #2 – Processor blocks all look the same instead of being more
>>>> visually distinct and easily recognizable by function
>>>> 
>>>> 
>>>> 
>>>> Our brains are wired for rapid pattern recognition of images, text takes
>>>> more cycle of higher-level reasoning to interpret.
>>>> 
>>>> 
>>>> 
>>>> Because processor blocks are shown, by default, with their I/O statistics
>>>> and textual names, they all kind of look the same.  This “runtime view” is
>>>> great for troubleshooting or monitoring, but not as useful when building
>>>> complex data flows.  An “iconic view” would provide an easier way to
>>>> visualize the structure and intent behind each graph and its flows,
>>>> especially when developing the flows.
>>>> 
>>>> 
>>>> 
>>>> Additionally, an icon representing each type of processor block would
>>>> make it much faster and easier to recognize what that processor block does,
>>>> versus having to read and interpret each one individually (especially for
>>>> complex graphs).  Processors that handle files could be represented by a
>>>> “file” icon, Hadoop by a Hadoop icon, HTTP by a globe icon, etc.
>>>> 
>>>> 
>>>> 
>>>> #3  - When dropping a new processor, open its Properties dialog
>>>> automatically (to avoid right-click, then “Configure” and choose tab steps)
>>>> 
>>>> 
>>>> 
>>>> Every time a new processor is dropped onto the canvass, we must go
>>>> through the process to select its type.  Then, the dialog closes and we’re
>>>> usually left with an incomplete processor with errors.  We know that most
>>>> processors require some initial Property configuration, so why not just
>>>> proceed to that dialog after choosing the processor type, so can finish
>>>> configuring it, then apply so we have a processor that’s ready to integrate?
>>>> 
>>>> 
>>>> 
>>>> Potential Solutions
>>>> 
>>>> 
>>>> 
>>>> Visio has a great model for addressing #1 that I would propose as a
>>>> starting point for resolving this issue – use of a “tool palette” that snaps
>>>> into place on the left side of the canvass.  Each group of tools (e.g.,
>>>> file-related processors, Hadoop processors, HTTP processors, etc.) would be
>>>> grouped together within a toolbox area, with an icon representing each
>>>> tool/processor with a brief description of each tool.
>>>> 
>>>> 
>>>> 
>>>> As with Visio, the user would open up several commonly used toolboxes and
>>>> then just drag and drop a tool from the toolbox directly onto the canvass,
>>>> with no need to select the processor type (each processor is shown as a
>>>> unique type in its toolbox).  This approach is very familiar to Visio users
>>>> and other tools that operate in a similar manner with object drag and drop.
>>>> Scrolling through lengthy lists is time-consuming and becomes tedious when
>>>> developing large graphs.
>>>> 
>>>> 
>>>> 
>>>> Once processors have icons associated with them, several things become
>>>> much easier:
>>>> 
>>>> 
>>>> 
>>>> 1.       The toolboxes are much easier to create, leveraging each
>>>> processors inherent icon representation
>>>> 
>>>> 2.       The runtime view (current view) could simply have each
>>>> processor’s icon shown (either in the white space to left of “5 mins” or in
>>>> the border area)
>>>> 
>>>> 3.       If a purely iconic view were added at some future point, then a
>>>> clear “as built” drawing of the data flow would make the graphs even more
>>>> self-documenting and obvious
>>>> 
>>>> 
>>>> 
>>>> Lastly, when the generic processor is dragged onto the canvass (as it is
>>>> today), and a processor type is selected, it would be very easy to proceed
>>>> next to the Property dialog (if there are any mandatory properties that must
>>>> be configured before first use), reducing the number of clicks required to
>>>> get a processor up and running.
>>>> 
>>>> 
>>>> 
>>>> I believe a usability study with target users would likely reveal the
>>>> above (or similar) conclusions.  In order for Nifi to scale with dozens or
>>>> even hundreds more processors, it’s clear that something has to give, as the
>>>> current method of choosing processor type has about run its course from a
>>>> usability standpoint IMHO.
>>>> 
>>>> 
>>>> 
>>>> Hope that’s helpful.
>>>> 
>>>> 
>>>> 
>>>> Rick
>> --
>> Rob

Re: Nifi UI Enhancements

[Rob Moran <rm...@gmail.com>]

I'll work on putting something together for discussion.

Rob
On Sep 4, 2015 11:20 PM, "Joe Witt" <jo...@gmail.com> wrote:

> Issue #1: Make it easier/more intuitive to find the desired processor
>
> Yep.  Several things we can do here.  All of the suggestions so far on
> this thread I personally like.
>
> Issue #2: Provide visual distinction to processors
>
> Yep.  Wanted this as far back as 2010.  In fact, the plan was to use
> the Enterprise Integration Patterns as the basis for it.  By having
> developers annotate the processor with its 'type' which would be
> 'route, transform, mediate' or more specific classifications thereof
> we could provide an intuitive icon.  But then we got tripped up on how
> much real estate it would take up, what to do if multiple
> classifications fit, etc..  And the idea just stalled.  If we did this
> though it would offer additional ways to tackle #1 as well.
>
> Issue #3:
>
> This is a user preference thing.
>
>     'I don't build flows often, but when I do I prefer breadth first'.
>
> As Rob points out some users prefer to lay the foundation first when
> building flows and then go back in and configure the details of each
> component.  If one is more of a depth first thinker the approach of
> having the config dialogue popup would be preferred.  Now having said
> this it is humorous to note that when drawing connections it does
> immediately go to a configuration dialogue and that is quite nice.
> Hmmm I might be changing my mind about which I prefer as I write
> this...
>
> --
>
> So, ultimately this thread should probably turn into a feature
> proposal around 'Streamlining visual flow design' perhaps and then
> spawn off a series of JIRAs.  Anyone want to volunteer to do that?
>
> Thanks
> Joe
>
> On Fri, Sep 4, 2015 at 7:02 PM, Rob Moran <rm...@gmail.com> wrote:
> > Rick - you bring up several very good usability issues.
> >
> > Re: Issue #1:
> >
> > I too believe the ‘generic processor picker’ is functional and should
> > remain. It’s useful for someone who doesn’t know what they need. However,
> > you’re right that there can be more efficient interaction for those users
> > who already know what they want.
> >
> > I like where Alan was headed with his thoughts - recognition of last
> used,
> > popular, or set by preference. To expand on user preferences, perhaps the
> > ability to set a default type accessed by a single click, or access
> > favorites via keyboard shortcuts. Also as Alan and others mentioned,
> logical
> > groupings of processors could help as well - this could be available as a
> > traditional menu from the current processor icon.
> >
> > Re: Issue #2:
> >
> > I agree. Unique icons and/or styling in general are ways we could start
> to
> > differentiate parts of a flow.
> >
> > I really like the idea of a more responsive interface that adapts to
> scaling
> > - only particular details of a component may be relevant at certain zoom
> > levels.
> >
> > Re: Issue #3:
> >
> > That’s a tough one for me. I understand your view and it makes sense - if
> > some standard configuration steps are necessary why not prompt them from
> the
> > start. However, I could also see someone building a flow and just want to
> > sort of visually create, or map out the entire flow without being
> > interrupted each time to configure. Again, maybe it comes down to a
> choice
> > (i.e., user preference).
> >
> > You also brought up usability studies and Issue #3 would be a good one to
> > gauge user opinion on.
> >
> >
> > On Fri, Sep 4, 2015 at 6:45 PM Alan Jackoway <al...@cloudera.com> wrote:
> >>
> >> Hello,
> >>
> >> I think that icons and opening the properties when you drop a processor
> >> (at least as an option) are both very good ideas. For the icons NiFi
> might
> >> have to deal with a little bit of licensing, but I'm sure we can figure
> that
> >> out. I think it should be somewhat easy for NiFi to provide some
> standard
> >> icons.
> >>
> >> I personally like the big boxes because I'm often interested in the
> >> numbers they contain, but since NiFi already has a zoom tool, I think an
> >> all-icon view when you zoom out far enough or some other kind of
> condensed
> >> view would make a lot of sense. In our deployment, we use process groups
> >> pretty heavily to augment the zooming, so I definitely understand how
> the UI
> >> can get overwhelmed by processors.
> >>
> >> Do you have specific ideas on how to improve the processor selector? The
> >> filtering usually works for me, but I can see what you're saying about
> it
> >> getting harder as the project expands. Here are two ideas off the top
> of my
> >> head:
> >> * Set favorite processors explicitly, and pin them to the top.
> >> * Sort processors based on things like "most commonly used", "last
> used",
> >> and "type/icon" (tying in with your previous idea).
> >>
> >> Alan
> >>
> >> On Sat, Sep 5, 2015 at 12:03 AM, Rick Braddy <rb...@softnas.com>
> wrote:
> >>>
> >>> Good point.
> >>>
> >>>
> >>>
> >>> I was not suggesting the generic processor picker be done away with at
> >>> all, as it’s very functional, just adding a more convenient way to
> drag and
> >>> drop processors that are most commonly used onto the canvas and adding
> an
> >>> iconic representation to processors.
> >>>
> >>>
> >>>
> >>> Rick
> >>>
> >>>
> >>>
> >>> From: Ian Ragsdale [mailto:ian.ragsdale@gmail.com]
> >>> Sent: Friday, September 04, 2015 3:48 PM
> >>> To: users@nifi.apache.org
> >>> Subject: Re: Nifi UI Enhancements
> >>>
> >>>
> >>>
> >>> Another way to look at silence is a lack of disagreement. :) I haven't
> >>> yet used NiFi enough to have these things become annoyances, but I
> can't
> >>> disagree with any of those general suggestions.
> >>>
> >>>
> >>>
> >>> A 2x2 grid of processors down the side that would let you choose the
> >>> processor type *before* dragging it onto the canvas sounds great to me,
> >>> assuming that the processors have distinctive icons. I think there
> would
> >>> likely still be a need for the processor picker, but the common use
> case
> >>> where you know exactly what you want to add could definitely be sped
> up.
> >>>
> >>>
> >>>
> >>> - Ian
> >>>
> >>>
> >>>
> >>> On Sep 4, 2015, at 4:37 PM, Rick Braddy <rb...@softnas.com> wrote:
> >>>
> >>>
> >>>
> >>> Okay.  Looks like I’m the only one who thinks this is an issue…
> >>>
> >>>
> >>>
> >>> Onward.
> >>>
> >>>
> >>>
> >>> From: Rick Braddy [mailto:rbraddy@softnas.com]
> >>> Sent: Wednesday, September 02, 2015 11:23 PM
> >>> To: users@nifi.apache.org
> >>> Subject: Nifi UI Enhancements
> >>>
> >>>
> >>>
> >>> After using Nifi for some weeks now, I have an enhancement request to
> >>> recommend for the UI that I believe will dramatically improve the
> usability.
> >>>
> >>>
> >>>
> >>> Before I jump into the problems, let me say this about the UI.  It’s
> very
> >>> intuitive, easy to learn and consistent.  It’s also very clean and
> >>> attractive from an appearance standpoint.  As described below, there
> are
> >>> some areas that are becoming tedious with dozens of processors today,
> and
> >>> that will become acute from a usability perspective if untreated.
> >>>
> >>>
> >>>
> >>> The Challenges
> >>>
> >>>
> >>>
> >>> Issue #1 – Processors take too much time to select and configure
> >>> initially
> >>>
> >>>
> >>>
> >>> Overall, the Nifi UI is fantastic from an ease of use perspective;
> >>> however, there is one area that’s problematic and that will become
> >>> increasingly challenging… adding new processor blocks and choosing the
> >>> processor type.
> >>>
> >>>
> >>>
> >>> The primary issue stems from the lengthy list of processors to scroll
> >>> through and pick from.  The filter cloud is helpful, but with time
> even this
> >>> mechanism is reaching its limits, as the number of processors
> continues to
> >>> grow with the success of the project.
> >>>
> >>>
> >>>
> >>> There needs to be an easier, more productive way to simply drag and
> drop
> >>> processors to the canvass.
> >>>
> >>>
> >>>
> >>> Issue #2 – Processor blocks all look the same instead of being more
> >>> visually distinct and easily recognizable by function
> >>>
> >>>
> >>>
> >>> Our brains are wired for rapid pattern recognition of images, text
> takes
> >>> more cycle of higher-level reasoning to interpret.
> >>>
> >>>
> >>>
> >>> Because processor blocks are shown, by default, with their I/O
> statistics
> >>> and textual names, they all kind of look the same.  This “runtime
> view” is
> >>> great for troubleshooting or monitoring, but not as useful when
> building
> >>> complex data flows.  An “iconic view” would provide an easier way to
> >>> visualize the structure and intent behind each graph and its flows,
> >>> especially when developing the flows.
> >>>
> >>>
> >>>
> >>> Additionally, an icon representing each type of processor block would
> >>> make it much faster and easier to recognize what that processor block
> does,
> >>> versus having to read and interpret each one individually (especially
> for
> >>> complex graphs).  Processors that handle files could be represented by
> a
> >>> “file” icon, Hadoop by a Hadoop icon, HTTP by a globe icon, etc.
> >>>
> >>>
> >>>
> >>> #3  - When dropping a new processor, open its Properties dialog
> >>> automatically (to avoid right-click, then “Configure” and choose tab
> steps)
> >>>
> >>>
> >>>
> >>> Every time a new processor is dropped onto the canvass, we must go
> >>> through the process to select its type.  Then, the dialog closes and
> we’re
> >>> usually left with an incomplete processor with errors.  We know that
> most
> >>> processors require some initial Property configuration, so why not just
> >>> proceed to that dialog after choosing the processor type, so can finish
> >>> configuring it, then apply so we have a processor that’s ready to
> integrate?
> >>>
> >>>
> >>>
> >>> Potential Solutions
> >>>
> >>>
> >>>
> >>> Visio has a great model for addressing #1 that I would propose as a
> >>> starting point for resolving this issue – use of a “tool palette” that
> snaps
> >>> into place on the left side of the canvass.  Each group of tools (e.g.,
> >>> file-related processors, Hadoop processors, HTTP processors, etc.)
> would be
> >>> grouped together within a toolbox area, with an icon representing each
> >>> tool/processor with a brief description of each tool.
> >>>
> >>>
> >>>
> >>> As with Visio, the user would open up several commonly used toolboxes
> and
> >>> then just drag and drop a tool from the toolbox directly onto the
> canvass,
> >>> with no need to select the processor type (each processor is shown as a
> >>> unique type in its toolbox).  This approach is very familiar to Visio
> users
> >>> and other tools that operate in a similar manner with object drag and
> drop.
> >>> Scrolling through lengthy lists is time-consuming and becomes tedious
> when
> >>> developing large graphs.
> >>>
> >>>
> >>>
> >>> Once processors have icons associated with them, several things become
> >>> much easier:
> >>>
> >>>
> >>>
> >>> 1.       The toolboxes are much easier to create, leveraging each
> >>> processors inherent icon representation
> >>>
> >>> 2.       The runtime view (current view) could simply have each
> >>> processor’s icon shown (either in the white space to left of “5 mins”
> or in
> >>> the border area)
> >>>
> >>> 3.       If a purely iconic view were added at some future point, then
> a
> >>> clear “as built” drawing of the data flow would make the graphs even
> more
> >>> self-documenting and obvious
> >>>
> >>>
> >>>
> >>> Lastly, when the generic processor is dragged onto the canvass (as it
> is
> >>> today), and a processor type is selected, it would be very easy to
> proceed
> >>> next to the Property dialog (if there are any mandatory properties
> that must
> >>> be configured before first use), reducing the number of clicks
> required to
> >>> get a processor up and running.
> >>>
> >>>
> >>>
> >>> I believe a usability study with target users would likely reveal the
> >>> above (or similar) conclusions.  In order for Nifi to scale with
> dozens or
> >>> even hundreds more processors, it’s clear that something has to give,
> as the
> >>> current method of choosing processor type has about run its course
> from a
> >>> usability standpoint IMHO.
> >>>
> >>>
> >>>
> >>> Hope that’s helpful.
> >>>
> >>>
> >>>
> >>> Rick
> >>>
> >>>
> >>
> >>
> > --
> > Rob
>

Re: Nifi UI Enhancements

[Joe Witt <jo...@gmail.com>]

Issue #1: Make it easier/more intuitive to find the desired processor

Yep.  Several things we can do here.  All of the suggestions so far on
this thread I personally like.

Issue #2: Provide visual distinction to processors

Yep.  Wanted this as far back as 2010.  In fact, the plan was to use
the Enterprise Integration Patterns as the basis for it.  By having
developers annotate the processor with its 'type' which would be
'route, transform, mediate' or more specific classifications thereof
we could provide an intuitive icon.  But then we got tripped up on how
much real estate it would take up, what to do if multiple
classifications fit, etc..  And the idea just stalled.  If we did this
though it would offer additional ways to tackle #1 as well.

Issue #3:

This is a user preference thing.

    'I don't build flows often, but when I do I prefer breadth first'.

As Rob points out some users prefer to lay the foundation first when
building flows and then go back in and configure the details of each
component.  If one is more of a depth first thinker the approach of
having the config dialogue popup would be preferred.  Now having said
this it is humorous to note that when drawing connections it does
immediately go to a configuration dialogue and that is quite nice.
Hmmm I might be changing my mind about which I prefer as I write
this...

--

So, ultimately this thread should probably turn into a feature
proposal around 'Streamlining visual flow design' perhaps and then
spawn off a series of JIRAs.  Anyone want to volunteer to do that?

Thanks
Joe

On Fri, Sep 4, 2015 at 7:02 PM, Rob Moran <rm...@gmail.com> wrote:
> Rick - you bring up several very good usability issues.
>
> Re: Issue #1:
>
> I too believe the ‘generic processor picker’ is functional and should
> remain. It’s useful for someone who doesn’t know what they need. However,
> you’re right that there can be more efficient interaction for those users
> who already know what they want.
>
> I like where Alan was headed with his thoughts - recognition of last used,
> popular, or set by preference. To expand on user preferences, perhaps the
> ability to set a default type accessed by a single click, or access
> favorites via keyboard shortcuts. Also as Alan and others mentioned, logical
> groupings of processors could help as well - this could be available as a
> traditional menu from the current processor icon.
>
> Re: Issue #2:
>
> I agree. Unique icons and/or styling in general are ways we could start to
> differentiate parts of a flow.
>
> I really like the idea of a more responsive interface that adapts to scaling
> - only particular details of a component may be relevant at certain zoom
> levels.
>
> Re: Issue #3:
>
> That’s a tough one for me. I understand your view and it makes sense - if
> some standard configuration steps are necessary why not prompt them from the
> start. However, I could also see someone building a flow and just want to
> sort of visually create, or map out the entire flow without being
> interrupted each time to configure. Again, maybe it comes down to a choice
> (i.e., user preference).
>
> You also brought up usability studies and Issue #3 would be a good one to
> gauge user opinion on.
>
>
> On Fri, Sep 4, 2015 at 6:45 PM Alan Jackoway <al...@cloudera.com> wrote:
>>
>> Hello,
>>
>> I think that icons and opening the properties when you drop a processor
>> (at least as an option) are both very good ideas. For the icons NiFi might
>> have to deal with a little bit of licensing, but I'm sure we can figure that
>> out. I think it should be somewhat easy for NiFi to provide some standard
>> icons.
>>
>> I personally like the big boxes because I'm often interested in the
>> numbers they contain, but since NiFi already has a zoom tool, I think an
>> all-icon view when you zoom out far enough or some other kind of condensed
>> view would make a lot of sense. In our deployment, we use process groups
>> pretty heavily to augment the zooming, so I definitely understand how the UI
>> can get overwhelmed by processors.
>>
>> Do you have specific ideas on how to improve the processor selector? The
>> filtering usually works for me, but I can see what you're saying about it
>> getting harder as the project expands. Here are two ideas off the top of my
>> head:
>> * Set favorite processors explicitly, and pin them to the top.
>> * Sort processors based on things like "most commonly used", "last used",
>> and "type/icon" (tying in with your previous idea).
>>
>> Alan
>>
>> On Sat, Sep 5, 2015 at 12:03 AM, Rick Braddy <rb...@softnas.com> wrote:
>>>
>>> Good point.
>>>
>>>
>>>
>>> I was not suggesting the generic processor picker be done away with at
>>> all, as it’s very functional, just adding a more convenient way to drag and
>>> drop processors that are most commonly used onto the canvas and adding an
>>> iconic representation to processors.
>>>
>>>
>>>
>>> Rick
>>>
>>>
>>>
>>> From: Ian Ragsdale [mailto:ian.ragsdale@gmail.com]
>>> Sent: Friday, September 04, 2015 3:48 PM
>>> To: users@nifi.apache.org
>>> Subject: Re: Nifi UI Enhancements
>>>
>>>
>>>
>>> Another way to look at silence is a lack of disagreement. :) I haven't
>>> yet used NiFi enough to have these things become annoyances, but I can't
>>> disagree with any of those general suggestions.
>>>
>>>
>>>
>>> A 2x2 grid of processors down the side that would let you choose the
>>> processor type *before* dragging it onto the canvas sounds great to me,
>>> assuming that the processors have distinctive icons. I think there would
>>> likely still be a need for the processor picker, but the common use case
>>> where you know exactly what you want to add could definitely be sped up.
>>>
>>>
>>>
>>> - Ian
>>>
>>>
>>>
>>> On Sep 4, 2015, at 4:37 PM, Rick Braddy <rb...@softnas.com> wrote:
>>>
>>>
>>>
>>> Okay.  Looks like I’m the only one who thinks this is an issue…
>>>
>>>
>>>
>>> Onward.
>>>
>>>
>>>
>>> From: Rick Braddy [mailto:rbraddy@softnas.com]
>>> Sent: Wednesday, September 02, 2015 11:23 PM
>>> To: users@nifi.apache.org
>>> Subject: Nifi UI Enhancements
>>>
>>>
>>>
>>> After using Nifi for some weeks now, I have an enhancement request to
>>> recommend for the UI that I believe will dramatically improve the usability.
>>>
>>>
>>>
>>> Before I jump into the problems, let me say this about the UI.  It’s very
>>> intuitive, easy to learn and consistent.  It’s also very clean and
>>> attractive from an appearance standpoint.  As described below, there are
>>> some areas that are becoming tedious with dozens of processors today, and
>>> that will become acute from a usability perspective if untreated.
>>>
>>>
>>>
>>> The Challenges
>>>
>>>
>>>
>>> Issue #1 – Processors take too much time to select and configure
>>> initially
>>>
>>>
>>>
>>> Overall, the Nifi UI is fantastic from an ease of use perspective;
>>> however, there is one area that’s problematic and that will become
>>> increasingly challenging… adding new processor blocks and choosing the
>>> processor type.
>>>
>>>
>>>
>>> The primary issue stems from the lengthy list of processors to scroll
>>> through and pick from.  The filter cloud is helpful, but with time even this
>>> mechanism is reaching its limits, as the number of processors continues to
>>> grow with the success of the project.
>>>
>>>
>>>
>>> There needs to be an easier, more productive way to simply drag and drop
>>> processors to the canvass.
>>>
>>>
>>>
>>> Issue #2 – Processor blocks all look the same instead of being more
>>> visually distinct and easily recognizable by function
>>>
>>>
>>>
>>> Our brains are wired for rapid pattern recognition of images, text takes
>>> more cycle of higher-level reasoning to interpret.
>>>
>>>
>>>
>>> Because processor blocks are shown, by default, with their I/O statistics
>>> and textual names, they all kind of look the same.  This “runtime view” is
>>> great for troubleshooting or monitoring, but not as useful when building
>>> complex data flows.  An “iconic view” would provide an easier way to
>>> visualize the structure and intent behind each graph and its flows,
>>> especially when developing the flows.
>>>
>>>
>>>
>>> Additionally, an icon representing each type of processor block would
>>> make it much faster and easier to recognize what that processor block does,
>>> versus having to read and interpret each one individually (especially for
>>> complex graphs).  Processors that handle files could be represented by a
>>> “file” icon, Hadoop by a Hadoop icon, HTTP by a globe icon, etc.
>>>
>>>
>>>
>>> #3  - When dropping a new processor, open its Properties dialog
>>> automatically (to avoid right-click, then “Configure” and choose tab steps)
>>>
>>>
>>>
>>> Every time a new processor is dropped onto the canvass, we must go
>>> through the process to select its type.  Then, the dialog closes and we’re
>>> usually left with an incomplete processor with errors.  We know that most
>>> processors require some initial Property configuration, so why not just
>>> proceed to that dialog after choosing the processor type, so can finish
>>> configuring it, then apply so we have a processor that’s ready to integrate?
>>>
>>>
>>>
>>> Potential Solutions
>>>
>>>
>>>
>>> Visio has a great model for addressing #1 that I would propose as a
>>> starting point for resolving this issue – use of a “tool palette” that snaps
>>> into place on the left side of the canvass.  Each group of tools (e.g.,
>>> file-related processors, Hadoop processors, HTTP processors, etc.) would be
>>> grouped together within a toolbox area, with an icon representing each
>>> tool/processor with a brief description of each tool.
>>>
>>>
>>>
>>> As with Visio, the user would open up several commonly used toolboxes and
>>> then just drag and drop a tool from the toolbox directly onto the canvass,
>>> with no need to select the processor type (each processor is shown as a
>>> unique type in its toolbox).  This approach is very familiar to Visio users
>>> and other tools that operate in a similar manner with object drag and drop.
>>> Scrolling through lengthy lists is time-consuming and becomes tedious when
>>> developing large graphs.
>>>
>>>
>>>
>>> Once processors have icons associated with them, several things become
>>> much easier:
>>>
>>>
>>>
>>> 1.       The toolboxes are much easier to create, leveraging each
>>> processors inherent icon representation
>>>
>>> 2.       The runtime view (current view) could simply have each
>>> processor’s icon shown (either in the white space to left of “5 mins” or in
>>> the border area)
>>>
>>> 3.       If a purely iconic view were added at some future point, then a
>>> clear “as built” drawing of the data flow would make the graphs even more
>>> self-documenting and obvious
>>>
>>>
>>>
>>> Lastly, when the generic processor is dragged onto the canvass (as it is
>>> today), and a processor type is selected, it would be very easy to proceed
>>> next to the Property dialog (if there are any mandatory properties that must
>>> be configured before first use), reducing the number of clicks required to
>>> get a processor up and running.
>>>
>>>
>>>
>>> I believe a usability study with target users would likely reveal the
>>> above (or similar) conclusions.  In order for Nifi to scale with dozens or
>>> even hundreds more processors, it’s clear that something has to give, as the
>>> current method of choosing processor type has about run its course from a
>>> usability standpoint IMHO.
>>>
>>>
>>>
>>> Hope that’s helpful.
>>>
>>>
>>>
>>> Rick
>>>
>>>
>>
>>
> --
> Rob

Re: Nifi UI Enhancements

[Rob Moran <rm...@gmail.com>]

Rick - you bring up several very good usability issues.

Re: Issue #1:

I too believe the ‘generic processor picker’ is functional and should
remain. It’s useful for someone who doesn’t know what they need. However,
you’re right that there can be more efficient interaction for those users
who already know what they want.

I like where Alan was headed with his thoughts - recognition of last used,
popular, or set by preference. To expand on user preferences, perhaps the
ability to set a default type accessed by a single click, or access
favorites via keyboard shortcuts. Also as Alan and others mentioned,
logical groupings of processors could help as well - this could be
available as a traditional menu from the current processor icon.

Re: Issue #2:

I agree. Unique icons and/or styling in general are ways we could start to
differentiate parts of a flow.

I really like the idea of a more responsive interface that adapts to
scaling - only particular details of a component may be relevant at certain
zoom levels.

Re: Issue #3:

That’s a tough one for me. I understand your view and it makes sense - if
some standard configuration steps are necessary why not prompt them from
the start. However, I could also see someone building a flow and just want
to sort of visually create, or map out the entire flow without being
interrupted each time to configure. Again, maybe it comes down to a choice
(i.e., user preference).

You also brought up usability studies and Issue #3 would be a good one to
gauge user opinion on.

On Fri, Sep 4, 2015 at 6:45 PM Alan Jackoway <al...@cloudera.com> wrote:

> Hello,
>
> I think that icons and opening the properties when you drop a processor
> (at least as an option) are both very good ideas. For the icons NiFi might
> have to deal with a little bit of licensing, but I'm sure we can figure
> that out. I think it should be somewhat easy for NiFi to provide some
> standard icons.
>
> I personally like the big boxes because I'm often interested in the
> numbers they contain, but since NiFi already has a zoom tool, I think an
> all-icon view when you zoom out far enough or some other kind of condensed
> view would make a lot of sense. In our deployment, we use process groups
> pretty heavily to augment the zooming, so I definitely understand how the
> UI can get overwhelmed by processors.
>
> Do you have specific ideas on how to improve the processor selector? The
> filtering usually works for me, but I can see what you're saying about it
> getting harder as the project expands. Here are two ideas off the top of my
> head:
> * Set favorite processors explicitly, and pin them to the top.
> * Sort processors based on things like "most commonly used", "last used",
> and "type/icon" (tying in with your previous idea).
>
> Alan
>
> On Sat, Sep 5, 2015 at 12:03 AM, Rick Braddy <rb...@softnas.com> wrote:
>
>> Good point.
>>
>>
>>
>> I was not suggesting the generic processor picker be done away with at
>> all, as it’s very functional, just adding a more convenient way to drag and
>> drop processors that are most commonly used onto the canvas and adding an
>> iconic representation to processors.
>>
>>
>>
>> Rick
>>
>>
>>
>> *From:* Ian Ragsdale [mailto:ian.ragsdale@gmail.com]
>> *Sent:* Friday, September 04, 2015 3:48 PM
>> *To:* users@nifi.apache.org
>> *Subject:* Re: Nifi UI Enhancements
>>
>>
>>
>> Another way to look at silence is a lack of disagreement. :) I haven't
>> yet used NiFi enough to have these things become annoyances, but I can't
>> disagree with any of those general suggestions.
>>
>>
>>
>> A 2x2 grid of processors down the side that would let you choose the
>> processor type *before* dragging it onto the canvas sounds great to me,
>> assuming that the processors have distinctive icons. I think there would
>> likely still be a need for the processor picker, but the common use case
>> where you know exactly what you want to add could definitely be sped up.
>>
>>
>>
>> - Ian
>>
>>
>>
>> On Sep 4, 2015, at 4:37 PM, Rick Braddy <rb...@softnas.com> wrote:
>>
>>
>>
>> Okay.  Looks like I’m the only one who thinks this is an issue…
>>
>>
>>
>> Onward.
>>
>>
>>
>> *From:* Rick Braddy [mailto:rbraddy@softnas.com <rb...@softnas.com>]
>> *Sent:* Wednesday, September 02, 2015 11:23 PM
>> *To:* users@nifi.apache.org
>> *Subject:* Nifi UI Enhancements
>>
>>
>>
>> After using Nifi for some weeks now, I have an enhancement request to
>> recommend for the UI that I believe will dramatically improve the usability.
>>
>>
>>
>> Before I jump into the problems, let me say this about the UI.  It’s very
>> intuitive, easy to learn and consistent.  It’s also very clean and
>> attractive from an appearance standpoint.  As described below, there are
>> some areas that are becoming tedious with dozens of processors today, and
>> that will become acute from a usability perspective if untreated.
>>
>>
>>
>> *The Challenges*
>>
>>
>>
>> *Issue #1 – Processors take too much time to select and configure
>> initially*
>>
>>
>>
>> Overall, the Nifi UI is fantastic from an ease of use perspective;
>> however, there is one area that’s problematic and that will become
>> increasingly challenging… adding new processor blocks and choosing the
>> processor type.
>>
>>
>>
>> The primary issue stems from the lengthy list of processors to scroll
>> through and pick from.  The filter cloud is helpful, but with time even
>> this mechanism is reaching its limits, as the number of processors
>> continues to grow with the success of the project.
>>
>>
>>
>> There needs to be an easier, more productive way to simply drag and drop
>> processors to the canvass.
>>
>>
>>
>> *Issue #2 – Processor blocks all look the same instead of being more
>> visually distinct and easily recognizable by function*
>>
>>
>>
>> Our brains are wired for rapid pattern recognition of images, text takes
>> more cycle of higher-level reasoning to interpret.
>>
>>
>>
>> Because processor blocks are shown, by default, with their I/O statistics
>> and textual names, they all kind of look the same.  This “runtime view” is
>> great for troubleshooting or monitoring, but not as useful when building
>> complex data flows.  An “iconic view” would provide an easier way to
>> visualize the structure and intent behind each graph and its flows,
>> especially when developing the flows.
>>
>>
>>
>> Additionally, an icon representing each type of processor block would
>> make it much faster and easier to recognize what that processor block does,
>> versus having to read and interpret each one individually (especially for
>> complex graphs).  Processors that handle files could be represented by a
>> “file” icon, Hadoop by a Hadoop icon, HTTP by a globe icon, etc.
>>
>>
>>
>> *#3  - When dropping a new processor, open its Properties dialog
>> automatically (to avoid right-click, then “Configure” and choose tab steps)*
>>
>>
>>
>> Every time a new processor is dropped onto the canvass, we must go
>> through the process to select its type.  Then, the dialog closes and we’re
>> usually left with an incomplete processor with errors.  We know that most
>> processors require some initial Property configuration, so why not just
>> proceed to that dialog after choosing the processor type, so can finish
>> configuring it, then apply so we have a processor that’s ready to integrate?
>>
>>
>>
>> *Potential Solutions*
>>
>>
>>
>> Visio has a great model for addressing #1 that I would propose as a
>> starting point for resolving this issue – use of a “tool palette” that
>> snaps into place on the left side of the canvass.  Each group of tools
>> (e.g., file-related processors, Hadoop processors, HTTP processors, etc.)
>> would be grouped together within a toolbox area, with an icon representing
>> each tool/processor with a brief description of each tool.
>>
>>
>>
>> As with Visio, the user would open up several commonly used toolboxes and
>> then just drag and drop a tool from the toolbox directly onto the canvass,
>> with no need to select the processor type (each processor is shown as a
>> unique type in its toolbox).  This approach is very familiar to Visio users
>> and other tools that operate in a similar manner with object drag and
>> drop.  Scrolling through lengthy lists is time-consuming and becomes
>> tedious when developing large graphs.
>>
>>
>>
>> Once processors have icons associated with them, several things become
>> much easier:
>>
>>
>>
>> 1.       The toolboxes are much easier to create, leveraging each
>> processors inherent icon representation
>>
>> 2.       The runtime view (current view) could simply have each
>> processor’s icon shown (either in the white space to left of “5 mins” or in
>> the border area)
>>
>> 3.       If a purely iconic view were added at some future point, then a
>> clear “as built” drawing of the data flow would make the graphs even more
>> self-documenting and obvious
>>
>>
>>
>> Lastly, when the generic processor is dragged onto the canvass (as it is
>> today), and a processor type is selected, it would be very easy to proceed
>> next to the Property dialog (if there are any mandatory properties that
>> must be configured before first use), reducing the number of clicks
>> required to get a processor up and running.
>>
>>
>>
>> I believe a usability study with target users would likely reveal the
>> above (or similar) conclusions.  In order for Nifi to scale with dozens or
>> even hundreds more processors, it’s clear that something has to give, as
>> the current method of choosing processor type has about run its course from
>> a usability standpoint IMHO.
>>
>>
>>
>> Hope that’s helpful.
>>
>>
>>
>> Rick
>>
>>
>>
>
> --
Rob

Re: Nifi UI Enhancements

[Alan Jackoway <al...@cloudera.com>]

Hello,

I think that icons and opening the properties when you drop a processor (at
least as an option) are both very good ideas. For the icons NiFi might have
to deal with a little bit of licensing, but I'm sure we can figure that
out. I think it should be somewhat easy for NiFi to provide some standard
icons.

I personally like the big boxes because I'm often interested in the numbers
they contain, but since NiFi already has a zoom tool, I think an all-icon
view when you zoom out far enough or some other kind of condensed view
would make a lot of sense. In our deployment, we use process groups pretty
heavily to augment the zooming, so I definitely understand how the UI can
get overwhelmed by processors.

Do you have specific ideas on how to improve the processor selector? The
filtering usually works for me, but I can see what you're saying about it
getting harder as the project expands. Here are two ideas off the top of my
head:
* Set favorite processors explicitly, and pin them to the top.
* Sort processors based on things like "most commonly used", "last used",
and "type/icon" (tying in with your previous idea).

Alan

On Sat, Sep 5, 2015 at 12:03 AM, Rick Braddy <rb...@softnas.com> wrote:

> Good point.
>
>
>
> I was not suggesting the generic processor picker be done away with at
> all, as it’s very functional, just adding a more convenient way to drag and
> drop processors that are most commonly used onto the canvas and adding an
> iconic representation to processors.
>
>
>
> Rick
>
>
>
> *From:* Ian Ragsdale [mailto:ian.ragsdale@gmail.com]
> *Sent:* Friday, September 04, 2015 3:48 PM
> *To:* users@nifi.apache.org
> *Subject:* Re: Nifi UI Enhancements
>
>
>
> Another way to look at silence is a lack of disagreement. :) I haven't yet
> used NiFi enough to have these things become annoyances, but I can't
> disagree with any of those general suggestions.
>
>
>
> A 2x2 grid of processors down the side that would let you choose the
> processor type *before* dragging it onto the canvas sounds great to me,
> assuming that the processors have distinctive icons. I think there would
> likely still be a need for the processor picker, but the common use case
> where you know exactly what you want to add could definitely be sped up.
>
>
>
> - Ian
>
>
>
> On Sep 4, 2015, at 4:37 PM, Rick Braddy <rb...@softnas.com> wrote:
>
>
>
> Okay.  Looks like I’m the only one who thinks this is an issue…
>
>
>
> Onward.
>
>
>
> *From:* Rick Braddy [mailto:rbraddy@softnas.com <rb...@softnas.com>]
> *Sent:* Wednesday, September 02, 2015 11:23 PM
> *To:* users@nifi.apache.org
> *Subject:* Nifi UI Enhancements
>
>
>
> After using Nifi for some weeks now, I have an enhancement request to
> recommend for the UI that I believe will dramatically improve the usability.
>
>
>
> Before I jump into the problems, let me say this about the UI.  It’s very
> intuitive, easy to learn and consistent.  It’s also very clean and
> attractive from an appearance standpoint.  As described below, there are
> some areas that are becoming tedious with dozens of processors today, and
> that will become acute from a usability perspective if untreated.
>
>
>
> *The Challenges*
>
>
>
> *Issue #1 – Processors take too much time to select and configure
> initially*
>
>
>
> Overall, the Nifi UI is fantastic from an ease of use perspective;
> however, there is one area that’s problematic and that will become
> increasingly challenging… adding new processor blocks and choosing the
> processor type.
>
>
>
> The primary issue stems from the lengthy list of processors to scroll
> through and pick from.  The filter cloud is helpful, but with time even
> this mechanism is reaching its limits, as the number of processors
> continues to grow with the success of the project.
>
>
>
> There needs to be an easier, more productive way to simply drag and drop
> processors to the canvass.
>
>
>
> *Issue #2 – Processor blocks all look the same instead of being more
> visually distinct and easily recognizable by function*
>
>
>
> Our brains are wired for rapid pattern recognition of images, text takes
> more cycle of higher-level reasoning to interpret.
>
>
>
> Because processor blocks are shown, by default, with their I/O statistics
> and textual names, they all kind of look the same.  This “runtime view” is
> great for troubleshooting or monitoring, but not as useful when building
> complex data flows.  An “iconic view” would provide an easier way to
> visualize the structure and intent behind each graph and its flows,
> especially when developing the flows.
>
>
>
> Additionally, an icon representing each type of processor block would make
> it much faster and easier to recognize what that processor block does,
> versus having to read and interpret each one individually (especially for
> complex graphs).  Processors that handle files could be represented by a
> “file” icon, Hadoop by a Hadoop icon, HTTP by a globe icon, etc.
>
>
>
> *#3  - When dropping a new processor, open its Properties dialog
> automatically (to avoid right-click, then “Configure” and choose tab steps)*
>
>
>
> Every time a new processor is dropped onto the canvass, we must go through
> the process to select its type.  Then, the dialog closes and we’re usually
> left with an incomplete processor with errors.  We know that most
> processors require some initial Property configuration, so why not just
> proceed to that dialog after choosing the processor type, so can finish
> configuring it, then apply so we have a processor that’s ready to integrate?
>
>
>
> *Potential Solutions*
>
>
>
> Visio has a great model for addressing #1 that I would propose as a
> starting point for resolving this issue – use of a “tool palette” that
> snaps into place on the left side of the canvass.  Each group of tools
> (e.g., file-related processors, Hadoop processors, HTTP processors, etc.)
> would be grouped together within a toolbox area, with an icon representing
> each tool/processor with a brief description of each tool.
>
>
>
> As with Visio, the user would open up several commonly used toolboxes and
> then just drag and drop a tool from the toolbox directly onto the canvass,
> with no need to select the processor type (each processor is shown as a
> unique type in its toolbox).  This approach is very familiar to Visio users
> and other tools that operate in a similar manner with object drag and
> drop.  Scrolling through lengthy lists is time-consuming and becomes
> tedious when developing large graphs.
>
>
>
> Once processors have icons associated with them, several things become
> much easier:
>
>
>
> 1.       The toolboxes are much easier to create, leveraging each
> processors inherent icon representation
>
> 2.       The runtime view (current view) could simply have each
> processor’s icon shown (either in the white space to left of “5 mins” or in
> the border area)
>
> 3.       If a purely iconic view were added at some future point, then a
> clear “as built” drawing of the data flow would make the graphs even more
> self-documenting and obvious
>
>
>
> Lastly, when the generic processor is dragged onto the canvass (as it is
> today), and a processor type is selected, it would be very easy to proceed
> next to the Property dialog (if there are any mandatory properties that
> must be configured before first use), reducing the number of clicks
> required to get a processor up and running.
>
>
>
> I believe a usability study with target users would likely reveal the
> above (or similar) conclusions.  In order for Nifi to scale with dozens or
> even hundreds more processors, it’s clear that something has to give, as
> the current method of choosing processor type has about run its course from
> a usability standpoint IMHO.
>
>
>
> Hope that’s helpful.
>
>
>
> Rick
>
>
>

RE: Nifi UI Enhancements

[Rick Braddy <rb...@softnas.com>]

Good point.

I was not suggesting the generic processor picker be done away with at all, as it’s very functional, just adding a more convenient way to drag and drop processors that are most commonly used onto the canvas and adding an iconic representation to processors.

Rick

From: Ian Ragsdale [mailto:ian.ragsdale@gmail.com]
Sent: Friday, September 04, 2015 3:48 PM
To: users@nifi.apache.org
Subject: Re: Nifi UI Enhancements

Another way to look at silence is a lack of disagreement. :) I haven't yet used NiFi enough to have these things become annoyances, but I can't disagree with any of those general suggestions.

A 2x2 grid of processors down the side that would let you choose the processor type *before* dragging it onto the canvas sounds great to me, assuming that the processors have distinctive icons. I think there would likely still be a need for the processor picker, but the common use case where you know exactly what you want to add could definitely be sped up.

- Ian

On Sep 4, 2015, at 4:37 PM, Rick Braddy <rb...@softnas.com>> wrote:

Okay.  Looks like I’m the only one who thinks this is an issue…

Onward.

From: Rick Braddy [mailto:rbraddy@softnas.com]
Sent: Wednesday, September 02, 2015 11:23 PM
To: users@nifi.apache.org<ma...@nifi.apache.org>
Subject: Nifi UI Enhancements

After using Nifi for some weeks now, I have an enhancement request to recommend for the UI that I believe will dramatically improve the usability.

Before I jump into the problems, let me say this about the UI.  It’s very intuitive, easy to learn and consistent.  It’s also very clean and attractive from an appearance standpoint.  As described below, there are some areas that are becoming tedious with dozens of processors today, and that will become acute from a usability perspective if untreated.

The Challenges

Issue #1 – Processors take too much time to select and configure initially

Overall, the Nifi UI is fantastic from an ease of use perspective; however, there is one area that’s problematic and that will become increasingly challenging… adding new processor blocks and choosing the processor type.

The primary issue stems from the lengthy list of processors to scroll through and pick from.  The filter cloud is helpful, but with time even this mechanism is reaching its limits, as the number of processors continues to grow with the success of the project.

There needs to be an easier, more productive way to simply drag and drop processors to the canvass.

Issue #2 – Processor blocks all look the same instead of being more visually distinct and easily recognizable by function

Our brains are wired for rapid pattern recognition of images, text takes more cycle of higher-level reasoning to interpret.

Because processor blocks are shown, by default, with their I/O statistics and textual names, they all kind of look the same.  This “runtime view” is great for troubleshooting or monitoring, but not as useful when building complex data flows.  An “iconic view” would provide an easier way to visualize the structure and intent behind each graph and its flows, especially when developing the flows.

Additionally, an icon representing each type of processor block would make it much faster and easier to recognize what that processor block does, versus having to read and interpret each one individually (especially for complex graphs).  Processors that handle files could be represented by a “file” icon, Hadoop by a Hadoop icon, HTTP by a globe icon, etc.

#3  - When dropping a new processor, open its Properties dialog automatically (to avoid right-click, then “Configure” and choose tab steps)

Every time a new processor is dropped onto the canvass, we must go through the process to select its type.  Then, the dialog closes and we’re usually left with an incomplete processor with errors.  We know that most processors require some initial Property configuration, so why not just proceed to that dialog after choosing the processor type, so can finish configuring it, then apply so we have a processor that’s ready to integrate?

Potential Solutions

Visio has a great model for addressing #1 that I would propose as a starting point for resolving this issue – use of a “tool palette” that snaps into place on the left side of the canvass.  Each group of tools (e.g., file-related processors, Hadoop processors, HTTP processors, etc.) would be grouped together within a toolbox area, with an icon representing each tool/processor with a brief description of each tool.

As with Visio, the user would open up several commonly used toolboxes and then just drag and drop a tool from the toolbox directly onto the canvass, with no need to select the processor type (each processor is shown as a unique type in its toolbox).  This approach is very familiar to Visio users and other tools that operate in a similar manner with object drag and drop.  Scrolling through lengthy lists is time-consuming and becomes tedious when developing large graphs.

Once processors have icons associated with them, several things become much easier:


1.       The toolboxes are much easier to create, leveraging each processors inherent icon representation

2.       The runtime view (current view) could simply have each processor’s icon shown (either in the white space to left of “5 mins” or in the border area)

3.       If a purely iconic view were added at some future point, then a clear “as built” drawing of the data flow would make the graphs even more self-documenting and obvious

Lastly, when the generic processor is dragged onto the canvass (as it is today), and a processor type is selected, it would be very easy to proceed next to the Property dialog (if there are any mandatory properties that must be configured before first use), reducing the number of clicks required to get a processor up and running.

I believe a usability study with target users would likely reveal the above (or similar) conclusions.  In order for Nifi to scale with dozens or even hundreds more processors, it’s clear that something has to give, as the current method of choosing processor type has about run its course from a usability standpoint IMHO.

Hope that’s helpful.

Rick

Re: Nifi UI Enhancements

[Ian Ragsdale <ia...@gmail.com>]

Another way to look at silence is a lack of disagreement. :) I haven't yet used NiFi enough to have these things become annoyances, but I can't disagree with any of those general suggestions.

A 2x2 grid of processors down the side that would let you choose the processor type *before* dragging it onto the canvas sounds great to me, assuming that the processors have distinctive icons. I think there would likely still be a need for the processor picker, but the common use case where you know exactly what you want to add could definitely be sped up.

- Ian

> On Sep 4, 2015, at 4:37 PM, Rick Braddy <rb...@softnas.com> wrote:
> 
> Okay.  Looks like I’m the only one who thinks this is an issue…
>  
> Onward.
>  
> From: Rick Braddy [mailto:rbraddy@softnas.com <ma...@softnas.com>] 
> Sent: Wednesday, September 02, 2015 11:23 PM
> To: users@nifi.apache.org <ma...@nifi.apache.org>
> Subject: Nifi UI Enhancements
>  
> After using Nifi for some weeks now, I have an enhancement request to recommend for the UI that I believe will dramatically improve the usability.
>  
> Before I jump into the problems, let me say this about the UI.  It’s very intuitive, easy to learn and consistent.  It’s also very clean and attractive from an appearance standpoint.  As described below, there are some areas that are becoming tedious with dozens of processors today, and that will become acute from a usability perspective if untreated.
>  
> The Challenges
>  
> Issue #1 – Processors take too much time to select and configure initially
>  
> Overall, the Nifi UI is fantastic from an ease of use perspective; however, there is one area that’s problematic and that will become increasingly challenging… adding new processor blocks and choosing the processor type.
>  
> The primary issue stems from the lengthy list of processors to scroll through and pick from.  The filter cloud is helpful, but with time even this mechanism is reaching its limits, as the number of processors continues to grow with the success of the project.
>  
> There needs to be an easier, more productive way to simply drag and drop processors to the canvass.
>  
> Issue #2 – Processor blocks all look the same instead of being more visually distinct and easily recognizable by function
>  
> Our brains are wired for rapid pattern recognition of images, text takes more cycle of higher-level reasoning to interpret.
>  
> Because processor blocks are shown, by default, with their I/O statistics and textual names, they all kind of look the same.  This “runtime view” is great for troubleshooting or monitoring, but not as useful when building complex data flows.  An “iconic view” would provide an easier way to visualize the structure and intent behind each graph and its flows, especially when developing the flows.
>  
> Additionally, an icon representing each type of processor block would make it much faster and easier to recognize what that processor block does, versus having to read and interpret each one individually (especially for complex graphs).  Processors that handle files could be represented by a “file” icon, Hadoop by a Hadoop icon, HTTP by a globe icon, etc.
>  
> #3  - When dropping a new processor, open its Properties dialog automatically (to avoid right-click, then “Configure” and choose tab steps)
>  
> Every time a new processor is dropped onto the canvass, we must go through the process to select its type.  Then, the dialog closes and we’re usually left with an incomplete processor with errors.  We know that most processors require some initial Property configuration, so why not just proceed to that dialog after choosing the processor type, so can finish configuring it, then apply so we have a processor that’s ready to integrate?
>  
> Potential Solutions
>  
> Visio has a great model for addressing #1 that I would propose as a starting point for resolving this issue – use of a “tool palette” that snaps into place on the left side of the canvass.  Each group of tools (e.g., file-related processors, Hadoop processors, HTTP processors, etc.) would be grouped together within a toolbox area, with an icon representing each tool/processor with a brief description of each tool.
>  
> As with Visio, the user would open up several commonly used toolboxes and then just drag and drop a tool from the toolbox directly onto the canvass, with no need to select the processor type (each processor is shown as a unique type in its toolbox).  This approach is very familiar to Visio users and other tools that operate in a similar manner with object drag and drop.  Scrolling through lengthy lists is time-consuming and becomes tedious when developing large graphs.
>  
> Once processors have icons associated with them, several things become much easier:
>  
> 1.       The toolboxes are much easier to create, leveraging each processors inherent icon representation
> 
> 2.       The runtime view (current view) could simply have each processor’s icon shown (either in the white space to left of “5 mins” or in the border area)
> 
> 3.       If a purely iconic view were added at some future point, then a clear “as built” drawing of the data flow would make the graphs even more self-documenting and obvious
> 
>  
> Lastly, when the generic processor is dragged onto the canvass (as it is today), and a processor type is selected, it would be very easy to proceed next to the Property dialog (if there are any mandatory properties that must be configured before first use), reducing the number of clicks required to get a processor up and running.
>  
> I believe a usability study with target users would likely reveal the above (or similar) conclusions.  In order for Nifi to scale with dozens or even hundreds more processors, it’s clear that something has to give, as the current method of choosing processor type has about run its course from a usability standpoint IMHO.
>  
> Hope that’s helpful.
>  
> Rick

RE: Nifi UI Enhancements

[Rick Braddy <rb...@softnas.com>]

Okay.  Looks like I'm the only one who thinks this is an issue...

Onward.

From: Rick Braddy [mailto:rbraddy@softnas.com]
Sent: Wednesday, September 02, 2015 11:23 PM
To: users@nifi.apache.org
Subject: Nifi UI Enhancements

After using Nifi for some weeks now, I have an enhancement request to recommend for the UI that I believe will dramatically improve the usability.

Before I jump into the problems, let me say this about the UI.  It's very intuitive, easy to learn and consistent.  It's also very clean and attractive from an appearance standpoint.  As described below, there are some areas that are becoming tedious with dozens of processors today, and that will become acute from a usability perspective if untreated.

The Challenges

Issue #1 - Processors take too much time to select and configure initially

Overall, the Nifi UI is fantastic from an ease of use perspective; however, there is one area that's problematic and that will become increasingly challenging... adding new processor blocks and choosing the processor type.

The primary issue stems from the lengthy list of processors to scroll through and pick from.  The filter cloud is helpful, but with time even this mechanism is reaching its limits, as the number of processors continues to grow with the success of the project.

There needs to be an easier, more productive way to simply drag and drop processors to the canvass.

Issue #2 - Processor blocks all look the same instead of being more visually distinct and easily recognizable by function

Our brains are wired for rapid pattern recognition of images, text takes more cycle of higher-level reasoning to interpret.

Because processor blocks are shown, by default, with their I/O statistics and textual names, they all kind of look the same.  This "runtime view" is great for troubleshooting or monitoring, but not as useful when building complex data flows.  An "iconic view" would provide an easier way to visualize the structure and intent behind each graph and its flows, especially when developing the flows.

Additionally, an icon representing each type of processor block would make it much faster and easier to recognize what that processor block does, versus having to read and interpret each one individually (especially for complex graphs).  Processors that handle files could be represented by a "file" icon, Hadoop by a Hadoop icon, HTTP by a globe icon, etc.

#3  - When dropping a new processor, open its Properties dialog automatically (to avoid right-click, then "Configure" and choose tab steps)

Every time a new processor is dropped onto the canvass, we must go through the process to select its type.  Then, the dialog closes and we're usually left with an incomplete processor with errors.  We know that most processors require some initial Property configuration, so why not just proceed to that dialog after choosing the processor type, so can finish configuring it, then apply so we have a processor that's ready to integrate?

Potential Solutions

Visio has a great model for addressing #1 that I would propose as a starting point for resolving this issue - use of a "tool palette" that snaps into place on the left side of the canvass.  Each group of tools (e.g., file-related processors, Hadoop processors, HTTP processors, etc.) would be grouped together within a toolbox area, with an icon representing each tool/processor with a brief description of each tool.

As with Visio, the user would open up several commonly used toolboxes and then just drag and drop a tool from the toolbox directly onto the canvass, with no need to select the processor type (each processor is shown as a unique type in its toolbox).  This approach is very familiar to Visio users and other tools that operate in a similar manner with object drag and drop.  Scrolling through lengthy lists is time-consuming and becomes tedious when developing large graphs.

Once processors have icons associated with them, several things become much easier:


1.       The toolboxes are much easier to create, leveraging each processors inherent icon representation

2.       The runtime view (current view) could simply have each processor's icon shown (either in the white space to left of "5 mins" or in the border area)

3.       If a purely iconic view were added at some future point, then a clear "as built" drawing of the data flow would make the graphs even more self-documenting and obvious

Lastly, when the generic processor is dragged onto the canvass (as it is today), and a processor type is selected, it would be very easy to proceed next to the Property dialog (if there are any mandatory properties that must be configured before first use), reducing the number of clicks required to get a processor up and running.

I believe a usability study with target users would likely reveal the above (or similar) conclusions.  In order for Nifi to scale with dozens or even hundreds more processors, it's clear that something has to give, as the current method of choosing processor type has about run its course from a usability standpoint IMHO.

Hope that's helpful.

Rick