Anybody reformatted the "explain" output to be more visual?

[Mark Bennett <mb...@ideaeng.com>]

I realize Solr just pulls the .toString from Lucene's explain.

It's helpful to a point, but quickly scanning over a results list doesn't
convey "the big picture".

Looking at Lucene's code, I don't think moving to .toHtml() would be much
better, as I read it you'd just get the same dense text in an HTML list
instead of a newline list.

So I guess reformatting it would likely involve parsing it first.  Getting
it into an XML snippet would allow for XSLT processing.

Some ideas that I had for visualization:
* Generally reformat each "match" paragraph to fit on one line
* Use font sizes and weights to convey relative importance.
* The field name could convey the norm for it.
* The term could possibly convey the density.
* For some of the multiplied products, use the length and width of a
rectangle, so the area is a visual aid (scaled appropriately)
* Having these various sizes of fonts and boxes on one line would also be
easier to visually "sum"
* Maybe a mouse-over for precise numerical values

Just some ideas.  Visually representing dismax would be another challenge.

--
Mark Bennett / New Idea Engineering, Inc. / mbennett@ideaeng.com
Direct: 408-733-0387 / Main: 866-IDEA-ENG / Cell: 408-829-6513

Re: Anybody reformatted the "explain" output to be more visual?

[Chris Hostetter <ho...@fucit.org>]

: So to be clear, you'd be in favor of adding some type of .toXml() to the
: explain classes, vs. trying to parse the string output.

...no i mean a helper method like...

  public static NamedList convert(Explanation exp) { ... }

...that walks the Explanation structure and converts it to a NmaedList 
structure.  that way it would convey the structure with any response 
writer (xml, json, etc...)

we can't add solr specific methods directly to the Explanation classes 
(they're in Lucene core and already provide a programatic way to inspect 
the explanations)



-Hoss

Re: Anybody reformatted the "explain" output to be more visual?

[Mark Bennett <mb...@ideaeng.com>]

Thanks Hos,

So to be clear, you'd be in favor of adding some type of .toXml() to the
explain classes, vs. trying to parse the string output.

And then I guess a default implementation nesting a string in XML for
operators that are complex and where the author hasn't added a .toXml() to
their stuff.  My only anxiety is the number of classes this might effect,
I'm not clear on the relationship between operators like "dismax" in Solr
and Lucene, and the base explain stuff in Lucene.  Would adding a .toXml()
method require modifications to all operator classes to be of use?

In the XML it'd be nice to identify numeric fields.  And to possibly call
out operands and results.  Maybe have a field that conveys "final score"
(for this branch), etc.

if there's interest, maybe I could mock up a draft of what the XML might
look like.

--
Mark Bennett / New Idea Engineering, Inc. / mbennett@ideaeng.com
Direct: 408-733-0387 / Main: 866-IDEA-ENG / Cell: 408-829-6513


On Tue, Jul 28, 2009 at 12:28 PM, Chris Hostetter
<ho...@fucit.org>wrote:

>
> once upon a time, pre Apache, i had some really simple code to walk an
> Explanation tree and convert it into nested NamedLists ... but back then
> there was a bug in
> the XmlresponseWRiter code that made it break on structures deeper then
> some magic number so i tossed the code and just used the
> explantation.toString in the debug output instead.
>
> a patch to add functionality like that back in would be fairly straight
> forwrad if someone wanted to tackle it.
>
>
>
> : Date: Tue, 28 Jul 2009 11:14:48 -0700
> : From: Mark Bennett <mb...@ideaeng.com>
> : Reply-To: solr-user@lucene.apache.org
> : To: solr-user@lucene.apache.org
> : Subject: Anybody reformatted the "explain" output to be more visual?
> :
> : I realize Solr just pulls the .toString from Lucene's explain.
> :
> : It's helpful to a point, but quickly scanning over a results list doesn't
> : convey "the big picture".
> :
> : Looking at Lucene's code, I don't think moving to .toHtml() would be much
> : better, as I read it you'd just get the same dense text in an HTML list
> : instead of a newline list.
> :
> : So I guess reformatting it would likely involve parsing it first.
>  Getting
> : it into an XML snippet would allow for XSLT processing.
> :
> : Some ideas that I had for visualization:
> : * Generally reformat each "match" paragraph to fit on one line
> : * Use font sizes and weights to convey relative importance.
> : * The field name could convey the norm for it.
> : * The term could possibly convey the density.
> : * For some of the multiplied products, use the length and width of a
> : rectangle, so the area is a visual aid (scaled appropriately)
> : * Having these various sizes of fonts and boxes on one line would also be
> : easier to visually "sum"
> : * Maybe a mouse-over for precise numerical values
> :
> : Just some ideas.  Visually representing dismax would be another
> challenge.
> :
> : --
> : Mark Bennett / New Idea Engineering, Inc. / mbennett@ideaeng.com
> : Direct: 408-733-0387 / Main: 866-IDEA-ENG / Cell: 408-829-6513
> :
>
>
>
> -Hoss
>
>

Re: Anybody reformatted the "explain" output to be more visual?

[Chris Hostetter <ho...@fucit.org>]

once upon a time, pre Apache, i had some really simple code to walk an 
Explanation tree and convert it into nested NamedLists ... but back then there was a bug in 
the XmlresponseWRiter code that made it break on structures deeper then 
some magic number so i tossed the code and just used the 
explantation.toString in the debug output instead.

a patch to add functionality like that back in would be fairly straight 
forwrad if someone wanted to tackle it.



: Date: Tue, 28 Jul 2009 11:14:48 -0700
: From: Mark Bennett <mb...@ideaeng.com>
: Reply-To: solr-user@lucene.apache.org
: To: solr-user@lucene.apache.org
: Subject: Anybody reformatted the "explain" output to be more visual?
: 
: I realize Solr just pulls the .toString from Lucene's explain.
: 
: It's helpful to a point, but quickly scanning over a results list doesn't
: convey "the big picture".
: 
: Looking at Lucene's code, I don't think moving to .toHtml() would be much
: better, as I read it you'd just get the same dense text in an HTML list
: instead of a newline list.
: 
: So I guess reformatting it would likely involve parsing it first.  Getting
: it into an XML snippet would allow for XSLT processing.
: 
: Some ideas that I had for visualization:
: * Generally reformat each "match" paragraph to fit on one line
: * Use font sizes and weights to convey relative importance.
: * The field name could convey the norm for it.
: * The term could possibly convey the density.
: * For some of the multiplied products, use the length and width of a
: rectangle, so the area is a visual aid (scaled appropriately)
: * Having these various sizes of fonts and boxes on one line would also be
: easier to visually "sum"
: * Maybe a mouse-over for precise numerical values
: 
: Just some ideas.  Visually representing dismax would be another challenge.
: 
: --
: Mark Bennett / New Idea Engineering, Inc. / mbennett@ideaeng.com
: Direct: 408-733-0387 / Main: 866-IDEA-ENG / Cell: 408-829-6513
: 



-Hoss