Test cases for Solr wiki/ref guide consolidation policy

[Jack Krupansky <ja...@basetechnology.com>]

[Is there an active Jira issue where these comments belong?]

Here are two great test case for the issue of how to switch over from the old Solr wiki to the new Confluence-based Solr Reference Guide: the terms component and the term vector component.

1. The ref guide is virtually identical to the wiki anyway for these two pages. No apparent need to move any info over or worry about loss of info. IOW, 100% overlap
2. If anybody does want to update either of these two pages, it would be nice to be clear what the status of the wiki page is. Maybe it should have a banner indicating that it is historic/archive.
3. Should the wiki be kept as is and simply add a “pointer” to the ref guide?
4. Or should the wiki be “stubbed out” and point to the ref guide only?
5. Or should the wiki page be deleted and any referencing pages in the wiki re-point to the ref guide. There may be non-Solr web pages that link to the wiki page. A 301 redirect would be nice.

There are three other interesting issues with these two test cases:

1. They both have Javadoc which, as they say, “needs some love” – should the Javadoc be updated, maintained, encouraged, etc?
2. Should a pointer to the ref guide be added to the Javadoc? There are plenty of cases in Solr where the Javadoc is spotty, missing or explicitly “TO DO”; a policy for dealing with it  is needed – may simply linking to the closest ref guide page is the next step.
3. Both have additional DEFINITIVE reference documentation... in the Solr example solrconfig.xml. How much info should go into the Javadoc vs. solrconfig vs. ref guide? Some day we will finally have multiple example config/schemas; then solrconfig might not be the best place to use as the master for doc info.

I did notice that the Javadoc for the term vector component has a copy-paste bug – it names the request handler “/terms”, while in solrconfig it is properly named “/tvrh”. The new ref guide has the same “bug”.

-- Jack Krupansky

Re: Test cases for Solr wiki/ref guide consolidation policy

[Cassandra Targett <ca...@gmail.com>]

I've been meaning to reply to this for a couple weeks now...I've been
thinking about it quite a bit.

On Sat, Aug 3, 2013 at 4:11 PM, Jack Krupansky <ja...@basetechnology.com> wrote:
> [Is there an active Jira issue where these comments belong?]

No, not really. Although SOLR-4618 is still open, that is really only
tracking issues with the original donation of the Ref Guide from
LucidWorks.

Going forward, I think it works well to have Jira issues for
individual items that need documentation (say, when it's getting time
to do a particular release and New Feature X has not been documented,
as Hoss did for the 4.4 release), or add comments to the Ref Guide
page if something needs to be improved, but open-ended questions
probably aren't really served well with never-ending Jira issues.

> Here are two great test case for the issue of how to switch over from the
> old Solr wiki to the new Confluence-based Solr Reference Guide: the terms
> component and the term vector component.
>
> 1. The ref guide is virtually identical to the wiki anyway for these two
> pages. No apparent need to move any info over or worry about loss of info.
> IOW, 100% overlap
> 2. If anybody does want to update either of these two pages, it would be
> nice to be clear what the status of the wiki page is. Maybe it should have a
> banner indicating that it is historic/archive.
> 3. Should the wiki be kept as is and simply add a “pointer” to the ref
> guide?
> 4. Or should the wiki be “stubbed out” and point to the ref guide only?
> 5. Or should the wiki page be deleted and any referencing pages in the wiki
> re-point to the ref guide. There may be non-Solr web pages that link to the
> wiki page. A 301 redirect would be nice.

All of these questions are actually addressed by this:
https://cwiki.apache.org/confluence/display/solr/Internal+-+Maintaining+Documentation#Internal-MaintainingDocumentation-Migrating%22Official%22DocumentationfromMoinMoin

It's high on my list to start work on that effort; part of the goal of
having the ref guide is to improve the available documentation and
this duplication between ref guide and wiki actually makes it
temporarily worse. The earliest authors of the Ref Guide did some
liberal copying from the wiki and for various reasons I wasn't ever
able to entirely erase it. Those pages are the easiest to migrate,
since, as with the cases you mention, they're nearly identical
already. If there aren't other priorities, then those are probably a
great place to start.

>
> There are three other interesting issues with these two test cases:
>
> 1. They both have Javadoc which, as they say, “needs some love” – should the
> Javadoc be updated, maintained, encouraged, etc?

IMO, yes. But, don't read anything into that...I'm just saying "yes, I
think Javadoc is important."

Javadoc is in the code, so it would require jira issues & patches,
etc., to fix anything...someday maybe I'll dip my toes into that but
definitely not in the short-term.

> 2. Should a pointer to the ref guide be added to the Javadoc? There are
> plenty of cases in Solr where the Javadoc is spotty, missing or explicitly
> “TO DO”; a policy for dealing with it  is needed – may simply linking to the
> closest ref guide page is the next step.

I hadn't thought about that before. My first instinct is that
"something" is better than nothing, so a link to the ref guide would
be better than "TO DO", but at the same time, that might not really
help another developer much...maybe more than nothing though.

> 3. Both have additional DEFINITIVE reference documentation... in the Solr
> example solrconfig.xml. How much info should go into the Javadoc vs.
> solrconfig vs. ref guide? Some day we will finally have multiple example
> config/schemas; then solrconfig might not be the best place to use as the
> master for doc info.

I don't think this needs to be too complicated.

Javadoc obviously is the definitive Java API documentation. If you're
developing customizations for Lucene/Solr (or contributing patches to
Lucene/Solr), it will be helpful to you. It should help you understand
what the code is doing and how, and it's generally written by
developers for other developers.

The Ref Guide is meant to give deeper context of how each feature
works, and how it interacts with other features. More of a classical
manual. You won't learn details of the code itself from the Ref Guide.
It's meant for people trying to *use* the software, as opposed to
people who want to extend the software. Some may find both useful at
different times, depending on what they're doing.

The comments in solrconfig.xml are good, I think, mostly because they
provide inline context when you're making changes. However, IMO,
solrconfig.xml (or schema.xml, or any other config) should not be the
only place something is explained - where that is the case (and there
are a few) there should be a page in the Ref Guide that explains the
same thing (and maybe also the Javadoc...but that probably depends).

Ideally, all these things are in sync. In reality, though, it's hard
to check everything in solrconfig.xml against what someone edited in
the wiki against what was in the ref guide for the last version
against what may or may not be written in the javadoc against Jira
comments and finally the actual patch...any one of those could be
out-of-date or misaligned. And I think that is why we've ended up in
this spot of having some definitive doc in solrconfig.xml or
schema.xml, some in the Javadoc, and finally some in the wiki. In some
cases they line up, in others they don't.

>
> I did notice that the Javadoc for the term vector component has a copy-paste
> bug – it names the request handler “/terms”, while in solrconfig it is
> properly named “/tvrh”. The new ref guide has the same “bug”.
>

So, this provides a good example to use - I would recommend that if
you notice that kind of thing, make a comment on the ref guide page.
Several people with edit rights get notifications of comments, and
we'll use those as a way to know what needs to get fixed. As for the
Javadoc errors, I'll defer to any of the committers who have ideas on
how to deal with that...obviously it's going to need a patch, but I
don't know how high that is on anyone's list to review, etc.

At any rate, some of this is the nature of the beast at the moment -
it's not going to be magically perfect overnight, but it should get
better over time. Suggestions from you and folks trying to use the
documentation are welcome - the doc is one area that needs attention,
and I'd like to focus on what needs fixing the most.

Cassandra

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org