You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bloodhound.apache.org by Apache Bloodhound <bl...@incubator.apache.org> on 2012/11/22 11:32:55 UTC
[Apache Bloodhound] Proposals/BEP-0004 added
Page "Proposals/BEP-0004" was added by andrej
Content:
-------8<------8<------8<------8<------8<------8<------8<------8<--------
= BEP 4 : Improved search architecture #overview
[[PageOutline]]
|| '''PEP''' || 4 ||
|| '''Title''' || Improved search architecture ||
|| '''Version''' || ||
|| '''Last-Modified''' || ||
|| '''Author''' || The Bloodhound project ||
|| '''Status''' || Draft ||
|| '''Type''' || Standards Trac ||
|| '''Content-Type''' || [wiki:PageTemplates/Proposals text/x-trac-wiki] ||
|| '''Created''' || ||
|| '''Post-History''' || ||
----
== Abstract #abstract
This document is meant to gather community consensus on the implementation of the improved search functionality. Simultaneously, we want to merge search and custom query functionality under the same umbrella. This is the place to discuss it and make proposals. Main discussion must be placed in mail list with [BEP-0004] subject.
== Motivation ==
Nowadays modern issue tracking system should provide advanced free text search functionality combine with flexible query mechanism. There are a lot of user requests asking for better and customizable search.
The term project is very generic and may be confusing considering the context. Therefore in this specification the word product is used instead.
== Rationale #rationale
=== Quick search box
Quick search box should be just frontend for adhoc query. For example, if user searches for “bla” string, user receives Search Result view with the following query: text~=”bla”. User can refine query in Query Result view.
Search box should accept the following search types:
- free-text search: “bla”
- free-text + query: bla status!=closed
- query only: status!=closed
Other nice to have features:
- Did you mean...
- Suggestions during typing
=== Query Result view
Query Result view represents consistent view of query result for different resources. Result may be represented as
- All resources result tab - default, common for all resources columns e.g. name and description
- Resource result tabs - resource specific fields are shown e.g. id, status, summary for ticket.
- Query Result view can be instructed to limit result to specific resource type e.g. show only tickets in result. In this case, resource tabs can be omitted.
User can refine search conditions either by editing query or by using Query Builder.
User can specify what fields should be selected and in what results order should be applied. A new order type is introduced for free-text search: score
Query Result view should support facets (http://searchhub.org/2009/09/02/faceted-search-with-solr/) for example:
- All resources result tab can show facets for resource type and product
- Ticket result tab can show facets for products and ticket status
Other nice to have features in no particular order:
- Possibility to save a query for specific user and sharing of saved queries
- Search highlighting
=== Resource Query
New Resource Query should provide the following functionality:
- free-text search support
- facet support
- it is a superset of TracQuery functionality
- basic query expressions AND, OR, NOT, search by specific field, search [FROM TO] - TBD (can be similar to SQL or lucene/solr like)
- search through different resources not only tickets: wiki, milestones, changesets and other pluggable resources
- search through all resource fields
- search through attachments, history and comments
- multi-product aware - apply security context etc
- order by free-text score. Score calculation can be configured, for example if found in summary: id: score:100, score*10, in keywords: score*5, in components: score*3 …
Nice to have features
- Support search in attachments of specific types e.g word, exel etc.
=== Resource Query macro
Similar to existing TracQuery, New macro must be introduced to enable access to Resource Query from Wiki.
=== Query Builder
A lot improvements may be introduced here, TBD
== Proposal #proposal
=== Implementation steps
We can start with small step by implementing search box functionality similar to trac:wiki:AdvancedSearch and then add support for query, wiki macro and query builder/wizard.
=== Possible free text platforms
A few possible solutions were discussed on the mail list:
==== Whoosh
Whoosh links:
- Product page: http://pypi.python.org/pypi/Whoosh/
- Discussion on implementing Trac search using Whoosh: https://groups.google.com/forum/?fromgroups=#!msg/trac-dev/sbU-g0C6kvk/1_juL29aAtQJ
==== !PyLucen
[http://lucene.apache.org/pylucene/ PyLucene] is not a Lucene port but a Python wrapper around Java Lucene. !PyLucene embeds a Java VM with Lucene into a Python process.
==== Java-based servers.
Solr or !ElasticSearch
== Rejected ideas
== Backwards Compatibility #backwards-compatibility
TracQuery implementation must be changed to use Resource Query for data retrieval.
Existing TracSearch syntax will be mapped to Resource Query.
== Reference Implementation #reference-implementation
== Resources #resources
== References #references
Proposals/BEP-0003 - Multi-product architecture
=== Trac proposals on this subject
- trac:wiki:AdvancedSearch
- trac:wiki:SearchRefactoring
=== Plugins on this subject
- http://trac-hacks.org/wiki/FullTextSearchPlugin from Logica
- http://trac-hacks.org/wiki/TracAdvancedSearchPlugin - uses Solr, claims to be designed with extensions points
=== Existing search and query implementation
- TracQuery
- TracSearch
== Copyright #copyright
Copyright © 2009-2012 The [http://www.apache.org Apache Software Foundation] [[BR]]
Licensed under the [http://www.apache.org/licenses/LICENSE-2.0 Apache License, Version 2.0].
Apache Bloodhound, Apache, the Apache feather logo, and the Apache Bloodhound project logo are trademarks of The Apache Software Foundation.
-------8<------8<------8<------8<------8<------8<------8<------8<--------
--
Page URL: <https://issues.apache.org/bloodhound/wiki/Proposals/BEP-0004>
Apache Bloodhound <https://issues.apache.org/bloodhound/>
The Apache Bloodhound (incubating) issue tracker
This is an automated message. Someone added your email address to be
notified of changes on 'Proposals/BEP-0004' page.
If it was not you, please report to .
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Gary Martin <ga...@wandisco.com>.
On 22/11/12 18:50, Peter Koželj wrote:
> Changing the tab also changes the columns displayed (based on what that
> resource supports).
> But we can do that with checkboxes as well (show intersection of columns)
> so maybe why not.
>
> At the end it comes down to what do you need more often.
> If we can come up with a way so that one does not need two clicks every
> time to switch from tickets to wikis for example,
> I welcome the idea.
I do agree with that. It would still be possible to have available
columns changing as you change the query, of course, but it seems to me
that this could be a bit confusing. I suspect that categorisation with
tabs would reduce confusion. Saying that, I am not an expert and I don't
mind seeing a few competing designs, as long as there is not too much
excess effort involved.
Cheers,
Gary
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Branko Čibej <br...@wandisco.com>.
On 22.11.2012 19:50, Peter Koželj wrote:
> Changing the tab also changes the columns displayed (based on what that
> resource supports).
> But we can do that with checkboxes as well (show intersection of columns)
> so maybe why not.
>
> At the end it comes down to what do you need more often.
> If we can come up with a way so that one does not need two clicks every
> time to switch from tickets to wikis for example,
> I welcome the idea.
Isn't it the Trac Way to just write five different plugins for the
various layout styles? :)
-- Brane
--
Branko Čibej
Director of Subversion | WANdisco | www.wandisco.com
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Joachim Dreimann <jo...@wandisco.com>.
On 23 November 2012 04:45, Olemis Lang <ol...@gmail.com> wrote:
> ike I mentioned before , even if queries and reports are yet another
> kind of search engine , there is a difference between them and free
> text search . For instance , reports and queries have some focus on
> presentation , hence visible columns may be selected to satisfy some
> information needs . Nonetheless in free text search findability is the
> main goal over everything else . That's why the first point mentioned
> above does not apply .
>
> The second suggestion I made seems more appropriate for products ,
> milestones , ... but in the case of tickets , for instance they may be
> annotated (in a similar way to Google search results I mentioned
> above) with # of comments , # of attachments , ... each pointing to
> the specific section within ticket view .
>
> To make myself clear , IMO none of them is suitable for query or
>
I agree that reports serve a different purpose (or rather will do by our
previous conversations).
Search and Custom Query are very similar though in regards to the users
initial intention for using them.
One issue with Custom Query is that it expects the user to know which
criteria to use beforehand - there's no real discoverability.
Facets are a great solution for that. They suggest what factors are in
common for all search results and allow the user to discover useful further
filters.
In addition to screen size, there's a big difference of context on mobile
and desktop search/queries: Most likely users will use mobile phones to do
navigational searches or check something quickly, not go through the
elaborate workflow of iterating query context and making batch changes.
It's okay if the experience is different at first because it will take us a
long time to refine the system sufficiently to expose all of the
functionality on mobiles. That shouldn't stop progress on desktop or laptop
sized screens.
- Joe
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Olemis Lang <ol...@gmail.com>.
On 11/22/12, Gary Martin <ga...@wandisco.com> wrote:
> On 23/11/12 00:34, Olemis Lang wrote:
>> Good answer , indeed !
>> :)
>>
>> On 11/22/12, Andrej Golcov <an...@digiverse.si> wrote:
[...]
>>>> what is the reason for using tables and columns to display search
>>>> results
>>> ?
>>> The idea is that we combine free-text search and query. If we want to
>>> provide readable query result it can be possible to represent it as
>>> table. It is aslo consistent to existing TracQuery result view.
>>>
>> I asked mainly because I've been taking a look at several search UX
>> solutions for mobiles and it turns out that they all have no more than
>> 3 columns ... and when they have the third one it's most of the time a
>> checkbox , download indicator , ... i.e. something tiny .
>
> Well, this is a problem that we would have to overcome for ticket
> queries anyway.
>
Maybe , yes , if we take a straight look at it . However it seems to
me that ticket queries serve to a different purpose than free text
search . In that case it is also important to make results look
somehow like users want to . With mobile devices it is still possible
to e.g. hide columns ... until everything fits . That's not the goal
of free text search .
[...]
>> For the debate : What about having a single column (e.g. to the right)
>> for any kinds of search results metadata displayed in the form of
>> clickable metadata [1]_ and stack it under summary + full text hit
>> snippets for tablets & smartphones ?
>>
>> OTOH , regarding meta-data , it'd be nice to have some other tags
>> attached to search results in a way similar to what is displayed for
>> Hadoop Common when performing this search
>>
>> https://www.google.com/search?client=opera&rls=en&q=jira+apache&sourceid=opera&ie=utf-8&oe=utf-8&channel=suggest
>
> I guess that there are a number of possible solutions for display. Are
> any of these an approach that you would be advocating for the current
> ticket queries and reports?
Like I mentioned before , even if queries and reports are yet another
kind of search engine , there is a difference between them and free
text search . For instance , reports and queries have some focus on
presentation , hence visible columns may be selected to satisfy some
information needs . Nonetheless in free text search findability is the
main goal over everything else . That's why the first point mentioned
above does not apply .
The second suggestion I made seems more appropriate for products ,
milestones , ... but in the case of tickets , for instance they may be
annotated (in a similar way to Google search results I mentioned
above) with # of comments , # of attachments , ... each pointing to
the specific section within ticket view .
To make myself clear , IMO none of them is suitable for query or
reports web UI , because that serves to slightly different purpose .
--
Regards,
Olemis.
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Featured article:
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Andrej Golcov <an...@digiverse.si>.
> Talking of this, are we including a means to specify which fields are
> displayed in the tickets tab as part of the requirements?
Resource Query should definitely support specifying which fields to return.
In BEP it is covered by "it is a superset of TracQuery functionality". So,
Query Result view can give possibility to user to select wich fileds to
query.
> I guess that there are a number of possible solutions for display. Are
> any of these an approach that you would be advocating for the current
> ticket queries and reports? I think I'll reserve judgement until I
> play around with some ideas for a bit longer.
Agreed. I suggest that we concentrate on Resource Query functionality and
provide minimalistic search result rendering, similar to existing search
and then play around it to see what is the best presentation approach. Any
way, that should be done it in pluggable way ;)
Regards, Andrej
On 23 November 2012 02:12, Gary Martin <ga...@wandisco.com> wrote:
> On 23/11/12 00:34, Olemis Lang wrote:
>
>> Good answer , indeed !
>> :)
>>
>> On 11/22/12, Andrej Golcov <an...@digiverse.si> wrote:
>>
>>> IMO when a user searches for something , (s)he might not know where
>>>> exactly to find it but have a fuzzy idea . IMO we should consider
>>>> having multiple selections via checkboxes rather than tabs .
>>>>
>>> If user does exactly know what resource (s)he is searching, "All" tab is
>>> used where query can be edited to show only tickets and milestones.
>>> IMHO, user usually knows what type (s)he needs (I believe, most of search
>>> is for tickets) and ,IMO, tabs make things easier and less confusing.
>>>
>>> Each tab has it's own fields set. Combination of checkboxes and different
>>> grids can confuse.
>>>
>>> Ok . This is something that depends on the type of application and its
>> users ... so no big deal about it .
>>
>> ;)
>>
>> what is the reason for using tables and columns to display search results
>>>>
>>> ?
>>> The idea is that we combine free-text search and query. If we want to
>>> provide readable query result it can be possible to represent it as
>>> table. It is aslo consistent to existing TracQuery result view.
>>>
>>> I asked mainly because I've been taking a look at several search UX
>> solutions for mobiles and it turns out that they all have no more than
>> 3 columns ... and when they have the third one it's most of the time a
>> checkbox , download indicator , ... i.e. something tiny .
>>
> Well, this is a problem that we would have to overcome for ticket queries
> anyway.
>
>
> Besides there is another fact . Free text search is a bit different to
>> a ticket report . Especially highlighting means that maybe snippets of
>> long text will be displayed to put the match in context . So in that
>> view there will be basically two kinds of data
>>
>> 1. Highlighted excerpts of long texts (wiki page body ,
>> ticket description , custom textarea ticket field ...)
>> 2. Relatively small (potentially tiny) text (i.e. ticket status,
>> ticket type, dates ...)
>>
>
> The highlighted excerpts approach would be a more natural fit for the
> 'All' tab than for the ticket query results. It doesn't mean that the free
> text search should not be able to apply to those results.
>
> From the ticket query side, one of the enhancements this work could
> provide, if accepted, is this free text search. It seems a much simpler
> query to specify than having to state all the fields you might expect to
> find the text in!
>
>
> They all will have relatively equal relevance competing for horizontal
>> space (i.e. width)
>>
>> I wander how will proposed solution (the user experience as a whole)
>> look like using responsive layout or for smartphones and tablets ?
>>
>> Another question I have is related to the purpose of the content
>> included in those columns ? What are they for ?
>>
>
> Talking of this, are we including a means to specify which fields are
> displayed in the tickets tab as part of the requirements?
>
>
> For the debate : What about having a single column (e.g. to the right)
>> for any kinds of search results metadata displayed in the form of
>> clickable metadata [1]_ and stack it under summary + full text hit
>> snippets for tablets & smartphones ?
>>
>> OTOH , regarding meta-data , it'd be nice to have some other tags
>> attached to search results in a way similar to what is displayed for
>> Hadoop Common when performing this search
>>
>> https://www.google.com/search?**client=opera&rls=en&q=jira+**
>> apache&sourceid=opera&ie=utf-**8&oe=utf-8&channel=suggest<https://www.google.com/search?client=opera&rls=en&q=jira+apache&sourceid=opera&ie=utf-8&oe=utf-8&channel=suggest>
>>
>
> I guess that there are a number of possible solutions for display. Are any
> of these an approach that you would be advocating for the current ticket
> queries and reports? I think I'll reserve judgement until I play around
> with some ideas for a bit longer.
>
> Cheers,
> Gary
>
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Gary Martin <ga...@wandisco.com>.
On 23/11/12 00:34, Olemis Lang wrote:
> Good answer , indeed !
> :)
>
> On 11/22/12, Andrej Golcov <an...@digiverse.si> wrote:
>>> IMO when a user searches for something , (s)he might not know where
>>> exactly to find it but have a fuzzy idea . IMO we should consider
>>> having multiple selections via checkboxes rather than tabs .
>> If user does exactly know what resource (s)he is searching, "All" tab is
>> used where query can be edited to show only tickets and milestones.
>> IMHO, user usually knows what type (s)he needs (I believe, most of search
>> is for tickets) and ,IMO, tabs make things easier and less confusing.
>>
>> Each tab has it's own fields set. Combination of checkboxes and different
>> grids can confuse.
>>
> Ok . This is something that depends on the type of application and its
> users ... so no big deal about it .
>
> ;)
>
>>> what is the reason for using tables and columns to display search results
>> ?
>> The idea is that we combine free-text search and query. If we want to
>> provide readable query result it can be possible to represent it as
>> table. It is aslo consistent to existing TracQuery result view.
>>
> I asked mainly because I've been taking a look at several search UX
> solutions for mobiles and it turns out that they all have no more than
> 3 columns ... and when they have the third one it's most of the time a
> checkbox , download indicator , ... i.e. something tiny .
Well, this is a problem that we would have to overcome for ticket
queries anyway.
> Besides there is another fact . Free text search is a bit different to
> a ticket report . Especially highlighting means that maybe snippets of
> long text will be displayed to put the match in context . So in that
> view there will be basically two kinds of data
>
> 1. Highlighted excerpts of long texts (wiki page body ,
> ticket description , custom textarea ticket field ...)
> 2. Relatively small (potentially tiny) text (i.e. ticket status,
> ticket type, dates ...)
The highlighted excerpts approach would be a more natural fit for the
'All' tab than for the ticket query results. It doesn't mean that the
free text search should not be able to apply to those results.
From the ticket query side, one of the enhancements this work could
provide, if accepted, is this free text search. It seems a much simpler
query to specify than having to state all the fields you might expect to
find the text in!
> They all will have relatively equal relevance competing for horizontal
> space (i.e. width)
>
> I wander how will proposed solution (the user experience as a whole)
> look like using responsive layout or for smartphones and tablets ?
>
> Another question I have is related to the purpose of the content
> included in those columns ? What are they for ?
Talking of this, are we including a means to specify which fields are
displayed in the tickets tab as part of the requirements?
> For the debate : What about having a single column (e.g. to the right)
> for any kinds of search results metadata displayed in the form of
> clickable metadata [1]_ and stack it under summary + full text hit
> snippets for tablets & smartphones ?
>
> OTOH , regarding meta-data , it'd be nice to have some other tags
> attached to search results in a way similar to what is displayed for
> Hadoop Common when performing this search
>
> https://www.google.com/search?client=opera&rls=en&q=jira+apache&sourceid=opera&ie=utf-8&oe=utf-8&channel=suggest
I guess that there are a number of possible solutions for display. Are
any of these an approach that you would be advocating for the current
ticket queries and reports? I think I'll reserve judgement until I play
around with some ideas for a bit longer.
Cheers,
Gary
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Olemis Lang <ol...@gmail.com>.
Good answer , indeed !
:)
On 11/22/12, Andrej Golcov <an...@digiverse.si> wrote:
>> IMO when a user searches for something , (s)he might not know where
>> exactly to find it but have a fuzzy idea . IMO we should consider
>> having multiple selections via checkboxes rather than tabs .
>
> If user does exactly know what resource (s)he is searching, "All" tab is
> used where query can be edited to show only tickets and milestones.
> IMHO, user usually knows what type (s)he needs (I believe, most of search
> is for tickets) and ,IMO, tabs make things easier and less confusing.
>
> Each tab has it's own fields set. Combination of checkboxes and different
> grids can confuse.
>
Ok . This is something that depends on the type of application and its
users ... so no big deal about it .
;)
>> what is the reason for using tables and columns to display search results
> ?
> The idea is that we combine free-text search and query. If we want to
> provide readable query result it can be possible to represent it as
> table. It is aslo consistent to existing TracQuery result view.
>
I asked mainly because I've been taking a look at several search UX
solutions for mobiles and it turns out that they all have no more than
3 columns ... and when they have the third one it's most of the time a
checkbox , download indicator , ... i.e. something tiny .
Besides there is another fact . Free text search is a bit different to
a ticket report . Especially highlighting means that maybe snippets of
long text will be displayed to put the match in context . So in that
view there will be basically two kinds of data
1. Highlighted excerpts of long texts (wiki page body ,
ticket description , custom textarea ticket field ...)
2. Relatively small (potentially tiny) text (i.e. ticket status,
ticket type, dates ...)
They all will have relatively equal relevance competing for horizontal
space (i.e. width)
I wander how will proposed solution (the user experience as a whole)
look like using responsive layout or for smartphones and tablets ?
Another question I have is related to the purpose of the content
included in those columns ? What are they for ?
For the debate : What about having a single column (e.g. to the right)
for any kinds of search results metadata displayed in the form of
clickable metadata [1]_ and stack it under summary + full text hit
snippets for tablets & smartphones ?
OTOH , regarding meta-data , it'd be nice to have some other tags
attached to search results in a way similar to what is displayed for
Hadoop Common when performing this search
https://www.google.com/search?client=opera&rls=en&q=jira+apache&sourceid=opera&ie=utf-8&oe=utf-8&channel=suggest
>> Is there any other search engine considered as a reference ?
> jira and youtrack return kind of table fro free text-search result.
>
about YouTrack , I implemented a client once upon a time , and I
really liked what I saw . AFAICR it had an interesting query syntax .
Maybe something to take a look at .
.. [1] Clickable metadata
(http://ebiinterfaces.files.wordpress.com/2010/08/webinar-designing-the-search-experience.pdf)
PS: I ask because I've been paying special attention to UI design
after theme work ... so I'm curious and interested in fleshing out UI
stuff ... and learn in the process . Hope you don't mind
;)
--
Regards,
Olemis.
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Featured article:
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Andrej Golcov <an...@digiverse.si>.
> IMO when a user searches for something , (s)he might not know where
> exactly to find it but have a fuzzy idea . IMO we should consider
> having multiple selections via checkboxes rather than tabs .
If user does exactly know what resource (s)he is searching, "All" tab is
used where query can be edited to show only tickets and milestones.
IMHO, user usually knows what type (s)he needs (I believe, most of search
is for tickets) and ,IMO, tabs make things easier and less confusing.
Each tab has it's own fields set. Combination of checkboxes and different
grids can confuse.
> what is the reason for using tables and columns to display search results
?
The idea is that we combine free-text search and query. If we want to
provide readable query result it can be possible to represent it as
table. It is aslo consistent to existing TracQuery result view.
> Is there any other search engine considered as a reference ?
jira and youtrack return kind of table fro free text-search result.
Regards, Andrej
On 22 November 2012 21:47, Olemis Lang <ol...@gmail.com> wrote:
> On 11/22/12, Peter Koželj <pe...@digiverse.si> wrote:
> > On 22 November 2012 19:19, Joachim Dreimann
> > <jo...@wandisco.com>wrote:
> >> On 22 November 2012 18:06, Olemis Lang <ol...@gmail.com> wrote:
> >> > >>> On 22/11/12 11:08, Andrej Golcov wrote:
> [...]
> >> > >>>>> Query Result view should support facets (
> >> > >>>>> http://searchhub.org/2009/09/**02/faceted-search-with-solr/<
> >> > http://searchhub.org/2009/09/02/faceted-search-with-solr/>)
> >> > >>>>> for example:
> >> > >>>>> - All resources result tab can show facets for resource type
> >> > >>>>> and
> >> > >>>>> product
> >> > >>>>> - Ticket result tab can show facets for products and ticket
> >> status
> >> > >>>>>
> >> >
> >> > +1
> >> >
> >> > Nonetheless I'd prefer checkboxes rather than tabs , in order to
> >> > choose multiple options (e.g. tickets and changesets ...) .
> >> >
> >> > What d'u think ?
> >>
> >> I missed that point earlier. Yes, I also prefer checkboxes over tabs for
> >> the extra flexibility they give to multi-select. I would have expected
> to
> >> be able to select multiple facets to show too.
> >>
> >> - Joe
> >>
> > Changing the tab also changes the columns displayed (based on what that
> > resource supports).
> > But we can do that with checkboxes as well (show intersection of columns)
> > so maybe why not.
> >
> > At the end it comes down to what do you need more often.
> > If we can come up with a way so that one does not need two clicks every
> > time to switch from tickets to wikis for example,
> > I welcome the idea.
> >
> >
>
> IMO when a user searches for something , (s)he might not know where
> exactly to find it but have a fuzzy idea . IMO we should consider
> having multiple selections via checkboxes rather than tabs .
>
> btw
>
> what is the reason for using tables and columns to display search results ?
> Is there any other search engine considered as a reference ?
>
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Branko Čibej <br...@wandisco.com>.
On 22.11.2012 21:47, Olemis Lang wrote:
> On 11/22/12, Peter Koželj <pe...@digiverse.si> wrote:
>> On 22 November 2012 19:19, Joachim Dreimann
>> <jo...@wandisco.com>wrote:
>>> On 22 November 2012 18:06, Olemis Lang <ol...@gmail.com> wrote:
>>>>>>> On 22/11/12 11:08, Andrej Golcov wrote:
> [...]
>>>>>>>>> Query Result view should support facets (
>>>>>>>>> http://searchhub.org/2009/09/**02/faceted-search-with-solr/<
>>>> http://searchhub.org/2009/09/02/faceted-search-with-solr/>)
>>>>>>>>> for example:
>>>>>>>>> - All resources result tab can show facets for resource type
>>>>>>>>> and
>>>>>>>>> product
>>>>>>>>> - Ticket result tab can show facets for products and ticket
>>> status
>>>> +1
>>>>
>>>> Nonetheless I'd prefer checkboxes rather than tabs , in order to
>>>> choose multiple options (e.g. tickets and changesets ...) .
>>>>
>>>> What d'u think ?
>>> I missed that point earlier. Yes, I also prefer checkboxes over tabs for
>>> the extra flexibility they give to multi-select. I would have expected to
>>> be able to select multiple facets to show too.
>>>
>>> - Joe
>>>
>> Changing the tab also changes the columns displayed (based on what that
>> resource supports).
>> But we can do that with checkboxes as well (show intersection of columns)
>> so maybe why not.
>>
>> At the end it comes down to what do you need more often.
>> If we can come up with a way so that one does not need two clicks every
>> time to switch from tickets to wikis for example,
>> I welcome the idea.
>>
>>
> IMO when a user searches for something , (s)he might not know where
> exactly to find it but have a fuzzy idea . IMO we should consider
> having multiple selections via checkboxes rather than tabs .
>
> btw
>
> what is the reason for using tables and columns to display search results ?
> Is there any other search engine considered as a reference ?
Yes, pretty much every issue tracker with a halfway useful search, and
even some without.
See for http://goo.gl/0eM3f or http://goo.gl/4KkTk and even
http://trac.edgewall.org/report/1 all of which return query results in
tabular format.
-- Brane
--
Branko Čibej
Director of Subversion | WANdisco | www.wandisco.com
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Olemis Lang <ol...@gmail.com>.
On 11/22/12, Peter Koželj <pe...@digiverse.si> wrote:
> On 22 November 2012 19:19, Joachim Dreimann
> <jo...@wandisco.com>wrote:
>> On 22 November 2012 18:06, Olemis Lang <ol...@gmail.com> wrote:
>> > >>> On 22/11/12 11:08, Andrej Golcov wrote:
[...]
>> > >>>>> Query Result view should support facets (
>> > >>>>> http://searchhub.org/2009/09/**02/faceted-search-with-solr/<
>> > http://searchhub.org/2009/09/02/faceted-search-with-solr/>)
>> > >>>>> for example:
>> > >>>>> - All resources result tab can show facets for resource type
>> > >>>>> and
>> > >>>>> product
>> > >>>>> - Ticket result tab can show facets for products and ticket
>> status
>> > >>>>>
>> >
>> > +1
>> >
>> > Nonetheless I'd prefer checkboxes rather than tabs , in order to
>> > choose multiple options (e.g. tickets and changesets ...) .
>> >
>> > What d'u think ?
>>
>> I missed that point earlier. Yes, I also prefer checkboxes over tabs for
>> the extra flexibility they give to multi-select. I would have expected to
>> be able to select multiple facets to show too.
>>
>> - Joe
>>
> Changing the tab also changes the columns displayed (based on what that
> resource supports).
> But we can do that with checkboxes as well (show intersection of columns)
> so maybe why not.
>
> At the end it comes down to what do you need more often.
> If we can come up with a way so that one does not need two clicks every
> time to switch from tickets to wikis for example,
> I welcome the idea.
>
>
IMO when a user searches for something , (s)he might not know where
exactly to find it but have a fuzzy idea . IMO we should consider
having multiple selections via checkboxes rather than tabs .
btw
what is the reason for using tables and columns to display search results ?
Is there any other search engine considered as a reference ?
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Peter Koželj <pe...@digiverse.si>.
Changing the tab also changes the columns displayed (based on what that
resource supports).
But we can do that with checkboxes as well (show intersection of columns)
so maybe why not.
At the end it comes down to what do you need more often.
If we can come up with a way so that one does not need two clicks every
time to switch from tickets to wikis for example,
I welcome the idea.
On 22 November 2012 19:19, Joachim Dreimann
<jo...@wandisco.com>wrote:
> I missed that point earlier. Yes, I also prefer checkboxes over tabs for
> the extra flexibility they give to multi-select. I would have expected to
> be able to select multiple facets to show too.
>
> - Joe
>
> On 22 November 2012 18:06, Olemis Lang <ol...@gmail.com> wrote:
>
> > On 11/22/12, Olemis Lang <ol...@gmail.com> wrote:
> > > On 11/22/12, Joachim Dreimann <jo...@wandisco.com> wrote:
> > >> On 22 November 2012 12:56, Gary Martin <ga...@wandisco.com>
> > wrote:
> > >>> On 22/11/12 11:08, Andrej Golcov wrote:
> > >>>
> > > [...]
> > >>>>>
> > >>>>> Query Result view should support facets (
> > >>>>> http://searchhub.org/2009/09/**02/faceted-search-with-solr/<
> > http://searchhub.org/2009/09/02/faceted-search-with-solr/>)
> > >>>>> for example:
> > >>>>> - All resources result tab can show facets for resource type and
> > >>>>> product
> > >>>>> - Ticket result tab can show facets for products and ticket
> status
> > >>>>>
> > >>>>
> > >>> Facets seem to me to be a very important enhancement.
> > >>>
> > >>
> > >> I agree. This would be a great feature to have and build upon.
> > >>
> > >
> > > +1
> > >
> >
> > Nonetheless I'd prefer checkboxes rather than tabs , in order to
> > choose multiple options (e.g. tickets and changesets ...) .
> >
> > What d'u think ?
> >
> > --
> > Regards,
> >
> > Olemis.
> >
> > Blog ES: http://simelo-es.blogspot.com/
> > Blog EN: http://simelo-en.blogspot.com/
> >
> > Featured article:
> >
>
>
>
> --
> Joe Dreimann
> UX Designer | WANdisco <http://www.wandisco.com/>
> *
> *
> *Transform your software development department. Register for a free SVN
> HealthCheck <http://go.wandisco.com/HealthCheck-Sig.html> *
>
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Joachim Dreimann <jo...@wandisco.com>.
I missed that point earlier. Yes, I also prefer checkboxes over tabs for
the extra flexibility they give to multi-select. I would have expected to
be able to select multiple facets to show too.
- Joe
On 22 November 2012 18:06, Olemis Lang <ol...@gmail.com> wrote:
> On 11/22/12, Olemis Lang <ol...@gmail.com> wrote:
> > On 11/22/12, Joachim Dreimann <jo...@wandisco.com> wrote:
> >> On 22 November 2012 12:56, Gary Martin <ga...@wandisco.com>
> wrote:
> >>> On 22/11/12 11:08, Andrej Golcov wrote:
> >>>
> > [...]
> >>>>>
> >>>>> Query Result view should support facets (
> >>>>> http://searchhub.org/2009/09/**02/faceted-search-with-solr/<
> http://searchhub.org/2009/09/02/faceted-search-with-solr/>)
> >>>>> for example:
> >>>>> - All resources result tab can show facets for resource type and
> >>>>> product
> >>>>> - Ticket result tab can show facets for products and ticket status
> >>>>>
> >>>>
> >>> Facets seem to me to be a very important enhancement.
> >>>
> >>
> >> I agree. This would be a great feature to have and build upon.
> >>
> >
> > +1
> >
>
> Nonetheless I'd prefer checkboxes rather than tabs , in order to
> choose multiple options (e.g. tickets and changesets ...) .
>
> What d'u think ?
>
> --
> Regards,
>
> Olemis.
>
> Blog ES: http://simelo-es.blogspot.com/
> Blog EN: http://simelo-en.blogspot.com/
>
> Featured article:
>
--
Joe Dreimann
UX Designer | WANdisco <http://www.wandisco.com/>
*
*
*Transform your software development department. Register for a free SVN
HealthCheck <http://go.wandisco.com/HealthCheck-Sig.html> *
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Olemis Lang <ol...@gmail.com>.
On 11/22/12, Olemis Lang <ol...@gmail.com> wrote:
> On 11/22/12, Joachim Dreimann <jo...@wandisco.com> wrote:
>> On 22 November 2012 12:56, Gary Martin <ga...@wandisco.com> wrote:
>>> On 22/11/12 11:08, Andrej Golcov wrote:
>>>
> [...]
>>>>>
>>>>> Query Result view should support facets (
>>>>> http://searchhub.org/2009/09/**02/faceted-search-with-solr/<http://searchhub.org/2009/09/02/faceted-search-with-solr/>)
>>>>> for example:
>>>>> - All resources result tab can show facets for resource type and
>>>>> product
>>>>> - Ticket result tab can show facets for products and ticket status
>>>>>
>>>>
>>> Facets seem to me to be a very important enhancement.
>>>
>>
>> I agree. This would be a great feature to have and build upon.
>>
>
> +1
>
Nonetheless I'd prefer checkboxes rather than tabs , in order to
choose multiple options (e.g. tickets and changesets ...) .
What d'u think ?
--
Regards,
Olemis.
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Featured article:
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Olemis Lang <ol...@gmail.com>.
On 11/22/12, Joachim Dreimann <jo...@wandisco.com> wrote:
> On 22 November 2012 12:56, Gary Martin <ga...@wandisco.com> wrote:
>> On 22/11/12 11:08, Andrej Golcov wrote:
>>
[...]
>>>>
>>>> Query Result view should support facets (
>>>> http://searchhub.org/2009/09/**02/faceted-search-with-solr/<http://searchhub.org/2009/09/02/faceted-search-with-solr/>)
>>>> for example:
>>>> - All resources result tab can show facets for resource type and
>>>> product
>>>> - Ticket result tab can show facets for products and ticket status
>>>>
>>>
>> Facets seem to me to be a very important enhancement.
>>
>
> I agree. This would be a great feature to have and build upon.
>
+1
--
Regards,
Olemis.
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Featured article:
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Joachim Dreimann <jo...@wandisco.com>.
On 22 November 2012 12:56, Gary Martin <ga...@wandisco.com> wrote:
> Next a few comments on the Query Result view and Resource Query sections:
>
> On 22/11/12 11:08, Andrej Golcov wrote:
>
>> Hi all,
>>
>> I've just added first draft of proposal for search improvements. So, let's
>> discuss it :)
>>
>> I suggest that we agree on functional requirements and then pass to
>> possible implementation.
>>
>> Regards, Andrej
>>
>>
>> On 22 November 2012 11:32, Apache Bloodhound <
>> bloodhound-dev@incubator.**apache.org<bl...@incubator.apache.org>>
>> wrote:
>>
>> Page "Proposals/BEP-0004" was added by andrej
>>> Content:
>>> -------8<------8<------8<-----**-8<------8<------8<------8<---**
>>> ---8<--------
>>>
>>> = BEP 4 : Improved search architecture #overview
>>>
>> [snip]
>
>>
>>> === Query Result view
>>>
>>> Query Result view represents consistent view of query result for
>>> different
>>> resources. Result may be represented as
>>> - All resources result tab - default, common for all resources columns
>>> e.g. name and description
>>> - Resource result tabs - resource specific fields are shown e.g. id,
>>> status, summary for ticket.
>>> - Query Result view can be instructed to limit result to specific
>>> resource type e.g. show only tickets in result. In this case, resource
>>> tabs
>>> can be omitted.
>>>
>>
> Presumably, this would be a part of the new query syntax so that it can be
> relatively easy to filter before getting to this page.
>
>> User can refine search conditions either by editing query or by using
>>> Query Builder.
>>>
>>
> I would like to see others comment on the mockup for the query builder.
> Consistency with other views suggest that there may be space on the right
> hand side, replacing the activity area - I assume that we will not be
> displying activity on any search/query screen.
>
I like that suggestion, replacing the activity area with a query builder in
the UI. I've raised ticket #272 to create a mockup for this.
>
> User can specify what fields should be selected and in what results order
>>> should be applied. A new order type is introduced for free-text search:
>>> score
>>>
>>> Query Result view should support facets (
>>> http://searchhub.org/2009/09/**02/faceted-search-with-solr/<http://searchhub.org/2009/09/02/faceted-search-with-solr/>)
>>> for example:
>>> - All resources result tab can show facets for resource type and
>>> product
>>> - Ticket result tab can show facets for products and ticket status
>>>
>>
> Facets seem to me to be a very important enhancement.
>
I agree. This would be a great feature to have and build upon.
>
>>> Other nice to have features in no particular order:
>>> - Possibility to save a query for specific user and sharing of saved
>>> queries
>>> - Search highlighting
>>>
>>
> Saved/stored queries have been discussed in the mailing list before a few
> times so I think this is probably an expectation beyond the ability to put
> a search macro on a wiki page. I think that this should probably be
> considered separately of this solution but we should make sure that such
> stored queries support any implementation that arises from this.
>
#230 is the suggestion for Custom Queries to show recent queries. There was
also a suggestion to be able to save queries. Just thought it may be
relevant here to mention it.
>
> Search highlighting might also be worth separating out so that we make
> sure that we focus on getting searches done correctly, even if it turns out
> it is just a small enhancement on top. I assume that this is referring to
> terms being highlighted when you click on a link result here so it is
> probably out of scope of the Query result view section anyway.
>
> === Resource Query
>>>
>>> New Resource Query should provide the following functionality:
>>> - free-text search support
>>> - facet support
>>> - it is a superset of TracQuery functionality
>>>
>> So we can embed TracQuery syntax within the new queries?
>
>> - basic query expressions AND, OR, NOT, search by specific field,
>>> search
>>> [FROM TO] - TBD (can be similar to SQL or lucene/solr like)
>>>
>>
> I think that this could also be considered from the point of view of a set
> operation syntax which could be interesting. Not sure if it is worth it of
> course. Anyway, in addition to logical operators, we do need to be able to
> express a grouping syntax to build more advanced expressions from the
> basics.
>
> It would also be nice if we were able to express these queries relatively
> easily in a query string so that it is possible to just write your query
> directly in a link and be understandable.
>
>> - search through different resources not only tickets: wiki,
>>> milestones,
>>> changesets and other pluggable resources
>>> - search through all resource fields
>>> - search through attachments, history and comments
>>> - multi-product aware - apply security context etc
>>> - order by free-text score. Score calculation can be configured, for
>>> example if found in summary: id: score:100, score*10, in keywords:
>>> score*5,
>>> in components: score*3 …
>>>
>>
> I would probably suggest that we keep configurability of scores in mind
> but don't provide the interface to change it until a bit later. A bit of
> clarification might be helpful here too. Presumably term found in summary
> would be score*=100 rather than just assigned 100. In addition, we could
> consider modifying the score based on the date. There is a further
> possibility of including fairly simple "favour [something]" (e.g. "newer",
> "older", etc) radio boxes to adjust the scoring dynamically as I am not
> convinced that users should be expected to adjust these values for
> themselves at the moment.
>
> Nice to have features
>>> - Support search in attachments of specific types e.g word, exel etc.
>>>
>> I seem to remember that you get this in solr for free but I am not so
> sure about the others.
>
> Cheers,
> Gary
>
--
Joe Dreimann
UX Designer | WANdisco <http://www.wandisco.com/>
*
*
*Transform your software development department. Register for a free SVN
HealthCheck <http://go.wandisco.com/HealthCheck-Sig.html> *
Re: Improved Search Architecture - Query Results View and Resource
Query (Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Andrej Golcov <an...@digiverse.si>.
Hi,
> I would like to see others comment on the mockup for the query builder.
Consistency with other
> views suggest that there may be space on the right hand side, replacing
the activity area - I
> assume that we will not be displying activity on any search/query screen.
+1
> Saved/stored queries have been discussed in the mailing list before a few
times so I think this is
> probably an expectation beyond the ability to put a search macro on a
wiki page. I think that this
> should probably be considered separately of this solution but we should
make sure that such
> stored queries support any implementation that arises from this.
Let's consider saved/stored queries as postponed feature and concentrate on
core changes.
> Search highlighting might also be worth separating out so that we make
sure that we focus on
> getting searches done correctly, even if it turns out it is just a small
enhancement on top. I assume
> that this is referring to terms being highlighted when you click on a
link result here so it is probably
> out of scope of the Query result view section anyway.
By "search highlighting" I meant highlighting of searching phrases in
result set. Let's consider this as "low priority" feature.
> I think that this could also be considered from the point of view of a
set operation syntax which
> could be interesting. Not sure if it is worth it of course. Anyway, in
addition to logical operators, we
> do need to be able to express a grouping syntax to build more advanced
expressions from the
> basics.
What do you mean by grouping? More SQL like COUNT(...) GROUP BY or display
results grouped by some fields?
I'm thinking about creating separate sub-page for Resource
Query requirements and syntax.
Regards, Andrej
On 22 November 2012 13:56, Gary Martin <ga...@wandisco.com> wrote:
> Next a few comments on the Query Result view and Resource Query sections:
>
> On 22/11/12 11:08, Andrej Golcov wrote:
>
>> Hi all,
>>
>> I've just added first draft of proposal for search improvements. So, let's
>> discuss it :)
>>
>> I suggest that we agree on functional requirements and then pass to
>> possible implementation.
>>
>> Regards, Andrej
>>
>>
>> On 22 November 2012 11:32, Apache Bloodhound <
>> bloodhound-dev@incubator.**apache.org<bl...@incubator.apache.org>>
>> wrote:
>>
>> Page "Proposals/BEP-0004" was added by andrej
>>> Content:
>>> -------8<------8<------8<-----**-8<------8<------8<------8<---**
>>> ---8<--------
>>>
>>> = BEP 4 : Improved search architecture #overview
>>>
>> [snip]
>
>>
>>> === Query Result view
>>>
>>> Query Result view represents consistent view of query result for
>>> different
>>> resources. Result may be represented as
>>> - All resources result tab - default, common for all resources columns
>>> e.g. name and description
>>> - Resource result tabs - resource specific fields are shown e.g. id,
>>> status, summary for ticket.
>>> - Query Result view can be instructed to limit result to specific
>>> resource type e.g. show only tickets in result. In this case, resource
>>> tabs
>>> can be omitted.
>>>
>>
> Presumably, this would be a part of the new query syntax so that it can be
> relatively easy to filter before getting to this page.
>
>> User can refine search conditions either by editing query or by using
>>> Query Builder.
>>>
>>
> I would like to see others comment on the mockup for the query builder.
> Consistency with other views suggest that there may be space on the right
> hand side, replacing the activity area - I assume that we will not be
> displying activity on any search/query screen.
>
> User can specify what fields should be selected and in what results order
>>> should be applied. A new order type is introduced for free-text search:
>>> score
>>>
>>> Query Result view should support facets (
>>> http://searchhub.org/2009/09/**02/faceted-search-with-solr/<http://searchhub.org/2009/09/02/faceted-search-with-solr/>)
>>> for example:
>>> - All resources result tab can show facets for resource type and
>>> product
>>> - Ticket result tab can show facets for products and ticket status
>>>
>>
> Facets seem to me to be a very important enhancement.
>
>>
>>> Other nice to have features in no particular order:
>>> - Possibility to save a query for specific user and sharing of saved
>>> queries
>>> - Search highlighting
>>>
>>
> Saved/stored queries have been discussed in the mailing list before a few
> times so I think this is probably an expectation beyond the ability to put
> a search macro on a wiki page. I think that this should probably be
> considered separately of this solution but we should make sure that such
> stored queries support any implementation that arises from this.
>
> Search highlighting might also be worth separating out so that we make
> sure that we focus on getting searches done correctly, even if it turns out
> it is just a small enhancement on top. I assume that this is referring to
> terms being highlighted when you click on a link result here so it is
> probably out of scope of the Query result view section anyway.
>
> === Resource Query
>>>
>>> New Resource Query should provide the following functionality:
>>> - free-text search support
>>> - facet support
>>> - it is a superset of TracQuery functionality
>>>
>> So we can embed TracQuery syntax within the new queries?
>
>> - basic query expressions AND, OR, NOT, search by specific field,
>>> search
>>> [FROM TO] - TBD (can be similar to SQL or lucene/solr like)
>>>
>>
> I think that this could also be considered from the point of view of a set
> operation syntax which could be interesting. Not sure if it is worth it of
> course. Anyway, in addition to logical operators, we do need to be able to
> express a grouping syntax to build more advanced expressions from the
> basics.
>
> It would also be nice if we were able to express these queries relatively
> easily in a query string so that it is possible to just write your query
> directly in a link and be understandable.
>
>> - search through different resources not only tickets: wiki,
>>> milestones,
>>> changesets and other pluggable resources
>>> - search through all resource fields
>>> - search through attachments, history and comments
>>> - multi-product aware - apply security context etc
>>> - order by free-text score. Score calculation can be configured, for
>>> example if found in summary: id: score:100, score*10, in keywords:
>>> score*5,
>>> in components: score*3 …
>>>
>>
> I would probably suggest that we keep configurability of scores in mind
> but don't provide the interface to change it until a bit later. A bit of
> clarification might be helpful here too. Presumably term found in summary
> would be score*=100 rather than just assigned 100. In addition, we could
> consider modifying the score based on the date. There is a further
> possibility of including fairly simple "favour [something]" (e.g. "newer",
> "older", etc) radio boxes to adjust the scoring dynamically as I am not
> convinced that users should be expected to adjust these values for
> themselves at the moment.
>
> Nice to have features
>>> - Support search in attachments of specific types e.g word, exel etc.
>>>
>> I seem to remember that you get this in solr for free but I am not so
> sure about the others.
>
> Cheers,
> Gary
>
Improved Search Architecture - Query Results View and Resource Query
(Was: Re: [Apache Bloodhound] Proposals/BEP-0004 added)
Posted by Gary Martin <ga...@wandisco.com>.
Next a few comments on the Query Result view and Resource Query sections:
On 22/11/12 11:08, Andrej Golcov wrote:
> Hi all,
>
> I've just added first draft of proposal for search improvements. So, let's
> discuss it :)
>
> I suggest that we agree on functional requirements and then pass to
> possible implementation.
>
> Regards, Andrej
>
>
> On 22 November 2012 11:32, Apache Bloodhound <
> bloodhound-dev@incubator.apache.org> wrote:
>
>> Page "Proposals/BEP-0004" was added by andrej
>> Content:
>> -------8<------8<------8<------8<------8<------8<------8<------8<--------
>>
>> = BEP 4 : Improved search architecture #overview
[snip]
>>
>> === Query Result view
>>
>> Query Result view represents consistent view of query result for different
>> resources. Result may be represented as
>> - All resources result tab - default, common for all resources columns
>> e.g. name and description
>> - Resource result tabs - resource specific fields are shown e.g. id,
>> status, summary for ticket.
>> - Query Result view can be instructed to limit result to specific
>> resource type e.g. show only tickets in result. In this case, resource tabs
>> can be omitted.
Presumably, this would be a part of the new query syntax so that it can
be relatively easy to filter before getting to this page.
>> User can refine search conditions either by editing query or by using
>> Query Builder.
I would like to see others comment on the mockup for the query builder.
Consistency with other views suggest that there may be space on the
right hand side, replacing the activity area - I assume that we will not
be displying activity on any search/query screen.
>> User can specify what fields should be selected and in what results order
>> should be applied. A new order type is introduced for free-text search:
>> score
>>
>> Query Result view should support facets (
>> http://searchhub.org/2009/09/02/faceted-search-with-solr/) for example:
>> - All resources result tab can show facets for resource type and product
>> - Ticket result tab can show facets for products and ticket status
Facets seem to me to be a very important enhancement.
>>
>> Other nice to have features in no particular order:
>> - Possibility to save a query for specific user and sharing of saved
>> queries
>> - Search highlighting
Saved/stored queries have been discussed in the mailing list before a
few times so I think this is probably an expectation beyond the ability
to put a search macro on a wiki page. I think that this should probably
be considered separately of this solution but we should make sure that
such stored queries support any implementation that arises from this.
Search highlighting might also be worth separating out so that we make
sure that we focus on getting searches done correctly, even if it turns
out it is just a small enhancement on top. I assume that this is
referring to terms being highlighted when you click on a link result
here so it is probably out of scope of the Query result view section anyway.
>> === Resource Query
>>
>> New Resource Query should provide the following functionality:
>> - free-text search support
>> - facet support
>> - it is a superset of TracQuery functionality
So we can embed TracQuery syntax within the new queries?
>> - basic query expressions AND, OR, NOT, search by specific field, search
>> [FROM TO] - TBD (can be similar to SQL or lucene/solr like)
I think that this could also be considered from the point of view of a
set operation syntax which could be interesting. Not sure if it is worth
it of course. Anyway, in addition to logical operators, we do need to be
able to express a grouping syntax to build more advanced expressions
from the basics.
It would also be nice if we were able to express these queries
relatively easily in a query string so that it is possible to just write
your query directly in a link and be understandable.
>> - search through different resources not only tickets: wiki, milestones,
>> changesets and other pluggable resources
>> - search through all resource fields
>> - search through attachments, history and comments
>> - multi-product aware - apply security context etc
>> - order by free-text score. Score calculation can be configured, for
>> example if found in summary: id: score:100, score*10, in keywords: score*5,
>> in components: score*3 …
I would probably suggest that we keep configurability of scores in mind
but don't provide the interface to change it until a bit later. A bit of
clarification might be helpful here too. Presumably term found in
summary would be score*=100 rather than just assigned 100. In addition,
we could consider modifying the score based on the date. There is a
further possibility of including fairly simple "favour [something]"
(e.g. "newer", "older", etc) radio boxes to adjust the scoring
dynamically as I am not convinced that users should be expected to
adjust these values for themselves at the moment.
>> Nice to have features
>> - Support search in attachments of specific types e.g word, exel etc.
I seem to remember that you get this in solr for free but I am not so
sure about the others.
Cheers,
Gary
Re: Improved Search Architecture - Quick Search Box (Was: Re: [Apache
Bloodhound] Proposals/BEP-0004 added)
Posted by Gary Martin <ga...@wandisco.com>.
On 22/11/12 17:08, Peter Koželj wrote:
> As I understood the BEP the syntax itself is TBD and totally open at the
> moment.
> I would actually prefere if we define the syntax in a subpage and use one
> of the standards grammars to define it.
That would be my understanding too. It is only a question of whether
there actually is a simple way to specify the search syntax that doesn't
require too much effort on our part. If we can stick close to standards,
that would be fantastic.
Re: Improved Search Architecture - Quick Search Box (Was: Re: [Apache
Bloodhound] Proposals/BEP-0004 added)
Posted by Peter Koželj <pe...@digiverse.si>.
As I understood the BEP the syntax itself is TBD and totally open at the
moment.
I would actually prefere if we define the syntax in a subpage and use one
of the standards grammars to define it.
On 22 November 2012 16:59, Gary Martin <ga...@wandisco.com> wrote:
> On 22/11/12 14:20, Andrej Golcov wrote:
>
>> On a technical note, is there any reason to specify text~= for a general
>>>
>> text query? I would have
>>
>>> thought that we could distinguish free text (as perhaps suggested below
>>>
>> in the examples below.)
>> Agreed and fixed the BEP accordingly.
>>
>
> I was hoping for a bit of a discussion around this point actually. I was
> wondering whether I had missed some occasions where there is a requirement
> for the ability to avoid ambiguity. In the end there is always quoting and
> there should be a bracket notation so there may be no requirement for
> text~= at all.
>
> We will of course also require the ability to escape quoting and other
> control characters within any new query language.
>
> I would if there is much support for defining query languages any of the
> suggested search backends.
>
> Cheers,
> Gary
>
Re: Improved Search Architecture - Quick Search Box (Was: Re: [Apache
Bloodhound] Proposals/BEP-0004 added)
Posted by Gary Martin <ga...@wandisco.com>.
On 22/11/12 14:20, Andrej Golcov wrote:
>> On a technical note, is there any reason to specify text~= for a general
> text query? I would have
>> thought that we could distinguish free text (as perhaps suggested below
> in the examples below.)
> Agreed and fixed the BEP accordingly.
I was hoping for a bit of a discussion around this point actually. I was
wondering whether I had missed some occasions where there is a
requirement for the ability to avoid ambiguity. In the end there is
always quoting and there should be a bracket notation so there may be no
requirement for text~= at all.
We will of course also require the ability to escape quoting and other
control characters within any new query language.
I would if there is much support for defining query languages any of the
suggested search backends.
Cheers,
Gary
Re: Improved Search Architecture - Quick Search Box (Was: Re: [Apache
Bloodhound] Proposals/BEP-0004 added)
Posted by Andrej Golcov <an...@digiverse.si>.
> On a technical note, is there any reason to specify text~= for a general
text query? I would have
> thought that we could distinguish free text (as perhaps suggested below
in the examples below.)
Agreed and fixed the BEP accordingly.
>> Other nice to have features:
>> - Did you mean...
>> - Suggestions during typing
>>
> > The second is probably more useful than the former but these are
probably worth considering later
Definitely, I meant "nice to have features" as possible by really but low
priority.
IMHO, after the discussion, we should come with list of first priority
tickets and left "nice to have features" for later.
Regards, Andrej
>
On 22 November 2012 13:01, Gary Martin <ga...@wandisco.com> wrote:
> Hi Andrej,
>
> Thanks for this. There are plenty of good ideas here but I will go through
> it in parts creating different threads. Hope that is good for everyone.
>
> On 22/11/12 11:08, Andrej Golcov wrote:
>
>> Hi all,
>>
>> I've just added first draft of proposal for search improvements. So, let's
>> discuss it :)
>>
>> I suggest that we agree on functional requirements and then pass to
>> possible implementation.
>>
>> Regards, Andrej
>>
>>
>> On 22 November 2012 11:32, Apache Bloodhound <
>> bloodhound-dev@incubator.**apache.org<bl...@incubator.apache.org>>
>> wrote:
>>
>> Page "Proposals/BEP-0004" was added by andrej
>>> Content:
>>> -------8<------8<------8<-----**-8<------8<------8<------8<---**
>>> ---8<--------
>>>
>>> = BEP 4 : Improved search architecture #overview
>>>
>> [snipped]
>
>>
>>> === Quick search box
>>>
>>> Quick search box should be just frontend for adhoc query. For example, if
>>> user searches for “bla” string, user receives Search Result view with the
>>> following query: text~=”bla”. User can refine query in Query Result
>>> view.
>>>
>> Quick search should probably maintain all the syntax that a query results
> view search box provides. The main reason for providing a different search
> box would be that you can construct a larger text query and keep it in
> view, which seems quite nice. It would also solve the current problem of
> attempting to refine the search in the quick search field resulting in the
> dropping of the filters (though this could be solved by adding the filters
> to the quick search query.)
>
> On a technical note, is there any reason to specify text~= for a general
> text query? I would have thought that we could distinguish free text (as
> perhaps suggested below in the examples below.)
>
>
>>> Search box should accept the following search types:
>>> - free-text search: “bla”
>>> - free-text + query: bla status!=closed
>>> - query only: status!=closed
>>>
>>
> This looks good. At the moment the quick search requires
> "query:status!=closed" for the last of those and to include the free-text
> search you would need to query a whole set of fields to see if they contain
> the text. This is clearly not as quick and intuitive. Obviously, as soon as
> you bring field queries into the problem you are automatically focusing on
> resources that have that field available (possibly relaxed when there is an
> OR operator involved - think "bla OR status!=closed")
>
>>
>>> Other nice to have features:
>>> - Did you mean...
>>> - Suggestions during typing
>>>
>>
> The second is probably more useful than the former but these are probably
> worth considering later.
>
> Cheers,
> Gary
>
Improved Search Architecture - Quick Search Box (Was: Re: [Apache
Bloodhound] Proposals/BEP-0004 added)
Posted by Gary Martin <ga...@wandisco.com>.
Hi Andrej,
Thanks for this. There are plenty of good ideas here but I will go
through it in parts creating different threads. Hope that is good for
everyone.
On 22/11/12 11:08, Andrej Golcov wrote:
> Hi all,
>
> I've just added first draft of proposal for search improvements. So, let's
> discuss it :)
>
> I suggest that we agree on functional requirements and then pass to
> possible implementation.
>
> Regards, Andrej
>
>
> On 22 November 2012 11:32, Apache Bloodhound <
> bloodhound-dev@incubator.apache.org> wrote:
>
>> Page "Proposals/BEP-0004" was added by andrej
>> Content:
>> -------8<------8<------8<------8<------8<------8<------8<------8<--------
>>
>> = BEP 4 : Improved search architecture #overview
[snipped]
>>
>> === Quick search box
>>
>> Quick search box should be just frontend for adhoc query. For example, if
>> user searches for “bla” string, user receives Search Result view with the
>> following query: text~=”bla”. User can refine query in Query Result view.
Quick search should probably maintain all the syntax that a query
results view search box provides. The main reason for providing a
different search box would be that you can construct a larger text query
and keep it in view, which seems quite nice. It would also solve the
current problem of attempting to refine the search in the quick search
field resulting in the dropping of the filters (though this could be
solved by adding the filters to the quick search query.)
On a technical note, is there any reason to specify text~= for a general
text query? I would have thought that we could distinguish free text (as
perhaps suggested below in the examples below.)
>>
>> Search box should accept the following search types:
>> - free-text search: “bla”
>> - free-text + query: bla status!=closed
>> - query only: status!=closed
This looks good. At the moment the quick search requires
"query:status!=closed" for the last of those and to include the
free-text search you would need to query a whole set of fields to see if
they contain the text. This is clearly not as quick and intuitive.
Obviously, as soon as you bring field queries into the problem you are
automatically focusing on resources that have that field available
(possibly relaxed when there is an OR operator involved - think "bla OR
status!=closed")
>>
>> Other nice to have features:
>> - Did you mean...
>> - Suggestions during typing
The second is probably more useful than the former but these are
probably worth considering later.
Cheers,
Gary
Re: [Apache Bloodhound] Proposals/BEP-0004 added
Posted by Andrej Golcov <an...@digiverse.si>.
Hi all,
I've just added first draft of proposal for search improvements. So, let's
discuss it :)
I suggest that we agree on functional requirements and then pass to
possible implementation.
Regards, Andrej
On 22 November 2012 11:32, Apache Bloodhound <
bloodhound-dev@incubator.apache.org> wrote:
> Page "Proposals/BEP-0004" was added by andrej
> Content:
> -------8<------8<------8<------8<------8<------8<------8<------8<--------
>
> = BEP 4 : Improved search architecture #overview
>
> [[PageOutline]]
>
> || '''PEP''' || 4 ||
> || '''Title''' || Improved search architecture ||
> || '''Version''' || ||
> || '''Last-Modified''' || ||
> || '''Author''' || The Bloodhound project ||
> || '''Status''' || Draft ||
> || '''Type''' || Standards Trac ||
> || '''Content-Type''' || [wiki:PageTemplates/Proposals text/x-trac-wiki] ||
> || '''Created''' || ||
> || '''Post-History''' || ||
>
> ----
>
> == Abstract #abstract
> This document is meant to gather community consensus on the implementation
> of the improved search functionality. Simultaneously, we want to merge
> search and custom query functionality under the same umbrella. This is the
> place to discuss it and make proposals. Main discussion must be placed in
> mail list with [BEP-0004] subject.
>
> == Motivation ==
>
> Nowadays modern issue tracking system should provide advanced free text
> search functionality combine with flexible query mechanism. There are a lot
> of user requests asking for better and customizable search.
>
> The term project is very generic and may be confusing considering the
> context. Therefore in this specification the word product is used instead.
>
> == Rationale #rationale
>
> === Quick search box
>
> Quick search box should be just frontend for adhoc query. For example, if
> user searches for “bla” string, user receives Search Result view with the
> following query: text~=”bla”. User can refine query in Query Result view.
>
> Search box should accept the following search types:
> - free-text search: “bla”
> - free-text + query: bla status!=closed
> - query only: status!=closed
>
> Other nice to have features:
> - Did you mean...
> - Suggestions during typing
>
> === Query Result view
>
> Query Result view represents consistent view of query result for different
> resources. Result may be represented as
> - All resources result tab - default, common for all resources columns
> e.g. name and description
> - Resource result tabs - resource specific fields are shown e.g. id,
> status, summary for ticket.
> - Query Result view can be instructed to limit result to specific
> resource type e.g. show only tickets in result. In this case, resource tabs
> can be omitted.
>
> User can refine search conditions either by editing query or by using
> Query Builder.
> User can specify what fields should be selected and in what results order
> should be applied. A new order type is introduced for free-text search:
> score
>
> Query Result view should support facets (
> http://searchhub.org/2009/09/02/faceted-search-with-solr/) for example:
> - All resources result tab can show facets for resource type and product
> - Ticket result tab can show facets for products and ticket status
>
> Other nice to have features in no particular order:
> - Possibility to save a query for specific user and sharing of saved
> queries
> - Search highlighting
>
> === Resource Query
>
> New Resource Query should provide the following functionality:
> - free-text search support
> - facet support
> - it is a superset of TracQuery functionality
> - basic query expressions AND, OR, NOT, search by specific field, search
> [FROM TO] - TBD (can be similar to SQL or lucene/solr like)
> - search through different resources not only tickets: wiki, milestones,
> changesets and other pluggable resources
> - search through all resource fields
> - search through attachments, history and comments
> - multi-product aware - apply security context etc
> - order by free-text score. Score calculation can be configured, for
> example if found in summary: id: score:100, score*10, in keywords: score*5,
> in components: score*3 …
>
> Nice to have features
> - Support search in attachments of specific types e.g word, exel etc.
>
> === Resource Query macro
>
> Similar to existing TracQuery, New macro must be introduced to enable
> access to Resource Query from Wiki.
>
> === Query Builder
>
> A lot improvements may be introduced here, TBD
>
> == Proposal #proposal
>
> === Implementation steps
>
> We can start with small step by implementing search box functionality
> similar to trac:wiki:AdvancedSearch and then add support for query, wiki
> macro and query builder/wizard.
>
> === Possible free text platforms
>
> A few possible solutions were discussed on the mail list:
>
> ==== Whoosh
>
> Whoosh links:
> - Product page: http://pypi.python.org/pypi/Whoosh/
> - Discussion on implementing Trac search using Whoosh:
> https://groups.google.com/forum/?fromgroups=#!msg/trac-dev/sbU-g0C6kvk/1_juL29aAtQJ
>
> ==== !PyLucen
>
> [http://lucene.apache.org/pylucene/ PyLucene] is not a Lucene port but a
> Python wrapper around Java Lucene. !PyLucene embeds a Java VM with Lucene
> into a Python process.
>
> ==== Java-based servers.
> Solr or !ElasticSearch
>
> == Rejected ideas
>
> == Backwards Compatibility #backwards-compatibility
>
> TracQuery implementation must be changed to use Resource Query for data
> retrieval.
>
> Existing TracSearch syntax will be mapped to Resource Query.
>
> == Reference Implementation #reference-implementation
>
> == Resources #resources
>
> == References #references
> Proposals/BEP-0003 - Multi-product architecture
>
> === Trac proposals on this subject
> - trac:wiki:AdvancedSearch
> - trac:wiki:SearchRefactoring
>
> === Plugins on this subject
> - http://trac-hacks.org/wiki/FullTextSearchPlugin from Logica
> - http://trac-hacks.org/wiki/TracAdvancedSearchPlugin - uses Solr,
> claims to be designed with extensions points
>
> === Existing search and query implementation
> - TracQuery
> - TracSearch
>
> == Copyright #copyright
>
> Copyright © 2009-2012 The [http://www.apache.org Apache Software
> Foundation] [[BR]]
> Licensed under the [http://www.apache.org/licenses/LICENSE-2.0 Apache
> License, Version 2.0].
>
> Apache Bloodhound, Apache, the Apache feather logo, and the Apache
> Bloodhound project logo are trademarks of The Apache Software Foundation.
> -------8<------8<------8<------8<------8<------8<------8<------8<--------
>
> --
> Page URL: <https://issues.apache.org/bloodhound/wiki/Proposals/BEP-0004>
> Apache Bloodhound <https://issues.apache.org/bloodhound/>
> The Apache Bloodhound (incubating) issue tracker
>
> This is an automated message. Someone added your email address to be
> notified of changes on 'Proposals/BEP-0004' page.
> If it was not you, please report to .
>