Filtering Documents

[Troy Martin <tr...@scriptedmotion.com>]

I've been using couchdb in a project of mine for awhile now with great success. I've run into a problem though that I can't seem to solve with any level of efficiency. 

I use couch to store workorder documents. I have a list page that outputs a list of workorders along with 6 workorder fields that the user can filter the list on. For example, the user can choose to filter workorders by workorderId, dateCreated, etc.

To enable filtering, I've been using elasticsearch instead of couch because I don't know a good way to filter the workorder documents using couch without creating a view for every combination of the filters. The problem I'm running into is elasticsearch is near real-time for indexing, so if I pull up the list page, access a workorder, update it and then return to the list page, there is up to a one-second delay due to the index refresh rate before the list page will contain the updated information. How to deal with that? 

I've been forcibly refreshing the elasticsearch index before returning to the list page, but I can't see that scaling well. Consequently, I've been thinking more and more about filtering documents in couch instead of relying on elasticsearch. Can anyone offer me any suggestions on how to go about this or have I run into a wall here?

Thanks,

Troy

Re: Filtering Documents

[Will Holley <wi...@gmail.com>]

Have you looked at CouchDB-Lucene (https://github.com/rnewson/couchdb-lucene)?

On 13 October 2014 13:30, Troy Martin <tr...@scriptedmotion.com> wrote:
> I'm using node as my server...I could filter on the server but there are potentially thousands of docs to go through so I'm hoping for a solution that will scale better.
>
> Multi range queries sound very promising...I'll look into those! Thanks for your help.
>
> Sent from my iPhone
>
> On Oct 13, 2014, at 5:13 AM, Mike Marino <mm...@gmail.com> wrote:
>
>>>
>>>
>>> Another option is to enable multirange-queries in the CouchDB. I know many
>>> people asking for them and there is existing a plugin/patch for - but there
>>> ends my knowledge. Maybe someone can help with a link.
>>
>> This should already be available in the master branch and should be
>> available in 2.0:
>>
>> https://issues.apache.org/jira/browse/COUCHDB-523
>>
>> Cheers,
>> Mike

Re: Filtering Documents

[Troy Martin <tr...@scriptedmotion.com>]

I'm using node as my server...I could filter on the server but there are potentially thousands of docs to go through so I'm hoping for a solution that will scale better.

Multi range queries sound very promising...I'll look into those! Thanks for your help.

Sent from my iPhone

On Oct 13, 2014, at 5:13 AM, Mike Marino <mm...@gmail.com> wrote:

>> 
>> 
>> Another option is to enable multirange-queries in the CouchDB. I know many
>> people asking for them and there is existing a plugin/patch for - but there
>> ends my knowledge. Maybe someone can help with a link.
> 
> This should already be available in the master branch and should be
> available in 2.0:
> 
> https://issues.apache.org/jira/browse/COUCHDB-523
> 
> Cheers,
> Mike

Re: Filtering Documents

[Mike Marino <mm...@gmail.com>]

>
>
> Another option is to enable multirange-queries in the CouchDB. I know many
> people asking for them and there is existing a plugin/patch for - but there
> ends my knowledge. Maybe someone can help with a link.
>

This should already be available in the master branch and should be
available in 2.0:

https://issues.apache.org/jira/browse/COUCHDB-523

Cheers,
Mike

Re: Filtering Documents

[Ingo Radatz <th...@googlemail.com>]

Hi Troy,

me in the same situation has finally implemented the filtering at client-side (in that case the browser). You haven't mentioned how our server-side filtering is used finally - so, take that simply as another option you have.

Another option is to enable multirange-queries in the CouchDB. I know many people asking for them and there is existing a plugin/patch for - but there ends my knowledge. Maybe someone can help with a link.

ingo

On 13 Oct 2014, at 13:15, Troy Martin <tr...@scriptedmotion.com> wrote:

> I've been using couchdb in a project of mine for awhile now with great success. I've run into a problem though that I can't seem to solve with any level of efficiency. 
> 
> I use couch to store workorder documents. I have a list page that outputs a list of workorders along with 6 workorder fields that the user can filter the list on. For example, the user can choose to filter workorders by workorderId, dateCreated, etc.
> 
> To enable filtering, I've been using elasticsearch instead of couch because I don't know a good way to filter the workorder documents using couch without creating a view for every combination of the filters. The problem I'm running into is elasticsearch is near real-time for indexing, so if I pull up the list page, access a workorder, update it and then return to the list page, there is up to a one-second delay due to the index refresh rate before the list page will contain the updated information. How to deal with that? 
> 
> I've been forcibly refreshing the elasticsearch index before returning to the list page, but I can't see that scaling well. Consequently, I've been thinking more and more about filtering documents in couch instead of relying on elasticsearch. Can anyone offer me any suggestions on how to go about this or have I run into a wall here?
> 
> Thanks,
> 
> Troy