[DISCUSS] Covering indexes for Mango RFC

[Mike Rhodes <co...@dx13.co.uk>]

All,

I've added an RFC to introduce covering index functionality to Mango JSON indexes. It's in a pull request at:

https://github.com/apache/couchdb/pull/4410

The aim is to allow users to create index and query combinations that operate as fast as views when searching for data (as opposed to aggregating it via reduces). It's a logical extension to my recent fields pushdown PR (https://github.com/apache/couchdb/pull/4394). Overall, I'm interested in addressing some of the low- and mid-hanging query performance fruit within Mango.

This work touches more places than the previous work, so I figured that it was RFC worthy. And it'd be nice not to create a PR with some fatal flaw to it :D 

One question I had on this is whether RFCs should be frozen once approved, or whether I should look to keep it up to date with changes in the code? The Bylaws don't really cover whether they are living documents or records of what the community agreed on at a point in time.

Anyway, cheers for having a look,
-- 
Mike.

Re: [DISCUSS] Covering indexes for Mango RFC

[Mike Rhodes <co...@dx13.co.uk>]

Things almost always change when one gets into the code, and I fear RFCs lose utility if they may mislead later explorers of the code. As the template implies that we should include areas of code change, for me it seems that if we can't update RFCs, they will fall out of sync with the code quite quickly.

Because of this, I am a fan of the living document approach. This comes from a belief that these documents should aid our future selves in understanding the code, as well as gaining approval for changes. Though this does make them play double-duty 🤔. 

As you suggest Nick, it might be a bit heavyweight for changes like mine to be formally versioned (and that likely discourages updates). Perhaps GitHub can help us instead. Which version gained approvals can be seen with Approve on the PRs the RFC was involved in. Later we might include updates to the RFC within PRs for the implementation. By approving the code changes, RFC changes are approved by implication.

If it's not essential that we have notes within the text itself on updates, but can instead rely on code history, I suggest we go that route -- at any rate, if we want to allow changes at all after approval!

-- 
Mike.

On Fri, 27 Jan 2023, at 17:05, Nick Vatamaniuc wrote:
> That looks like a nice improvement, Mike. Thanks for making the RFC.
>
>> One question I had on this is whether RFCs should be frozen once approved, or whether I should look to keep it up to date with changes in the code? The Bylaws don't really cover whether they are living documents or records of what the community agreed on at a point in time.
>
> It could make sense to keep them updated with major changes to the
> design, along with the documentation. They are in the main code
> repository so that should make it easier to keep them in sync. We
> could add version updates / notes to the RFC sections to make the
> design evolution more apparent. IETF uses "updates #ref" or "obsoletes
> #ref" format https://www.rfc-editor.org/rfc/rfc7322.html#section-4.1.4
> with newer ones replacing the older ones. But that might work better
> for a complete redesign or larger changes.
>
> Cheers,
> -Nick
>
> On Fri, Jan 27, 2023 at 8:49 AM Mike Rhodes <co...@dx13.co.uk> wrote:
>>
>> All,
>>
>> I've added an RFC to introduce covering index functionality to Mango JSON indexes. It's in a pull request at:
>>
>> https://github.com/apache/couchdb/pull/4410
>>
>> The aim is to allow users to create index and query combinations that operate as fast as views when searching for data (as opposed to aggregating it via reduces). It's a logical extension to my recent fields pushdown PR (https://github.com/apache/couchdb/pull/4394). Overall, I'm interested in addressing some of the low- and mid-hanging query performance fruit within Mango.
>>
>> This work touches more places than the previous work, so I figured that it was RFC worthy. And it'd be nice not to create a PR with some fatal flaw to it :D
>>
>> One question I had on this is whether RFCs should be frozen once approved, or whether I should look to keep it up to date with changes in the code? The Bylaws don't really cover whether they are living documents or records of what the community agreed on at a point in time.
>>
>> Anyway, cheers for having a look,
>> --
>> Mike.

Re: [DISCUSS] Covering indexes for Mango RFC

[Nick Vatamaniuc <va...@gmail.com>]

That looks like a nice improvement, Mike. Thanks for making the RFC.

> One question I had on this is whether RFCs should be frozen once approved, or whether I should look to keep it up to date with changes in the code? The Bylaws don't really cover whether they are living documents or records of what the community agreed on at a point in time.

It could make sense to keep them updated with major changes to the
design, along with the documentation. They are in the main code
repository so that should make it easier to keep them in sync. We
could add version updates / notes to the RFC sections to make the
design evolution more apparent. IETF uses "updates #ref" or "obsoletes
#ref" format https://www.rfc-editor.org/rfc/rfc7322.html#section-4.1.4
with newer ones replacing the older ones. But that might work better
for a complete redesign or larger changes.

Cheers,
-Nick

On Fri, Jan 27, 2023 at 8:49 AM Mike Rhodes <co...@dx13.co.uk> wrote:
>
> All,
>
> I've added an RFC to introduce covering index functionality to Mango JSON indexes. It's in a pull request at:
>
> https://github.com/apache/couchdb/pull/4410
>
> The aim is to allow users to create index and query combinations that operate as fast as views when searching for data (as opposed to aggregating it via reduces). It's a logical extension to my recent fields pushdown PR (https://github.com/apache/couchdb/pull/4394). Overall, I'm interested in addressing some of the low- and mid-hanging query performance fruit within Mango.
>
> This work touches more places than the previous work, so I figured that it was RFC worthy. And it'd be nice not to create a PR with some fatal flaw to it :D
>
> One question I had on this is whether RFCs should be frozen once approved, or whether I should look to keep it up to date with changes in the code? The Bylaws don't really cover whether they are living documents or records of what the community agreed on at a point in time.
>
> Anyway, cheers for having a look,
> --
> Mike.