Some random user questions

[Vincent van der Leun <vi...@fastmail.nl>]

Hello, 

I am a Dutch user of CouchDB for quite some time. I know I won't make headlines, given that I only run a single-node CouchDB instance that runs on a small VPS that also runs my small web-applications as well (my DB's are only up to dozens of MBs in size), but that wouldn't stop me from mailing the mailing list :)

Recently upgraded my CouchDB 2 instance to the latest 3 version, it was about time... It was an easy process and v3 seemingly fixed some bugs that were plaguing me, so I am happy with my choice.

I was mostly wondering how others are using their CouchDB setup, so that's why I am writing. I have some questions to the community in general:

Coming late to the CouchDB party (around the v2 days), Mango querying is one of my favorite features. Tutorials/videos always seem to talk about JavaScript filters functions and dismiss Mango as a minor, unimportant feature (IF they mention Mango at all). Was just wondering how long-time users are querying their JSON documents mostly nowadays?

I saw somewhere in the v3 release notes that the Javascript "update" functions were deprecated in v3, although I didn't see a Deprecation warning in the https://docs.couchdb.org/en/stable/ddocs/ddocs.html#updatefun section. Is it indeed deprecated? I kinda liked that feature, for example to update creation/update timestamp fields in documents automatically. 

Am I correct in thinking that CouchDB is moving away from the idea that web applications should run on top of CouchDB (since I use Nginx, I personally never even considered it) and logic like updating timestamps, should, according to the CouchDB devs, ideally be implemented in the client applications themselves? Also, are "Validate Document Update Functions" also part of the  "Update functions" that are deprecated?

Are members of this mailinglist using custom Query Servers in practice and if so, what are your use-cases? It seems so interesting and I wonder what people are doing with it.

Finally, one of the limitations that I know of, that deleted documents stay in the CouchDB database "forever", is one that stops me from choosing CouchDB for some applications, that have to deal with temporary data. Is that limitation still in place (now and/or for the foreseeable future?).  

To the CouchDB team members, past and present: many thanks for your efforts! I wish I was more familiar with Erlang and could contribute somehow.

Kind regards,
Vincent

Re: Some random user questions

[ermouth <er...@gmail.com>]

Hi Vincent,

I have somewhat different long standing opinion on several topics touched
by Jan, so I decided to add some notes )

> small VPS that also runs my small web-applications as well

I also have a lot of small instances, however we also use Couch for much
larger projects. Couch scales well not only as a cluster, but also as a
mesh, i.e. a set of nodes of very different perf and workload, more or less
independent, and only syncing small portions of data or design docs.

> how long-time users are querying their JSON documents mostly nowadays

Mostly map/reduce, mostly JS.

Mango is attractive if your docs are small and are in a way ‘atomized’
(docs with a single data record, not an aggregate of records). A single doc
can only emit a single key into a given Mango index, but map function can
emit several index keys with different values for each item of a complex
doc.

Couch is great for large complex docs of flexible structure, map/reduce
approach fits them remarkably well. Couch is ok for small docs of
predefined structure if performance and storage isn’t an issue. Otherwise
SQL-like DBs are generally much faster.

> are "Validate Document Update Functions" also part of the  "Update
functions"
>> update functions receive any HTTP requests and transform them into
CouchDB docs

Actually, rewrite + update functions provide that feature. Update function
alone, without rewrite, only accepts requests of predefined syntax, which
limits it immensely in terms of transforming any HTTP request into CouchDB
doc.

> since I use Nginx, I personally never even considered it

Well, if you build small webapps using nginx and Couch, consider looking at
rewrite as a function. Moving from long nginx configs to JS code was one of
the incentives to implement rewrite-as-js-function feature. See
https://docs.couchdb.org/en/stable/api/ddoc/rewrites.html#using-a-stringified-function-for-rewrites
for more details.

Rewrite functions are for making simple APIs right on top of Couch, so they
can be useful for established Mango queries – you can have a much simpler
API for known requests in your webapp.

Cheers,
ermouth

Re: Some random user questions

[Jan Lehnardt <ja...@apache.org>]

Heya Vincent,

apologies for taking so long to respond :)

> On 14. Jan 2023, at 23:08, Vincent van der Leun <vi...@fastmail.nl> wrote:
> 
> Hello, 
> 
> I am a Dutch user of CouchDB for quite some time. I know I won't make headlines, given that I only run a single-node CouchDB instance that runs on a small VPS that also runs my small web-applications as well (my DB's are only up to dozens of MBs in size), but that wouldn't stop me from mailing the mailing list :)

Any size is a good size for CouchDB :)

> Recently upgraded my CouchDB 2 instance to the latest 3 version, it was about time... It was an easy process and v3 seemingly fixed some bugs that were plaguing me, so I am happy with my choice.

\o/

> I was mostly wondering how others are using their CouchDB setup, so that's why I am writing. I have some questions to the community in general:
> 
> Coming late to the CouchDB party (around the v2 days), Mango querying is one of my favorite features. Tutorials/videos always seem to talk about JavaScript filters functions and dismiss Mango as a minor, unimportant feature (IF they mention Mango at all). Was just wondering how long-time users are querying their JSON documents mostly nowadays?

From my observation, most folks still use JavaScript MapReduce Views for querying, but as Mango provides a substantial indexing performance increase, and some additional easy of use, it is not unpopular, even if not as feature-rich as JS views. Mango also receives updates[1] like the rest of the system.

> I saw somewhere in the v3 release notes that the Javascript "update" functions were deprecated in v3, although I didn't see a Deprecation warning in the https://docs.couchdb.org/en/stable/ddocs/ddocs.html#updatefun section. Is it indeed deprecated? I kinda liked that feature, for example to update creation/update timestamp fields in documents automatically. 

We have deprecated show, list & rewrite functions, but not filters or update or validate_doc_update functions. Update functions were considered for deprecation, but we left them in for now.

> Am I correct in thinking that CouchDB is moving away from the idea that web applications should run on top of CouchDB (since I use Nginx, I personally never even considered it) and logic like updating timestamps, should, according to the CouchDB devs, ideally be implemented in the client applications themselves? Also, are "Validate Document Update Functions" also part of the  "Update functions" that are deprecated?

You are correct. CouchDB and its JS functions was conceived in a world before Node.js, which is now the de-facto standard for running server-side JS and had it been around, we’d likely have made use of it then. These days, using a Node.js process alongside your CouchDB installation can be indefinitely more useful and a lot more efficient than the deprecated functions could ever be.

Update functions and validate_doc_update functions are different things:

- update functions receive any HTTP requests and transform them into CouchDB docs
- validate_doc_update functions receive a single document update and allow or deny it being written in the database

The latter is also used for access control and validation purposes and they are not deprecated. Although we are (slowly) designing more efficient alternatives[2].

> Are members of this mailinglist using custom Query Servers in practice and if so, what are your use-cases? It seems so interesting and I wonder what people are doing with it.

The only ones I’m aware of are some folks that run a Python query server, so they have an all-python development environment.

> Finally, one of the limitations that I know of, that deleted documents stay in the CouchDB database "forever", is one that stops me from choosing CouchDB for some applications, that have to deal with temporary data. Is that limitation still in place (now and/or for the foreseeable future?).  

We are discussing an auto-prune features on the dev@ mailing list, here is a summary[3] of our current thinking.

> To the CouchDB team members, past and present: many thanks for your efforts! I wish I was more familiar with Erlang and could contribute somehow.

Thank you! [4]

[1]: https://github.com/apache/couchdb/pull/4410
[2]: https://github.com/apache/couchdb/issues/1554 <https://github.com/apache/couchdb/issues?q=is%3Aissue+is%3Aopen+validation+>
[3]: https://docs.google.com/document/d/1ZkJs3Lrk8YOmPHeVYSe_yTp5rkqqAQyAnSQ-VqZ0R-0/edit
[4]: https://learnyousomeerlang.com/content

Best
Jan
—
> 
> Kind regards,
> Vincent
>