[GitHub] eiri opened a new pull request #1470: Fix calculation of external size for attachments

[GitBox <gi...@apache.org>]

eiri opened a new pull request #1470: Fix calculation of external size for attachments
URL: https://github.com/apache/couchdb/pull/1470
 
 
   ## Overview
   
   A database external size is growing on any operation with attachments.
   
   The external size of a database defined as an uncompressed size of its content, so expectation would be that with updating or deleting a document's attachment the external size would change accordingly, i.e. increase or decrease to a size of the attachment.
   
   The external size, however, _always_ growing and shrinks to the expected values only on a database compression.
   
   Here is an example:
   ```bash
   $ curl -X PUT http://127.0.0.1:15984/koi
   {"ok":true}
   $ curl -X PUT http://127.0.0.1:15984/koi/holder -d'{}'
   {"ok":true,"id":"holder","rev":"1-967a00dff5e02add41819138abb3284d"}
   $ curl http://127.0.0.1:15984/koi | jq .sizes.external
   4
   $ curl -X PUT http://127.0.0.1:15984/koi/holder/keeper.bin?rev=1-967a00dff5e02add41819138abb3284d -H'Content-Type:application/octet-stream' --data-binary @data2K.bin
   {"ok":true,"id":"holder","rev":"2-9125078ed1a87fcfca3d32d0834e41c8"}
   $ curl http://127.0.0.1:15984/koi | jq .sizes.external
   2052
   $ curl -X PUT http://127.0.0.1:15984/koi/holder/filler.bin?rev=2-9125078ed1a87fcfca3d32d0834e41c8 -H'Content-Type:application/octet-stream' --data-binary @data8K.bin
   {"ok":true,"id":"holder","rev":"3-eb058b53377ede2720530b6ff073813a"}
   $ curl http://127.0.0.1:15984/koi | jq .sizes.external
   10244
   ```
   
   So far everything goes as expected and external size is growing in sync with attachment sizes. Let's try to update big attachment with a smaller file and then just delete the document completely:
   ```bash
   $ curl -X PUT http://127.0.0.1:15984/koi/holder/filler.bin?rev=3-eb058b53377ede2720530b6ff073813a -H'Content-Type:application/octet-stream' --data-binary @data4K.bin
   {"ok":true,"id":"holder","rev":"4-e1ce5b2ae946a8a64f3503cb6d646686"}
   $ curl http://127.0.0.1:15984/koi | jq .sizes.external
   14340
   $ curl -X POST http://127.0.0.1:15984/koi/_compact
   {"ok":true}
   $ curl http://127.0.0.1:15984/koi | jq .sizes.external
   6148
   $ curl -X DELETE http://127.0.0.1:15984/koi/holder?rev=4-e1ce5b2ae946a8a64f3503cb6d646686
   {"ok":true,"id":"holder","rev":"5-548de2faf802ebcca8de0c3ca3c32c09"}
   $ curl http://127.0.0.1:15984/koi | jq .sizes.external
   6148
   $ curl -X POST http://127.0.0.1:15984/koi/_compact
   {"ok":true}
   $ curl http://127.0.0.1:15984/koi | jq .sizes.external
   4
   ```
   
   ## Testing recommendations
   
   Surprisingly we don't have any tests for database info and I couldn't find an appropriate place to add new tests, so I just wrote a new test suite.
   ```bash
   $ make eunit apps=couch suites=couch_db_info_tests
   ...
   ======================== EUnit ========================
   module 'couch_db_info_tests'
   Application crypto was left running!
     couch_db_info_tests: db_info_sizes_test_...[0.014 s] ok
     couch_db_info_tests: db_info_sizes_test_...[0.023 s] ok
     couch_db_info_tests: db_info_sizes_test_...[0.015 s] ok
     couch_db_info_tests: db_info_sizes_test_...[0.010 s] ok
     couch_db_info_tests: db_info_sizes_test_...[0.010 s] ok
     couch_db_info_tests: db_info_sizes_test_...[0.010 s] ok
     couch_db_info_tests: db_info_sizes_test_...[0.101 s] ok
   [os_mon] cpu supervisor port (cpu_sup): Erlang has closed
     [done in 0.204 s]
   =======================================================
     All 7 tests passed.
   ```
   
   Naturally Travis and Jenkins should pass too.
   
   ## Checklist
   
   - [x] Code is written and works correctly;
   - [x] Changes are covered by tests;
   - [ ] Documentation reflects the changes;
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services