You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by va...@apache.org on 2023/02/21 18:55:13 UTC
[couchdb] branch main updated (f7cafa30a -> 187d9337e)
This is an automated email from the ASF dual-hosted git repository.
vatamane pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb.git
from f7cafa30a Fix typo in "feature_flags" section comment (#4433)
new b3890b5d3 Fix and extend developer instructions
new 9cde18150 mango: skip the `$keymapMatch` test for text indexes
new 1278cf19f mango: switch to UTF-8 encoding for every test
new 187d9337e Consolidate Mango integration tests
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
Makefile | 6 +-
Makefile.win | 6 +-
README-DEV.rst | 113 ++++++++++++++++++++++----------
src/mango/test/02-basic-find-test.py | 1 -
src/mango/test/03-operator-test.py | 4 ++
src/mango/test/04-key-tests.py | 1 -
src/mango/test/13-users-db-find-test.py | 1 -
src/mango/test/README.md | 29 --------
src/mango/test/mango.py | 44 ++++++-------
src/mango/test/user_docs.py | 1 -
10 files changed, 115 insertions(+), 91 deletions(-)
delete mode 100644 src/mango/test/README.md
[couchdb] 04/04: Consolidate Mango integration tests
Posted by va...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vatamane pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 187d9337ef918b8b7feb77d04f56bc524ff461a9
Author: Gabor Pali <ga...@ibm.com>
AuthorDate: Mon Feb 20 15:10:39 2023 +0100
Consolidate Mango integration tests
This commit groups a couple of major changes that aim to amend
many pain points in making the Mango integration test suites
more accessible.
- The test framework behind the Mango integration test suite
provides a lot of flags that are not currently exposed on
the level of the main `Makefile`. Change this for the
greater flexilibility.
- Mango's test suite documentation is buried in the source tree,
which is not common for other kind of tests. To increase its
visibility and unify the style, move the contents of this file
over to the general developer documentation.
- Promote the use of the `mango-test` target instead of setting
up the related machinery manually. The commands recorded in
the original documentation are out of date and only minor
implementation details anyway.
- Retire the explicit control over the activation of Mango
integration tests that require support for text indexes.
Instead learn the availability of this feature from the current
CouchDB instance and run tests based on that. This effectively
makes the activation automated, which could be controlled
implicitly by either hooking up of a Clouseau instance or not.
- Running the Mango integration tests do not remove the databases
on their completion, which can inadvertently pollute the local
data store. To avoid this, enforce removal of test databases but
allow it to be disabled on demand.
---
Makefile | 6 +++++-
Makefile.win | 6 +++++-
README-DEV.rst | 26 ++++++++++++++++++++++++++
src/mango/test/README.md | 29 -----------------------------
src/mango/test/mango.py | 44 +++++++++++++++++++++-----------------------
5 files changed, 57 insertions(+), 54 deletions(-)
diff --git a/Makefile b/Makefile
index f44075d86..726d38d86 100644
--- a/Makefile
+++ b/Makefile
@@ -328,7 +328,11 @@ mango-test: devclean all
@cd src/mango && \
python3 -m venv .venv && \
.venv/bin/python3 -m pip install -r requirements.txt
- @cd src/mango && ../../dev/run "$(TEST_OPTS)" -n 1 --admin=testuser:testpass '.venv/bin/python3 -m nose2'
+ @cd src/mango && \
+ ../../dev/run "$(TEST_OPTS)" \
+ -n 1 \
+ --admin=adm:pass \
+ 'COUCH_USER=adm COUCH_PASS=pass .venv/bin/python3 -m nose2 $(MANGO_TEST_OPTS)'
.PHONY: weatherreport-test
diff --git a/Makefile.win b/Makefile.win
index c13528bba..458b61d47 100644
--- a/Makefile.win
+++ b/Makefile.win
@@ -294,7 +294,11 @@ mango-test: devclean all
@cd src\mango && \
python.exe -m venv .venv && \
.venv\Scripts\pip.exe install -r requirements.txt
- @cd src\mango && .venv\Scripts\python.exe ..\..\dev\run -n 1 --admin=testuser:testpass .venv\Scripts\nose2
+ @cd src\mango && \
+ ..\..\dev\run $(TEST_OPTS) \
+ -n 1 \
+ --admin=adm:pass \
+ "env COUCH_USER=adm COUCH_PASS=pass .venv\Scripts\nose2 $(MANGO_TEST_OPTS)"
################################################################################
diff --git a/README-DEV.rst b/README-DEV.rst
index d8fd299d8..dd527337a 100644
--- a/README-DEV.rst
+++ b/README-DEV.rst
@@ -239,6 +239,32 @@ Scala, and Maven) have been installed successfully, e.g.::
git clone https://github.com/cloudant-labs/clouseau
mvn -f clouseau/pom.xml scala:run
+Mango Integration Tests
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Tests for the Mango interface can be run individually with the help of
+the ``mango-test`` target and they can be narrowed down to specific
+test suites via the ``MANGO_TEST_OPTS`` variable::
+
+ make mango-test \
+ MANGO_TEST_OPTS="--pretty-assert --verbose 03-operator-test"
+
+The value of the ``MANGO_TEST_OPTS`` variable will be passed down to
+the `Nose 2 <https://nose2.io/>`_ testing framework which is used for
+the implementation. Consult its documentation for more information.
+
+Tests that rely on text indexes are run only if the ``search`` feature
+is reported to be available (i.e. a working Clouseau instance is
+connected), otherwise they will be skipped.
+
+Note that the databases that are created during the tests will be all
+removed after each of the suites completed. However, with the help of
+the ``MANGO_TESTS_KEEP_DBS`` environment variable, it can be requested
+to keep those databases around for further investigation::
+
+ MANGO_TESTS_KEEP_DBS=please \
+ make mango-test MANGO_TEST_OPTS='03-operator-test'
+
Static Code Analysis
~~~~~~~~~~~~~~~~~~~~
diff --git a/src/mango/test/README.md b/src/mango/test/README.md
deleted file mode 100644
index c2f8aada2..000000000
--- a/src/mango/test/README.md
+++ /dev/null
@@ -1,29 +0,0 @@
-Mango Tests
-===========
-
-CouchDB should be started with `./dev/run -a testuser:testpass`.
-
-To run these, do this in the Mango top level directory:
-
- $ python3 -m venv venv
- $ . venv/bin/activate
- $ pip3 install -r requirements.txt
- $ venv/bin/nose2
-
-To run an individual test suite:
- nose2 12-use-correct-index-test
-
-To run the tests with text index support:
- MANGO_TEXT_INDEXES=1 nose2 test
-
-
-Test configuration
-==================
-
-The following environment variables can be used to configure the test fixtures:
-
- * `COUCH_HOST` - root url (including port) of the CouchDB instance to run the tests against. Default is `"http://127.0.0.1:15984"`.
- * `COUCH_USER` - CouchDB username (with admin premissions). Default is `"testuser"`.
- * `COUCH_PASSWORD` - CouchDB password. Default is `"testpass"`.
- * `COUCH_AUTH_HEADER` - Optional Authorization header value. If specified, this is used instead of basic authentication with the username/password variables above.
- * `MANGO_TEXT_INDEXES` - Set to `"1"` to run the tests only applicable to text indexes.
diff --git a/src/mango/test/mango.py b/src/mango/test/mango.py
index 03cb85f48..81bc082b6 100644
--- a/src/mango/test/mango.py
+++ b/src/mango/test/mango.py
@@ -23,17 +23,22 @@ import user_docs
import limit_docs
+COUCH_HOST = "http://127.0.0.1:15984"
+COUCH_USER = os.environ.get("COUCH_USER")
+COUCH_PASS = os.environ.get("COUCH_PASS")
+
+
def random_db_name():
return "mango_test_" + uuid.uuid4().hex
def has_text_service():
- return os.environ.get("MANGO_TEXT_INDEXES") == "1"
+ features = requests.get(COUCH_HOST).json()["features"]
+ return "search" in features
-def get_from_environment(key, default):
- value = os.environ.get(key)
- return value if value is not None else default
+def clean_up_dbs():
+ return not os.environ.get("MANGO_TESTS_KEEP_DBS")
# add delay functionality
@@ -46,32 +51,15 @@ class Database(object):
def __init__(
self,
dbname,
- host="127.0.0.1",
- port="15984",
- user="testuser",
- password="testpass",
):
- root_url = get_from_environment("COUCH_HOST", "http://{}:{}".format(host, port))
- auth_header = get_from_environment("COUCH_AUTH_HEADER", None)
- user = get_from_environment("COUCH_USER", user)
- password = get_from_environment("COUCH_PASSWORD", password)
-
- self.root_url = root_url
self.dbname = dbname
self.sess = requests.session()
-
- # allow explicit auth header to be set to enable testing
- # against deployments where basic auth isn't available
- if auth_header is not None:
- self.sess.headers["Authorization"] = auth_header
- else:
- self.sess.auth = (user, password)
-
+ self.sess.auth = (COUCH_USER, COUCH_PASS)
self.sess.headers["Content-Type"] = "application/json"
@property
def url(self):
- return "{}/{}".format(self.root_url, self.dbname)
+ return "{}/{}".format(COUCH_HOST, self.dbname)
def path(self, parts):
if isinstance(parts, ("".__class__, "".__class__)):
@@ -299,6 +287,11 @@ class UsersDbTests(unittest.TestCase):
klass.db = Database("_users")
user_docs.setup_users(klass.db)
+ @classmethod
+ def tearDownClass(klass):
+ if clean_up_dbs():
+ klass.db.delete()
+
def setUp(self):
self.db = self.__class__.db
@@ -309,6 +302,11 @@ class DbPerClass(unittest.TestCase):
klass.db = Database(random_db_name())
klass.db.create(q=1, n=1)
+ @classmethod
+ def tearDownClass(klass):
+ if clean_up_dbs():
+ klass.db.delete()
+
def setUp(self):
self.db = self.__class__.db
[couchdb] 01/04: Fix and extend developer instructions
Posted by va...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vatamane pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit b3890b5d3ce1d7715f18d4db6f74de087cb7b5e7
Author: Gabor Pali <ga...@ibm.com>
AuthorDate: Mon Feb 20 15:46:41 2023 +0100
Fix and extend developer instructions
- Various whitespace and formatting fixes.
- Fix the reference to `EUNIT_OPTS`. The variable name has a
typo (it is missing the 'S' from the end) and it is not an
environment variable but a make(1) variable.
- Create subsections of the different kind of tests.
- Update Homebrew home page link.
- Fix Fauxton readme link. The source code for Fauxton now lives
in a separate repository, make the corresponding link reflect
this.
---
README-DEV.rst | 87 ++++++++++++++++++++++++++++++++++++----------------------
1 file changed, 54 insertions(+), 33 deletions(-)
diff --git a/README-DEV.rst b/README-DEV.rst
index 7ef844096..d8fd299d8 100644
--- a/README-DEV.rst
+++ b/README-DEV.rst
@@ -1,9 +1,9 @@
Apache CouchDB DEVELOPERS
=========================
-Before you start here, read `INSTALL.Unix` (or `INSTALL.Windows`) and
-follow the setup instructions including the installation of all the
-listed dependencies for your system.
+Before you start here, read `<INSTALL.Unix.md>`_ (or
+`<INSTALL.Windows.md>`_) and follow the setup instructions including
+the installation of all the listed dependencies for your system.
Only follow these instructions if you are building from a source checkout.
@@ -31,7 +31,7 @@ You need the following optionally to build releases:
You need the following optionally to build Fauxton:
* `nodejs <http://nodejs.org/>`_
-* `npm <https://www.npmjs.com/>`_
+* `npm <https://www.npmjs.com/>`_
You will need these optional dependencies installed if:
@@ -46,8 +46,10 @@ However, you do not need them if:
If you intend to build Fauxton, you will also need to install its
dependencies. After running ``./configure`` to download all of the
dependent repositories, you can read about required dependencies in
-`src/fauxton/readme.md`. Typically, installing npm and node.js are
-sufficient to enable a Fauxton build.
+`its readme
+<https://github.com/apache/couchdb-fauxton/blob/main/readme.md>`_. Typically,
+installing ``npm`` and node.js are sufficient to enable a Fauxton
+build.
Here is a list of *optional* dependencies for various operating systems.
Installation will be easiest, when you install them all.
@@ -68,16 +70,20 @@ or `Visual Studio Code <https://code.visualstudio.com/docs/remote/containers>`_.
.. image:: https://img.shields.io/static/v1?label=Remote%20-%20Containers&message=Open&color=blue&logo=visualstudiocode
:target: https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/apache/couchdb
-If you already have VS Code and Docker installed, you can click the badge above or
-`here <https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/apache/couchdb>`_
-to get started. Clicking these links will cause VS Code to automatically install the
-Remote - Containers extension if needed, clone the source code into a container volume,
-and spin up a dev container for use.
+If you already have VS Code and Docker installed, you can click the
+badge above or `here
+<https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/apache/couchdb>`_
+to get started. Clicking these links will cause VS Code to
+automatically install the Remote - Containers extension if needed,
+clone the source code into a container volume, and spin up a dev
+container for use.
-This ``devcontainer`` will automatically run ``./configure && make`` the first time it is created.
-While this may take some extra time to spin up, this tradeoff means you will be able to
-run things like ``./dev/run``, ``./dev/run --admin=admin:admin``, ``./dev/run --with-admin-party-please``,
-and ``make check`` straight away. Subsequent startups should be quick.
+This ``devcontainer`` will automatically run ``./configure && make``
+the first time it is created. While this may take some extra time to
+spin up, this tradeoff means you will be able to run things like
+``./dev/run``, ``./dev/run --admin=admin:admin``, ``./dev/run
+--with-admin-party-please``, and ``make check`` straight away.
+Subsequent startups should be quick.
Debian-based (inc. Ubuntu) Systems
@@ -108,8 +114,7 @@ Centos 7 and RHEL 7
Mac OS X
~~~~~~~~
-Install `Homebrew <https://github.com/mxcl/homebrew>`_, if you do not have
-it already.
+Install `Homebrew <https://brew.sh/>`_, if you do not have it already.
Unless you want to install the optional dependencies, skip to the next section.
@@ -136,8 +141,9 @@ FreeBSD
Windows
~~~~~~~
-Follow the instructions in `INSTALL.Windows` and build all components from
-source, using the same Visual C++ compiler and runtime.
+Follow the instructions in `<INSTALL.Windows.md>`_ and build all
+components from source, using the same Visual C++ compiler and
+runtime.
Configuring
-----------
@@ -185,6 +191,9 @@ You can also run each test suite individually via the ``eunit``, ``mango-test``,
make elixir-suite
make weatherreport-test
+Erlang Unit Tests
+~~~~~~~~~~~~~~~~~
+
If you need to run specific Erlang tests, you can pass special "options"
to make targets::
@@ -200,28 +209,39 @@ to make targets::
# Ignore tests for specified apps
make eunit skip_deps=couch_log,couch_epi
-The ``apps``, ``suites``, ``tests`` and ``skip_deps`` could be combined in any
-way. These are mimics to ``rebar eunit`` arguments. If you're not satisfied by
-these, you can use EUNIT_OPT environment variable to specify exact `rebar eunit`
-options::
+The ``apps``, ``suites``, ``tests`` and ``skip_deps`` could be
+combined in any way. These are mimics to ``rebar eunit`` arguments. If
+you're not satisfied by these, you can use the ``EUNIT_OPTS`` variable
+to specify exact ``rebar eunit`` options::
make eunit EUNIT_OPTS="apps=couch,chttpd"
-There is an additional Elixir-based integration test suite for
-Dreyfus, which is not run automatically by the ``check`` target but it
-could be done manually via the corresponding target::
+Elixir Integration Tests
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+All the Elixir-based integration tests could be by the `elixir-suite`
+target::
+
+ make elixir-suite
+
+There is an additional suite for Dreyfus, which is not run
+automatically by either the ``elixir-suite`` or the ``check`` target
+but it could be done manually via the corresponding target::
make elixir-search
Note that this requires a running Clouseau instance with the name
-`clouseau@127.0.0.1`. The easiest way to get it is to clone the
-[`cloudant-labs/clouseau`](https://github.com/cloudant-labs/clouseau)
+``clouseau@127.0.0.1``. The easiest way to get it is to clone the
+`cloudant-labs/clouseau <https://github.com/cloudant-labs/clouseau>`_
repository and launch it run there once all the prerequisites (JDK,
Scala, and Maven) have been installed successfully, e.g.::
git clone https://github.com/cloudant-labs/clouseau
mvn -f clouseau/pom.xml scala:run
+Static Code Analysis
+~~~~~~~~~~~~~~~~~~~~
+
Code analyzer could be run by::
make dialyze
@@ -233,7 +253,8 @@ If you need to analyze only specific apps, you can specify them in familiar way
See ``make help`` for more info and useful commands.
-Please report any problems to the developer's mailing list.
+Please report any problems to the `developer's mailing list
+<ht...@couchdb.apache.org>`_.
Releasing
---------
@@ -257,13 +278,13 @@ built by running::
make release
-The release can then be found in the rel/couchdb directory.
+The release can then be found in the ``rel/couchdb`` directory.
Microsoft Windows
~~~~~~~~~~~~~~~~~
The release tarball and Erlang CouchDB release commands work on
Microsoft Windows the same as they do on Unix-like systems. To create
-a full installer, the separate couchdb-glazier repository is required.
-Full instructions are available in that repository's README file.
-
+a full installer, the separate `couchdb-glazier repository
+<https://github.com/apache/couchdb-glazier>`_ is required. Full
+instructions are available in that repository's ``README`` file.
[couchdb] 03/04: mango: switch to UTF-8 encoding for every test
Posted by va...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vatamane pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 1278cf19fed35582b8e1270e42c7f05e08e42361
Author: Gabor Pali <ga...@ibm.com>
AuthorDate: Mon Feb 20 11:01:29 2023 +0100
mango: switch to UTF-8 encoding for every test
Python 3 uses UTF-8 encoding on reading the source files by default
and UTF-8 itself has become more widely adopted in the recent years
therefore it makes sense to remove the associated annotations.
At the same time, it helps to unbreak the Unicode key tests where
the Apple logo ('', as Unicode character) is featured and then got
butchered by forcing the ISO-8859-1 encoding on it.
---
src/mango/test/02-basic-find-test.py | 1 -
src/mango/test/04-key-tests.py | 1 -
src/mango/test/13-users-db-find-test.py | 1 -
src/mango/test/user_docs.py | 1 -
4 files changed, 4 deletions(-)
diff --git a/src/mango/test/02-basic-find-test.py b/src/mango/test/02-basic-find-test.py
index afdba03a2..059e1fc80 100644
--- a/src/mango/test/02-basic-find-test.py
+++ b/src/mango/test/02-basic-find-test.py
@@ -1,4 +1,3 @@
-# -*- coding: latin-1 -*-
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
# use this file except in compliance with the License. You may obtain a copy of
# the License at
diff --git a/src/mango/test/04-key-tests.py b/src/mango/test/04-key-tests.py
index a9551c6f8..0cc410321 100644
--- a/src/mango/test/04-key-tests.py
+++ b/src/mango/test/04-key-tests.py
@@ -1,4 +1,3 @@
-# -*- coding: latin-1 -*-
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
# use this file except in compliance with the License. You may obtain a copy of
# the License at
diff --git a/src/mango/test/13-users-db-find-test.py b/src/mango/test/13-users-db-find-test.py
index 73d15ea1a..7daf2ee7f 100644
--- a/src/mango/test/13-users-db-find-test.py
+++ b/src/mango/test/13-users-db-find-test.py
@@ -1,4 +1,3 @@
-# -*- coding: latin-1 -*-
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
# use this file except in compliance with the License. You may obtain a copy of
# the License at
diff --git a/src/mango/test/user_docs.py b/src/mango/test/user_docs.py
index 4f06e0342..a1b6c6e1a 100644
--- a/src/mango/test/user_docs.py
+++ b/src/mango/test/user_docs.py
@@ -1,4 +1,3 @@
-# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
# use this file except in compliance with the License. You may obtain a copy of
# the License at
[couchdb] 02/04: mango: skip the `$keymapMatch` test for text indexes
Posted by va...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
vatamane pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 9cde1815031cf8947d82fe2e83c3d45664b593e1
Author: Gabor Pali <ga...@ibm.com>
AuthorDate: Mon Feb 20 10:56:14 2023 +0100
mango: skip the `$keymapMatch` test for text indexes
Text indexes do not support the `$keymapMatch` operator thus let
the test suite know about this limitation to avoid the related
error.
---
src/mango/test/03-operator-test.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/mango/test/03-operator-test.py b/src/mango/test/03-operator-test.py
index a67ef91f3..ed5eb3847 100644
--- a/src/mango/test/03-operator-test.py
+++ b/src/mango/test/03-operator-test.py
@@ -66,6 +66,10 @@ class OperatorTests:
docs = self.db.find({"emptybang": {"$allMatch": {"foo": {"$eq": 2}}}})
self.assertEqual(len(docs), 0)
+ @unittest.skipUnless(
+ not mango.has_text_service(),
+ "text indexes do not support the $keyMapMatch operator",
+ )
def test_keymap_match(self):
amdocs = [
{"foo": {"aa": "bar", "bb": "bang"}},