You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2021/04/09 15:17:52 UTC
[GitHub] [superset] EBoisseauSierra opened a new issue #14052: Unable to run tests successfully as per documentation
EBoisseauSierra opened a new issue #14052:
URL: https://github.com/apache/superset/issues/14052
I am trying to follow the documentation to setup my VM and successfully run the test suite, yet I'm facing multiple hurdles along the way.
### Expected results
* follow [the instructions](https://github.com/apache/superset/blob/master/CONTRIBUTING.md#setup-local-environment-for-development) only,
* run the test suite successfully.
### Actual results
I'm facing some issues:
1. one OS dependency is not specified,
2. one needs an admin actually named `admin` to load examples
2. examples can't be all loaded in one go,
3. tests fail
#### How to reproduce the bug
On a new VM running Fedora 32:
1. OSError: mysql_config not found
```shell
# Following https://github.com/apache/superset/blob/master/CONTRIBUTING.md#setup-local-environment-for-development
git clone git@github.com:apache/superset.git
cd superset
git checkout 1.0.1 # to make things reproducible
sudo dnf install -y gcc gcc-c++ libffi-devel python-devel python-pip python-wheel openssl-devel cyrus-sasl-devel openldap-devel # as per https://superset.apache.org/docs/installation/installing-superset-from-scratch#os-dependencies
python3.8 -m venv venv
source venv/bin/activate
pip install -r requirements/local.txt
```
This throws an `OSError: mysql_config not found`
As per [this SO question](https://stackoverflow.com/q/22949654/5433628), **the solution is to install `mariadb-devel` — what actually solves the issue.**
2. Fail to load the examples
```shell
# Following https://github.com/apache/superset/blob/master/CONTRIBUTING.md#setup-local-environment-for-development
git clone git@github.com:apache/superset.git
cd superset
git checkout 1.0.1 # to make things reproducible
sudo dnf install -y gcc gcc-c++ libffi-devel python-devel python-pip python-wheel openssl-devel cyrus-sasl-devel openldap-devel # as per https://superset.apache.org/docs/installation/installing-superset-from-scratch#os-dependencies
sudo dnf -y install mariadb-devel # FIX as per https://stackoverflow.com/q/22949654/5433628
python3.8 -m venv venv
source venv/bin/activate
pip install -r requirements/local.txt
pip install .
superset fab create-admin --username adent --firstname arthur --lastname dent --email dontp@n.ic --password 42
superset db upgrade
superset init
superset load-examples
```
This returns the following error:
```
INFO:superset.utils.core:Creating database reference for examples
Loading examples metadata and related data into examples
Traceback (most recent call last):
File "/home/fedora/superset/venv/bin/superset", line 11, in <module>
load_entry_point('apache-superset==1.0.1', 'console_scripts', 'superset')()
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/flask/cli.py", line 426, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/cli.py", line 187, in load_examples
load_examples_run(load_test_data, only_metadata, force)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/cli.py", line 120, in load_examples_run
from superset import examples
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/examples/__init__.py", line 18, in <module>
from .birth_names import load_birth_names
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/examples/birth_names.py", line 46, in <module>
raise NoDataException(
superset.exceptions.NoDataException: Admin user does not exist. Please, check if test users are properly loaded (`superset load_test_users`).
```
We can then explicitely create an admin user:
```shell
# Following https://github.com/apache/superset/blob/master/CONTRIBUTING.md#setup-local-environment-for-development
git clone git@github.com:apache/superset.git
cd superset
git checkout 1.0.1 # to make things reproducible
sudo dnf install -y gcc gcc-c++ libffi-devel python-devel python-pip python-wheel openssl-devel cyrus-sasl-devel openldap-devel # as per https://superset.apache.org/docs/installation/installing-superset-from-scratch#os-dependencies
sudo dnf -y install mariadb-devel # FIX as per https://stackoverflow.com/q/22949654/5433628
python3.8 -m venv venv
source venv/bin/activate
pip install -r requirements/local.txt
pip install .
superset fab create-admin --username admin --firstname Ad --lastname Ministrator --email admin@email.com --password secret # FIX explicitely create an admin user with username exactly `admin`
superset db upgrade
superset init
superset load-examples
```
But the following error is still returned
```
[…]
INFO:superset.models.helpers:Updating table_columns gender
Loading data inside the import transaction
INFO:superset.datasets.commands.importers.v1.utils:Loading data inside the import transaction
Traceback (most recent call last):
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: too many SQL variables
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/commands/importers/v1/examples.py", line 67, in run
self._import(db.session, self._configs, self.overwrite, self.force_data)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/commands/importers/v1/examples.py", line 99, in _import
dataset = import_dataset(
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 119, in import_dataset
load_data(data_uri, dataset, example_database, session)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 146, in load_data
df.to_sql(
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/pandas/core/generic.py", line 2605, in to_sql
sql.to_sql(
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/pandas/io/sql.py", line 589, in to_sql
pandas_sql.to_sql(
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/pandas/io/sql.py", line 1406, in to_sql
raise err
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/pandas/io/sql.py", line 1398, in to_sql
table.insert(chunksize, method=method)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/pandas/io/sql.py", line 830, in insert
exec_insert(conn, keys, chunk_iter)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/pandas/io/sql.py", line 758, in _execute_insert_multi
conn.execute(self.table.insert(data))
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
util.raise_(
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) too many SQL variables
[SQL: INSERT INTO "FCC 2018 Survey" ("ID", is_software_dev, is_first_dev_job, months_job_search, job_pref, job_intr_fllstck, job_intr_backend, job_intr_frntend, job_intr_mobile, job_intr_devops, job_intr_datasci, job_intr_teacher, job_intr_qa_engn, job_intr_ux_engn, job_intr_projm, job_intr_gamedev, job_intr_infosec, job_intr_dataengn, job_intr_other, when_appl_job, expected_earn, job_lctn_pref, job_relocate, reasons_to_code, reasons_to_code_other, rsrc_fcc, rsrc_mdn, rsrc_so, rsrc_edx, rsrc_coursera, rsrc_khan_acdm, rsrc_pluralsght, rsrc_codeacdm, rsrc_udacity, rsrc_udemy, rsrc_code_wars, rsrc_treehouse, rsrc_hackerrank, rsrc_frntendmstr, rsrc_lynda, rsrc_egghead, rsrc_css_tricks, rsrc_other, codeevnt_fcc, codeevnt_hackthn, codeevnt_confs, codeevnt_workshps, codeevnt_startupwknd, codeevnt_nodeschl, codeevnt_womenwc, codeevnt_girldevit, codeevnt_coderdojo, codeevnt_meetup, codeevnt_railsbrdg, codeevnt_gamejam, codeevnt_railsgrls, codeevnt_djangogrls, codeevnt_wkndbtcmp, codeevnt_o
ther, podcast_fcc, podcast_codenewbie, podcast_changelog, podcast_sedaily, podcast_js_jabber, podcast_syntaxfm, podcast_ltcwm, podcast_fullstckrd, podcast_frnthppyhr, podcast_codingblcks, podcast_shoptalk, podcast_devtea, podcast_progthrwdwn, podcast_geekspeak, podcast_hanselmnts, podcast_talkpythonme, podcast_rubyrogues, podcast_codepenrd, podcast_seradio, podcast_other, yt_mit_ocw, yt_fcc, yt_computerphile, yt_devtips, yt_cs_dojo, yt_engn_truth, yt_learncodeacdm, yt_lvluptuts, yt_funfunfunct, yt_codingtuts360, yt_codingtrain, yt_derekbanas, yt_simplilearn, yt_simpleprog, yt_mozillahacks, yt_googledevs, yt_other, hours_learning, months_programming, bootcamp_attend, bootcamp_name, bootcamp_finished, bootcamp_have_loan, bootcamp_recommend, money_for_learning, age, gender, gender_other, country_citizen, country_live, live_city_population, is_ethnic_minority, lang_at_home, school_degree, school_major, marital_status, has_finance_depends, has_children, num_children, do_finance_support,
debt_amt, home_mrtg_has, home_mrtg_owe, student_debt_has, student_debt_amt, curr_emplymnt, curr_emplymnt_other, curr_field, last_yr_income, communite_time, is_self_employed, has_served_military, is_recv_disab_bnft, has_high_spd_ntnet, time_start, time_end, network_id, time_total_sec) VALUES
[…]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/fedora/superset/venv/bin/superset", line 11, in <module>
load_entry_point('apache-superset==1.0.1', 'console_scripts', 'superset')()
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/flask/cli.py", line 426, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/cli.py", line 187, in load_examples
load_examples_run(load_test_data, only_metadata, force)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/cli.py", line 171, in load_examples_run
examples.load_from_configs(force)
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/examples/utils.py", line 30, in load_from_configs
command.run()
File "/home/fedora/superset/venv/lib64/python3.8/site-packages/superset/commands/importers/v1/examples.py", line 71, in run
raise self.import_error()
superset.commands.exceptions.CommandException
```
Weirdly, the solution seems to be… running `superset load-examples` again.
3. Test suite not passing
```shell
# Following https://github.com/apache/superset/blob/master/CONTRIBUTING.md#setup-local-environment-for-development
git clone git@github.com:apache/superset.git
cd superset
git checkout 1.0.1 # to make things reproducible
sudo dnf install -y gcc gcc-c++ libffi-devel python-devel python-pip python-wheel openssl-devel cyrus-sasl-devel openldap-devel # as per https://superset.apache.org/docs/installation/installing-superset-from-scratch#os-dependencies
sudo dnf -y install mariadb-devel # FIX as per https://stackoverflow.com/q/22949654/5433628
python3.8 -m venv venv
source venv/bin/activate
pip install -r requirements/local.txt
pip install .
superset fab create-admin --username admin --firstname Ad --lastname Ministrator --email admin@email.com --password secret # FIX explicitely create an admin user with username exactly `admin`
superset db upgrade
superset init
superset load-examples || superset load-examples # FIX
pip install -r requirements/testing.txt # FIX as, btw, this isn't specified but otherwise `tox` isn't installed
tox -e py38
```
and this fails:
```
[…]
======================================================= test session starts =======================================================
platform linux -- Python 3.8.7, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
cachedir: .tox/py38/.pytest_cache
rootdir: /home/fedora/superset, configfile: pytest.ini, testpaths: tests
plugins: cov-2.10.1, celery-4.4.7
collected 1194 items
[…]
======================== 60 failed, 1007 passed, 16 skipped, 220 warnings, 115 errors in 931.35s (0:15:31) ========================
ERROR: InvocationError for command /home/fedora/superset/.tox/py38/bin/pytest -s (exited with code 1)
```
One could also run the [shell script to run tests faster](https://github.com/apache/superset/tree/master/scripts/tests):
```shell
# Following https://github.com/apache/superset/blob/master/CONTRIBUTING.md#setup-local-environment-for-development
git clone git@github.com:apache/superset.git
cd superset
git checkout 1.0.1 # to make things reproducible
sudo dnf install -y gcc gcc-c++ libffi-devel python-devel python-pip python-wheel openssl-devel cyrus-sasl-devel openldap-devel # as per https://superset.apache.org/docs/installation/installing-superset-from-scratch#os-dependencies
sudo dnf -y install mariadb-devel # FIX as per https://stackoverflow.com/q/22949654/5433628
python3.8 -m venv venv
source venv/bin/activate
pip install -r requirements/local.txt
pip install .
superset fab create-admin --username admin --firstname Ad --lastname Ministrator --email admin@email.com --password secret # FIX explicitely create an admin user with username exactly `admin`
superset db upgrade
superset init
superset load-examples || superset load-examples # FIX
pip install -r requirements/testing.txt # FIX as, btw, this isn't specified but otherwise `tox` isn't installed
scripts/tests/run.sh
```
but then it misses some OS dependencies:
```
------------------------------------
DB_URI=postgresql+psycopg2://superset:superset@localhost/test
Superset config module=tests.superset_test_config
Run init procedures=1
Run reset DB=1
Test to run:tests
------------------------------------
--------------------
Reseting test DB
--------------------
scripts/tests/run.sh: line 29: docker-compose: command not found
scripts/tests/run.sh: line 40: docker: command not found
```
So after installing it and setting it up:
```shell
# Following https://github.com/apache/superset/blob/master/CONTRIBUTING.md#setup-local-environment-for-development
git clone git@github.com:apache/superset.git
cd superset
git checkout 1.0.1 # to make things reproducible
sudo dnf install -y gcc gcc-c++ libffi-devel python-devel python-pip python-wheel openssl-devel cyrus-sasl-devel openldap-devel # as per https://superset.apache.org/docs/installation/installing-superset-from-scratch#os-dependencies
sudo dnf -y install mariadb-devel # FIX as per https://stackoverflow.com/q/22949654/5433628
# FIX install docker as per https://docs.docker.com/engine/install/fedora/
sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
sudo dnf install docker-ce docker-ce-cli containerd.io
sudo systemctl start docker
sudo systemctl enable docker
sudo groupadd docker
sudo usermod -aG docker $USER
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo reboot
python3.8 -m venv venv
source venv/bin/activate
pip install -r requirements/local.txt
pip install .
superset fab create-admin --username admin --firstname Ad --lastname Ministrator --email admin@email.com --password secret # FIX explicitely create an admin user with username exactly `admin`
superset db upgrade
superset init
superset load-examples || superset load-examples # FIX
pip install -r requirements/testing.txt # FIX as, btw, this isn't specified but otherwise `tox` isn't installed
scripts/tests/run.sh
```
But I get
```
scripts/tests/run.sh
------------------------------------
DB_URI=postgresql+psycopg2://superset:superset@localhost/test
Superset config module=tests.superset_test_config
Run init procedures=1
Run reset DB=1
Test to run:tests
------------------------------------
--------------------
Reseting test DB
--------------------
Error: No such container: superset_db
```
### Environment
(please complete the following information):
- superset version: 1.0.1 (tested with `667eb83` — latest commit passing CI at 2021-04-09T14:58:35Z — as well)
- python version: 3.8.7
- node.js version: n/a
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [x] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional information
It may well be that I'm not reading the instructions correctly!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] EBoisseauSierra edited a comment on issue #14052: Unable to run tests successfully as per documentation
Posted by GitBox <gi...@apache.org>.
EBoisseauSierra edited a comment on issue #14052:
URL: https://github.com/apache/superset/issues/14052#issuecomment-833435226
Thanks @etr2460 for having had a deeper look.
I have uploaded [here](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/superset.db) (:warning: 91MiB file) the SQLite db generated (in `/home/fedora/.superset/superset.db`) by the script detailed [above](https://github.com/apache/superset/issues/14052#issuecomment-828994879).
I'm afraid I won't be able to prioritise investigating this (i.e. why running the tests fails when installing Superset bare metal) further at the moment, as I've been able to solve my actual issue (i.e. “how to run the test suite locally, so that if it passes there, [the Github CI](https://github.com/apache/superset/actions) will be ok too?”).
To do so, I've given up on the “bare metal” install, and went for docker instead. The process is detailed [here](https://gist.github.com/EBoisseauSierra/65ba8238fa641138bd6e8e6532a5b0b9) (kudos to @kleegnik for this!). I now intend to improve [the related PR](https://github.com/apache/superset/pull/14102) a bit further and close the whole matter once done.
How does that sound to you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] EBoisseauSierra commented on issue #14052: Unable to run tests successfully as per documentation
Posted by GitBox <gi...@apache.org>.
EBoisseauSierra commented on issue #14052:
URL: https://github.com/apache/superset/issues/14052#issuecomment-833435226
Thanks @etr2460 for having had a deeper look.
I have uploaded [here](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/superset.db) (:warning: 91MiB file) the SQLite db generated (in `/home/fedora/.superset/superset.db`) by the script detailed [above](https://github.com/apache/superset/issues/14052#issuecomment-828994879).
I'm afraid I won't be able to prioritise investigating this (i.e. why running the tests fails when installing Superset bare metal) further at the moment, as I've been able to solve my actual issue (i.e. “how to run the test suite locally, so that if it passes there, [the Github CI](https://github.com/apache/superset/actions) will be ok too?”).
To do so, I've given up on the “bare metal” install, and went for docker instead. The process is detailed [here](https://gist.github.com/EBoisseauSierra/65ba8238fa641138bd6e8e6532a5b0b9). I now intend to improve [the related PR](https://github.com/apache/superset/pull/14102) a bit further and close the whole matter once done.
How does that sound to you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] etr2460 commented on issue #14052: Unable to run tests successfully as per documentation
Posted by GitBox <gi...@apache.org>.
etr2460 commented on issue #14052:
URL: https://github.com/apache/superset/issues/14052#issuecomment-832267829
@EBoisseauSierra looking at your logs, it appears that the example database is not being properly populated, causing the errors throughout.
Perhaps it could be easier to debug this from within superset itself. Can you run the flask and npm dev server and inspect the example database in SQL Lab? That could help indicate what's properly functioning vs. what's broken
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] EBoisseauSierra commented on issue #14052: Unable to run tests successfully as per documentation
Posted by GitBox <gi...@apache.org>.
EBoisseauSierra commented on issue #14052:
URL: https://github.com/apache/superset/issues/14052#issuecomment-828994879
So I've been able to narrow down things a little.
Using following setup on a Fedora32 VM:
```shell
sudo dnf install -y git
git clone git@github.com:apache/superset.git
cd superset
git heavy_check_markout 1.1.0 # to make things reproducible
sudo dnf install -y gcc gcc-c++ libffi-devel python-devel python-pip python-wheel openssl-devel cyrus-sasl-devel openldap-devel # as per https://superset.apache.org/docs/installation/installing-superset-from-scratch#os-dependencies
sudo dnf -y install mariadb-devel # FIX as per https://stackoverflow.com/q/22949654/5433628
python3.8 -m venv venv
source venv/bin/activate
pip install -r requirements/testing.txt
pip install .
superset fab create-admin --username admin --firstname Ad --lastname Ministrator --email admin@email.com --password secret # FIX explicitely create an admin user with username exactly `admin`
superset db upgrade
superset init
superset load-examples || superset load-examples # FIX
sudo dnf -y install redis
sudo systemctl enable --now redis
```
I can detail the result of various tests, that I run using:
```
tox -e py38 -- tests/<filename>.py 2>&1 | tee tox38_<filename>.log
```
| Status | Test file | Result | Log file |
| :---: | :---- | :---- | :---- |
| :warning: | `access_tests.py` | 9P, 9W, 1E | [tox38_access_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_access_tests.log)|
| :x: | `alerts_tests.py` | 20F, 6P, 6W | [tox38_alerts_tests.log](https://chat.unipart.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_alerts_tests.log)|
| :warning: | `base_api_tests.py` | 3P, 9W, 2E | [tox38_base_api_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_base_api_tests.log)|
| :heavy_check_mark: | `base_tests.py` | | [tox38_base_tests.log](https://chat.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_base_tests.log) |
| :heavy_check_mark: | `cache_tests.py` | | [tox38_cache_tests.log](https://chat.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_cache_tests.log)|
| :x: | `celery_tests.py` | 4F, 5P, 12W | [tox38_celery_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_celery_tests.log) |
| :heavy_check_mark: | `commands_test.py` | 2P, 7W | [tox38_commands_tests.log](https://chat.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_commands_tests.log)|
| :heavy_check_mark: | `config_tests.py` | 6P, 8W | [tox38_config_tests.log](https://chat.unipart.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_config_tests.log)|
| :gear: | `conftest.py` | 6W | [tox38_conftest.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_conftest.log)|
| :x: | `core_tests.py` | 61P, 38W, 1E | [tox38_core_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_core_tests.log)|
| :heavy_check_mark: | `csv_upload_tests.py` | 2P, 2S, 10W | [tox38_csv_upload_tests.log](https://chat.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_csv_upload_tests.log)|
| :warning: | `dashboard_tests.py` | 17P, 7W, 2E | [tox38_dashboard_tests.log](https://chat.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_dashboard_tests.log)|
| :gear: | `dashboard_utils.py` | | |
| :heavy_check_mark: | `dataframe_test.py` | 2P, 7W | [tox38_dataframe_test.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_dataframe_test.log)|
| :x: | `datasource_tests.py` | 1F, 2P, 7W | [tox38_datasource_tests.log](https://chat.unipart.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_datasource_tests.log)|
| :heavy_check_mark: | `dict_import_export_tests.py` | 13P, 7W | [tox38_dict_import_export_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_dict_import_export_tests.log)|
| :heavy_check_mark: | `druid_func_tests.py` | 33P, 7W | [tox38_druid_func_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_druid_func_tests.log)|
| :heavy_check_mark: | `druid_func_tests_sip38.py` | 33P, 7W | [tox38_druid_func_tests_sip38.log](https://chat.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_druid_func_tests_sip38.log)|
| :heavy_check_mark: | `druid_tests.py` | 9P, 1S, 63W | [tox38_druid_tests.log](https://chat.unipart.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_druid_tests.log)|
| :heavy_check_mark: | `email_tests.py` | 8P, 7W | [tox38_email_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_email_tests.log)|
| :heavy_check_mark: | `event_logger_tests.py` | 5P, 7W | [tox38_event_logger_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_event_logger_tests.log)|
| :heavy_check_mark: | `feature_flag_tests.py` | 3P, 7W | [tox38_feature_flag_tests.log](https://chat.unipart.io/github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_feature_flag_tests.log)|
| :heavy_check_mark: | `form_tests.py` | 2P, 7W | [tox38_form_tests.log](https://chat.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_form_tests.log)|
| :x: | `import_export_tests.py` | 3F, 15P, 7W, 4E | [tox38_import_export_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_import_export_tests.log)|
| :gear: | `__init__.py` | | |
| :gear: | `insert_chart_mixin.py` | | |
| :heavy_check_mark: | `jinja_context_tests.py` | 30P, 7W | [tox38_jinja_context_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_jinja_context_tests.log)|
| :heavy_check_mark: | `log_api_tests.py` | 5P, 7W | [tox38_log_api_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_log_api_tests.log)|
| :heavy_check_mark: | `logging_configurator_tests.py` | 1P, 6W | [tox38_logging_configurator_tests.log](https://chat.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_logging_configurator_tests.log)|
| :heavy_check_mark: | `migration_tests.py` | 1P, 7W | [tox38_migration_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_migration_tests.log)|
| :heavy_check_mark: | `model_tests.py` | 18P, 7W | [tox38_model_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_model_tests.log)|
| :heavy_check_mark: | `pandas_postprocessing_tests.py` | 24P, 1S, 7W | [tox38_pandas_postprocessing_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_pandas_postprocessing_tests.log)|
| :x: | `query_context_tests.py` | 2F, 13P, 61W | [tox38_query_context_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_query_context_tests.log)|
| :heavy_check_mark: | `result_set_tests.py` | 15P, 7W | [tox38_result_set_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_result_set_tests.log)|
| :warning: | `schedules_test.py` | 5P, 7W, 8E | [tox38_schedules_test.log](https://chat.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_schedules_test.log) |
| :x: | `security_tests.py` | 24F, 13P, 7W, 4E | [tox38_security_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_security_tests.log)|
| :heavy_check_mark: | `sqla_models_tests.py` | 9P, 7W | [tox38_sqla_models_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_sqla_models_tests.log)|
| :heavy_check_mark: | `sqllab_tests.py` | 30P, 8W | [tox38_sqllab_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_sqllab_tests.log)|
| :heavy_check_mark: | `sql_parse_tests.py` | 42P, 6W | [tox38_sql_parse_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_sql_parse_tests.log)|
| :heavy_check_mark: | `sql_validator_tests.py` | 11P, 7W | [tox38_sql_validator_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_sql_validator_tests.log)|
| :heavy_check_mark: | `stats_logger_tests.py` | 2P, 6W | [tox38_stats_logger_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_stats_logger_tests.log)|
| :heavy_check_mark: | `strategy_tests.py` | 9P, 6W | [tox38_stats_logger_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_stats_logger_tests.log)|
| :gear: | `superset_test_config.py` | | |
| :gear: | `superset_test_config_sqllab_backend_persist.py` | | |
| :gear: | `superset_test_config_thumbnails.py` | | |
| :gear: | `superset_test_custom_template_processors.py` | | |
| :heavy_check_mark: | `tagging_tests.py` | 1P, 1S, 7W | [tox38_tagging_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_tagging_tests.log)|
| :gear: | `test_app.py` | | |
| :x: | `thumbnails_tests.py` | 2F, 10S, 7W | [tox38_thumbnails_tests.log](https://chat.github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_thumbnails_tests.log)|
| :warning: | `utils_tests.py` | 51P, 11W, 1E | [tox38_thumbnails_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_thumbnails_tests.log)|
| :heavy_check_mark: | `viz_tests.py` | 55P, 20W | [tox38_viz_tests.log](https://github.com/EBoisseauSierra/superset-tox38-logs/blob/master/tox38_viz_tests.log)|
Where:
* :x: tests failed (possibly with errors as well),
* :warning: : tests didn't failed, yet had errors,
* :heavy_check_mark: : tests ran fine (possibly with warnings),
* :gear: : not a test file.
I still need to investigate the faulty ones more closely to understand why they fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] EBoisseauSierra edited a comment on issue #14052: Unable to run tests successfully as per documentation
Posted by GitBox <gi...@apache.org>.
EBoisseauSierra edited a comment on issue #14052:
URL: https://github.com/apache/superset/issues/14052#issuecomment-828994879
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] EBoisseauSierra commented on issue #14052: Unable to run tests successfully as per documentation
Posted by GitBox <gi...@apache.org>.
EBoisseauSierra commented on issue #14052:
URL: https://github.com/apache/superset/issues/14052#issuecomment-833451122
I hadn't seen #13970 , what is a great initiative. I suppose that helps too!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org