You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2021/06/27 08:30:29 UTC
[GitHub] [superset] ofekisr opened a new pull request #15405: refactor load configuration
ofekisr opened a new pull request #15405:
URL: https://github.com/apache/superset/pull/15405
### SUMMARY
<!--- Describe the change below, including rationale and design decisions -->
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
<!--- Skip this if not applicable -->
### TESTING INSTRUCTIONS
<!--- Required! What steps can be taken to manually verify the changes? -->
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-870077561
@ofekisr currently my `superset_config.py` has this line:
```python
RESULTS_BACKEND = RedisCache(host="localhost", port=6379, key_prefix="superset_results")
```
And it failes to run because `pydash` tries to make a deepcopy of it:
```
Traceback (most recent call last):
File "/Users/beto/.pyenv/versions/superset/bin/superset", line 33, in <module>
sys.exit(load_entry_point('apache-superset', 'console_scripts', 'superset')())
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/core.py", line 1256, in invoke
Command.invoke(self, ctx)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/flask/cli.py", line 425, in decorator
with __ctx.ensure_object(ScriptInfo).load_app().app_context():
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/flask/cli.py", line 381, in load_app
app = call_factory(self, self.create_app)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/flask/cli.py", line 119, in call_factory
return app_factory()
File "/Users/beto/Projects/incubator-superset/superset/app.py", line 52, in create_app
raise ex
File "/Users/beto/Projects/incubator-superset/superset/app.py", line 41, in create_app
config = init_config()
File "/Users/beto/Projects/incubator-superset/superset/app.py", line 62, in init_config
return merge(config, override_conf)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/pydash-5.0.1-py3.8.egg/pydash/objects.py", line 859, in merge
return merge_with(obj, *sources)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/pydash-5.0.1-py3.8.egg/pydash/objects.py", line 907, in merge_with
sources = [copy.deepcopy(source) for source in sources]
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/pydash-5.0.1-py3.8.egg/pydash/objects.py", line 907, in <listcomp>
sources = [copy.deepcopy(source) for source in sources]
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 161, in deepcopy
rv = reductor(4)
TypeError: cannot pickle '_thread.lock' object
Exception ignored in: <function Redis.__del__ at 0x1109a7790>
Traceback (most recent call last):
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/redis/client.py", line 885, in __del__
self.close()
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/redis/client.py", line 888, in close
conn = self.connection
AttributeError: 'Redis' object has no attribute 'connection'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on a change in pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on a change in pull request #15405:
URL: https://github.com/apache/superset/pull/15405#discussion_r660151549
##########
File path: superset/app.py
##########
@@ -44,5 +52,51 @@ def create_app() -> Flask:
raise ex
+def init_config() -> Dict[Any, Any]:
+ config = convert_to_dict(load_default_config())
+ override_conf = convert_to_dict(load_override_config())
+ return merge(config, override_conf)
Review comment:
I think bringing a new dependency (pydash) just to merge two dicts here is overkill. Why not just use `dict.update`?
I'm also not sure if a recursive merge is the right strategy here. If we have:
```python
# superset/config.py
a = {"b": "c"}
```
And I create a custom config:
```python
# superset_config.py
a = {"d": "e"}
```
This would result in the following configuration:
```python
{'a': {'b': 'c', 'd': 'e'}}
```
Which is probably not what we want.
As a concrete example, we currently have languages configured in `superset/config.py`:
```python
LANGUAGES = {
"en": {"flag": "us", "name": "English"},
"es": {"flag": "es", "name": "Spanish"},
"it": {"flag": "it", "name": "Italian"},
"fr": {"flag": "fr", "name": "French"},
"zh": {"flag": "cn", "name": "Chinese"},
"ja": {"flag": "jp", "name": "Japanese"},
"de": {"flag": "de", "name": "German"},
"pt": {"flag": "pt", "name": "Portuguese"},
"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
"ru": {"flag": "ru", "name": "Russian"},
"ko": {"flag": "kr", "name": "Korean"},
"sl": {"flag": "si", "name": "Slovenian"},
}
```
If I want to have just Portuguese in my Superset instance I would add to my config:
```python
LANGUAGES = [
"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
]
```
But the `merge` would bring back all languages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] john-bodley edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
john-bodley edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-875075296
@ofekisr @amitmiran137 this updated logic breaks existing config functionality, specifically where callable configuration references other configuration variables:
- [ALLOWED_USER_CSV_SCHEMA_FUNC](https://github.com/apache/superset/blob/d4480f5c9aae474942ff35dc9932673942f81c22/superset/config.py#L828-L832)
- [CSV_TO_HIVE_UPLOAD_DIRECTORY_FUNC](https://github.com/apache/superset/blob/d4480f5c9aae474942ff35dc9932673942f81c22/superset/config.py#L808-L816)
i.e., locally overriding the `CSV_TO_HIVE_UPLOAD_DIRECTORY ` has no effect.
Note I'm supportive of the general direction of this PR, especially given the previous logic is atypical from a Flask perspective, however the regression needs to be addressed by either i) amending the logic or ii) reverting this PR (alongside @betodealmeida's and @villebro's PRs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-870077561
@ofekisr currently my `superset_config.py` has this line:
```python
RESULTS_BACKEND = RedisCache(host="localhost", port=6379, key_prefix="superset_results")
```
And it failes to run because `pydash` tries to make a deepcopy of it:
```
Traceback (most recent call last):
File "/Users/beto/.pyenv/versions/superset/bin/superset", line 33, in <module>
sys.exit(load_entry_point('apache-superset', 'console_scripts', 'superset')())
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/core.py", line 1256, in invoke
Command.invoke(self, ctx)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/flask/cli.py", line 425, in decorator
with __ctx.ensure_object(ScriptInfo).load_app().app_context():
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/flask/cli.py", line 381, in load_app
app = call_factory(self, self.create_app)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/flask/cli.py", line 119, in call_factory
return app_factory()
File "/Users/beto/Projects/incubator-superset/superset/app.py", line 52, in create_app
raise ex
File "/Users/beto/Projects/incubator-superset/superset/app.py", line 41, in create_app
config = init_config()
File "/Users/beto/Projects/incubator-superset/superset/app.py", line 62, in init_config
return merge(config, override_conf)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/pydash-5.0.1-py3.8.egg/pydash/objects.py", line 859, in merge
return merge_with(obj, *sources)
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/pydash-5.0.1-py3.8.egg/pydash/objects.py", line 907, in merge_with
sources = [copy.deepcopy(source) for source in sources]
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/pydash-5.0.1-py3.8.egg/pydash/objects.py", line 907, in <listcomp>
sources = [copy.deepcopy(source) for source in sources]
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/Users/beto/.pyenv/versions/3.8.8/lib/python3.8/copy.py", line 161, in deepcopy
rv = reductor(4)
TypeError: cannot pickle '_thread.lock' object
Exception ignored in: <function Redis.__del__ at 0x1109a7790>
Traceback (most recent call last):
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/redis/client.py", line 885, in __del__
self.close()
File "/Users/beto/.pyenv/versions/3.8.8/envs/superset/lib/python3.8/site-packages/redis/client.py", line 888, in close
conn = self.connection
AttributeError: 'Redis' object has no attribute 'connection'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] john-bodley edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
john-bodley edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-875059621
@ofekisr @amitmiran137 @betodealmeida @villebro I'm all for standardizing the configuration import logic but why don't we simply use the [recommended Flask pattern](https://flask.palletsprojects.com/en/2.0.x/config/#configuring-from-python-files) , i.e.,
```python
app = Flask(__name__)
app.config.from_object("superset.config")
app.config.from_envvar("SUPERSET_CONFIG_PATH", silent=True)
```
instead of writing our own import/merge logic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-869124850
# [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15405](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94ee62b) into [master](https://codecov.io/gh/apache/superset/commit/09c44d05fd57db48c4af02500c745d2900fa463b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (09c44d0) will **increase** coverage by `0.01%`.
> The diff coverage is `58.33%`.
> :exclamation: Current head 94ee62b differs from pull request most recent head 5cedd56. Consider uploading reports for the commit 5cedd56 to get more accurate results
[](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15405 +/- ##
==========================================
+ Coverage 77.23% 77.24% +0.01%
==========================================
Files 975 975
Lines 50610 50623 +13
Branches 6212 6212
==========================================
+ Hits 39090 39106 +16
+ Misses 11313 11310 -3
Partials 207 207
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `81.46% <58.33%> (+0.02%)` | :arrow_up: |
| mysql | `81.73% <58.33%> (+0.02%)` | :arrow_up: |
| postgres | `81.75% <58.33%> (+0.02%)` | :arrow_up: |
| presto | `81.45% <58.33%> (+0.02%)` | :arrow_up: |
| python | `82.27% <58.33%> (+0.02%)` | :arrow_up: |
| sqlite | `81.37% <58.33%> (+0.02%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `97.06% <ø> (+5.91%)` | :arrow_up: |
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `65.38% <58.33%> (-17.95%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [09c44d0...5cedd56](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-869124850
# [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15405](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94ee62b) into [master](https://codecov.io/gh/apache/superset/commit/09c44d05fd57db48c4af02500c745d2900fa463b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (09c44d0) will **decrease** coverage by `0.14%`.
> The diff coverage is `58.33%`.
[](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15405 +/- ##
==========================================
- Coverage 77.16% 77.02% -0.15%
==========================================
Files 975 975
Lines 50610 50623 +13
Branches 6212 6212
==========================================
- Hits 39051 38990 -61
- Misses 11352 11426 +74
Partials 207 207
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `81.73% <58.33%> (+0.02%)` | :arrow_up: |
| postgres | `81.75% <58.33%> (+0.02%)` | :arrow_up: |
| python | `81.83% <58.33%> (-0.28%)` | :arrow_down: |
| sqlite | `81.37% <58.33%> (+0.02%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `97.06% <ø> (+5.91%)` | :arrow_up: |
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `65.38% <58.33%> (-17.95%)` | :arrow_down: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [superset/db\_engine\_specs/hive.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2hpdmUucHk=) | `69.20% <0.00%> (-17.21%)` | :arrow_down: |
| [superset/views/database/mixins.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvbWl4aW5zLnB5) | `81.03% <0.00%> (-1.73%)` | :arrow_down: |
| [superset/db\_engine\_specs/presto.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL3ByZXN0by5weQ==) | `83.36% <0.00%> (-1.06%)` | :arrow_down: |
| [superset/db\_engine\_specs/base.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2Jhc2UucHk=) | `87.97% <0.00%> (-0.41%)` | :arrow_down: |
| [superset/connectors/sqla/models.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29ubmVjdG9ycy9zcWxhL21vZGVscy5weQ==) | `88.22% <0.00%> (-0.24%)` | :arrow_down: |
| [superset/utils/core.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvY29yZS5weQ==) | `88.97% <0.00%> (-0.13%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [09c44d0...94ee62b](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] john-bodley edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
john-bodley edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-875075296
@ofekisr @amitmiran137 this updated logic breaks existing config functionality, specifically where callable configuration references other configuration variables:
- [ALLOWED_USER_CSV_SCHEMA_FUNC](https://github.com/apache/superset/blob/d4480f5c9aae474942ff35dc9932673942f81c22/superset/config.py#L828-L832)
- [CSV_TO_HIVE_UPLOAD_DIRECTORY_FUNC](https://github.com/apache/superset/blob/d4480f5c9aae474942ff35dc9932673942f81c22/superset/config.py#L808-L816)
i.e., locally overriding the `CSV_TO_HIVE_UPLOAD_DIRECTORY ` has no effect.
Note I'm supportive of the general direction of this PR, especially given the previous logic is atypical from a Flask perspective, however the regression needs to be addressed by either i) amending the logic or ii) reverting this PR (alongside @betodealmeida's and @villebro's fixes).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-869124850
# [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15405](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94ee62b) into [master](https://codecov.io/gh/apache/superset/commit/a3d91aee28a5e1bffc79ff27de6feaa2edb611fc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a3d91ae) will **increase** coverage by `0.01%`.
> The diff coverage is `58.33%`.
> :exclamation: Current head 94ee62b differs from pull request most recent head 603bd29. Consider uploading reports for the commit 603bd29 to get more accurate results
[](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15405 +/- ##
==========================================
+ Coverage 77.23% 77.24% +0.01%
==========================================
Files 975 975
Lines 50615 50623 +8
Branches 6212 6212
==========================================
+ Hits 39094 39106 +12
+ Misses 11314 11310 -4
Partials 207 207
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `81.46% <58.33%> (+0.02%)` | :arrow_up: |
| mysql | `81.73% <58.33%> (+0.02%)` | :arrow_up: |
| postgres | `81.75% <58.33%> (+0.02%)` | :arrow_up: |
| presto | `81.45% <58.33%> (+0.02%)` | :arrow_up: |
| python | `82.27% <58.33%> (+0.02%)` | :arrow_up: |
| sqlite | `81.37% <58.33%> (+0.02%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `97.06% <ø> (+5.91%)` | :arrow_up: |
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `65.38% <58.33%> (-17.95%)` | :arrow_down: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.05% <0.00%> (+0.14%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a3d91ae...603bd29](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] commented on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-869124850
# [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15405](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94ee62b) into [master](https://codecov.io/gh/apache/superset/commit/09c44d05fd57db48c4af02500c745d2900fa463b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (09c44d0) will **decrease** coverage by `0.15%`.
> The diff coverage is `58.33%`.
[](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15405 +/- ##
==========================================
- Coverage 77.16% 77.00% -0.16%
==========================================
Files 975 975
Lines 50610 50594 -16
Branches 6212 6212
==========================================
- Hits 39051 38962 -89
- Misses 11352 11425 +73
Partials 207 207
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `81.73% <58.33%> (+0.02%)` | :arrow_up: |
| postgres | `81.73% <58.33%> (+<0.01%)` | :arrow_up: |
| python | `81.82% <58.33%> (-0.29%)` | :arrow_down: |
| sqlite | `81.37% <58.33%> (+0.02%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `97.06% <ø> (+5.91%)` | :arrow_up: |
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `65.38% <58.33%> (-17.95%)` | :arrow_down: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [superset/db\_engine\_specs/hive.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2hpdmUucHk=) | `69.20% <0.00%> (-17.21%)` | :arrow_down: |
| [superset/models/dashboard.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvbW9kZWxzL2Rhc2hib2FyZC5weQ==) | `73.42% <0.00%> (-2.45%)` | :arrow_down: |
| [superset/views/database/mixins.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvbWl4aW5zLnB5) | `81.03% <0.00%> (-1.73%)` | :arrow_down: |
| [superset/db\_engine\_specs/presto.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL3ByZXN0by5weQ==) | `83.36% <0.00%> (-1.06%)` | :arrow_down: |
| [superset/db\_engine\_specs/base.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2Jhc2UucHk=) | `87.97% <0.00%> (-0.41%)` | :arrow_down: |
| [superset/connectors/sqla/models.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29ubmVjdG9ycy9zcWxhL21vZGVscy5weQ==) | `88.20% <0.00%> (-0.25%)` | :arrow_down: |
| [superset/reports/notifications/base.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvcmVwb3J0cy9ub3RpZmljYXRpb25zL2Jhc2UucHk=) | `95.23% <0.00%> (-0.22%)` | :arrow_down: |
| ... and [7 more](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [09c44d0...94ee62b](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on a change in pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on a change in pull request #15405:
URL: https://github.com/apache/superset/pull/15405#discussion_r660151549
##########
File path: superset/app.py
##########
@@ -44,5 +52,51 @@ def create_app() -> Flask:
raise ex
+def init_config() -> Dict[Any, Any]:
+ config = convert_to_dict(load_default_config())
+ override_conf = convert_to_dict(load_override_config())
+ return merge(config, override_conf)
Review comment:
I think bringing a new dependency (pydash) just to merge two dicts here is overkill. Why not just use `dict.update`?
I'm also not sure if a recursive merge is the right strategy here. If we have:
```python
# superset/config.py
a = {"b": "c"}
```
And I create a custom config:
```python
# superset_config.py
a = {"d": "e"}
```
This would result in the following configuration:
```python
{'a': {'b': 'c', 'd': 'e'}}
```
Which is probably not what we want.
As a concrete example, we currently have languages configured in `superset/config.py`:
```python
LANGUAGES = {
"en": {"flag": "us", "name": "English"},
"es": {"flag": "es", "name": "Spanish"},
"it": {"flag": "it", "name": "Italian"},
"fr": {"flag": "fr", "name": "French"},
"zh": {"flag": "cn", "name": "Chinese"},
"ja": {"flag": "jp", "name": "Japanese"},
"de": {"flag": "de", "name": "German"},
"pt": {"flag": "pt", "name": "Portuguese"},
"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
"ru": {"flag": "ru", "name": "Russian"},
"ko": {"flag": "kr", "name": "Korean"},
"sl": {"flag": "si", "name": "Slovenian"},
}
```
If I want to have just Portuguese in my Superset instance I would add to my config:
```python
LANGUAGES = {
"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
}
```
But the `merge` would bring back all languages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-869124850
# [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15405](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94ee62b) into [master](https://codecov.io/gh/apache/superset/commit/09c44d05fd57db48c4af02500c745d2900fa463b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (09c44d0) will **increase** coverage by `0.01%`.
> The diff coverage is `58.33%`.
[](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15405 +/- ##
==========================================
+ Coverage 77.16% 77.17% +0.01%
==========================================
Files 975 975
Lines 50610 50623 +13
Branches 6212 6212
==========================================
+ Hits 39051 39067 +16
+ Misses 11352 11349 -3
Partials 207 207
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `81.46% <58.33%> (+0.02%)` | :arrow_up: |
| mysql | `81.73% <58.33%> (+0.02%)` | :arrow_up: |
| postgres | `81.75% <58.33%> (+0.02%)` | :arrow_up: |
| python | `82.13% <58.33%> (+0.02%)` | :arrow_up: |
| sqlite | `81.37% <58.33%> (+0.02%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `97.06% <ø> (+5.91%)` | :arrow_up: |
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `65.38% <58.33%> (-17.95%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [09c44d0...94ee62b](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-869124850
# [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15405](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (57f80e3) into [master](https://codecov.io/gh/apache/superset/commit/a3d91aee28a5e1bffc79ff27de6feaa2edb611fc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a3d91ae) will **decrease** coverage by `0.21%`.
> The diff coverage is `64.28%`.
[](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15405 +/- ##
==========================================
- Coverage 77.23% 77.02% -0.22%
==========================================
Files 975 975
Lines 50615 50630 +15
Branches 6212 6212
==========================================
- Hits 39094 38996 -98
- Misses 11314 11427 +113
Partials 207 207
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `81.73% <64.28%> (+0.02%)` | :arrow_up: |
| postgres | `81.75% <64.28%> (+0.02%)` | :arrow_up: |
| presto | `?` | |
| python | `81.83% <64.28%> (-0.42%)` | :arrow_down: |
| sqlite | `81.37% <64.28%> (+0.02%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `68.42% <63.41%> (-14.92%)` | :arrow_down: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `97.03% <100.00%> (+5.88%)` | :arrow_up: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [superset/db\_engine\_specs/hive.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2hpdmUucHk=) | `69.20% <0.00%> (-17.21%)` | :arrow_down: |
| [superset/db\_engine\_specs/presto.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL3ByZXN0by5weQ==) | `83.36% <0.00%> (-6.53%)` | :arrow_down: |
| [superset/views/database/mixins.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvbWl4aW5zLnB5) | `81.03% <0.00%> (-1.73%)` | :arrow_down: |
| [superset/connectors/sqla/models.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29ubmVjdG9ycy9zcWxhL21vZGVscy5weQ==) | `88.22% <0.00%> (-1.65%)` | :arrow_down: |
| [superset/db\_engine\_specs/base.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2Jhc2UucHk=) | `87.97% <0.00%> (-0.41%)` | :arrow_down: |
| [superset/models/core.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvbW9kZWxzL2NvcmUucHk=) | `89.76% <0.00%> (-0.27%)` | :arrow_down: |
| [superset/utils/core.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvY29yZS5weQ==) | `88.97% <0.00%> (-0.13%)` | :arrow_down: |
| ... and [1 more](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a3d91ae...57f80e3](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] john-bodley edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
john-bodley edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-875059621
@ofekisr @amitmiran137 @betodealmeida @villebro I'm all for standardizing the configuration import logic but why don't we simply use the [recommended Flask pattern](https://flask.palletsprojects.com/en/2.0.x/config/#configuring-from-python-files) , i.e.,
```python
app = Flask(__name__)
app.config.from_object("yourapplication.default_settings")
app.config.from_envvar("YOURAPPLICATION_SETTINGS", silent=True)
```
instead of writing our own import/merge logic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] john-bodley commented on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
john-bodley commented on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-875059621
@ofekisr @amitmiran137 @betodealmeida @villebro I'm all for standardizing the configuration import logic but why don't we simply use the [recommended Flask pattern](https://flask.palletsprojects.com/en/2.0.x/config/#configuring-from-python-files) rather than writing our own import/merge logic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-869124850
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-869124850
# [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15405](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94ee62b) into [master](https://codecov.io/gh/apache/superset/commit/a3d91aee28a5e1bffc79ff27de6feaa2edb611fc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a3d91ae) will **increase** coverage by `0.01%`.
> The diff coverage is `58.33%`.
> :exclamation: Current head 94ee62b differs from pull request most recent head e9ba24b. Consider uploading reports for the commit e9ba24b to get more accurate results
[](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15405 +/- ##
==========================================
+ Coverage 77.23% 77.24% +0.01%
==========================================
Files 975 975
Lines 50615 50623 +8
Branches 6212 6212
==========================================
+ Hits 39094 39106 +12
+ Misses 11314 11310 -4
Partials 207 207
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `81.46% <58.33%> (+0.02%)` | :arrow_up: |
| mysql | `81.73% <58.33%> (+0.02%)` | :arrow_up: |
| postgres | `81.75% <58.33%> (+0.02%)` | :arrow_up: |
| presto | `81.45% <58.33%> (+0.02%)` | :arrow_up: |
| python | `82.27% <58.33%> (+0.02%)` | :arrow_up: |
| sqlite | `81.37% <58.33%> (+0.02%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `97.06% <ø> (+5.91%)` | :arrow_up: |
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `65.38% <58.33%> (-17.95%)` | :arrow_down: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.05% <0.00%> (+0.14%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a3d91ae...e9ba24b](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] amitmiran137 merged pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
amitmiran137 merged pull request #15405:
URL: https://github.com/apache/superset/pull/15405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-875093649
@john-bodley I have PR reverting the changes: https://github.com/apache/superset/pull/15558
I think we need to evaluate this better, since we already tried to fix it twice and it's still breaking our deployments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-869124850
# [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15405](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94ee62b) into [master](https://codecov.io/gh/apache/superset/commit/09c44d05fd57db48c4af02500c745d2900fa463b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (09c44d0) will **increase** coverage by `0.01%`.
> The diff coverage is `58.33%`.
[](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15405 +/- ##
==========================================
+ Coverage 77.23% 77.24% +0.01%
==========================================
Files 975 975
Lines 50610 50623 +13
Branches 6212 6212
==========================================
+ Hits 39090 39106 +16
+ Misses 11313 11310 -3
Partials 207 207
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `81.46% <58.33%> (+0.02%)` | :arrow_up: |
| mysql | `81.73% <58.33%> (+0.02%)` | :arrow_up: |
| postgres | `81.75% <58.33%> (+0.02%)` | :arrow_up: |
| presto | `81.45% <58.33%> (+0.02%)` | :arrow_up: |
| python | `82.27% <58.33%> (+0.02%)` | :arrow_up: |
| sqlite | `81.37% <58.33%> (+0.02%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `97.06% <ø> (+5.91%)` | :arrow_up: |
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `65.38% <58.33%> (-17.95%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [09c44d0...94ee62b](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] john-bodley edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
john-bodley edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-875059621
@ofekisr @amitmiran137 @betodealmeida @villebro I'm all for standardizing the configuration import logic but why don't we simply use the [recommended Flask pattern](https://flask.palletsprojects.com/en/2.0.x/config/#configuring-from-python-files) instead of writing our own import/merge logic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-869124850
# [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15405](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94ee62b) into [master](https://codecov.io/gh/apache/superset/commit/a3d91aee28a5e1bffc79ff27de6feaa2edb611fc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a3d91ae) will **increase** coverage by `0.01%`.
> The diff coverage is `58.33%`.
> :exclamation: Current head 94ee62b differs from pull request most recent head 3079f0e. Consider uploading reports for the commit 3079f0e to get more accurate results
[](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15405 +/- ##
==========================================
+ Coverage 77.23% 77.24% +0.01%
==========================================
Files 975 975
Lines 50615 50623 +8
Branches 6212 6212
==========================================
+ Hits 39094 39106 +12
+ Misses 11314 11310 -4
Partials 207 207
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `81.46% <58.33%> (+0.02%)` | :arrow_up: |
| mysql | `81.73% <58.33%> (+0.02%)` | :arrow_up: |
| postgres | `81.75% <58.33%> (+0.02%)` | :arrow_up: |
| presto | `81.45% <58.33%> (+0.02%)` | :arrow_up: |
| python | `82.27% <58.33%> (+0.02%)` | :arrow_up: |
| sqlite | `81.37% <58.33%> (+0.02%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `97.06% <ø> (+5.91%)` | :arrow_up: |
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `65.38% <58.33%> (-17.95%)` | :arrow_down: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/15405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.05% <0.00%> (+0.14%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a3d91ae...3079f0e](https://codecov.io/gh/apache/superset/pull/15405?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] john-bodley commented on pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
john-bodley commented on pull request #15405:
URL: https://github.com/apache/superset/pull/15405#issuecomment-875075296
@ofekisr @amitmiran137 this updated logic breaks existing config functionality, specifically where callable configuration references other configuration variables:
- [ALLOWED_USER_CSV_SCHEMA_FUNC](https://github.com/apache/superset/blob/d4480f5c9aae474942ff35dc9932673942f81c22/superset/config.py#L828-L832)
- [CSV_TO_HIVE_UPLOAD_DIRECTORY_FUNC](https://github.com/apache/superset/blob/d4480f5c9aae474942ff35dc9932673942f81c22/superset/config.py#L808-L816)
i.e., locally overriding the `CSV_TO_HIVE_UPLOAD_DIRECTORY ` has no effect.
Note I'm supportive of the general direction of this PR, especially given the previous logic is atypical from a Flask perspective, however the regression needs to be addressed by either i) amending the logic or ii) reverting this (and @betodealmeida's and @villebro's PRs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on a change in pull request #15405: chore(refactor): load configuration and merge recursively
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on a change in pull request #15405:
URL: https://github.com/apache/superset/pull/15405#discussion_r660151549
##########
File path: superset/app.py
##########
@@ -44,5 +52,51 @@ def create_app() -> Flask:
raise ex
+def init_config() -> Dict[Any, Any]:
+ config = convert_to_dict(load_default_config())
+ override_conf = convert_to_dict(load_override_config())
+ return merge(config, override_conf)
Review comment:
I think bringing a new dependency (pydash) just to merge two dicts here is overkill. Why not just use `dict.update`?
I'm also not sure if a recursive merge is the right strategy here. If we have:
```python
# superset/config.py
a = {"b": "c"}
```
And I create a custom config:
```python
# superset_config.py
a = {"d": "e"}
```
This would result in the following configuration:
```python
{'a': {'b': 'c', 'd': 'e'}}
```
Which is probably not what we want.
As a concrete example, we currently have languages configured in `superset/config.py`:
```python
LANGUAGES = {
"en": {"flag": "us", "name": "English"},
"es": {"flag": "es", "name": "Spanish"},
"it": {"flag": "it", "name": "Italian"},
"fr": {"flag": "fr", "name": "French"},
"zh": {"flag": "cn", "name": "Chinese"},
"ja": {"flag": "jp", "name": "Japanese"},
"de": {"flag": "de", "name": "German"},
"pt": {"flag": "pt", "name": "Portuguese"},
"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
"ru": {"flag": "ru", "name": "Russian"},
"ko": {"flag": "kr", "name": "Korean"},
"sl": {"flag": "si", "name": "Slovenian"},
}
```
If I want to have just Portuguese in my Superset instance I would add to my config:
```python
LANGUAGES = [
"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
]
```
But the `merge` would bring back all languages.
##########
File path: superset/app.py
##########
@@ -44,5 +52,51 @@ def create_app() -> Flask:
raise ex
+def init_config() -> Dict[Any, Any]:
+ config = convert_to_dict(load_default_config())
+ override_conf = convert_to_dict(load_override_config())
+ return merge(config, override_conf)
Review comment:
I think bringing a new dependency (pydash) just to merge two dicts here is overkill. Why not just use `dict.update`?
I'm also not sure if a recursive merge is the right strategy here. If we have:
```python
# superset/config.py
a = {"b": "c"}
```
And I create a custom config:
```python
# superset_config.py
a = {"d": "e"}
```
This would result in the following configuration:
```python
{'a': {'b': 'c', 'd': 'e'}}
```
Which is probably not what we want.
As a concrete example, we currently have languages configured in `superset/config.py`:
```python
LANGUAGES = {
"en": {"flag": "us", "name": "English"},
"es": {"flag": "es", "name": "Spanish"},
"it": {"flag": "it", "name": "Italian"},
"fr": {"flag": "fr", "name": "French"},
"zh": {"flag": "cn", "name": "Chinese"},
"ja": {"flag": "jp", "name": "Japanese"},
"de": {"flag": "de", "name": "German"},
"pt": {"flag": "pt", "name": "Portuguese"},
"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
"ru": {"flag": "ru", "name": "Russian"},
"ko": {"flag": "kr", "name": "Korean"},
"sl": {"flag": "si", "name": "Slovenian"},
}
```
If I want to have just Portuguese in my Superset instance I would add to my config:
```python
LANGUAGES = {
"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
}
```
But the `merge` would bring back all languages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org