You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2020/01/24 02:21:07 UTC
[GitHub] [incubator-superset] etr2460 opened a new pull request #9012: [SQL
Lab] Add function names to autocomplete
etr2460 opened a new pull request #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012
### CATEGORY
Choose one
- [ ] Bug Fix
- [x] Enhancement (new features, refinement)
- [ ] Refactor
- [ ] Add tests
- [ ] Build / Development Environment
- [ ] Documentation
### SUMMARY
Adds a new function to `db_engine_specs` for getting function names available in a database. Also implements this function for Presto and Hive.
Then passes this result through the database API and adds it to the SQL Lab editor autocomplete
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Hive:
<img width="326" alt="Screen Shot 2020-01-23 at 6 17 31 PM" src="https://user-images.githubusercontent.com/7409244/73040036-fccc5280-3e0c-11ea-8767-3cb7517d664a.png">
Presto:
<img width="353" alt="Screen Shot 2020-01-23 at 6 17 45 PM" src="https://user-images.githubusercontent.com/7409244/73040037-fccc5280-3e0c-11ea-9907-183ac92219f6.png">
### TEST PLAN
Test with Hive and Presto and see function autocompletes. Test with mysql and ensure that autocompletes still work
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Changes UI
- [ ] Requires DB Migration.
- [ ] Confirm DB Migration upgrade and downgrade tested.
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
### REVIEWERS
to: @dpgaspar @villebro @john-bodley @graceguo-supercat @betodealmeida
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] etr2460 merged pull request #9012: [SQL Lab]
Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
etr2460 merged pull request #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] etr2460 commented on issue #9012: [SQL Lab]
Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
etr2460 commented on issue #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012#issuecomment-578306032
Sounds good! Feel free to tag me on your PR and i'll review
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] villebro commented on issue #9012: [SQL Lab]
Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
villebro commented on issue #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012#issuecomment-578303467
No panic, I'm pretty excited about getting these in for Snowflake, so can do the necessary changes in a separate PR 👍
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] villebro commented on issue #9012: [SQL Lab]
Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
villebro commented on issue #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012#issuecomment-578306764
Thanks, will do!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] villebro commented on issue #9012: [SQL Lab]
Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
villebro commented on issue #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012#issuecomment-578265785
I seem to have just missed the train.. LGTM, but something worth considering: some dbs support defining UDFs per account/database/schema (e.g. `SHOW USER FUNCTIONS IN SCHEMA 'my_schema'`). While fetching global functions is most likely 99 % of use cases, adding the schema parameter to `get_function_names` could be used to add UDFs that are available in the chosen schema for the current user.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] etr2460 commented on a change in pull request
#9012: [SQL Lab] Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
etr2460 commented on a change in pull request #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012#discussion_r370449801
##########
File path: superset/models/core.py
##########
@@ -320,7 +324,7 @@ def get_quoter(self):
return self.get_dialect().identifier_preparer.quote
def get_df( # pylint: disable=too-many-locals
- self, sql: str, schema: str, mutator: Optional[Callable] = None
+ self, sql: str, schema: Optional[str] = None, mutator: Optional[Callable] = None
Review comment:
`self.get_sla_engine` (which is called on line 336) sets schema to None by default so I moved this up a level to make my db query easier (since it's not schema dependent)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] etr2460 commented on issue #9012: [SQL Lab]
Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
etr2460 commented on issue #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012#issuecomment-578302607
eek, sorry about that @villebro! I didn't realize some dbs supported UDFs per schema. Do you think it's worth adding that functionality now, given that i've only implemented hive and presto, or should we wait for someone more familiar with Snowflake or other dbs to update the function signature for what they require?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] etr2460 commented on a change in pull request
#9012: [SQL Lab] Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
etr2460 commented on a change in pull request #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012#discussion_r370449801
##########
File path: superset/models/core.py
##########
@@ -320,7 +324,7 @@ def get_quoter(self):
return self.get_dialect().identifier_preparer.quote
def get_df( # pylint: disable=too-many-locals
- self, sql: str, schema: str, mutator: Optional[Callable] = None
+ self, sql: str, schema: Optional[str] = None, mutator: Optional[Callable] = None
Review comment:
`self.get_sla_engine` sets schema to None by default so I moved this up a level to make my db query easier (since it's not schema dependent)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] villebro edited a comment on issue #9012: [SQL
Lab] Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
villebro edited a comment on issue #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012#issuecomment-578265785
I seem to have just missed the train.. LGTM, but something worth considering: some dbs support defining UDFs per account/database/schema (e.g. Snowflake: `SHOW USER FUNCTIONS IN SCHEMA 'my_schema'`). While fetching global functions is most likely 99 % of use cases, adding the schema parameter to `get_function_names` could be used to add UDFs that are available in the chosen schema for the current user.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull
request #9012: [SQL Lab] Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
john-bodley commented on a change in pull request #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012#discussion_r370451868
##########
File path: superset/db_engine_specs/hive.py
##########
@@ -422,3 +422,15 @@ def execute( # type: ignore
): # pylint: disable=arguments-differ
kwargs = {"async": async_}
cursor.execute(query, **kwargs)
+
+ @classmethod
+ def get_function_names(cls, database: "Database") -> List[str]:
+ """
+ Get a list of function names that are able to be called on the database.
+ Used for SQL Lab autocomplete.
+
+ :param database: The database to get functions for
+ :return: A list of function names useable in the database
+ """
+ df = database.get_df("SHOW FUNCTIONS")
Review comment:
Can we remove the interim `df` variable, i.e.,
```
return database.get_df("SHOW FUNCTIONS")["tab_name"].tolist()
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] codecov-io commented on issue #9012: [SQL Lab]
Add function names to autocomplete
Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #9012: [SQL Lab] Add function names to autocomplete
URL: https://github.com/apache/incubator-superset/pull/9012#issuecomment-578225853
# [Codecov](https://codecov.io/gh/apache/incubator-superset/pull/9012?src=pr&el=h1) Report
> Merging [#9012](https://codecov.io/gh/apache/incubator-superset/pull/9012?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-superset/commit/d0bc04fc93f0f96b603b0c4f6304cab9a802d0c1?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `60%`.
[](https://codecov.io/gh/apache/incubator-superset/pull/9012?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #9012 +/- ##
==========================================
+ Coverage 59.15% 59.15% +<.01%
==========================================
Files 367 367
Lines 11682 11686 +4
Branches 2864 2866 +2
==========================================
+ Hits 6910 6913 +3
- Misses 4593 4594 +1
Partials 179 179
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-superset/pull/9012?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...uperset/assets/src/SqlLab/components/SqlEditor.jsx](https://codecov.io/gh/apache/incubator-superset/pull/9012/diff?src=pr&el=tree#diff-c3VwZXJzZXQvYXNzZXRzL3NyYy9TcWxMYWIvY29tcG9uZW50cy9TcWxFZGl0b3IuanN4) | `51.94% <100%> (+0.31%)` | :arrow_up: |
| [superset/assets/src/SqlLab/constants.js](https://codecov.io/gh/apache/incubator-superset/pull/9012/diff?src=pr&el=tree#diff-c3VwZXJzZXQvYXNzZXRzL3NyYy9TcWxMYWIvY29uc3RhbnRzLmpz) | `100% <100%> (ø)` | :arrow_up: |
| [.../assets/src/SqlLab/components/AceEditorWrapper.jsx](https://codecov.io/gh/apache/incubator-superset/pull/9012/diff?src=pr&el=tree#diff-c3VwZXJzZXQvYXNzZXRzL3NyYy9TcWxMYWIvY29tcG9uZW50cy9BY2VFZGl0b3JXcmFwcGVyLmpzeA==) | `54.21% <33.33%> (-0.11%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-superset/pull/9012?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-superset/pull/9012?src=pr&el=footer). Last update [d0bc04f...99a14c7](https://codecov.io/gh/apache/incubator-superset/pull/9012?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org