You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "justinpark (via GitHub)" <gi...@apache.org> on 2023/09/19 22:27:49 UTC
[GitHub] [superset] justinpark opened a new pull request, #25344: feat(sqllab): Format sql
justinpark opened a new pull request, #25344:
URL: https://github.com/apache/superset/pull/25344
### SUMMARY
This commit adds "Format SQL" feature in sql editor which help to format sql code in the sql editor.
This commit also adds a new shortcut (ctrl + shift + f) for this feature.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
https://github.com/apache/superset/assets/1392866/24a54a46-5b2c-494f-9c46-cfbf8f30a193
### TESTING INSTRUCTIONS
Go to SQL Lab and type a long-run sql
Hit Ctrl + Shift + f or click "Format SQL" in the action dropdown
Check the formatted sql
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [x] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on code in PR #25344:
URL: https://github.com/apache/superset/pull/25344#discussion_r1370942959
##########
superset-frontend/src/SqlLab/components/SqlEditor/index.tsx:
##########
@@ -384,8 +389,16 @@ const SqlEditor: React.FC<Props> = ({
}),
func: stopQuery,
},
+ {
+ name: 'formatQuery',
+ key: KeyboardShortcut.CTRL_SHIFT_F,
+ descr: KEY_MAP[KeyboardShortcut.CTRL_SHIFT_F],
+ func: () => {
+ formatCurrentQuery();
+ },
Review Comment:
```suggestion
func: formatCurrentQuery,
```
##########
superset/sqllab/api.py:
##########
@@ -185,6 +189,54 @@ def estimate_query_cost(self) -> Response:
result = command.run()
return self.response(200, result=result)
+ @expose("/format/", methods=("POST",))
+ @statsd_metrics
+ @protect()
+ @permission_name("read")
+ @event_logger.log_this_with_context(
+ action=lambda self, *args, **kwargs: f"{self.__class__.__name__}" f".format",
+ log_to_statsd=False,
+ )
+ def format(self) -> FlaskResponse:
Review Comment:
Maybe rename the endpoint to `format_sql`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] justinpark commented on a diff in pull request #25344: feat(sqllab): Format sql
Posted by "justinpark (via GitHub)" <gi...@apache.org>.
justinpark commented on code in PR #25344:
URL: https://github.com/apache/superset/pull/25344#discussion_r1330770998
##########
superset-frontend/src/SqlLab/components/AceEditorWrapper/index.tsx:
##########
@@ -144,8 +144,10 @@ const AceEditorWrapper = ({
};
const onChangeText = (text: string) => {
- setSql(text);
- onChange(text);
+ if (text !== sql) {
Review Comment:
This change prevents the overwrite the sql by `setQueryEditorAndSaveSqlWithDebounce`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "john-bodley (via GitHub)" <gi...@apache.org>.
john-bodley commented on code in PR #25344:
URL: https://github.com/apache/superset/pull/25344#discussion_r1380743029
##########
superset/sqllab/api.py:
##########
@@ -185,6 +189,54 @@ def estimate_query_cost(self) -> Response:
result = command.run()
return self.response(200, result=result)
+ @expose("/format_sql/", methods=("POST",))
+ @statsd_metrics
+ @protect()
+ @permission_name("read")
+ @event_logger.log_this_with_context(
+ action=lambda self, *args, **kwargs: f"{self.__class__.__name__}" f".format",
+ log_to_statsd=False,
+ )
+ def format_sql(self) -> FlaskResponse:
+ """Format the SQL query.
+ ---
+ post:
+ summary: Format SQL code
+ requestBody:
+ description: SQL query
+ required: true
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/FormatQueryPayloadSchema'
+ responses:
+ 200:
+ description: Format SQL result
+ content:
+ application/json:
+ schema:
+ type: object
+ properties:
+ result:
+ type: string
+ 400:
+ $ref: '#/components/responses/400'
+ 401:
+ $ref: '#/components/responses/401'
+ 403:
+ $ref: '#/components/responses/403'
+ 500:
+ $ref: '#/components/responses/500'
+ """
+ try:
+ model = self.format_model_schema.load(request.json)
+ result = sqlparse.format(
+ model.get("sql", ""), reindent=True, keyword_case="upper"
Review Comment:
If the `sql` field is required (per the schema) is `mode["sql"]` suffice?
##########
superset/sqllab/api.py:
##########
@@ -185,6 +189,54 @@ def estimate_query_cost(self) -> Response:
result = command.run()
return self.response(200, result=result)
+ @expose("/format_sql/", methods=("POST",))
+ @statsd_metrics
+ @protect()
+ @permission_name("read")
+ @event_logger.log_this_with_context(
+ action=lambda self, *args, **kwargs: f"{self.__class__.__name__}" f".format",
+ log_to_statsd=False,
+ )
+ def format_sql(self) -> FlaskResponse:
+ """Format the SQL query.
+ ---
+ post:
+ summary: Format SQL code
+ requestBody:
+ description: SQL query
+ required: true
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/FormatQueryPayloadSchema'
+ responses:
+ 200:
+ description: Format SQL result
+ content:
+ application/json:
+ schema:
+ type: object
+ properties:
+ result:
+ type: string
+ 400:
+ $ref: '#/components/responses/400'
+ 401:
+ $ref: '#/components/responses/401'
+ 403:
+ $ref: '#/components/responses/403'
+ 500:
+ $ref: '#/components/responses/500'
+ """
+ try:
+ model = self.format_model_schema.load(request.json)
+ result = sqlparse.format(
+ model.get("sql", ""), reindent=True, keyword_case="upper"
Review Comment:
If the `sql` field is required (per the schema) is `model["sql"]` suffice?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25344:
URL: https://github.com/apache/superset/pull/25344#issuecomment-1792779027
Ephemeral environment shutdown and build artifacts deleted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "justinpark (via GitHub)" <gi...@apache.org>.
justinpark merged PR #25344:
URL: https://github.com/apache/superset/pull/25344
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] michael-s-molina commented on a diff in pull request #25344: feat(sqllab): Format sql
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on code in PR #25344:
URL: https://github.com/apache/superset/pull/25344#discussion_r1331937144
##########
superset/sqllab/api.py:
##########
@@ -183,6 +187,54 @@ def estimate_query_cost(self) -> Response:
result = command.run()
return self.response(200, result=result)
+ @expose("/format/", methods=("POST",))
+ @statsd_metrics
+ @protect()
+ @permission_name("read")
+ @event_logger.log_this_with_context(
+ action=lambda self, *args, **kwargs: f"{self.__class__.__name__}" f".format",
+ log_to_statsd=False,
+ )
+ def format(self) -> FlaskResponse:
+ """Format the SQL query.
+ ---
+ post:
+ summary: Format SQL code
+ requestBody:
+ description: SQL query
+ required: true
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/EstimateQueryCostSchema'
Review Comment:
```suggestion
$ref: '#/components/schemas/FormatQueryPayloadSchema'
```
##########
superset-frontend/src/SqlLab/actions/sqlLab.js:
##########
@@ -1026,6 +1026,20 @@ export function queryEditorSetSql(queryEditor, sql) {
return { type: QUERY_EDITOR_SET_SQL, queryEditor, sql };
}
+export function formatQuery(queryEditor) {
+ return function (dispatch, getState) {
+ const { sql } = getUpToDateQuery(getState(), queryEditor);
+
Review Comment:
```suggestion
```
##########
superset/sqllab/api.py:
##########
@@ -183,6 +187,54 @@ def estimate_query_cost(self) -> Response:
result = command.run()
return self.response(200, result=result)
+ @expose("/format/", methods=("POST",))
+ @statsd_metrics
+ @protect()
+ @permission_name("read")
+ @event_logger.log_this_with_context(
+ action=lambda self, *args, **kwargs: f"{self.__class__.__name__}" f".format",
+ log_to_statsd=False,
+ )
+ def format(self) -> FlaskResponse:
+ """Format the SQL query.
+ ---
+ post:
+ summary: Format SQL code
+ requestBody:
+ description: SQL query
+ required: true
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/EstimateQueryCostSchema'
+ responses:
+ 200:
+ description: Format SQL result
+ content:
+ application/json:
+ schema:
+ type: object
+ properties:
+ result:
+ type: string
+ 400:
+ $ref: '#/components/responses/400'
+ 401:
+ $ref: '#/components/responses/401'
+ 403:
+ $ref: '#/components/responses/403'
+ 500:
+ $ref: '#/components/responses/500'
+ """
+ try:
+ model = self.format_model_schema.load(request.json)
+ except ValidationError as error:
+ return self.response_400(message=error.messages)
+ result = sqlparse.format(
+ model.get("sql", ""), reindent=True, keyword_case="upper"
+ )
+ return self.response(200, result=result)
Review Comment:
```suggestion
try:
model = self.format_model_schema.load(request.json)
result = sqlparse.format(
model.get("sql", ""), reindent=True, keyword_case="upper"
)
return self.response(200, result=result)
except ValidationError as error:
return self.response_400(message=error.messages)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] michael-s-molina commented on pull request #25344: feat(sqllab): Format sql
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on PR #25344:
URL: https://github.com/apache/superset/pull/25344#issuecomment-1729326444
/testenv up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25344:
URL: https://github.com/apache/superset/pull/25344#issuecomment-1778222729
@michael-s-molina Ephemeral environment spinning up at http://18.237.171.147:8080. Credentials are `admin`/`admin`. Please allow several minutes for bootstrapping and startup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "justinpark (via GitHub)" <gi...@apache.org>.
justinpark commented on code in PR #25344:
URL: https://github.com/apache/superset/pull/25344#discussion_r1378139656
##########
superset-frontend/src/SqlLab/actions/sqlLab.js:
##########
@@ -1033,6 +1033,19 @@ export function queryEditorSetSql(queryEditor, sql) {
return { type: QUERY_EDITOR_SET_SQL, queryEditor, sql };
}
+export function formatQuery(queryEditor) {
+ return function (dispatch, getState) {
+ const { sql } = getUpToDateQuery(getState(), queryEditor);
+ return SupersetClient.post({
Review Comment:
> why couldn't we just us a npm package to do this work vs. creating a whole new endpoint?
@hughhhh We chose server-side formatting over client-side formatting for several reasons:
1. It reduces the need to maintain additional NPM dependencies.
2. It reduces the package size.
3. It ensures that SQL formatting is consistent across surfaces in "view query", using sqlparse.
If you and other community members believe that an NPM package solution is better, I am happy to make changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] github-actions[bot] commented on pull request #25344: feat(sqllab): Format sql
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25344:
URL: https://github.com/apache/superset/pull/25344#issuecomment-1729329263
@michael-s-molina Ephemeral environment spinning up at http://34.220.51.49:8080. Credentials are `admin`/`admin`. Please allow several minutes for bootstrapping and startup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "john-bodley (via GitHub)" <gi...@apache.org>.
john-bodley commented on code in PR #25344:
URL: https://github.com/apache/superset/pull/25344#discussion_r1379176886
##########
superset-frontend/src/SqlLab/actions/sqlLab.js:
##########
@@ -1033,6 +1033,19 @@ export function queryEditorSetSql(queryEditor, sql) {
return { type: QUERY_EDITOR_SET_SQL, queryEditor, sql };
}
+export function formatQuery(queryEditor) {
+ return function (dispatch, getState) {
+ const { sql } = getUpToDateQuery(getState(), queryEditor);
+ return SupersetClient.post({
Review Comment:
There seems to be a number of pros with having this on the backend including: custom formatting, dialect specific formatting, and consistency throughout the application.
@villebro has experience working with frontend SQL formatting and ran into a number of issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "justinpark (via GitHub)" <gi...@apache.org>.
justinpark commented on PR #25344:
URL: https://github.com/apache/superset/pull/25344#issuecomment-1760335913
@michael-s-molina I also set up `SQLLAB_FORMAT_SQL_OPTIONS` in config to setup custom SQL format option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "justinpark (via GitHub)" <gi...@apache.org>.
justinpark commented on PR #25344:
URL: https://github.com/apache/superset/pull/25344#issuecomment-1759958479
@michael-s-molina could you stamp this PR if the updates looks okay?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on PR #25344:
URL: https://github.com/apache/superset/pull/25344#issuecomment-1778218941
/testenv up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on code in PR #25344:
URL: https://github.com/apache/superset/pull/25344#discussion_r1371104984
##########
superset-frontend/src/SqlLab/actions/sqlLab.js:
##########
@@ -1033,6 +1033,19 @@ export function queryEditorSetSql(queryEditor, sql) {
return { type: QUERY_EDITOR_SET_SQL, queryEditor, sql };
}
+export function formatQuery(queryEditor) {
+ return function (dispatch, getState) {
+ const { sql } = getUpToDateQuery(getState(), queryEditor);
+ return SupersetClient.post({
Review Comment:
why couldn't we just us a npm package to do this work vs. creating a whole new endpoint?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "justinpark (via GitHub)" <gi...@apache.org>.
justinpark commented on code in PR #25344:
URL: https://github.com/apache/superset/pull/25344#discussion_r1378139656
##########
superset-frontend/src/SqlLab/actions/sqlLab.js:
##########
@@ -1033,6 +1033,19 @@ export function queryEditorSetSql(queryEditor, sql) {
return { type: QUERY_EDITOR_SET_SQL, queryEditor, sql };
}
+export function formatQuery(queryEditor) {
+ return function (dispatch, getState) {
+ const { sql } = getUpToDateQuery(getState(), queryEditor);
+ return SupersetClient.post({
Review Comment:
> why couldn't we just us a npm package to do this work vs. creating a whole new endpoint?
@hughhhh We chose server-side formatting over client-side formatting for several reasons:
1. It reduces the need to maintain additional NPM dependencies.
2. It reduces the package size.
3. It ensures that SQL formatting is consistent across surfaces in "view query", using sqlparse.
If you and other community members believe that an NPM package solution is better, I am happy to make changes.
@betodealmeida @eschutho @michael-s-molina do you have an opinion on the frontend approach?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(sqllab): Format sql [superset]
Posted by "justinpark (via GitHub)" <gi...@apache.org>.
justinpark commented on PR #25344:
URL: https://github.com/apache/superset/pull/25344#issuecomment-1766887011
> I also set up `SQLLAB_FORMAT_SQL_OPTIONS` in config to setup custom SQL format option.
Revert this config option after offline discussion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org