You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/05/20 07:20:28 UTC
[GitHub] [superset] niravpeak opened a new issue, #20137: redshift_connector Driver throws error while running SELECT 1 (or any query)
niravpeak opened a new issue, #20137:
URL: https://github.com/apache/superset/issues/20137
A clear and concise description of what the bug is.
We were using driver `redshift+psycopg2` , that worked well so far. As part of enhanced security we moved from that driver to `redshift+redshift_connector` driver. Although it does successful connect & dropdown of tables. it is unable to display correct dataset & getting mentioned error below:
#### How to reproduce the bug
1. install redshift_connector
bootstrapScript: |
#!/bin/bash
rm -rf /var/lib/apt/lists/* && \
pip install \
psycopg2-binary==2.9.1 \
redis==3.5.3 \
sqlalchemy-redshift==0.8.9 \
redshift-connector==2.0.907 && \
if [ ! -f ~/bootstrap ]; then echo "Running Superset with uid {{ .Values.runAsUser }}" > ~/bootstrap; fi
2. Go to 'databases' => Add database using username / password or IAM based. test connect
3. Go to sql editor page
4. select database & relevant schema.
5. On editor type "SELECT 1"
6. See below error:
First element of field tuple is neither a tuple nor str
### Expected results
Output with data 1
what you expected to happen.
### Actual results
what actually happens.
On pod we are getting below error trace:
raceback (most recent call last):
File "/app/superset/views/base.py", line 207, in wraps
return f(self, *args, **kwargs)
File "/app/superset/utils/log.py", line 245, in wrapper
value = f(*args, **kwargs)
File "/app/superset/views/core.py", line 2393, in sql_json
command_result: CommandResult = command.run()
File "/app/superset/sqllab/command.py", line 104, in run
raise ex
File "/app/superset/sqllab/command.py", line 96, in run
status = self._run_sql_json_exec_from_scratch()
File "/app/superset/sqllab/command.py", line 138, in _run_sql_json_exec_from_scratch
raise ex
File "/app/superset/sqllab/command.py", line 133, in _run_sql_json_exec_from_scratch
return self._sql_json_executor.execute(
File "/app/superset/sqllab/sql_json_executer.py", line 111, in execute
raise SupersetErrorsException(
superset.exceptions.SupersetErrorsException: [SupersetError(message='First element of field tuple is neither a tuple nor str', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Amazon Redshift', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
#### Screenshots
If applicable, add screenshots to help explain your problem.
### Environment
(please complete the following information):
- browser type and version: chrome:
- superset version: `superset version`: Superset 0.0.0dev
- python version: `python --version`: Python 3.8.12
- node.js version: `node -v`:
- any feature flags active: [ installed with helm chart ]
helm install superset . --values=values.yaml -n superset-experiment
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [x] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
Add any other context about the problem here.
Few more details about debug trace from redshift_connector can be seen as below:
2022-05-17 05:50:12,363:DEBUG:redshift_connector:===================================
2022-05-17 05:50:12,363:DEBUG:redshift_connector.cursor:Cursor.paramstyle=named
2022-05-17 05:50:12,363:DEBUG:redshift_connector.core:===================================
2022-05-17 05:50:12,363:DEBUG:redshift_connector.core:Establishing a connection
2022-05-17 05:50:12,363:DEBUG:redshift_connector.core:{'user': 'IAM:uksegmentexplorer', 'database': 'dev', 'application_name': 'sqlalchemy-redshift', 'replication': None, 'client_protocol_version': '2', 'driver_version': 'Redshift Python Driver 2.0.907', 'os_version': 'Linux-5.4.181-99.354.amzn2.x86_64-x86_64-with-glibc2.2.5'}
2022-05-17 05:50:12,363:DEBUG:redshift_connector.core:===================================
2022-05-17 05:50:12,369:DEBUG:redshift_connector.cursor:Cursor.paramstyle=format
2022-05-17 05:50:12,369:DEBUG:redshift_connector.core:Sending start-up message
2022-05-17 05:50:12,551:DEBUG:redshift_connector.core:Server indicated EXTENDED_RESULT_METADATA transfer protocol will be used rather than protocol requested by client: BINARY
2022-05-17 05:50:12,551:DEBUG:redshift_connector.cursor:Cursor.paramstyle=format
2022-05-17 05:50:12,559:DEBUG:redshift_connector.core:field count=1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Brooke-white commented on issue #20137: redshift_connector Driver throws error while running SELECT 1 (or any query)
Posted by GitBox <gi...@apache.org>.
Brooke-white commented on issue #20137:
URL: https://github.com/apache/superset/issues/20137#issuecomment-1150446638
please let me know if there is anything needed from redshift_connector to support this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho commented on issue #20137: redshift_connector Driver throws error while running SELECT 1 (or any query)
Posted by GitBox <gi...@apache.org>.
eschutho commented on issue #20137:
URL: https://github.com/apache/superset/issues/20137#issuecomment-1150435690
`psycopg2` is the default driver for redshift on Superset, so it's definitely likely that we would need to make some small adjustments to use the `redshift_connector` driver. If we were to support it natively, I believe right now we would have to create a new db engine spec for redshift+redshift_connector. @betodealmeida do you have any additional insight on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Brooke-white commented on issue #20137: redshift_connector Driver throws error while running SELECT 1 (or any query)
Posted by GitBox <gi...@apache.org>.
Brooke-white commented on issue #20137:
URL: https://github.com/apache/superset/issues/20137#issuecomment-1148961520
Hi @rusackas & @eschutho , I maintain redshift_connector. We haven't tested integration with superset, so I'm new to the superset codebase. Is this issue something you folks have seen in the past?
I'm working on reproducing this issue locally and will update any more details I find here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Brooke-white commented on issue #20137: redshift_connector Driver throws error while running SELECT 1 (or any query)
Posted by GitBox <gi...@apache.org>.
Brooke-white commented on issue #20137:
URL: https://github.com/apache/superset/issues/20137#issuecomment-1155426450
Thanks for the heads up @betodealmeida -- we are working with @niravpeak to determine what a migration path looks like for retrieving column names as strings
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho commented on issue #20137: redshift_connector Driver throws error while running SELECT 1 (or any query)
Posted by GitBox <gi...@apache.org>.
eschutho commented on issue #20137:
URL: https://github.com/apache/superset/issues/20137#issuecomment-1166144160
I'm going to close this ticket given @betodealmeida's fix, but feel free to comment if you feel that action was premature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho closed issue #20137: redshift_connector Driver throws error while running SELECT 1 (or any query)
Posted by GitBox <gi...@apache.org>.
eschutho closed issue #20137: redshift_connector Driver throws error while running SELECT 1 (or any query)
URL: https://github.com/apache/superset/issues/20137
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] rusackas commented on issue #20137: redshift_connector Driver throws error while running SELECT 1 (or any query)
Posted by GitBox <gi...@apache.org>.
rusackas commented on issue #20137:
URL: https://github.com/apache/superset/issues/20137#issuecomment-1137434066
@eschutho do you think this change in driver is something we should look at and support?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on issue #20137: redshift_connector Driver throws error while running SELECT 1 (or any query)
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on issue #20137:
URL: https://github.com/apache/superset/issues/20137#issuecomment-1151687592
@Brooke-white the cursor description returned by the`redshift_connector` has the column name as bytes, and we expect it to be a string. For example:
```python
[
(b"date", 1043, None, None, None, None, None),
(b"open", 701, None, None, None, None, None),
(b"high", 701, None, None, None, None, None),
(b"low", 701, None, None, None, None, None),
(b"close", 701, None, None, None, None, None),
(b"adj close", 701, None, None, None, None, None),
(b"volume", 20, None, None, None, None, None),
]
```
There's no standard type for the column name (https://peps.python.org/pep-0249/#cursor-attributes), but in all DB engine specs I've seen it's returned as a string, so you might want to change that. I'll also update Superset so that it works with bytes, just in case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org