[GitHub] [incubator-superset] john-bodley commented on issue #7413: feature: see Presto row and array data types

[GitBox <gi...@apache.org>]

john-bodley commented on issue #7413: feature: see Presto row and array data types
URL: https://github.com/apache/incubator-superset/pull/7413#issuecomment-521397338
 
 
   @betodealmeida @khtruong @DiggidyDave we're running into issues with this feature as it doesn't seem to handle all the Presto [structural types](https://prestodb.github.io/docs/current/language/types) i.e., it's missing the the `MAP`. This isn't problematic in itself but it's an issue when the `ROW` or `ARRAY` contains a `MAP` object, i.e, `array(map(varchar, varchar))`. 
   
   The failure is:
   
   ```
   {"error": "list index out of range", "stacktrace": "Traceback (most recent call last):\n  File \"/srv/superset/superset/views/core.py\", line 2284, in table\n    columns = mydb.get_columns(table_name, schema)\n  File \"/srv/superset/superset/models/core.py\", line 1154, in get_columns\n    return self.db_engine_spec.get_columns(self.inspector, table_name, schema)\n  File \"/srv/superset/superset/db_engine_specs/presto.py\", line 235, in get_columns\n    cls._parse_structural_column(column.Column, column.Type, result)\n  File \"/srv/superset/superset/db_engine_specs/presto.py\", line 157, in _parse_structural_column\n    if field_info[1] == \"array\" or field_info[1] == \"row\":\nIndexError: list index out of range\n"}
   ```
   
   I looked into trying to fix (checking the length of `field_info`) but I don't fully follow the logic and I'm unsure whether a `MAP` needs to be handled like an `ARRAY` or `ROW` and thus where I should fix this. Note that a `MAP` can consist of other structural types. 
   
   Note I'm uncertain how complex the fix is but could Lyft either i) look into fixing this in the next 48 hours, ii) put the feature behind a feature flag, or iii) revert the change whilst a long term solution is found. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org