You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/12/21 08:26:17 UTC

[GitHub] [superset] applike-ss opened a new issue, #22487: No tables listed when using pyathena adapter with federated query (lambda)

applike-ss opened a new issue, #22487:
URL: https://github.com/apache/superset/issues/22487

   I am in the process of setting up a superset instance.
   For doing that, i added PyAthena and PyAthenaJDBC via pip and they seems to work as i can connect to athena and querying the default awsdatacatalog seems to work (can create datasets and make charts based on those).
   Now the next step i need to take is to add another data catalog which is queried via the query federation of athena.
   That requires athena to use a lambda function to connect to an AWS RDS instance.
   The process itself is working from within athena and i can create a database within superset.
   However when i try to create a dataset it is loading endlessly for the tables and eventually stops.
   There are a lot of tables and i see that it is querying the metadata for them.
   I saw a lot of TooManyRequestsException when trying, so i retried in hope that superset does cache the table metadata it fetched already and would eventually succeed.
   That seems to be the case, as i can not see any more errors.
   I see no tables listed for selecting though.
   
   #### How to reproduce the bug
   
   1. Create a RDS with >50 tables
   2. Create a athena datasource
   3. Create a federated query lambda
   4. Check that it works from athena
   5. Add database in superset
   6. Try to create a dataset
   
   ### Expected results
   
   I should see a list of tables from which i can pick which one to use for the dataset
   
   ### Actual results
   
   It loads for some time and eventually finishes with "no data" and no errors in the logs (after retrying some times)
   
   #### Screenshots
   ![Auswahl_1513](https://user-images.githubusercontent.com/31846129/208855792-9a313fe8-45d8-4fb2-a26f-3bbbdae669e5.png)
   
   ### Environment
   
   (please complete the following information):
   
   - browser type and version: Chrome 108
   - superset version: 1.5.2
   - python version: 3.8.13
   - node.js version: -
   - any feature flags active: DYNAMIC_PLUGINS 
   
   ### Checklist
   
   Make sure to follow these steps before submitting your issue - thank you!
   
   - [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
   - [x] I have reproduced the issue with at least the latest released version of superset.
   - [x] I have checked the issue tracker for the same issue and I haven't found one similar.
   
   ### Additional context
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] mai-nguyen-coinhako commented on issue #22487: No tables listed when using pyathena adapter with federated query (lambda)

Posted by "mai-nguyen-coinhako (via GitHub)" <gi...@apache.org>.
mai-nguyen-coinhako commented on issue #22487:
URL: https://github.com/apache/superset/issues/22487#issuecomment-1428465194

   I have made this [PR 410](https://github.com/laughingman7743/PyAthena/pull/410/files) to address this issue. Hopefully, it is fix.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] mai-nguyen-coinhako commented on issue #22487: No tables listed when using pyathena adapter with federated query (lambda)

Posted by "mai-nguyen-coinhako (via GitHub)" <gi...@apache.org>.
mai-nguyen-coinhako commented on issue #22487:
URL: https://github.com/apache/superset/issues/22487#issuecomment-1429224061

   The fix [PR #410](https://github.com/laughingman7743/PyAthena/pull/410/files) has been bumped to PyAthena v2.23.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] mai-nguyen-coinhako commented on issue #22487: No tables listed when using pyathena adapter with federated query (lambda)

Posted by "mai-nguyen-coinhako (via GitHub)" <gi...@apache.org>.
mai-nguyen-coinhako commented on issue #22487:
URL: https://github.com/apache/superset/issues/22487#issuecomment-1429271737

   Testing is successful with my Superset environment. Now the table list appears properly in SQLab and when creating a new dataset. 
   
   ![SQLab](https://user-images.githubusercontent.com/99168209/218671290-d6a5e9c0-3f5d-48d0-8fc9-60f3de9f734f.png)
   
   ![Add a new dataset](https://user-images.githubusercontent.com/99168209/218671332-698f6de3-5b9f-46c3-9653-2f854ef55b1c.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] No tables listed when using pyathena adapter with federated query (lambda) [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #22487:
URL: https://github.com/apache/superset/issues/22487#issuecomment-1986220258

   Hi there, and sorry this has gone quiet for so long. We're no longer supporting Superset 2.x or prior, and it's been a while since this thread saw any activity, so I'm closing this as stale. If it is still an issue in Superset 3.x, we can re-open this, or feel free to open a new issue with updated context.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] No tables listed when using pyathena adapter with federated query (lambda) [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #22487: No tables listed when using pyathena adapter with federated query (lambda)
URL: https://github.com/apache/superset/issues/22487


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] applike-ss commented on issue #22487: No tables listed when using pyathena adapter with federated query (lambda)

Posted by "applike-ss (via GitHub)" <gi...@apache.org>.
applike-ss commented on issue #22487:
URL: https://github.com/apache/superset/issues/22487#issuecomment-1427814275

   PyAthena has updated to v2.21.0 to include the necessary changes, how it seems to be up to the superset side. However in the Helm `boostrapScript` value you can let pip install the new version. I did not check myself yet whether it works.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] anikamurarka-ai commented on issue #22487: No tables listed when using pyathena adapter with federated query (lambda)

Posted by "anikamurarka-ai (via GitHub)" <gi...@apache.org>.
anikamurarka-ai commented on issue #22487:
URL: https://github.com/apache/superset/issues/22487#issuecomment-1667596533

   Hi, any update on this? Facing same issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] mai-nguyen-coinhako commented on issue #22487: No tables listed when using pyathena adapter with federated query (lambda)

Posted by "mai-nguyen-coinhako (via GitHub)" <gi...@apache.org>.
mai-nguyen-coinhako commented on issue #22487:
URL: https://github.com/apache/superset/issues/22487#issuecomment-1427782315

   Hi, any updates on this bug?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] mai-nguyen-coinhako commented on issue #22487: No tables listed when using pyathena adapter with federated query (lambda)

Posted by "mai-nguyen-coinhako (via GitHub)" <gi...@apache.org>.
mai-nguyen-coinhako commented on issue #22487:
URL: https://github.com/apache/superset/issues/22487#issuecomment-1428395013

   Hi, I have updated `PyAthena` to v2.22.0 and test in Superset, which is deployed in Kubernetes, but it is still not working. The table list still does not show up in the UI of SQLab. In my case, I use Athena to query data from AWS Timestream database through its query federation feature. From following the stack trace in Superset logging, an event `request-created.athena.ListTableMetadata` was created, and it successfully received a Json response from Athena with all the table metadata in the database as below: 
   
   ```
   2023-02-13 17:09:30,995:DEBUG:botocore.hooks:Event request-created.athena.ListTableMetadata: calling handler <function add_retry_headers at 0x7fe1cc285ca0>
   2023-02-13 17:09:30,996:DEBUG:botocore.endpoint:Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://athena.ap-southeast-2.amazonaws.com/, headers={'X-Amz-Target': b'AmazonAthena.ListTableMetadata', 'Content-Type': b'application/x-amz-json-1.1', 'User-Agent': b'Boto3/1.26.69 Python/3.8.12 Linux/5.4.219-126.411.amzn2.x86_64 Botocore/1.29.69 PyAthena/2.22.0', 'X-Amz-Date': b'20230213T170930Z'
   023-02-13 17:09:30,996:DEBUG:botocore.httpsession:Certificate path: /usr/local/lib/python3.8/site-packages/certifi/cacert.pem
   2023-02-13 17:09:30,997:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): athena.ap-southeast-2.amazonaws.com:443
   2023-02-13 17:09:34,597:DEBUG:urllib3.connectionpool:https://athena.ap-southeast-2.amazonaws.com:443 "POST / HTTP/1.1" 200 13012
   2023-02-13 17:09:34,602:DEBUG:botocore.parsers:Response headers: {'Date': 'Mon, 13 Feb 2023 17:09:34 GMT', 'Content-Type': 'application/x-amz-json-1.1', 'Content-Length': '13012', 'Connection': 'keep-alive', 'x-amzn-RequestId': '27544755-53ff-43bd-9fd1-6d79cba83684'}
   2023-02-13 17:09:34,603:DEBUG:botocore.parsers:Response body:
   b'{"TableMetadataList":[{"Columns":[{"Name":"id","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"coin","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"jump_eth","Type":"varchar"},{"Name":"event_datetime","Type":"varchar"},{"Name":"id","Type":"varchar"},{"Name":"jump_btc","Type":"varchar"},{"Name":"version","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"curve_gen_calendar","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"jump_eth","Type":"varchar"},{"Name":"event_datetime","Type":"varchar"},{"Name":"id","Type":"varchar"},{"Name":"jump_btc","Type":"varchar"},{"Name":"version","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"curve_gen_calend
 ar_test","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"dbt_strikes","Type":"varchar"},{"Name":"maturity_date","Type":"varchar"},{"Name":"all_strikes","Type":"varchar"},{"Name":"dbt_bids","Type":"varchar"},{"Name":"sabr_vols","Type":"varchar"},{"Name":"dbt_dn_strike","Type":"varchar"},{"Name":"dbt_asks","Type":"varchar"},{"Name":"dbt_10c_strike","Type":"varchar"},{"Name":"dbt_vols","Type":"varchar"},{"Name":"dbt_25p_strike","Type":"varchar"},{"Name":"dbt_10p_strike","Type":"varchar"},{"Name":"coin","Type":"varchar"},{"Name":"dbt_25c_strike","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"curve_gen_plotting_params","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"coin","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::double","Type":"double"}],"Name":"curve_parameters","PartitionKe
 ys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"btc_currency","Type":"varchar"},{"Name":"eth_equity","Type":"varchar"},{"Name":"ethw_available_funds","Type":"varchar"},{"Name":"usdc_margin_balance","Type":"varchar"},{"Name":"usdc_initial_margin","Type":"varchar"},{"Name":"receive_notifications","Type":"varchar"},{"Name":"type","Type":"varchar"},{"Name":"ethw_margin_balance","Type":"varchar"},{"Name":"eth_balance","Type":"varchar"},{"Name":"sol_available_withdrawal_funds","Type":"varchar"},{"Name":"security_keys_enabled","Type":"varchar"},{"Name":"sol_available_funds","Type":"varchar"},{"Name":"eth_available_withdrawal_funds","Type":"varchar"},{"Name":"ethw_equity","Type":"varchar"},{"Name":"id","Type":"varchar"},{"Name":"login_enabled","Type":"varchar"},{"Name":"btc_equity","Type":"varchar"},{"Name":"btc_available_funds","Type":"varchar"},{"Name":"is_password","Type":"varchar"},{"Name":"security_keys_assignments","Type":"varchar"},{"Name":"usdc_equity","Type":"varchar"},{"Name":
 "eth_available_funds","Type":"varchar"},{"Name":"ethw_balance","Type":"varchar"},{"Name":"referrals_count","Type":"varchar"},{"Name":"ethw_initial_margin","Type":"varchar"},{"Name":"ethw_available_withdrawal_funds","Type":"varchar"},{"Name":"sol_equity","Type":"varchar"},{"Name":"btc_margin_balance","Type":"varchar"},{"Name":"usdc_available_funds","Type":"varchar"},{"Name":"sol_currency","Type":"varchar"},{"Name":"btc_initial_margin","Type":"varchar"},{"Name":"eth_currency","Type":"varchar"},{"Name":"proof_id","Type":"varchar"},{"Name":"btc_balance","Type":"varchar"},{"Name":"usdc_balance","Type":"varchar"},{"Name":"usdc_maintenance_margin","Type":"varchar"},{"Name":"email","Type":"varchar"},{"Name":"ethw_currency","Type":"varchar"},{"Name":"sol_margin_balance","Type":"varchar"},{"Name":"ethw_maintenance_margin","Type":"varchar"},{"Name":"btc_maintenance_margin","Type":"varchar"},{"Name":"proof_id_signature","Type":"varchar"},{"Name":"sol_balance","Type":"varchar"},{"Name":"eth_main
 tenance_margin","Type":"varchar"},{"Name":"system_name","Type":"varchar"},{"Name":"sol_initial_margin","Type":"varchar"},{"Name":"usdc_available_withdrawal_funds","Type":"varchar"},{"Name":"sol_maintenance_margin","Type":"varchar"},{"Name":"eth_initial_margin","Type":"varchar"},{"Name":"usdc_currency","Type":"varchar"},{"Name":"btc_available_withdrawal_funds","Type":"varchar"},{"Name":"eth_margin_balance","Type":"varchar"},{"Name":"username","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"deribit_account_info","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"leverage","Type":"varchar"},{"Name":"initial_margin","Type":"varchar"},{"Name":"realized_profit_loss","Type":"varchar"},{"Name":"settlement_price","Type":"varchar"},{"Name":"maintenance_margin","Type":"varchar"},{"Name":"delta","Type":"varchar"},{"Name":"index_price","Type":"varchar"},{"Name":"floating_profit_
 loss_usd","Type":"varchar"},{"Name":"theta","Type":"varchar"},{"Name":"uid","Type":"varchar"},{"Name":"total_profit_loss","Type":"varchar"},{"Name":"direction","Type":"varchar"},{"Name":"open_orders_margin","Type":"varchar"},{"Name":"kind","Type":"varchar"},{"Name":"mark_price","Type":"varchar"},{"Name":"average_price","Type":"varchar"},{"Name":"estimated_liquidation_price","Type":"varchar"},{"Name":"interest_value","Type":"varchar"},{"Name":"size_currency","Type":"varchar"},{"Name":"floating_profit_loss","Type":"varchar"},{"Name":"size","Type":"varchar"},{"Name":"instrument_name","Type":"varchar"},{"Name":"average_price_usd","Type":"varchar"},{"Name":"realized_funding","Type":"varchar"},{"Name":"gamma","Type":"varchar"},{"Name":"vega","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"deribit_account_position","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"stats_v
 olume_usd","Type":"varchar"},{"Name":"stats_low","Type":"varchar"},{"Name":"settlement_price","Type":"varchar"},{"Name":"index_price","Type":"varchar"},{"Name":"open_interest","Type":"varchar"},{"Name":"best_ask_amount","Type":"varchar"},{"Name":"greeks_delta","Type":"varchar"},{"Name":"stats_high","Type":"varchar"},{"Name":"underlying_index","Type":"varchar"},{"Name":"greeks_gamma","Type":"varchar"},{"Name":"current_funding","Type":"varchar"},{"Name":"best_bid_amount","Type":"varchar"},{"Name":"greeks_theta","Type":"varchar"},{"Name":"funding_8h","Type":"varchar"},{"Name":"estimated_delivery_price","Type":"varchar"},{"Name":"interest_rate","Type":"varchar"},{"Name":"mark_iv","Type":"varchar"},{"Name":"state","Type":"varchar"},{"Name":"bid_iv","Type":"varchar"},{"Name":"timestamp","Type":"varchar"},{"Name":"underlying_price","Type":"varchar"},{"Name":"mark_price","Type":"varchar"},{"Name":"greeks_rho","Type":"varchar"},{"Name":"interest_value","Type":"varchar"},{"Name":"best_bid_pri
 ce","Type":"varchar"},{"Name":"greeks_vega","Type":"varchar"},{"Name":"stats_price_change","Type":"varchar"},{"Name":"max_price","Type":"varchar"},{"Name":"min_price","Type":"varchar"},{"Name":"instrument_name","Type":"varchar"},{"Name":"best_ask_price","Type":"varchar"},{"Name":"stats_volume","Type":"varchar"},{"Name":"last_price","Type":"varchar"},{"Name":"ask_iv","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"deribit_option_price","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"}],"Name":"deribit_transaction_log","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"total_pl","Type":"varchar"},{"Name":"options_session_upl","Type":"varchar"},{"Name":"initial_margin","Type":"varchar"},{"Name":"options_session_rpl","Type":"varchar"},{"Name":"referrer_id","Type":"varchar"},{"Name":"mainten
 ance_margin","Type":"varchar"},{"Name":"futures_session_upl","Type":"varchar"},{"Name":"futures_session_rpl","Type":"varchar"},{"Name":"ETH DVOL","Type":"varchar"},{"Name":"portfolio_margining_enabled","Type":"varchar"},{"Name":"type","Type":"varchar"},{"Name":"equity","Type":"varchar"},{"Name":"ETH Price","Type":"varchar"},{"Name":"options_pl","Type":"varchar"},{"Name":"projected_maintenance_margin","Type":"varchar"},{"Name":"fee_balance","Type":"varchar"},{"Name":"futures_pl","Type":"varchar"},{"Name":"security_keys_enabled","Type":"varchar"},{"Name":"balance","Type":"varchar"},{"Name":"projected_initial_margin","Type":"varchar"},{"Name":"projected_delta_total","Type":"varchar"},{"Name":"margin_balance","Type":"varchar"},{"Name":"currency","Type":"varchar"},{"Name":"delta_total_map","Type":"varchar"},{"Name":"available_withdrawal_funds","Type":"varchar"},{"Name":"delta_total","Type":"varchar"},{"Name":"id","Type":"varchar"},{"Name":"login_enabled","Type":"varchar"},{"Name":"email"
 ,"Type":"varchar"},{"Name":"limits","Type":"varchar"},{"Name":"options_gamma","Type":"varchar"},{"Name":"options_vega","Type":"varchar"},{"Name":"session_upl","Type":"varchar"},{"Name":"creation_timestamp","Type":"varchar"},{"Name":"interuser_transfers_enabled","Type":"varchar"},{"Name":"system_name","Type":"varchar"},{"Name":"session_rpl","Type":"varchar"},{"Name":"options_theta","Type":"varchar"},{"Name":"options_delta","Type":"varchar"},{"Name":"options_value","Type":"varchar"},{"Name":"available_funds","Type":"varchar"},{"Name":"username","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::double","Type":"double"}],"Name":"historical","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"jump_eth","Type":"varchar"},{"Name":"event_datetime","Type":"varchar"},{"Name":"id","Type":"varchar"},{"Name":"jump_btc","Type":"varchar"},{"Name":"version","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Na
 me":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"last_occurrence_test","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"gen_25rr","Type":"varchar"},{"Name":"dbt_atm","Type":"varchar"},{"Name":"dbt_ask","Type":"varchar"},{"Name":"maturity_date","Type":"varchar"},{"Name":"gen_25bfly","Type":"varchar"},{"Name":"atm_strike","Type":"varchar"},{"Name":"dbt_bid","Type":"varchar"},{"Name":"dbt_25rr","Type":"varchar"},{"Name":"gen_10bfly","Type":"varchar"},{"Name":"dbt_10bfly","Type":"varchar"},{"Name":"dbt_10rr","Type":"varchar"},{"Name":"dbt_25bfly","Type":"varchar"},{"Name":"gen_10rr","Type":"varchar"},{"Name":"id","Type":"varchar"},{"Name":"gen_atm","Type":"varchar"},{"Name":"maturity_date_ts","Type":"varchar"},{"Name":"coin","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"voltable","PartitionKeys":[],"TableType":"EXTERNAL"},{"
 Columns":[{"Name":"dbt_ask","Type":"varchar"},{"Name":"all_strikes","Type":"varchar"},{"Name":"gen_25bfly","Type":"varchar"},{"Name":"atm_strike","Type":"varchar"},{"Name":"dbt_bids","Type":"varchar"},{"Name":"dbt_bid","Type":"varchar"},{"Name":"gen_10bfly","Type":"varchar"},{"Name":"dbt_10bfly","Type":"varchar"},{"Name":"dbt_10rr","Type":"varchar"},{"Name":"dbt_vols","Type":"varchar"},{"Name":"gen_10rr","Type":"varchar"},{"Name":"id","Type":"varchar"},{"Name":"gen_atm","Type":"varchar"},{"Name":"dbt_strikes","Type":"varchar"},{"Name":"gen_25rr","Type":"varchar"},{"Name":"dbt_atm","Type":"varchar"},{"Name":"sabr_vols","Type":"varchar"},{"Name":"dbt_dn_strike","Type":"varchar"},{"Name":"dbt_25rr","Type":"varchar"},{"Name":"dbt_asks","Type":"varchar"},{"Name":"dbt_10c_strike","Type":"varchar"},{"Name":"dbt_25p_strike","Type":"varchar"},{"Name":"dbt_25bfly","Type":"varchar"},{"Name":"dbt_10p_strike","Type":"varchar"},{"Name":"coin","Type":"varchar"},{"Name":"dbt_25c_strike","Type":"var
 char"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"voltable_merged","PartitionKeys":[],"TableType":"EXTERNAL"},{"Columns":[{"Name":"gen_25rr","Type":"varchar"},{"Name":"dbt_atm","Type":"varchar"},{"Name":"dbt_ask","Type":"varchar"},{"Name":"maturity_date","Type":"varchar"},{"Name":"gen_25bfly","Type":"varchar"},{"Name":"atm_strike","Type":"varchar"},{"Name":"dbt_bid","Type":"varchar"},{"Name":"dbt_25rr","Type":"varchar"},{"Name":"gen_10bfly","Type":"varchar"},{"Name":"dbt_10bfly","Type":"varchar"},{"Name":"dbt_10rr","Type":"varchar"},{"Name":"dbt_25bfly","Type":"varchar"},{"Name":"gen_10rr","Type":"varchar"},{"Name":"id","Type":"varchar"},{"Name":"gen_atm","Type":"varchar"},{"Name":"maturity_date_ts","Type":"varchar"},{"Name":"coin","Type":"varchar"},{"Name":"measure_name","Type":"varchar"},{"Name":"time","Type":"timestamp"},{"Name":"measure_value::varchar","Type":"varchar"}],"Name":"voltable
 _ray_test","PartitionKeys":[],"TableType":"EXTERNAL"}]}'
   2023-02-13 17:09:34,609:DEBUG:botocore.hooks:Event needs-retry.athena.ListTableMetadata: calling handler <botocore.retryhandler.RetryHandler object at 0x7fe148ed3d00>
   2023-02-13 17:09:34,610:DEBUG:botocore.retryhandler:No retry needed.
   ```
   It can be seen that there is a new Table Type `EXTERNAL` for Athena federation query. The fix from this PR [#403](https://github.com/laughingman7743/PyAthena/pull/403) in PyAthena only supports **EXTERNAL_TABLE**  and **MANAGED_TABLE**, which is created via Spark when Glue is enabled as hivemeta store, though `EXTERNAL` are fully queryable in Athena. 
   
   On the bright side, I still can query data following the below format:
   ```
   Select * 
   from catalog_name.database_name.table_name 
   limit 10
   ```
   **How to reproduce the bug**
   
   1. Create an AWS Timestream database with around 10 tables
   2. Create a Athena data source
   3. Create a federated query lambda
   4. Check that it works from Athena
   5. Add database in Superset
   6. Go to SQLab, choose Athena database and its schema 
   7. Try to create a dataset in Superset 
   
   ### Expected results
   1. In SQLab, a list of tables should be seen from the drop-down list 
   2. A list of tables from which I can pick which one to use to create a dataset
   
   ### Actual results
   
   When creating a dataset, it loads for some time and eventually finishes with "no data" and no errors in the logs (after retrying some times)
   
   In SQLab,  it loads for some time and eventually finishes with "no data" even though the API called to Athena TableMetadata is successful in the logs. 
   
   ### Screenshots
   <img width="691" alt="image" src="https://user-images.githubusercontent.com/99168209/218532826-75d1856d-6937-4f01-8136-c30f63c634fe.png">
   
   ### Environment
   (please complete the following information):
   
    1. Browser type and version: Gecko/20100101 Firefox/109.0
    2. Superset version: 2.0.1
    3. Python version: Python 3.8.12
    4. Node.js version: -
    5. Any feature flags active: 
    - GENERIC_CHART_AXES
    - ALERT_REPORTS
    - THUMBNAILS
    - GLOBAL_ASYNC_QUERIES
    - DASHBOARD_CACHE
   
   ### Checklist
   
   Make sure to follow these steps before submitting your issue - thank you!
   
   - [ ]     I have checked the superset logs for python stacktraces and included it here as text if there are any.
   - [ ]     I have reproduced the issue with at least the latest released version of superset.
   - [ ]     I have checked the issue tracker for the same issue and I haven't found one similar.
   
   ### Additional context


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org