You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "KhaledBenZina (via GitHub)" <gi...@apache.org> on 2024/04/11 12:58:53 UTC
[I] Error message when trying to use mongodb connection with superset meta database functionality [superset]
KhaledBenZina opened a new issue, #27989:
URL: https://github.com/apache/superset/issues/27989
### Bug description
Hello, I am running Superset on docker compose along with Apache Drill and a mongodb instance. I am trying to query the MongoDB database using Superset meta database for multiple data sources dashboards. The query: SELECT * from "mongodb_connection.mongo.my_database.my_table";
I am having this error message when I try to query mongo: "superset error: Catalogs are not currently supported". Is there a workaround or is something wrong with my query? Btw everything works fine when I query the mongodb using the classic mongodb database connection (with Apache Drill)
### How to reproduce the bug
1. Run Apache Drill on top of mongodb
2. Create a Mongodb connection using Apache Drill this will be called mongo
3. Create a Superset meta database connection with allowed_db the mongodb connection in engine parameters
4. Query the mongodb database on SQL lab using the syntax : SELECT * from "mongodb_connection.mongo.my_database.my_table";
5. Error message superset error: Catalogs are not currently supported
### Screenshots/recordings

### Superset version
4.0.0
### Python version
3.9
### Node version
16
### Browser
Chrome
### Additional context
I am using docker compose for superset&Drill&mongodb
### Checklist
- [X] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [X] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Error message when trying to query mongodb databse using superset meta database connection [superset]
Posted by "mistercrunch (via GitHub)" <gi...@apache.org>.
mistercrunch commented on issue #27989:
URL: https://github.com/apache/superset/issues/27989#issuecomment-2064995165
Just a thought, but I've seen hacks that try to do best of both worlds (perf and realtime) by creating view that that UNION ALL as in
```sql
SELECT * FROM highperf_materalized_table WHERE dt < DATE('{hard_coded_last_etl_job}')
UNION ALL
SELECT * FROM realtime_virtual_external table WHERE dt >= DATE('{hard_coded_last_etl_job}')
```
You need to make sure predicate is properly applied by the optimizer / excution-engine and leveraging some indexed/partitioned field, but it's doable in some systems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Error message when trying to query mongodb databse using superset meta database connection [superset]
Posted by "mistercrunch (via GitHub)" <gi...@apache.org>.
mistercrunch closed issue #27989: Error message when trying to query mongodb databse using superset meta database connection
URL: https://github.com/apache/superset/issues/27989
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Error message when trying to query mongodb databse using superset meta database connection [superset]
Posted by "KhaledBenZina (via GitHub)" <gi...@apache.org>.
KhaledBenZina commented on issue #27989:
URL: https://github.com/apache/superset/issues/27989#issuecomment-2063479734
I found a workaround actually.
I prepare my dataset on Drill directly by writing my query, store it in parquet using Drill filesystem, then query the DFS on Superset. The only difference is that I wont have my data live as I need to recreate the dataset on Drill each time (I will use a seperate python job for that)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Error message when trying to query mongodb databse using superset meta database connection [superset]
Posted by "mistercrunch (via GitHub)" <gi...@apache.org>.
mistercrunch commented on issue #27989:
URL: https://github.com/apache/superset/issues/27989#issuecomment-2065000846
Closing for now as I'm pretty sure it's a driver/database error message, but feel free to reopen if it's not the case. The py script I posted above should make that clear, and could be used to open a ticket in the proper place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Error message when trying to query mongodb databse using superset meta database connection [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #27989:
URL: https://github.com/apache/superset/issues/27989#issuecomment-2050660610
Hopefully @betodealmeida can help a bit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Error message when trying to query mongodb databse using superset meta database connection [superset]
Posted by "mistercrunch (via GitHub)" <gi...@apache.org>.
mistercrunch commented on issue #27989:
URL: https://github.com/apache/superset/issues/27989#issuecomment-2062672036
This a driver issue? Drill issue? My guess is you can validate with this simple python script (thanks gpt!)
```python
from sqlalchemy import create_engine
engine = create_engine('drill://{{THE_CONN_STR_YOU_PUT_IN_SUPERSET}}')
with engine.connect() as connection:
result = connection.execute("SELECT * FROM cat.schema.table") # Replace 'your_table' with your actual table name
for row in result:
print(row)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org