[GitHub] [superset] Animesh3193 opened a new issue #15255: DB Connection pool throws error when executing raw sql in Celery job

[GitBox <gi...@apache.org>]

Animesh3193 opened a new issue #15255:
URL: https://github.com/apache/superset/issues/15255


   When running Superset version 0.36 coupled with postgres to schedule jobs that runs raw SQL postgres queries in kubernetes environemnt. The query execution fails as follow :- 
   >psycopg2.DatabaseError: error with status PGRES_TUPLES_OK and no message from the libpq
   
   ### Expected results
   
   - When Raw SQL is run in the db session object using the execute method
   - By using fetchall to fetch all row from the postgres database, result is expected to be a list of tuples containing value of columns
   
   ### Actual results
   
   - When Raw SQL is run in the db session object using the execute method
   - Then using fetchall to fetch all row we encounter below error
   - result that we get is the Postgres error as psycopg2.DatabaseError: error with status PGRES_TUPLES_OK and no message from the libpq
   
   #### Screenshots
   
   ![image](https://user-images.githubusercontent.com/16374123/122551767-0fbf7280-d053-11eb-8233-6775af67cd88.png)
   
   #### How to reproduce the bug
   
   - In a celery job initialize db session
   - DB session is initialized as db.session() or db.create_scoped_session()
   - raw sql is used as sqlalchemy text e.g. sqlalchemy.text('Select * from Employees')
   - raw sql is executed in the db object to give result as list of tuples
   - The code being used is db.create_scoped_session().execute(sqlalchemy.text('Select * from Employees')).fetchall()
   - The code fails as psycopg2.DatabaseError: error with status PGRES_TUPLES_OK and no message from the libpq.
   
   ### Environment
   
   (please complete the following information):
   
   - superset version: `0.36`
   - python version: `3.7.6`
   
   We use postgres version 9.2 with the celery running in kubernetes within docker container.
   Celery has redis configured as Backend. Below are few python module with their respective version
   
   celery==4.4.2
   SQLAlchemy==1.3.17
   Flask==1.1.2
   Flask-AppBuilder==2.3.4
   kombu==4.6.8
   psycopg2==2.8.5
   redis == 3.5.0
   
   ### Checklist
   
   Make sure to follow these steps before submitting your issue - thank you!
   
   - [ * ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
   - [ x ] I have reproduced the issue with at least the latest released version of superset.
   - [ * ] I have checked the issue tracker for the same issue and I haven't found one similar.
   
   ### Additional context
   
   1. For the superset version we are using is 0.36 release which we do not have any plans to upgrade as of now.
   2. The main use is to fetch details from postgres table that are not currently part of superset model architecture and process the same every 5min to 1 hour which takes around 20 minutes to get completed, for which trying to create a celery job.
   3. Already went through the PR request :- [GitHub]https://github.com/apache/superset/pull/13350, which could be a solution, but unsure how to approach with the changes as version cannot be upgarded to 1.1
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

[GitHub] [superset] Animesh3193 closed issue #15255: DB Connection pool throws error when executing raw sql in Celery job

[GitBox <gi...@apache.org>]

Animesh3193 closed issue #15255:
URL: https://github.com/apache/superset/issues/15255


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org