You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@airavata.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2019/01/16 20:12:00 UTC
[jira] [Commented] (AIRAVATA-2692) Django: hitting a Thrift 15
second timeout with large file uploads
[ https://issues.apache.org/jira/browse/AIRAVATA-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744421#comment-16744421 ]
ASF subversion and git services commented on AIRAVATA-2692:
-----------------------------------------------------------
Commit dabc08f52b938fb5c8f6e29d849fa3bc4a6811f7 in airavata-django-portal's branch refs/heads/master from Marcus Christie
[ https://gitbox.apache.org/repos/asf?p=airavata-django-portal.git;h=dabc08f ]
AIRAVATA-2692 Switch to thrift client connection pool
> Django: hitting a Thrift 15 second timeout with large file uploads
> ------------------------------------------------------------------
>
> Key: AIRAVATA-2692
> URL: https://issues.apache.org/jira/browse/AIRAVATA-2692
> Project: Airavata
> Issue Type: Bug
> Reporter: Marcus Christie
> Assignee: Marcus Christie
> Priority: Major
>
> For some reason it fails to get the project when doing a file upload
> {noformat}
> [Fri Feb 23 03:46:37.783086 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] [2018-02-23 03:46:37,782 django_airavata.apps.api.views:652 ERROR] Failed to upload file
> [Fri Feb 23 03:46:37.783108 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] Traceback (most recent call last):
> [Fri Feb 23 03:46:37.783113 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] File "/var/www/portals/django-seagrid/django-airavata-gateway/django_airavata/apps/api/views.py", line 642, in upload_input_file
> [Fri Feb 23 03:46:37.783118 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] request.authz_token, project_id)
> [Fri Feb 23 03:46:37.783122 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] File "/var/www/portals/django-seagrid/django-airavata-gateway/airavata/api/Airavata.py", line 4733, in getProject
> [Fri Feb 23 03:46:37.783126 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] return self.recv_getProject()
> [Fri Feb 23 03:46:37.783130 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] File "/var/www/portals/django-seagrid/django-airavata-gateway/airavata/api/Airavata.py", line 4746, in recv_getProject
> [Fri Feb 23 03:46:37.783134 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] (fname, mtype, rseqid) = iprot.readMessageBegin()
> [Fri Feb 23 03:46:37.783138 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] File "/var/www/portals/django-seagrid/venv/lib64/python3.4/site-packages/thrift/protocol/TBinaryProtocol.py", line 134, in readMessageBegin
> [Fri Feb 23 03:46:37.783152 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] sz = self.readI32()
> [Fri Feb 23 03:46:37.783185 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] File "/var/www/portals/django-seagrid/venv/lib64/python3.4/site-packages/thrift/protocol/TBinaryProtocol.py", line 217, in readI32
> [Fri Feb 23 03:46:37.783190 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] buff = self.trans.readAll(4)
> [Fri Feb 23 03:46:37.783194 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] File "/var/www/portals/django-seagrid/venv/lib64/python3.4/site-packages/thrift/transport/TTransport.py", line 60, in readAll
> [Fri Feb 23 03:46:37.783198 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] chunk = self.read(sz - have)
> [Fri Feb 23 03:46:37.783202 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] File "/var/www/portals/django-seagrid/venv/lib64/python3.4/site-packages/thrift/transport/TTransport.py", line 161, in read
> [Fri Feb 23 03:46:37.783206 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] self.__rbuf = BufferIO(self.__trans.read(max(sz, self.__rbuf_size)))
> [Fri Feb 23 03:46:37.783210 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] File "/var/www/portals/django-seagrid/venv/lib64/python3.4/site-packages/thrift/transport/TSocket.py", line 132, in read
> [Fri Feb 23 03:46:37.783239 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] message='TSocket read 0 bytes')
> [Fri Feb 23 03:46:37.783251 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341] thrift.transport.TTransport.TTransportException: TSocket read 0 bytes
> [Fri Feb 23 03:46:37.783263 2018] [wsgi:error] [pid 5145] [remote 68.45.52.105:56341]
> {noformat}
> This is hitting a 15 second timeout. Not sure where it is at.
> Maybe in mod_wsgi?
> https://groups.google.com/forum/#!topic/modwsgi/zuhFREjOE8M
> http://modwsgi.readthedocs.io/en/master/configuration-directives/WSGIDaemonProcess.html?highlight=timeout
> I know it is a 15 second timeout thing and not related to the size of the file. If I put {{time.sleep(15)}} in my Django view handler and upload a very small file it triggers the same Thrift error. Weird that it causes Thrift to generate this error.
> h5. TODO
> * [ ] integrate Thrift connection pool (https://github.com/Thriftpy/thrift_connector)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)