You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by Suresh Marru <sm...@apache.org> on 2015/03/18 19:13:45 UTC
[DISCUSS] Migrate to thrift 0.9.2 and multiplex API's
Hi All,
Airavata API grew larger which causes an issue with handler classes and the service is also getting unmanageable. Latest thrift 0.9.2 release supports multiplexing for all the clients [1]. I did a quick POC to verify it indeed works well Java, PHP, CPP and Python [2] [3].
How about we start migrating thrift 0.9.2? and also in the process multiplex API’s? I will start a refactoring in Airavata-Labs repo[4] to verify if its really helps our issues.
Suresh
[1] - https://issues.apache.org/jira/browse/THRIFT-1915 <https://issues.apache.org/jira/browse/THRIFT-1915>
[2] - https://issues.apache.org/jira/browse/AIRAVATA-1632 <https://issues.apache.org/jira/browse/AIRAVATA-1632>
[3] - https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api>
[4] - https://github.com/apache/airavata-labs <https://github.com/apache/airavata-labs>
Re: [DISCUSS] Migrate to thrift 0.9.2 and multiplex API's
Posted by Suresh Marru <sm...@apache.org>.
Hi All,
I moved the master to use thrift 0.9.2. The integration tests build fine, but lets please test more and make sure there are no other issues.
Lets to the changes incrementally so we can make quick progress to 0.16. We will need to confirm all the Airavata component to data models, I will send a separate thread on this start once we draft the models.
Suresh
> On Jun 9, 2015, at 7:48 AM, Suresh Marru <sm...@apache.org> wrote:
>
> Hi All,
>
> Here is a draft of the multiplex API blocks, please comment on the list or preferably on the JIRA issue [1]
>
> airavata-super-admin (targeted for Airavata Super Admins to create and manage gateways, and any other API methods requiring super admin credentials)
> gateway-management (manage gateway credentials, gateway resource profile)
> application-execution (previous experiment functions made explicit for single application)
> workflow-execution
> app-catalog
> resource-catalog (splitting resources from app catalog so availability, health, queue information can be managed efficiently).
>
> Suresh
> [1] - https://issues.apache.org/jira/browse/AIRAVATA-1637 <https://issues.apache.org/jira/browse/AIRAVATA-1637>
>
>
>> On Jun 5, 2015, at 3:59 PM, Suresh Marru <smarru@apache.org <ma...@apache.org>> wrote:
>>
>> Hi All,
>>
>> We are good to move to thrift 0.9.2 for master. On hip chat I suggested there is a blocker with multiple suppress warnings, but that was added from airavata thrift shell script and not thrift code. Thats cleared now. There are no blockers for 0.9.2. Lets switch to it soon.
>>
>> Suresh
>>
>>> On Jun 1, 2015, at 2:08 PM, Suresh Marru <smarru@apache.org <ma...@apache.org>> wrote:
>>>
>>> Hi All,
>>>
>>> If there are no objections I shall upgrade 0.16 (latest master) to thrift 0.9.2 and start multiplexing API’s as discussed below.
>>>
>>> Hi Kenneth,
>>>
>>> Sorry I missed to answer your query earlier, please see below:
>>>
>>>> On Mar 18, 2015, at 2:38 PM, K Yoshimoto <kenneth@sdsc.edu <ma...@sdsc.edu>> wrote:
>>>>
>>>> What are multiplex APIs?
>>>
>>> Once we write API functions in the thrift IDL, we add the implementations into the server side skeletons. As the API methods grow, the server implementation becomes unmanageable. So can break down the API into small categories. But this will force us to run a server for each of this API blocks. This will be very cumbersome for clients to figure our which service to use for which API methods. Thrift multi-plexing provides us a way to run multiple API services in one service. So the code managed in nice modular ways but the service is one which multiplexes into multiple implementations.
>>>
>>>> Does this affect how a client is written to use a thrift-generated API?
>>>
>>> Yes there are subtle changes but they are very minor and they will be absorbed in the Airavata Client wrapper provided for all supported languages. I can walk through Airavata mock up examples below, or you might find a better example in the thrift python patch at - https://issues.apache.org/jira/browse/THRIFT-1914 <https://issues.apache.org/jira/browse/THRIFT-1914>
>>>
>>> Suresh
>>>
>>>> On Wed, Mar 18, 2015 at 02:13:45PM -0400, Suresh Marru wrote:
>>>>> Hi All,
>>>>>
>>>>> Airavata API grew larger which causes an issue with handler classes and the service is also getting unmanageable. Latest thrift 0.9.2 release supports multiplexing for all the clients [1]. I did a quick POC to verify it indeed works well Java, PHP, CPP and Python [2] [3].
>>>>>
>>>>> How about we start migrating thrift 0.9.2? and also in the process multiplex API’s? I will start a refactoring in Airavata-Labs repo[4] to verify if its really helps our issues.
>>>>>
>>>>> Suresh
>>>>> [1] - https://issues.apache.org/jira/browse/THRIFT-1915 <https://issues.apache.org/jira/browse/THRIFT-1915> <https://issues.apache.org/jira/browse/THRIFT-1915 <https://issues.apache.org/jira/browse/THRIFT-1915>>
>>>>> [2] - https://issues.apache.org/jira/browse/AIRAVATA-1632 <https://issues.apache.org/jira/browse/AIRAVATA-1632> <https://issues.apache.org/jira/browse/AIRAVATA-1632 <https://issues.apache.org/jira/browse/AIRAVATA-1632>>
>>>>> [3] - https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api> <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api>>
>>>>> [4] - https://github.com/apache/airavata-labs <https://github.com/apache/airavata-labs> <https://github.com/apache/airavata-labs <https://github.com/apache/airavata-labs>>
>>>>>
>>>
>>
>
Re: [DISCUSS] Migrate to thrift 0.9.2 and multiplex API's
Posted by Suresh Marru <sm...@apache.org>.
Hi All,
Here is a draft of the multiplex API blocks, please comment on the list or preferably on the JIRA issue [1]
airavata-super-admin (targeted for Airavata Super Admins to create and manage gateways, and any other API methods requiring super admin credentials)
gateway-management (manage gateway credentials, gateway resource profile)
application-execution (previous experiment functions made explicit for single application)
workflow-execution
app-catalog
resource-catalog (splitting resources from app catalog so availability, health, queue information can be managed efficiently).
Suresh
[1] - https://issues.apache.org/jira/browse/AIRAVATA-1637 <https://issues.apache.org/jira/browse/AIRAVATA-1637>
> On Jun 5, 2015, at 3:59 PM, Suresh Marru <sm...@apache.org> wrote:
>
> Hi All,
>
> We are good to move to thrift 0.9.2 for master. On hip chat I suggested there is a blocker with multiple suppress warnings, but that was added from airavata thrift shell script and not thrift code. Thats cleared now. There are no blockers for 0.9.2. Lets switch to it soon.
>
> Suresh
>
>> On Jun 1, 2015, at 2:08 PM, Suresh Marru <smarru@apache.org <ma...@apache.org>> wrote:
>>
>> Hi All,
>>
>> If there are no objections I shall upgrade 0.16 (latest master) to thrift 0.9.2 and start multiplexing API’s as discussed below.
>>
>> Hi Kenneth,
>>
>> Sorry I missed to answer your query earlier, please see below:
>>
>>> On Mar 18, 2015, at 2:38 PM, K Yoshimoto <kenneth@sdsc.edu <ma...@sdsc.edu>> wrote:
>>>
>>> What are multiplex APIs?
>>
>> Once we write API functions in the thrift IDL, we add the implementations into the server side skeletons. As the API methods grow, the server implementation becomes unmanageable. So can break down the API into small categories. But this will force us to run a server for each of this API blocks. This will be very cumbersome for clients to figure our which service to use for which API methods. Thrift multi-plexing provides us a way to run multiple API services in one service. So the code managed in nice modular ways but the service is one which multiplexes into multiple implementations.
>>
>>> Does this affect how a client is written to use a thrift-generated API?
>>
>> Yes there are subtle changes but they are very minor and they will be absorbed in the Airavata Client wrapper provided for all supported languages. I can walk through Airavata mock up examples below, or you might find a better example in the thrift python patch at - https://issues.apache.org/jira/browse/THRIFT-1914 <https://issues.apache.org/jira/browse/THRIFT-1914>
>>
>> Suresh
>>
>>> On Wed, Mar 18, 2015 at 02:13:45PM -0400, Suresh Marru wrote:
>>>> Hi All,
>>>>
>>>> Airavata API grew larger which causes an issue with handler classes and the service is also getting unmanageable. Latest thrift 0.9.2 release supports multiplexing for all the clients [1]. I did a quick POC to verify it indeed works well Java, PHP, CPP and Python [2] [3].
>>>>
>>>> How about we start migrating thrift 0.9.2? and also in the process multiplex API’s? I will start a refactoring in Airavata-Labs repo[4] to verify if its really helps our issues.
>>>>
>>>> Suresh
>>>> [1] - https://issues.apache.org/jira/browse/THRIFT-1915 <https://issues.apache.org/jira/browse/THRIFT-1915> <https://issues.apache.org/jira/browse/THRIFT-1915 <https://issues.apache.org/jira/browse/THRIFT-1915>>
>>>> [2] - https://issues.apache.org/jira/browse/AIRAVATA-1632 <https://issues.apache.org/jira/browse/AIRAVATA-1632> <https://issues.apache.org/jira/browse/AIRAVATA-1632 <https://issues.apache.org/jira/browse/AIRAVATA-1632>>
>>>> [3] - https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api> <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api>>
>>>> [4] - https://github.com/apache/airavata-labs <https://github.com/apache/airavata-labs> <https://github.com/apache/airavata-labs <https://github.com/apache/airavata-labs>>
>>>>
>>
>
Re: [DISCUSS] Migrate to thrift 0.9.2 and multiplex API's
Posted by Suresh Marru <sm...@apache.org>.
Hi All,
We are good to move to thrift 0.9.2 for master. On hip chat I suggested there is a blocker with multiple suppress warnings, but that was added from airavata thrift shell script and not thrift code. Thats cleared now. There are no blockers for 0.9.2. Lets switch to it soon.
Suresh
> On Jun 1, 2015, at 2:08 PM, Suresh Marru <sm...@apache.org> wrote:
>
> Hi All,
>
> If there are no objections I shall upgrade 0.16 (latest master) to thrift 0.9.2 and start multiplexing API’s as discussed below.
>
> Hi Kenneth,
>
> Sorry I missed to answer your query earlier, please see below:
>
>> On Mar 18, 2015, at 2:38 PM, K Yoshimoto <kenneth@sdsc.edu <ma...@sdsc.edu>> wrote:
>>
>> What are multiplex APIs?
>
> Once we write API functions in the thrift IDL, we add the implementations into the server side skeletons. As the API methods grow, the server implementation becomes unmanageable. So can break down the API into small categories. But this will force us to run a server for each of this API blocks. This will be very cumbersome for clients to figure our which service to use for which API methods. Thrift multi-plexing provides us a way to run multiple API services in one service. So the code managed in nice modular ways but the service is one which multiplexes into multiple implementations.
>
>> Does this affect how a client is written to use a thrift-generated API?
>
> Yes there are subtle changes but they are very minor and they will be absorbed in the Airavata Client wrapper provided for all supported languages. I can walk through Airavata mock up examples below, or you might find a better example in the thrift python patch at - https://issues.apache.org/jira/browse/THRIFT-1914 <https://issues.apache.org/jira/browse/THRIFT-1914>
>
> Suresh
>
>> On Wed, Mar 18, 2015 at 02:13:45PM -0400, Suresh Marru wrote:
>>> Hi All,
>>>
>>> Airavata API grew larger which causes an issue with handler classes and the service is also getting unmanageable. Latest thrift 0.9.2 release supports multiplexing for all the clients [1]. I did a quick POC to verify it indeed works well Java, PHP, CPP and Python [2] [3].
>>>
>>> How about we start migrating thrift 0.9.2? and also in the process multiplex API’s? I will start a refactoring in Airavata-Labs repo[4] to verify if its really helps our issues.
>>>
>>> Suresh
>>> [1] - https://issues.apache.org/jira/browse/THRIFT-1915 <https://issues.apache.org/jira/browse/THRIFT-1915> <https://issues.apache.org/jira/browse/THRIFT-1915 <https://issues.apache.org/jira/browse/THRIFT-1915>>
>>> [2] - https://issues.apache.org/jira/browse/AIRAVATA-1632 <https://issues.apache.org/jira/browse/AIRAVATA-1632> <https://issues.apache.org/jira/browse/AIRAVATA-1632 <https://issues.apache.org/jira/browse/AIRAVATA-1632>>
>>> [3] - https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api> <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api>>
>>> [4] - https://github.com/apache/airavata-labs <https://github.com/apache/airavata-labs> <https://github.com/apache/airavata-labs <https://github.com/apache/airavata-labs>>
>>>
>
Re: [DISCUSS] Migrate to thrift 0.9.2 and multiplex API's
Posted by Suresh Marru <sm...@apache.org>.
Hi All,
If there are no objections I shall upgrade 0.16 (latest master) to thrift 0.9.2 and start multiplexing API’s as discussed below.
Hi Kenneth,
Sorry I missed to answer your query earlier, please see below:
> On Mar 18, 2015, at 2:38 PM, K Yoshimoto <ke...@sdsc.edu> wrote:
>
> What are multiplex APIs?
Once we write API functions in the thrift IDL, we add the implementations into the server side skeletons. As the API methods grow, the server implementation becomes unmanageable. So can break down the API into small categories. But this will force us to run a server for each of this API blocks. This will be very cumbersome for clients to figure our which service to use for which API methods. Thrift multi-plexing provides us a way to run multiple API services in one service. So the code managed in nice modular ways but the service is one which multiplexes into multiple implementations.
> Does this affect how a client is written to use a thrift-generated API?
Yes there are subtle changes but they are very minor and they will be absorbed in the Airavata Client wrapper provided for all supported languages. I can walk through Airavata mock up examples below, or you might find a better example in the thrift python patch at - https://issues.apache.org/jira/browse/THRIFT-1914 <https://issues.apache.org/jira/browse/THRIFT-1914>
Suresh
> On Wed, Mar 18, 2015 at 02:13:45PM -0400, Suresh Marru wrote:
>> Hi All,
>>
>> Airavata API grew larger which causes an issue with handler classes and the service is also getting unmanageable. Latest thrift 0.9.2 release supports multiplexing for all the clients [1]. I did a quick POC to verify it indeed works well Java, PHP, CPP and Python [2] [3].
>>
>> How about we start migrating thrift 0.9.2? and also in the process multiplex API’s? I will start a refactoring in Airavata-Labs repo[4] to verify if its really helps our issues.
>>
>> Suresh
>> [1] - https://issues.apache.org/jira/browse/THRIFT-1915 <https://issues.apache.org/jira/browse/THRIFT-1915>
>> [2] - https://issues.apache.org/jira/browse/AIRAVATA-1632 <https://issues.apache.org/jira/browse/AIRAVATA-1632>
>> [3] - https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api>
>> [4] - https://github.com/apache/airavata-labs <https://github.com/apache/airavata-labs>
>>
Re: [DISCUSS] Migrate to thrift 0.9.2 and multiplex API's
Posted by K Yoshimoto <ke...@sdsc.edu>.
What are multiplex APIs? Does this affect how a client is
written to use a thrift-generated API?
On Wed, Mar 18, 2015 at 02:13:45PM -0400, Suresh Marru wrote:
> Hi All,
>
> Airavata API grew larger which causes an issue with handler classes and the service is also getting unmanageable. Latest thrift 0.9.2 release supports multiplexing for all the clients [1]. I did a quick POC to verify it indeed works well Java, PHP, CPP and Python [2] [3].
>
> How about we start migrating thrift 0.9.2? and also in the process multiplex API’s? I will start a refactoring in Airavata-Labs repo[4] to verify if its really helps our issues.
>
> Suresh
> [1] - https://issues.apache.org/jira/browse/THRIFT-1915 <https://issues.apache.org/jira/browse/THRIFT-1915>
> [2] - https://issues.apache.org/jira/browse/AIRAVATA-1632 <https://issues.apache.org/jira/browse/AIRAVATA-1632>
> [3] - https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api <https://github.com/apache/airavata-sandbox/tree/master/airavata-mock-multiplexed-api>
> [4] - https://github.com/apache/airavata-labs <https://github.com/apache/airavata-labs>
>