You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@thrift.apache.org by Deeraj <de...@tataelxsi.co.in> on 2015/04/20 06:27:18 UTC
Thrift IDL underlying working
Hi,
I would like to know the working of Thrift IDL, the underlying
architecture.I know that Thrift is using socket for communication.
I want to know whether if the server side is a bit slow to read the
data, then whether there would be a data loss or will it be there in the
queue ?
--
Thanks & Regards,
Deeraj Soman
Senior Engineer - TBU | Embedded Product Design
TATA ELXSI
Technopark CampusKariyavattomTrivandrum 695 581India
Phone:+91 471 666 1124
Mobile: +91 9895703852
*www.tataelxsi.com*
<https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f>
Re: Thrift IDL underlying working
Posted by Deeraj <de...@tataelxsi.co.in>.
Thanks Henrique.
On 4/21/2015 10:49 AM, Henrique Mendonça wrote:
> AFAIK all provided socket transports use TCP only.
> UDP can be done (implementing yourself) but it generally only supports
> oneway methods...
>
> On 21 April 2015 at 14:46, Deeraj <de...@tataelxsi.co.in> wrote:
>
>> Ya it clarified a lot. I have one more doubt. In Thrift white paper it
>> says about TCP sockets.
>> Kindly confirm whether the communication is happening over the TCP ? Or
>> can is it in UDP ?
>>
>>
>> On 4/20/2015 3:31 PM, Henrique Mendonça wrote:
>>
>>> It will depend a lot on which language and server you are using but
>>> normally the client will send the whole serialized data at once and wait
>>> for the server return. There is no data lost in the void :)
>>> There is also no streaming in Thrift, so the data to be sent is well
>>> defined and server knows exactly what to expect.
>>>
>>> If you need to implement a processing queue for a very slow server (or
>>> very
>>> fast clients) I'd recommend you to do it outside the thrift server, so
>>> that
>>> the thrift server thread[s] are free to process the upcoming requests.
>>>
>>> I hope it clarified a bit
>>>
>>> Best,
>>> Henrique
>>>
>>> On 20 April 2015 at 19:08, Deeraj <de...@tataelxsi.co.in> wrote:
>>>
>>> Let me elaborate.
>>>> Suppose my server and clients are up and running. Ideally whatever data
>>>> the client transmits, the server will read it from the transport layer.
>>>> In
>>>> such a case there is no data loss.
>>>>
>>>> Suppose the server is reading the data in a slower rate compared to the
>>>> client's transmission rate, there is a chance for data loss.
>>>> Is such a case possible when we implement the server and client using
>>>> thrift ?
>>>>
>>>> In case of message queue, the lost data will be held in the queue and the
>>>> same can be read later. Is there any such mechanism in thrift server/
>>>> client to read the lost data if such a case exists ?
>>>>
>>>> I hope the question is clear now
>>>>
>>>>
>>>>
>>>>
>>>> On 4/20/2015 2:28 PM, Henrique Mendonça wrote:
>>>>
>>>> Hi Deeraj,
>>>>> Sorry, but you might need to elaborate a bit more your question. I am
>>>>> not
>>>>> sure which queue or reading data you are talking about. Generally, there
>>>>> is
>>>>> no partially transmitted data, if that's what you're referring.
>>>>>
>>>>> Regards,
>>>>> Henrique
>>>>>
>>>>> ps.: sockets are used on the transport level
>>>>>
>>>>>
>>>>> On 20 April 2015 at 15:06, Deeraj <de...@tataelxsi.co.in> wrote:
>>>>>
>>>>> Hi Henrique,
>>>>>
>>>>>> Suppose if I am using socket as my protocol. Will the data be there in
>>>>>> the
>>>>>> queue if the server fails to read the data ?
>>>>>>
>>>>>> I have already implemented the server and client by reading the
>>>>>> tutorial.
>>>>>>
>>>>>>
>>>>>> On 4/20/2015 10:19 AM, Henrique Mendonça wrote:
>>>>>>
>>>>>> Hi Deeraj,
>>>>>>
>>>>>>> Apache Thrift can use sockets but need not. You are free to choose
>>>>>>> from
>>>>>>> a
>>>>>>> list of supported transports and protocols for each target language.
>>>>>>> You
>>>>>>> can even use it only for data serialization if you want.
>>>>>>>
>>>>>>> The documentation is unfortunately still a bit sparse but you can get
>>>>>>> the
>>>>>>> idea following the home page:
>>>>>>>
>>>>>>>
>>>>>>> *Getting Started...*
>>>>>>>
>>>>>>> - *Writing a .thrift file*
>>>>>>>
>>>>>>> *After the Thrift compiler is installed you will need to create
>>>>>>> a
>>>>>>> thrift
>>>>>>> file. This file is an interface definition
>>>>>>> <https://thrift.apache.org/docs/idl>made up of thrift types
>>>>>>> <https://thrift.apache.org/docs/types> and Services. The
>>>>>>> services
>>>>>>> you
>>>>>>> define in this file are implemented by the server and are
>>>>>>> called by
>>>>>>> any
>>>>>>> clients. The Thrift compiler is used to generate your Thrift
>>>>>>> File
>>>>>>> into
>>>>>>> source code which is used by the different client libraries and
>>>>>>> the
>>>>>>> server
>>>>>>> you write. To generate the source from a thrift file run*
>>>>>>>
>>>>>>> * thrift --gen <language> <Thrift filename>
>>>>>>> *
>>>>>>>
>>>>>>> *The sample tutorial.thrift file used for all the client and
>>>>>>> server
>>>>>>> tutorials can be found here
>>>>>>> <
>>>>>>>
>>>>>>>
>>>>>>> https://git-wip-us.apache.org/repos/asf/thrift/?p=thrift.git;a=tree;f=tutorial
>>>>>>>
>>>>>>> .*
>>>>>>>> *To learn more about Apache Thrift Read the Whitepaper
>>>>>>> <https://thrift.apache.org/static/files/thrift-20070401.pdf>*
>>>>>>>
>>>>>>>
>>>>>>> The above mentioned white paper is a good start, Randy's book is also
>>>>>>> very
>>>>>>> handy: http://www.manning.com/abernethy/
>>>>>>> His first [free] chapter might cover some of your questions.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Henrique
>>>>>>>
>>>>>>>
>>>>>>> On 20 April 2015 at 14:27, Deeraj <de...@tataelxsi.co.in> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I would like to know the working of Thrift IDL, the underlying
>>>>>>>> architecture.I know that Thrift is using socket for communication.
>>>>>>>>
>>>>>>>> I want to know whether if the server side is a bit slow to read the
>>>>>>>> data,
>>>>>>>> then whether there would be a data loss or will it be there in the
>>>>>>>> queue
>>>>>>>> ?
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks & Regards,
>>>>>>>> Deeraj Soman
>>>>>>>> Senior Engineer - TBU | Embedded Product Design
>>>>>>>> TATA ELXSI
>>>>>>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>>>>>>> Phone:+91 471 666 1124
>>>>>>>> Mobile: +91 9895703852
>>>>>>>> *www.tataelxsi.com* <
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>> Thanks & Regards,
>>>>>> Deeraj Soman
>>>>>> Senior Engineer - TBU | Embedded Product Design
>>>>>> TATA ELXSI
>>>>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>>>>> Phone:+91 471 666 1124
>>>>>> Mobile: +91 9895703852
>>>>>> *www.tataelxsi.com* <
>>>>>>
>>>>>>
>>>>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>> Thanks & Regards,
>>>> Deeraj Soman
>>>> Senior Engineer - TBU | Embedded Product Design
>>>> TATA ELXSI
>>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>>> Phone:+91 471 666 1124
>>>> Mobile: +91 9895703852
>>>> *www.tataelxsi.com* <
>>>>
>>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>>
>>>>
>>>>
>> --
>> Thanks & Regards,
>> Deeraj Soman
>> Senior Engineer - TBU | Embedded Product Design
>> TATA ELXSI
>> Technopark CampusKariyavattomTrivandrum 695 581India
>> Phone:+91 471 666 1124
>> Mobile: +91 9895703852
>> *www.tataelxsi.com* <
>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>
>>
--
Thanks & Regards,
Deeraj Soman
Senior Engineer - TBU | Embedded Product Design
TATA ELXSI
Technopark CampusKariyavattomTrivandrum 695 581India
Phone:+91 471 666 1124
Mobile: +91 9895703852
*www.tataelxsi.com*
<https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f>
Re: Thrift IDL underlying working
Posted by Henrique Mendonça <hm...@gmail.com>.
AFAIK all provided socket transports use TCP only.
UDP can be done (implementing yourself) but it generally only supports
oneway methods...
On 21 April 2015 at 14:46, Deeraj <de...@tataelxsi.co.in> wrote:
> Ya it clarified a lot. I have one more doubt. In Thrift white paper it
> says about TCP sockets.
> Kindly confirm whether the communication is happening over the TCP ? Or
> can is it in UDP ?
>
>
> On 4/20/2015 3:31 PM, Henrique Mendonça wrote:
>
>> It will depend a lot on which language and server you are using but
>> normally the client will send the whole serialized data at once and wait
>> for the server return. There is no data lost in the void :)
>> There is also no streaming in Thrift, so the data to be sent is well
>> defined and server knows exactly what to expect.
>>
>> If you need to implement a processing queue for a very slow server (or
>> very
>> fast clients) I'd recommend you to do it outside the thrift server, so
>> that
>> the thrift server thread[s] are free to process the upcoming requests.
>>
>> I hope it clarified a bit
>>
>> Best,
>> Henrique
>>
>> On 20 April 2015 at 19:08, Deeraj <de...@tataelxsi.co.in> wrote:
>>
>> Let me elaborate.
>>>
>>> Suppose my server and clients are up and running. Ideally whatever data
>>> the client transmits, the server will read it from the transport layer.
>>> In
>>> such a case there is no data loss.
>>>
>>> Suppose the server is reading the data in a slower rate compared to the
>>> client's transmission rate, there is a chance for data loss.
>>> Is such a case possible when we implement the server and client using
>>> thrift ?
>>>
>>> In case of message queue, the lost data will be held in the queue and the
>>> same can be read later. Is there any such mechanism in thrift server/
>>> client to read the lost data if such a case exists ?
>>>
>>> I hope the question is clear now
>>>
>>>
>>>
>>>
>>> On 4/20/2015 2:28 PM, Henrique Mendonça wrote:
>>>
>>> Hi Deeraj,
>>>>
>>>> Sorry, but you might need to elaborate a bit more your question. I am
>>>> not
>>>> sure which queue or reading data you are talking about. Generally, there
>>>> is
>>>> no partially transmitted data, if that's what you're referring.
>>>>
>>>> Regards,
>>>> Henrique
>>>>
>>>> ps.: sockets are used on the transport level
>>>>
>>>>
>>>> On 20 April 2015 at 15:06, Deeraj <de...@tataelxsi.co.in> wrote:
>>>>
>>>> Hi Henrique,
>>>>
>>>>> Suppose if I am using socket as my protocol. Will the data be there in
>>>>> the
>>>>> queue if the server fails to read the data ?
>>>>>
>>>>> I have already implemented the server and client by reading the
>>>>> tutorial.
>>>>>
>>>>>
>>>>> On 4/20/2015 10:19 AM, Henrique Mendonça wrote:
>>>>>
>>>>> Hi Deeraj,
>>>>>
>>>>>> Apache Thrift can use sockets but need not. You are free to choose
>>>>>> from
>>>>>> a
>>>>>> list of supported transports and protocols for each target language.
>>>>>> You
>>>>>> can even use it only for data serialization if you want.
>>>>>>
>>>>>> The documentation is unfortunately still a bit sparse but you can get
>>>>>> the
>>>>>> idea following the home page:
>>>>>>
>>>>>>
>>>>>> *Getting Started...*
>>>>>>
>>>>>> - *Writing a .thrift file*
>>>>>>
>>>>>> *After the Thrift compiler is installed you will need to create
>>>>>> a
>>>>>> thrift
>>>>>> file. This file is an interface definition
>>>>>> <https://thrift.apache.org/docs/idl>made up of thrift types
>>>>>> <https://thrift.apache.org/docs/types> and Services. The
>>>>>> services
>>>>>> you
>>>>>> define in this file are implemented by the server and are
>>>>>> called by
>>>>>> any
>>>>>> clients. The Thrift compiler is used to generate your Thrift
>>>>>> File
>>>>>> into
>>>>>> source code which is used by the different client libraries and
>>>>>> the
>>>>>> server
>>>>>> you write. To generate the source from a thrift file run*
>>>>>>
>>>>>> * thrift --gen <language> <Thrift filename>
>>>>>> *
>>>>>>
>>>>>> *The sample tutorial.thrift file used for all the client and
>>>>>> server
>>>>>> tutorials can be found here
>>>>>> <
>>>>>>
>>>>>>
>>>>>> https://git-wip-us.apache.org/repos/asf/thrift/?p=thrift.git;a=tree;f=tutorial
>>>>>>
>>>>>> .*
>>>>>>>
>>>>>>> *To learn more about Apache Thrift Read the Whitepaper
>>>>>> <https://thrift.apache.org/static/files/thrift-20070401.pdf>*
>>>>>>
>>>>>>
>>>>>> The above mentioned white paper is a good start, Randy's book is also
>>>>>> very
>>>>>> handy: http://www.manning.com/abernethy/
>>>>>> His first [free] chapter might cover some of your questions.
>>>>>>
>>>>>> Cheers,
>>>>>> Henrique
>>>>>>
>>>>>>
>>>>>> On 20 April 2015 at 14:27, Deeraj <de...@tataelxsi.co.in> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I would like to know the working of Thrift IDL, the underlying
>>>>>>> architecture.I know that Thrift is using socket for communication.
>>>>>>>
>>>>>>> I want to know whether if the server side is a bit slow to read the
>>>>>>> data,
>>>>>>> then whether there would be a data loss or will it be there in the
>>>>>>> queue
>>>>>>> ?
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thanks & Regards,
>>>>>>> Deeraj Soman
>>>>>>> Senior Engineer - TBU | Embedded Product Design
>>>>>>> TATA ELXSI
>>>>>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>>>>>> Phone:+91 471 666 1124
>>>>>>> Mobile: +91 9895703852
>>>>>>> *www.tataelxsi.com* <
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>> Thanks & Regards,
>>>>> Deeraj Soman
>>>>> Senior Engineer - TBU | Embedded Product Design
>>>>> TATA ELXSI
>>>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>>>> Phone:+91 471 666 1124
>>>>> Mobile: +91 9895703852
>>>>> *www.tataelxsi.com* <
>>>>>
>>>>>
>>>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>>>
>>>>>
>>>>>
>>>>> --
>>> Thanks & Regards,
>>> Deeraj Soman
>>> Senior Engineer - TBU | Embedded Product Design
>>> TATA ELXSI
>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>> Phone:+91 471 666 1124
>>> Mobile: +91 9895703852
>>> *www.tataelxsi.com* <
>>>
>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>
>>>
>>>
> --
> Thanks & Regards,
> Deeraj Soman
> Senior Engineer - TBU | Embedded Product Design
> TATA ELXSI
> Technopark CampusKariyavattomTrivandrum 695 581India
> Phone:+91 471 666 1124
> Mobile: +91 9895703852
> *www.tataelxsi.com* <
> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
> >
>
>
>
Re: Thrift IDL underlying working
Posted by Deeraj <de...@tataelxsi.co.in>.
Ya it clarified a lot. I have one more doubt. In Thrift white paper it
says about TCP sockets.
Kindly confirm whether the communication is happening over the TCP ? Or
can is it in UDP ?
On 4/20/2015 3:31 PM, Henrique Mendonça wrote:
> It will depend a lot on which language and server you are using but
> normally the client will send the whole serialized data at once and wait
> for the server return. There is no data lost in the void :)
> There is also no streaming in Thrift, so the data to be sent is well
> defined and server knows exactly what to expect.
>
> If you need to implement a processing queue for a very slow server (or very
> fast clients) I'd recommend you to do it outside the thrift server, so that
> the thrift server thread[s] are free to process the upcoming requests.
>
> I hope it clarified a bit
>
> Best,
> Henrique
>
> On 20 April 2015 at 19:08, Deeraj <de...@tataelxsi.co.in> wrote:
>
>> Let me elaborate.
>>
>> Suppose my server and clients are up and running. Ideally whatever data
>> the client transmits, the server will read it from the transport layer. In
>> such a case there is no data loss.
>>
>> Suppose the server is reading the data in a slower rate compared to the
>> client's transmission rate, there is a chance for data loss.
>> Is such a case possible when we implement the server and client using
>> thrift ?
>>
>> In case of message queue, the lost data will be held in the queue and the
>> same can be read later. Is there any such mechanism in thrift server/
>> client to read the lost data if such a case exists ?
>>
>> I hope the question is clear now
>>
>>
>>
>>
>> On 4/20/2015 2:28 PM, Henrique Mendonça wrote:
>>
>>> Hi Deeraj,
>>>
>>> Sorry, but you might need to elaborate a bit more your question. I am not
>>> sure which queue or reading data you are talking about. Generally, there
>>> is
>>> no partially transmitted data, if that's what you're referring.
>>>
>>> Regards,
>>> Henrique
>>>
>>> ps.: sockets are used on the transport level
>>>
>>>
>>> On 20 April 2015 at 15:06, Deeraj <de...@tataelxsi.co.in> wrote:
>>>
>>> Hi Henrique,
>>>> Suppose if I am using socket as my protocol. Will the data be there in
>>>> the
>>>> queue if the server fails to read the data ?
>>>>
>>>> I have already implemented the server and client by reading the tutorial.
>>>>
>>>>
>>>> On 4/20/2015 10:19 AM, Henrique Mendonça wrote:
>>>>
>>>> Hi Deeraj,
>>>>> Apache Thrift can use sockets but need not. You are free to choose from
>>>>> a
>>>>> list of supported transports and protocols for each target language. You
>>>>> can even use it only for data serialization if you want.
>>>>>
>>>>> The documentation is unfortunately still a bit sparse but you can get
>>>>> the
>>>>> idea following the home page:
>>>>>
>>>>>
>>>>> *Getting Started...*
>>>>>
>>>>> - *Writing a .thrift file*
>>>>>
>>>>> *After the Thrift compiler is installed you will need to create a
>>>>> thrift
>>>>> file. This file is an interface definition
>>>>> <https://thrift.apache.org/docs/idl>made up of thrift types
>>>>> <https://thrift.apache.org/docs/types> and Services. The services
>>>>> you
>>>>> define in this file are implemented by the server and are called by
>>>>> any
>>>>> clients. The Thrift compiler is used to generate your Thrift File
>>>>> into
>>>>> source code which is used by the different client libraries and the
>>>>> server
>>>>> you write. To generate the source from a thrift file run*
>>>>>
>>>>> * thrift --gen <language> <Thrift filename>
>>>>> *
>>>>>
>>>>> *The sample tutorial.thrift file used for all the client and server
>>>>> tutorials can be found here
>>>>> <
>>>>>
>>>>> https://git-wip-us.apache.org/repos/asf/thrift/?p=thrift.git;a=tree;f=tutorial
>>>>>
>>>>>> .*
>>>>>>
>>>>> *To learn more about Apache Thrift Read the Whitepaper
>>>>> <https://thrift.apache.org/static/files/thrift-20070401.pdf>*
>>>>>
>>>>>
>>>>> The above mentioned white paper is a good start, Randy's book is also
>>>>> very
>>>>> handy: http://www.manning.com/abernethy/
>>>>> His first [free] chapter might cover some of your questions.
>>>>>
>>>>> Cheers,
>>>>> Henrique
>>>>>
>>>>>
>>>>> On 20 April 2015 at 14:27, Deeraj <de...@tataelxsi.co.in> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>>> I would like to know the working of Thrift IDL, the underlying
>>>>>> architecture.I know that Thrift is using socket for communication.
>>>>>>
>>>>>> I want to know whether if the server side is a bit slow to read the
>>>>>> data,
>>>>>> then whether there would be a data loss or will it be there in the
>>>>>> queue
>>>>>> ?
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Thanks & Regards,
>>>>>> Deeraj Soman
>>>>>> Senior Engineer - TBU | Embedded Product Design
>>>>>> TATA ELXSI
>>>>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>>>>> Phone:+91 471 666 1124
>>>>>> Mobile: +91 9895703852
>>>>>> *www.tataelxsi.com* <
>>>>>>
>>>>>>
>>>>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>> Thanks & Regards,
>>>> Deeraj Soman
>>>> Senior Engineer - TBU | Embedded Product Design
>>>> TATA ELXSI
>>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>>> Phone:+91 471 666 1124
>>>> Mobile: +91 9895703852
>>>> *www.tataelxsi.com* <
>>>>
>>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>>
>>>>
>>>>
>> --
>> Thanks & Regards,
>> Deeraj Soman
>> Senior Engineer - TBU | Embedded Product Design
>> TATA ELXSI
>> Technopark CampusKariyavattomTrivandrum 695 581India
>> Phone:+91 471 666 1124
>> Mobile: +91 9895703852
>> *www.tataelxsi.com* <
>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>
>>
--
Thanks & Regards,
Deeraj Soman
Senior Engineer - TBU | Embedded Product Design
TATA ELXSI
Technopark CampusKariyavattomTrivandrum 695 581India
Phone:+91 471 666 1124
Mobile: +91 9895703852
*www.tataelxsi.com*
<https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f>
Re: Thrift IDL underlying working
Posted by Henrique Mendonça <hm...@gmail.com>.
It will depend a lot on which language and server you are using but
normally the client will send the whole serialized data at once and wait
for the server return. There is no data lost in the void :)
There is also no streaming in Thrift, so the data to be sent is well
defined and server knows exactly what to expect.
If you need to implement a processing queue for a very slow server (or very
fast clients) I'd recommend you to do it outside the thrift server, so that
the thrift server thread[s] are free to process the upcoming requests.
I hope it clarified a bit
Best,
Henrique
On 20 April 2015 at 19:08, Deeraj <de...@tataelxsi.co.in> wrote:
> Let me elaborate.
>
> Suppose my server and clients are up and running. Ideally whatever data
> the client transmits, the server will read it from the transport layer. In
> such a case there is no data loss.
>
> Suppose the server is reading the data in a slower rate compared to the
> client's transmission rate, there is a chance for data loss.
> Is such a case possible when we implement the server and client using
> thrift ?
>
> In case of message queue, the lost data will be held in the queue and the
> same can be read later. Is there any such mechanism in thrift server/
> client to read the lost data if such a case exists ?
>
> I hope the question is clear now
>
>
>
>
> On 4/20/2015 2:28 PM, Henrique Mendonça wrote:
>
>> Hi Deeraj,
>>
>> Sorry, but you might need to elaborate a bit more your question. I am not
>> sure which queue or reading data you are talking about. Generally, there
>> is
>> no partially transmitted data, if that's what you're referring.
>>
>> Regards,
>> Henrique
>>
>> ps.: sockets are used on the transport level
>>
>>
>> On 20 April 2015 at 15:06, Deeraj <de...@tataelxsi.co.in> wrote:
>>
>> Hi Henrique,
>>>
>>> Suppose if I am using socket as my protocol. Will the data be there in
>>> the
>>> queue if the server fails to read the data ?
>>>
>>> I have already implemented the server and client by reading the tutorial.
>>>
>>>
>>> On 4/20/2015 10:19 AM, Henrique Mendonça wrote:
>>>
>>> Hi Deeraj,
>>>>
>>>> Apache Thrift can use sockets but need not. You are free to choose from
>>>> a
>>>> list of supported transports and protocols for each target language. You
>>>> can even use it only for data serialization if you want.
>>>>
>>>> The documentation is unfortunately still a bit sparse but you can get
>>>> the
>>>> idea following the home page:
>>>>
>>>>
>>>> *Getting Started...*
>>>>
>>>> - *Writing a .thrift file*
>>>>
>>>> *After the Thrift compiler is installed you will need to create a
>>>> thrift
>>>> file. This file is an interface definition
>>>> <https://thrift.apache.org/docs/idl>made up of thrift types
>>>> <https://thrift.apache.org/docs/types> and Services. The services
>>>> you
>>>> define in this file are implemented by the server and are called by
>>>> any
>>>> clients. The Thrift compiler is used to generate your Thrift File
>>>> into
>>>> source code which is used by the different client libraries and the
>>>> server
>>>> you write. To generate the source from a thrift file run*
>>>>
>>>> * thrift --gen <language> <Thrift filename>
>>>> *
>>>>
>>>> *The sample tutorial.thrift file used for all the client and server
>>>> tutorials can be found here
>>>> <
>>>>
>>>> https://git-wip-us.apache.org/repos/asf/thrift/?p=thrift.git;a=tree;f=tutorial
>>>>
>>>>> .*
>>>>>
>>>>
>>>> *To learn more about Apache Thrift Read the Whitepaper
>>>> <https://thrift.apache.org/static/files/thrift-20070401.pdf>*
>>>>
>>>>
>>>> The above mentioned white paper is a good start, Randy's book is also
>>>> very
>>>> handy: http://www.manning.com/abernethy/
>>>> His first [free] chapter might cover some of your questions.
>>>>
>>>> Cheers,
>>>> Henrique
>>>>
>>>>
>>>> On 20 April 2015 at 14:27, Deeraj <de...@tataelxsi.co.in> wrote:
>>>>
>>>> Hi,
>>>>
>>>>> I would like to know the working of Thrift IDL, the underlying
>>>>> architecture.I know that Thrift is using socket for communication.
>>>>>
>>>>> I want to know whether if the server side is a bit slow to read the
>>>>> data,
>>>>> then whether there would be a data loss or will it be there in the
>>>>> queue
>>>>> ?
>>>>>
>>>>>
>>>>> --
>>>>> Thanks & Regards,
>>>>> Deeraj Soman
>>>>> Senior Engineer - TBU | Embedded Product Design
>>>>> TATA ELXSI
>>>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>>>> Phone:+91 471 666 1124
>>>>> Mobile: +91 9895703852
>>>>> *www.tataelxsi.com* <
>>>>>
>>>>>
>>>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>>>
>>>>>
>>>>>
>>>>> --
>>> Thanks & Regards,
>>> Deeraj Soman
>>> Senior Engineer - TBU | Embedded Product Design
>>> TATA ELXSI
>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>> Phone:+91 471 666 1124
>>> Mobile: +91 9895703852
>>> *www.tataelxsi.com* <
>>>
>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>
>>>
>>>
> --
> Thanks & Regards,
> Deeraj Soman
> Senior Engineer - TBU | Embedded Product Design
> TATA ELXSI
> Technopark CampusKariyavattomTrivandrum 695 581India
> Phone:+91 471 666 1124
> Mobile: +91 9895703852
> *www.tataelxsi.com* <
> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
> >
>
>
>
Re: Thrift IDL underlying working
Posted by Deeraj <de...@tataelxsi.co.in>.
Let me elaborate.
Suppose my server and clients are up and running. Ideally whatever data
the client transmits, the server will read it from the transport layer.
In such a case there is no data loss.
Suppose the server is reading the data in a slower rate compared to the
client's transmission rate, there is a chance for data loss.
Is such a case possible when we implement the server and client using
thrift ?
In case of message queue, the lost data will be held in the queue and
the same can be read later. Is there any such mechanism in thrift
server/ client to read the lost data if such a case exists ?
I hope the question is clear now
On 4/20/2015 2:28 PM, Henrique Mendonça wrote:
> Hi Deeraj,
>
> Sorry, but you might need to elaborate a bit more your question. I am not
> sure which queue or reading data you are talking about. Generally, there is
> no partially transmitted data, if that's what you're referring.
>
> Regards,
> Henrique
>
> ps.: sockets are used on the transport level
>
>
> On 20 April 2015 at 15:06, Deeraj <de...@tataelxsi.co.in> wrote:
>
>> Hi Henrique,
>>
>> Suppose if I am using socket as my protocol. Will the data be there in the
>> queue if the server fails to read the data ?
>>
>> I have already implemented the server and client by reading the tutorial.
>>
>>
>> On 4/20/2015 10:19 AM, Henrique Mendonça wrote:
>>
>>> Hi Deeraj,
>>>
>>> Apache Thrift can use sockets but need not. You are free to choose from a
>>> list of supported transports and protocols for each target language. You
>>> can even use it only for data serialization if you want.
>>>
>>> The documentation is unfortunately still a bit sparse but you can get the
>>> idea following the home page:
>>>
>>>
>>> *Getting Started...*
>>>
>>> - *Writing a .thrift file*
>>>
>>> *After the Thrift compiler is installed you will need to create a
>>> thrift
>>> file. This file is an interface definition
>>> <https://thrift.apache.org/docs/idl>made up of thrift types
>>> <https://thrift.apache.org/docs/types> and Services. The services you
>>> define in this file are implemented by the server and are called by
>>> any
>>> clients. The Thrift compiler is used to generate your Thrift File into
>>> source code which is used by the different client libraries and the
>>> server
>>> you write. To generate the source from a thrift file run*
>>>
>>> * thrift --gen <language> <Thrift filename>
>>> *
>>>
>>> *The sample tutorial.thrift file used for all the client and server
>>> tutorials can be found here
>>> <
>>> https://git-wip-us.apache.org/repos/asf/thrift/?p=thrift.git;a=tree;f=tutorial
>>>> .*
>>>
>>> *To learn more about Apache Thrift Read the Whitepaper
>>> <https://thrift.apache.org/static/files/thrift-20070401.pdf>*
>>>
>>>
>>> The above mentioned white paper is a good start, Randy's book is also very
>>> handy: http://www.manning.com/abernethy/
>>> His first [free] chapter might cover some of your questions.
>>>
>>> Cheers,
>>> Henrique
>>>
>>>
>>> On 20 April 2015 at 14:27, Deeraj <de...@tataelxsi.co.in> wrote:
>>>
>>> Hi,
>>>> I would like to know the working of Thrift IDL, the underlying
>>>> architecture.I know that Thrift is using socket for communication.
>>>>
>>>> I want to know whether if the server side is a bit slow to read the data,
>>>> then whether there would be a data loss or will it be there in the queue
>>>> ?
>>>>
>>>>
>>>> --
>>>> Thanks & Regards,
>>>> Deeraj Soman
>>>> Senior Engineer - TBU | Embedded Product Design
>>>> TATA ELXSI
>>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>>> Phone:+91 471 666 1124
>>>> Mobile: +91 9895703852
>>>> *www.tataelxsi.com* <
>>>>
>>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>>
>>>>
>>>>
>> --
>> Thanks & Regards,
>> Deeraj Soman
>> Senior Engineer - TBU | Embedded Product Design
>> TATA ELXSI
>> Technopark CampusKariyavattomTrivandrum 695 581India
>> Phone:+91 471 666 1124
>> Mobile: +91 9895703852
>> *www.tataelxsi.com* <
>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>
>>
--
Thanks & Regards,
Deeraj Soman
Senior Engineer - TBU | Embedded Product Design
TATA ELXSI
Technopark CampusKariyavattomTrivandrum 695 581India
Phone:+91 471 666 1124
Mobile: +91 9895703852
*www.tataelxsi.com*
<https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f>
Re: Thrift IDL underlying working
Posted by Henrique Mendonça <he...@apache.org>.
Hi Deeraj,
Sorry, but you might need to elaborate a bit more your question. I am not
sure which queue or reading data you are talking about. Generally, there is
no partially transmitted data, if that's what you're referring.
Regards,
Henrique
ps.: sockets are used on the transport level
On 20 April 2015 at 15:06, Deeraj <de...@tataelxsi.co.in> wrote:
> Hi Henrique,
>
> Suppose if I am using socket as my protocol. Will the data be there in the
> queue if the server fails to read the data ?
>
> I have already implemented the server and client by reading the tutorial.
>
>
> On 4/20/2015 10:19 AM, Henrique Mendonça wrote:
>
>> Hi Deeraj,
>>
>> Apache Thrift can use sockets but need not. You are free to choose from a
>> list of supported transports and protocols for each target language. You
>> can even use it only for data serialization if you want.
>>
>> The documentation is unfortunately still a bit sparse but you can get the
>> idea following the home page:
>>
>>
>> *Getting Started...*
>>
>> - *Writing a .thrift file*
>>
>> *After the Thrift compiler is installed you will need to create a
>> thrift
>> file. This file is an interface definition
>> <https://thrift.apache.org/docs/idl>made up of thrift types
>> <https://thrift.apache.org/docs/types> and Services. The services you
>> define in this file are implemented by the server and are called by
>> any
>> clients. The Thrift compiler is used to generate your Thrift File into
>> source code which is used by the different client libraries and the
>> server
>> you write. To generate the source from a thrift file run*
>>
>> * thrift --gen <language> <Thrift filename>
>> *
>>
>> *The sample tutorial.thrift file used for all the client and server
>> tutorials can be found here
>> <
>> https://git-wip-us.apache.org/repos/asf/thrift/?p=thrift.git;a=tree;f=tutorial
>> >.*
>>
>>
>> *To learn more about Apache Thrift Read the Whitepaper
>> <https://thrift.apache.org/static/files/thrift-20070401.pdf>*
>>
>>
>> The above mentioned white paper is a good start, Randy's book is also very
>> handy: http://www.manning.com/abernethy/
>> His first [free] chapter might cover some of your questions.
>>
>> Cheers,
>> Henrique
>>
>>
>> On 20 April 2015 at 14:27, Deeraj <de...@tataelxsi.co.in> wrote:
>>
>> Hi,
>>>
>>> I would like to know the working of Thrift IDL, the underlying
>>> architecture.I know that Thrift is using socket for communication.
>>>
>>> I want to know whether if the server side is a bit slow to read the data,
>>> then whether there would be a data loss or will it be there in the queue
>>> ?
>>>
>>>
>>> --
>>> Thanks & Regards,
>>> Deeraj Soman
>>> Senior Engineer - TBU | Embedded Product Design
>>> TATA ELXSI
>>> Technopark CampusKariyavattomTrivandrum 695 581India
>>> Phone:+91 471 666 1124
>>> Mobile: +91 9895703852
>>> *www.tataelxsi.com* <
>>>
>>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>>
>>>
>>>
> --
> Thanks & Regards,
> Deeraj Soman
> Senior Engineer - TBU | Embedded Product Design
> TATA ELXSI
> Technopark CampusKariyavattomTrivandrum 695 581India
> Phone:+91 471 666 1124
> Mobile: +91 9895703852
> *www.tataelxsi.com* <
> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
> >
>
>
>
Re: Thrift IDL underlying working
Posted by Deeraj <de...@tataelxsi.co.in>.
Hi Henrique,
Suppose if I am using socket as my protocol. Will the data be there in
the queue if the server fails to read the data ?
I have already implemented the server and client by reading the tutorial.
On 4/20/2015 10:19 AM, Henrique Mendonça wrote:
> Hi Deeraj,
>
> Apache Thrift can use sockets but need not. You are free to choose from a
> list of supported transports and protocols for each target language. You
> can even use it only for data serialization if you want.
>
> The documentation is unfortunately still a bit sparse but you can get the
> idea following the home page:
>
>
> *Getting Started...*
>
> - *Writing a .thrift file*
>
> *After the Thrift compiler is installed you will need to create a thrift
> file. This file is an interface definition
> <https://thrift.apache.org/docs/idl>made up of thrift types
> <https://thrift.apache.org/docs/types> and Services. The services you
> define in this file are implemented by the server and are called by any
> clients. The Thrift compiler is used to generate your Thrift File into
> source code which is used by the different client libraries and the server
> you write. To generate the source from a thrift file run*
>
> * thrift --gen <language> <Thrift filename>
> *
>
> *The sample tutorial.thrift file used for all the client and server
> tutorials can be found here
> <https://git-wip-us.apache.org/repos/asf/thrift/?p=thrift.git;a=tree;f=tutorial>.*
>
>
> *To learn more about Apache Thrift Read the Whitepaper
> <https://thrift.apache.org/static/files/thrift-20070401.pdf>*
>
> The above mentioned white paper is a good start, Randy's book is also very
> handy: http://www.manning.com/abernethy/
> His first [free] chapter might cover some of your questions.
>
> Cheers,
> Henrique
>
>
> On 20 April 2015 at 14:27, Deeraj <de...@tataelxsi.co.in> wrote:
>
>> Hi,
>>
>> I would like to know the working of Thrift IDL, the underlying
>> architecture.I know that Thrift is using socket for communication.
>>
>> I want to know whether if the server side is a bit slow to read the data,
>> then whether there would be a data loss or will it be there in the queue ?
>>
>>
>> --
>> Thanks & Regards,
>> Deeraj Soman
>> Senior Engineer - TBU | Embedded Product Design
>> TATA ELXSI
>> Technopark CampusKariyavattomTrivandrum 695 581India
>> Phone:+91 471 666 1124
>> Mobile: +91 9895703852
>> *www.tataelxsi.com* <
>> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
>>
>>
--
Thanks & Regards,
Deeraj Soman
Senior Engineer - TBU | Embedded Product Design
TATA ELXSI
Technopark CampusKariyavattomTrivandrum 695 581India
Phone:+91 471 666 1124
Mobile: +91 9895703852
*www.tataelxsi.com*
<https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f>
Re: Thrift IDL underlying working
Posted by Henrique Mendonça <hm...@gmail.com>.
Hi Deeraj,
Apache Thrift can use sockets but need not. You are free to choose from a
list of supported transports and protocols for each target language. You
can even use it only for data serialization if you want.
The documentation is unfortunately still a bit sparse but you can get the
idea following the home page:
*Getting Started...*
- *Writing a .thrift file*
*After the Thrift compiler is installed you will need to create a thrift
file. This file is an interface definition
<https://thrift.apache.org/docs/idl>made up of thrift types
<https://thrift.apache.org/docs/types> and Services. The services you
define in this file are implemented by the server and are called by any
clients. The Thrift compiler is used to generate your Thrift File into
source code which is used by the different client libraries and the server
you write. To generate the source from a thrift file run*
* thrift --gen <language> <Thrift filename>
*
*The sample tutorial.thrift file used for all the client and server
tutorials can be found here
<https://git-wip-us.apache.org/repos/asf/thrift/?p=thrift.git;a=tree;f=tutorial>.*
*To learn more about Apache Thrift Read the Whitepaper
<https://thrift.apache.org/static/files/thrift-20070401.pdf>*
The above mentioned white paper is a good start, Randy's book is also very
handy: http://www.manning.com/abernethy/
His first [free] chapter might cover some of your questions.
Cheers,
Henrique
On 20 April 2015 at 14:27, Deeraj <de...@tataelxsi.co.in> wrote:
> Hi,
>
> I would like to know the working of Thrift IDL, the underlying
> architecture.I know that Thrift is using socket for communication.
>
> I want to know whether if the server side is a bit slow to read the data,
> then whether there would be a data loss or will it be there in the queue ?
>
>
> --
> Thanks & Regards,
> Deeraj Soman
> Senior Engineer - TBU | Embedded Product Design
> TATA ELXSI
> Technopark CampusKariyavattomTrivandrum 695 581India
> Phone:+91 471 666 1124
> Mobile: +91 9895703852
> *www.tataelxsi.com* <
> https://sinprd0410.outlook.com/owa/redir.aspx?C=qyQ5HANN90SvDNp5pyzlQgCjW3rPss8IRh9kaHIpdT-pocCvJc2bGjG5OjXXp8HUHmQTB2YZ2Kw.&URL=http%3a%2f%2fwww.tataelxsi.com%2f
> >
>
>
>