You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@thrift.apache.org by Yuri Bochkarev <ba...@gmail.com> on 2011/06/30 18:29:51 UTC
Send large files
Hello.
Can you tell me please if Thrift is good for sending large files in
LAN? If so, can you point me to some examples?
I have a server written in Erlang and a client written in Python.
Beside of simple communication protocol which Thrift is perfectly good
for, server also needs to send large (up to several gigabytes:
50Mb..5Gb) files to the client and to get same large files back. The
process must be reliable (we must detect network failures and send
file again) and resumable (if network connection drops, it would be
nice to continue sending from the point where is was stopped, like
FTP). The latter requirement is not that important, actually.
The communication is expected to happen in LAN. Will Thrift
implementation properly handle such a large binary messages?
Thanks in advance,
Yuri
Re: Send large files
Posted by Rory McGuire <rm...@clearformat.com>.
Hessian does I believe.
http://hessian.caucho.com/
-Rory
On Fri, 01 Jul 2011 11:39:40 +0200, Florian Weimer <fw...@bfk.de> wrote:
> * Bryan Duxbury:
>
>> It does not naturally support chunking.
>
> Are there any alternatives which do? It seems that this feature is
> extremely rare among (structured) transport protocols---even many HTTP
> implementations do not support incremental processing of incoming data
> during arrival.
Re: Send large files
Posted by Dmitriy Kargapolov <dm...@gmail.com>.
On 07/01/2011 05:39 AM, Florian Weimer wrote:
> * Bryan Duxbury:
>
>> It does not naturally support chunking.
> Are there any alternatives which do? It seems that this feature is
> extremely rare among (structured) transport protocols---even many HTTP
> implementations do not support incremental processing of incoming data
> during arrival.
>
Adding transfer large data feature to the Thrift would kill it as a
protocol. It's a nice smart framework targeting multi-platform RPC based
by DDL and code generation. It's not a panacea for all possible
scenarios and should not be.
Re: Send large files
Posted by Florian Weimer <fw...@bfk.de>.
* Bryan Duxbury:
> It does not naturally support chunking.
Are there any alternatives which do? It seems that this feature is
extremely rare among (structured) transport protocols---even many HTTP
implementations do not support incremental processing of incoming data
during arrival.
--
Florian Weimer <fw...@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99
Re: Send large files
Posted by Bryan Duxbury <br...@rapleaf.com>.
It does not naturally support chunking.
On Thu, Jun 30, 2011 at 12:51 PM, Dheerajgoswami
<dh...@yahoo.com>wrote:
> I had an impression that thrift does support chunking mechanism for large
> data transfer. Is that not true?
>
> Thanks
> Dheeraj
>
>
> On Jun 30, 2011, at 11:50 AM, Bryan Duxbury <br...@rapleaf.com> wrote:
>
> > Thrift is not particularly suited to sending large files. In particular,
> > Thrift structures need to be fully in-memory before they can be
> serialized
> > and sent. You could implement a multi-message transfer mechanism, though.
> >
> > On Thu, Jun 30, 2011 at 9:29 AM, Yuri Bochkarev <baltazar.bz@gmail.com
> >wrote:
> >
> >> Hello.
> >>
> >>
> >> Can you tell me please if Thrift is good for sending large files in
> >> LAN? If so, can you point me to some examples?
> >>
> >> I have a server written in Erlang and a client written in Python.
> >> Beside of simple communication protocol which Thrift is perfectly good
> >> for, server also needs to send large (up to several gigabytes:
> >> 50Mb..5Gb) files to the client and to get same large files back. The
> >> process must be reliable (we must detect network failures and send
> >> file again) and resumable (if network connection drops, it would be
> >> nice to continue sending from the point where is was stopped, like
> >> FTP). The latter requirement is not that important, actually.
> >>
> >> The communication is expected to happen in LAN. Will Thrift
> >> implementation properly handle such a large binary messages?
> >>
> >>
> >> Thanks in advance,
> >> Yuri
> >>
>
Re: Send large files
Posted by Dheerajgoswami <dh...@yahoo.com>.
I had an impression that thrift does support chunking mechanism for large data transfer. Is that not true?
Thanks
Dheeraj
On Jun 30, 2011, at 11:50 AM, Bryan Duxbury <br...@rapleaf.com> wrote:
> Thrift is not particularly suited to sending large files. In particular,
> Thrift structures need to be fully in-memory before they can be serialized
> and sent. You could implement a multi-message transfer mechanism, though.
>
> On Thu, Jun 30, 2011 at 9:29 AM, Yuri Bochkarev <ba...@gmail.com>wrote:
>
>> Hello.
>>
>>
>> Can you tell me please if Thrift is good for sending large files in
>> LAN? If so, can you point me to some examples?
>>
>> I have a server written in Erlang and a client written in Python.
>> Beside of simple communication protocol which Thrift is perfectly good
>> for, server also needs to send large (up to several gigabytes:
>> 50Mb..5Gb) files to the client and to get same large files back. The
>> process must be reliable (we must detect network failures and send
>> file again) and resumable (if network connection drops, it would be
>> nice to continue sending from the point where is was stopped, like
>> FTP). The latter requirement is not that important, actually.
>>
>> The communication is expected to happen in LAN. Will Thrift
>> implementation properly handle such a large binary messages?
>>
>>
>> Thanks in advance,
>> Yuri
>>
Re: Send large files
Posted by Bryan Duxbury <br...@rapleaf.com>.
Thrift is not particularly suited to sending large files. In particular,
Thrift structures need to be fully in-memory before they can be serialized
and sent. You could implement a multi-message transfer mechanism, though.
On Thu, Jun 30, 2011 at 9:29 AM, Yuri Bochkarev <ba...@gmail.com>wrote:
> Hello.
>
>
> Can you tell me please if Thrift is good for sending large files in
> LAN? If so, can you point me to some examples?
>
> I have a server written in Erlang and a client written in Python.
> Beside of simple communication protocol which Thrift is perfectly good
> for, server also needs to send large (up to several gigabytes:
> 50Mb..5Gb) files to the client and to get same large files back. The
> process must be reliable (we must detect network failures and send
> file again) and resumable (if network connection drops, it would be
> nice to continue sending from the point where is was stopped, like
> FTP). The latter requirement is not that important, actually.
>
> The communication is expected to happen in LAN. Will Thrift
> implementation properly handle such a large binary messages?
>
>
> Thanks in advance,
> Yuri
>