Use Kafka To Send Files

[hu...@live.com]

Hi     Sometimes we want use kafka to send files(like textfile,xml...), but I didn't see this in documention.
     Can kafka use to tansfer files? If can ,how can I do
    Thanks 		 	   		  

Re: Use Kafka To Send Files

[Guozhang Wang <wa...@gmail.com>]

We have seen people sending files with tens of MB size like messages (i.e.
one file as a single message) using Kafka.

Guozhang

On Sun, Jun 15, 2014 at 7:41 PM, <hu...@live.com> wrote:

> Hi     Sometimes we want use kafka to send files(like textfile,xml...),
> but I didn't see this in documention.
>      Can kafka use to tansfer files? If can ,how can I do
>     Thanks




-- 
-- Guozhang

Re: Use Kafka To Send Files

[Steve Morin <st...@gmail.com>]

You also wouldn't have any meta data about the file so I would avoid doing this.

> On Jun 15, 2014, at 20:51, Mark Roberts <wi...@gmail.com> wrote:
> 
> You would ship the contents of the file across as a message. In general this would mean that your maximum file size must be smaller than your maximum message size. It would generally be a better choice to put a pointer to the file in some shared location on the queue.
> 
> -Mark
> 
>> On Jun 15, 2014, at 19:41, <hu...@live.com> wrote:
>> 
>> Hi     Sometimes we want use kafka to send files(like textfile,xml...), but I didn't see this in documention.
>>    Can kafka use to tansfer files? If can ,how can I do
>>   Thanks                         

Re: Use Kafka To Send Files

[Mark Roberts <wi...@gmail.com>]

You would ship the contents of the file across as a message. In general this would mean that your maximum file size must be smaller than your maximum message size. It would generally be a better choice to put a pointer to the file in some shared location on the queue.

-Mark

> On Jun 15, 2014, at 19:41, <hu...@live.com> wrote:
> 
> Hi     Sometimes we want use kafka to send files(like textfile,xml...), but I didn't see this in documention.
>     Can kafka use to tansfer files? If can ,how can I do
>    Thanks