You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by SADHNA <sa...@gmail.com> on 2017/11/09 10:15:40 UTC
Regarding kafka produce Image(18mb)
Hi Team,
I am trying to send images using kafka. I have
used value_serializer=lambda v: json.dumps(v).encode('utf-8') and sending
the messages like message = {"cam_id" : 1, "frame" : image, "frame_no": i}
but getting TypeError: array([[[ 26, 36, 31],
[ 16, 26, 21],
[ 11, 21, 16],
...,
[ 0, 19, 2],
[ 3, 25, 11],
[ 18, 40, 26]],
[[ 18, 28, 23],
[ 10, 20, 15],
[ 9, 19, 14],
...,
[ 9, 32, 15],
[ 14, 36, 22],
[ 25, 47, 33]],
[[ 13, 23, 18],
[ 9, 19, 14],
[ 8, 18, 13],
...,
[ 24, 47, 30],
[ 30, 52, 38],
[ 35, 57, 43]],
...,
[[125, 133, 136],
[127, 135, 138],
[131, 139, 142],
...,
[118, 136, 146],
[112, 130, 140],
[121, 139, 149]],
[[129, 137, 140],
[129, 137, 140],
[130, 138, 141],
...,
[144, 162, 172],
[132, 150, 160],
[128, 146, 156]],
[[136, 144, 147],
[134, 142, 145],
[133, 141, 144],
...,
[153, 171, 181],
[145, 163, 173],
[137, 155, 165]]], dtype=uint8) is not JSON serializable.
Can you please help me to resolve this issue. Each image if of 18mb in size.
Thanks and Regards,
Sadhna
Re: Regarding kafka produce Image(18mb)
Posted by Ted Yu <yu...@gmail.com>.
bq. Each image if of 18mb in size.
Seems the image size is non-trivial.
Can you tell us more about use case ?
e.g. what systems (Apache projects) are involved.
Just want to see if there is another route for sending the images.
On Thu, Nov 9, 2017 at 2:15 AM, SADHNA <sa...@gmail.com>
wrote:
> Hi Team,
>
> I am trying to send images using kafka. I have
> used value_serializer=lambda v: json.dumps(v).encode('utf-8') and sending
> the messages like message = {"cam_id" : 1, "frame" : image, "frame_no": i}
>
> but getting TypeError: array([[[ 26, 36, 31],
> [ 16, 26, 21],
> [ 11, 21, 16],
> ...,
> [ 0, 19, 2],
> [ 3, 25, 11],
> [ 18, 40, 26]],
>
> [[ 18, 28, 23],
> [ 10, 20, 15],
> [ 9, 19, 14],
> ...,
> [ 9, 32, 15],
> [ 14, 36, 22],
> [ 25, 47, 33]],
>
> [[ 13, 23, 18],
> [ 9, 19, 14],
> [ 8, 18, 13],
> ...,
> [ 24, 47, 30],
> [ 30, 52, 38],
> [ 35, 57, 43]],
>
> ...,
> [[125, 133, 136],
> [127, 135, 138],
> [131, 139, 142],
> ...,
> [118, 136, 146],
> [112, 130, 140],
> [121, 139, 149]],
>
> [[129, 137, 140],
> [129, 137, 140],
> [130, 138, 141],
> ...,
> [144, 162, 172],
> [132, 150, 160],
> [128, 146, 156]],
>
> [[136, 144, 147],
> [134, 142, 145],
> [133, 141, 144],
> ...,
> [153, 171, 181],
> [145, 163, 173],
> [137, 155, 165]]], dtype=uint8) is not JSON serializable.
>
>
> Can you please help me to resolve this issue. Each image if of 18mb in
> size.
>
> Thanks and Regards,
> Sadhna
>