You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by "Sandesh (JIRA)" <ji...@apache.org> on 2016/06/27 18:46:52 UTC
[jira] [Updated] (APEXCORE-478) SerDe based on the types
[ https://issues.apache.org/jira/browse/APEXCORE-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandesh updated APEXCORE-478:
-----------------------------
Description:
During deploy time, Apex already knows the type of the object to be serialized and deserialized, so can optimize the SerDe used. I am quoting Vlad here,
"
+1 for type based serialization. Tuples in most cases are flat
records/pojo and it should be possible programmatically construct a
codec that will significantly outperform Kryo. It should also reduce
amount of data passed over the wire. I started to look in that direction
as well as Kryo serialization is one of bottlenecks that limits Apex
throughput when operators are deployed into different containers
including NODE_LOCAL case."
Link to mail thread on this discussion
http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAJhKGzqxknF_7Wj2T2Vfp%3D35cziT0hTJoQTnWopET%2Bj8OgpUJw%40mail.gmail.com%3E
was:
During deploy time, Apex already knows the type of the object to be serialized and deserialized. I am quoting Vlad here,
"
+1 for type based serialization. Tuples in most cases are flat
records/pojo and it should be possible programmatically construct a
codec that will significantly outperform Kryo. It should also reduce
amount of data passed over the wire. I started to look in that direction
as well as Kryo serialization is one of bottlenecks that limits Apex
throughput when operators are deployed into different containers
including NODE_LOCAL case."
Link to mail thread on this discussion
http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAJhKGzqxknF_7Wj2T2Vfp%3D35cziT0hTJoQTnWopET%2Bj8OgpUJw%40mail.gmail.com%3E
> SerDe based on the types
> ------------------------
>
> Key: APEXCORE-478
> URL: https://issues.apache.org/jira/browse/APEXCORE-478
> Project: Apache Apex Core
> Issue Type: Improvement
> Reporter: Sandesh
>
> During deploy time, Apex already knows the type of the object to be serialized and deserialized, so can optimize the SerDe used. I am quoting Vlad here,
> "
> +1 for type based serialization. Tuples in most cases are flat
> records/pojo and it should be possible programmatically construct a
> codec that will significantly outperform Kryo. It should also reduce
> amount of data passed over the wire. I started to look in that direction
> as well as Kryo serialization is one of bottlenecks that limits Apex
> throughput when operators are deployed into different containers
> including NODE_LOCAL case."
> Link to mail thread on this discussion
> http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAJhKGzqxknF_7Wj2T2Vfp%3D35cziT0hTJoQTnWopET%2Bj8OgpUJw%40mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)