[jira] [Commented] (SAMOA-77) Rebuild and unification of data serialization

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMOA-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328518#comment-16328518 ] 

ASF GitHub Bot commented on SAMOA-77:
-------------------------------------

GitHub user pwawrzyniak opened a pull request:

    https://github.com/apache/incubator-samoa/pull/76

    SAMOA-77: Rebuild and unification of data serialization

    This PR provides unified data serialization pattern for SAMOA project. In particular, the currently distributed code is groupped into samoa-instances and unified to single Loader pattern (derived from existing one). The readers can be created by factory class. This PR also clean some of the Kafka-related code and introduces new structure to samoa-instances class/packets structure.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/pwawrzyniak/incubator-samoa SAMOA-77

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-samoa/pull/76.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #76
    
----
commit 1eb5d7c5f71eb3c6ed40a5ffd26e2504373d8b92
Author: Cezar <ce...@...>
Date:   2017-11-30T11:27:02Z

    loader refactor, loaders package creation, loader type creation, loader factory creation

commit a2e1793433c51584765d5f92e3db6ff84481b7a7
Author: Cezar <ce...@...>
Date:   2017-12-04T07:57:24Z

    space refactor

commit af493cd355595097e131bf385e7de9cb58695dbe
Author: Cezar <ce...@...>
Date:   2017-12-04T09:22:43Z

    refactor, clean-up

commit d5b9b067444185782a1e94bb52287739aed62820
Author: Cezar <ce...@...>
Date:   2017-12-04T10:03:56Z

    instances refactor

commit 0de64a288f7b3573200982085608bc57055f38bb
Author: Cezar <ce...@...>
Date:   2017-12-04T10:10:42Z

    interfaces clean up

commit 0f5db023d5a923481bb8a29630cea8e171fab0cd
Author: Cezar <ce...@...>
Date:   2017-12-04T14:47:30Z

    loader factory implementation for avro and arff

commit 80479f86f8fa8c42d84e59431f33c528eb108d52
Author: Cezar <ce...@...>
Date:   2017-12-06T17:50:45Z

    refactor, clean up

commit 5d90e57dd18e4ce1d2a9c4ed4bab54c90b28128a
Author: Cezar <ce...@...>
Date:   2017-12-08T11:08:26Z

    arffLoader refactor, clean up

commit 3c00a52a5e1b49662832c1c85dbdfa036a2e1e43
Author: Cezar <ce...@...>
Date:   2017-12-08T15:55:48Z

    refactor, clean up

commit 70e1bfde3b0d3b8272c992394519cf9bbcd63269
Author: Cezar <ce...@...>
Date:   2017-12-19T11:55:10Z

    kafka newest version added

commit 6a202d7f502771541003037c61553dcb96fac800
Author: Cezar <ce...@...>
Date:   2017-12-19T12:05:02Z

    kafka newest version aded

commit 07b2b50c293c51b65fb43714b27c33694463b7a3
Author: Cezar <ce...@...>
Date:   2017-12-20T12:41:15Z

    kafka loader added

commit 2bf171070e3d40c7b520b71f3042ec1be4d30e47
Author: Cezar <ce...@...>
Date:   2017-12-22T10:06:25Z

    kafka loader test added

----


> Rebuild and unification of data serialization
> ---------------------------------------------
>
>                 Key: SAMOA-77
>                 URL: https://issues.apache.org/jira/browse/SAMOA-77
>             Project: SAMOA
>          Issue Type: New Feature
>          Components: SAMOA-API, SAMOA-Instances
>            Reporter: Piotr Wawrzyniak
>            Priority: Major
>   Original Estimate: 840h
>  Remaining Estimate: 840h
>
> Currently all Loader instances are created in the Instances class constructors, by providing proper parameters in constructor different implementation of Loader is instantiated (and Loader implementations handle deserialization of “sources”- arff file, avro files etc.). In our opinion this could be improved by creating a factory object designed to create proper Loader instances. Such LoaderFactory could accept number of parameters and would return most appropriate Loader implementation. This solution will benefit in the future when there will be need to add new loaders/sources. Also the code will be cleaner and the process of adding another loaders will be simplified, also thanks to ’strict’ use of factory design pattern.
> So to sum up, it would mean we would keep all the serialization-related code in the samoa-instances project, and we will use Loader interface (possible extended to support reading and writing of data), and LoaderFactory class. Arguments for factory method would be passed as task options, which means adoption of new serialization formats and data sources would be also simplified. This would mean that we will redesign Kafka components so they will follow this pattern, but on the other hand the pattern would be shared among all the possible sources of data. Moreover, Instances class would be modified by removing some of its constructors and replacing them by single constructor accepting Loader implementation as created by the LoaderFactory.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)