You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Maxim Muzafarov (JIRA)" <ji...@apache.org> on 2019/05/15 15:36:00 UTC
[jira] [Updated] (IGNITE-10619) Add support files transmission
between nodes over connection via CommunicationSpi
[ https://issues.apache.org/jira/browse/IGNITE-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maxim Muzafarov updated IGNITE-10619:
-------------------------------------
Description:
Partition preloader must support cache partition file relocation from one cluster node to another (the zero copy algorithm [1] assume to be used by default). To achieve this, the file transfer machinery must be implemented at Apache Ignite over Communication SPI.
_CommunicationSpi_
Ignite's Comminication SPI must support to:
* establishing channel connections to the remote node to an arbitrary topic (GridTopic) with predefined processing policy;
* listening incoming channel creation events and registering connection handlers on the particular node;
* an arbitrary set of channel parameters on connection handshake;
_FileTransmitProcessor_
The file transmission manager must support to:
* using different approaches of incoming data handling – buffered and direct (zero-copy approach of FileChannel#transferTo);
* transferring data by chunks of predefined size with saving intermediate results;
* re-establishing connection if an error occurs and continue file upload\download;
* limiting connection bandwidth (upload and download) at runtime;
was:
To benefit from zero copy we must delegate the file transferring to FileChannel#transferTo(long, long, java.nio.channels.WritableByteChannel) because the fast path of transferTo method is only executed if the destination buffer inherits from an internal JDK class.
The {{CommunicationSpi}} needs to support pipe connections between two nodes;
* The WritableByteChannel needs to be accessed on the supplier side;
* The ReadableByteChannel needs to be read on the demander side;
* The CommunicationListener must be extended to respond on new incoming pipe connections;
> Add support files transmission between nodes over connection via CommunicationSpi
> ---------------------------------------------------------------------------------
>
> Key: IGNITE-10619
> URL: https://issues.apache.org/jira/browse/IGNITE-10619
> Project: Ignite
> Issue Type: Sub-task
> Components: persistence
> Reporter: Maxim Muzafarov
> Assignee: Maxim Muzafarov
> Priority: Major
> Labels: iep-28
>
> Partition preloader must support cache partition file relocation from one cluster node to another (the zero copy algorithm [1] assume to be used by default). To achieve this, the file transfer machinery must be implemented at Apache Ignite over Communication SPI.
> _CommunicationSpi_
> Ignite's Comminication SPI must support to:
> * establishing channel connections to the remote node to an arbitrary topic (GridTopic) with predefined processing policy;
> * listening incoming channel creation events and registering connection handlers on the particular node;
> * an arbitrary set of channel parameters on connection handshake;
> _FileTransmitProcessor_
> The file transmission manager must support to:
> * using different approaches of incoming data handling – buffered and direct (zero-copy approach of FileChannel#transferTo);
> * transferring data by chunks of predefined size with saving intermediate results;
> * re-establishing connection if an error occurs and continue file upload\download;
> * limiting connection bandwidth (upload and download) at runtime;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)