You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Nikolay Izhikov <ni...@apache.org> on 2018/11/02 10:23:23 UTC
IGNITE-2.7. New Features
Hello, Guys.
Good news! We have 2 final tickets for 2.7.
So release date is very near!
Let's collect new features and improvements of Ignite 2.7 and includes it to release notes and other documents.
Can you answer and describe your contributions?
Re: IGNITE-2.7. New Features
Posted by Maxim Muzafarov <ma...@gmail.com>.
Vladimir,
We have a major IgniteSet improvement `Separate cache for non-collocated
IgniteSet.` [1] which is not mentioned yet.
Should we also include it to the release notes?
[1] https://issues.apache.org/jira/browse/IGNITE-7823
On Tue, 6 Nov 2018 at 16:50 Vladimir Ozerov <vo...@gridgain.com> wrote:
> Igniters,
>
> Any more comments on release notes?
>
> On Tue, Nov 6, 2018 at 9:01 AM Nikolay Izhikov <ni...@apache.org>
> wrote:
>
> > Vladimir, Thank you so much for this list!
> >
> > пн, 5 нояб. 2018 г., 23:25 Vladimir Ozerov vozerov@gridgain.com:
> >
> > > Igniters,
> > >
> > > I assembled information about new features I am aware of. Please find
> the
> > > list below and review it, so that we do not miss anything important.
> Note
> > > that normally we do not include information about every single fix or
> > minor
> > > feature, as it will make release notes too big. Instead, we strive to
> > keep
> > > only the most prominent features/fixes in release notes. Minor things
> are
> > > grouped into generic points such as "Stability and performance
> > > improvements". Also we typically try to place the most important
> features
> > > first, to catch user's attention as quickly as possible.
> > >
> > > Ignite:
> > > * Added experimental support for multi-version concurrency control with
> > > snapshot isolation
> > > - available for both cache API and SQL
> > > - use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
> > > - not production ready, data consistency is not guaranteed in case of
> > > node failures
> > > * Implemented Transparent Data Encryption based on JKS certificates
> > > * Implemented Node.JS Thin Client
> > > * Implemented Python Thin Client
> > > * Implemented PHP Thin Client
> > > * Ignite start scripts now support Java 9 and higher
> > > * Added ability to set WAL history size in bytes
> > > * Added SslContextFactory.protocols and SslContextFactory.cipherSuites
> > > properties to control which SSL encryption algorithms can be used
> > > * Added JCache 1.1 compliance
> > > * Added IgniteCompute.withNoResultCache method with semantics similar
> to
> > > ComputeTaskNoResultCache annotation
> > > * Spring Data 2.0 is now supported in separate module
> > > 'ignite-spring-data_2.0'
> > > * Added monitoring of critical system workers
> > > * Added ability to provide custom implementations of ExceptionListener
> > for
> > > JmsStreamer
> > > * Ignite KafkaStreamer was upgraded to use new KafkaConsmer
> configuration
> > > * S3 IP Finder now supports subfolder usage instead of bucket root
> > > * Improved dynamic cache start speed
> > > * Improved checkpoint performance by decreasing mark duration.
> > > * Added ability to manage compression level for compressed WAL
> archives.
> > > * Added metrics for Entry Processor invocations.
> > > * Added JMX metrics: ClusterMetricsMXBean.getTotalBaselineNodes and
> > > ClusterMetricsMXBean.getActiveBaselineNodes
> > > * Node uptime metric now includes days count
> > > * Exposed info about thin client connections through JMX
> > > * Inrtoduced new system property IGNITE_REUSE_MEMORY_ON_DEACTIVATE to
> > > enable reuse of allocated memory on node deactivation (disabled by
> > default)
> > > * Optimistic transaction now will be properly rolled back if waiting
> too
> > > long for new topology on remap
> > > * ScanQuery with setLocal flag now checks if partition is actually
> > present
> > > on local node
> > > * Improved cluster behavior when left node does not cause partition
> > > affinity assignment changes
> > > * Interrupting user thread during partition initialization will no
> longer
> > > cause node to stop
> > > * Fixed problem when partition lost event were not triggered if
> multiple
> > > nodes left cluster
> > > * Fixed massive node drop from cluster on temporary network issues
> > > * Fixed service redeployment on cluster reactivation
> > > * Fixed client node stability under ZooKeeper discovery
> > > * Massive performance and stability improvements
> > >
> > > Ignite .Net:
> > > * Add .NET Core 2.1 support
> > > * Added thin client connection failover
> > >
> > > Ignite C++:
> > > * Implemented Thin Client with base cache operations
> > > * Implemented smart affinity routing for Thin Client to send requests
> > > directly to nodes containing data when possible
> > > * Added Clang compiler support
> > >
> > > SQL:
> > > * Added experimental support for fully ACID transactional SQL with
> > snapshot
> > > isolation:
> > > - use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
> > > - transaction can be started through native API (IgniteTransactions),
> > > thin JDBC driver or ODBC driver
> > > - not production ready, data consistency is not guaranteed in case of
> > > node failures
> > > * Added a set of system views located in "IGNITE" schema to view
> cluster
> > > information (NODES, NODE_ATTRIBUTES, NODE_METRICS, BASELINE_NODES)
> > > * Added ability to create predefined SQL schemas
> > > * Added GROUP_CONCAT function support
> > > * Added string length constraint
> > > * Custom Java objects are now inlined into primary and secondary
> indexes
> > > what may significantly improve performance when AFFINITY_KEY is used
> > > * Added timeout to fail query execution in case it cannot be mapped to
> > > topology
> > > * Restricted number of cores allocated for CREATE INDEX by default to 4
> > to
> > > avoid contention on index tree
> > > Fixed transaction hanging during runtime error on commit.
> > > * Fixed possible memory leak when result set size is multiple of page
> > size
> > > * Fixed situation when data may be returned from cache partitoins in
> LOST
> > > state even when PartitionLossPolicy doesn't permit it
> > > * Fixed "Caches have distinct sets of data nodes" during SQL JOIN query
> > > execution between REPLICATED and PARTITIONED caches
> > > * Fixed wrong result for sql queries when item size exceeds page size
> > > * Fixed error during SQL query from client node with local flag set to
> > > "true"
> > > * Fixed handling UUID as column type
> > >
> > > JDBC:
> > > * Implemented DataSource interface for thin driver
> > >
> > > ODBC:
> > > * Added streaming mode support
> > > * Fixed crash in Linux when there is more than 1023 open file
> descriptors
> > > * Fixed bug that prevented cursors on server from being closed
> > > * Fixed segmentation fault when reusing a closed connection
> > >
> > > Web Console:
> > > * Added new metrics: WAL and Data size on disk
> > > * Added support for "collocated" query mode on Query screen
> > > * Added support for Java 9+ for Web Agent.
> > > * Added ability to show/hide password field value
> > > * Implemented execution of selected part of SQL query
> > > * Implemented explain of selected part of SQL query
> > > * Implemented connection to secured cluster
> > > * Implemented responsive full screen layout
> > > * Split "Sign In" page to three separate pages
> > > * UI updated to modern look and feel
> > > * Improved backend stability
> > > * Fixed fail when working with web sockets
> > >
> > > REST:
> > > * Added option IGNITE_REST_GETALL_AS_ARRAY for array format in "getAll"
> > > call
> > >
> > > Visor:
> > > * Added output of node "Consistent ID"
> > > * Visor now collects information about cache groups instead of separate
> > > caches to reduce memory consumption
> > > * Improved help for "start" command
> > > * Fixed output of cache metrics
> > >
> > > Control utility:
> > > * Added information about transaction start time
> > > * Added command to collect information about distribution of partitions
> > > * Added command to reset lost partitions
> > > * Added support for empty label (control.sh --tx label null)
> > > * Added atomicity mode to utility output.
> > > * Added orphaned local and remote transactions and ability to rollback
> > them
> > > * Added "--dump" flag to dump current partition state to file.
> > > * Renamed command argument '--force' to '--yes'
> > > * Removed "initOrder" and "loc keys" from info
> > > * Fixed control utility hanging when connected to a joining node with
> PME
> > >
> > > ML:
> > > * Added TensorFlow integration
> > > * Added Estimator API support to TensorFlow cluster on top of Apache
> > Ignite
> > > * Added ANN algorithm based on ACD concept
> > > * Added Random Forest algorithm
> > > * Added OneHotEncoder for categorical features
> > > * Added model estimation
> > > * Added K-fold cross validation for ML models
> > > * Added splitter for splitting dataset on test and train subsets
> > > * Added ability of filtering data during datasets creation
> > > * Added encoding categorical features with One-of-K Encoder
> > > * Added MinMax scaler preprocessor
> > > * Added gradient boosting for trees
> > > * Added indexing for decision trees
> > > * Added GDB convergence by error support
> > > * Added ability to build pipeline of data preprocessing and model
> > training
> > > * Added ability to start and maintain TensorFlow cluster on top of
> Apache
> > > Ignite
> > > * Added support of Multi-Class for Logistic Regression
> > > * Implemented distributed binary logistic regression
> > >
> > > De[endency updates:
> > > * Apache Camel updated to 2.22.0
> > > * Apache Commons Beanutils updated to 1.9.3
> > > * Apache Hadoop Yarn updated to 2.7.7
> > > * Apache Kafka updated to 1.1.0
> > > * Apache Lucene updated to 7.4.0
> > > * Apache Mesos updated to 1.5.0
> > > * Apache Tomcat updated to 9.0.10
> > > * Apache Zookeeper updated to 3.4.13
> > > * Guava updated to 25.1-jre
> > > * Jackson Databind updated to 2.9.6
> > > * Jackson 1 usages replaced with Jackson 2
> > > * JCraft updated to 0.1.54
> > > * H2 version updated to 1.4.197
> > > * Log4j 2.x updated to 2.11.0
> > > * Netty updated to 4.1.27.Final
> > > * RocketMQ updated to 4.3.0
> > > * Scala 2.10.x was updated to 2.10.7
> > > * Scala 2.11.x updated to 2.11.12
> > >
> > > On Fri, Nov 2, 2018 at 6:47 PM David Harvey <sy...@gmail.com>
> > wrote:
> > >
> > > > IGNITE-9365 Force backups to different AWS availability zones using
> > only
> > > > Spring XML
> > > >
> > > > This deserves documentation enhancements:
> > > > https://apacheignite.readme.io/docs/affinity-collocation
> > > >
> > > > What is the mechanism for proposing an update for 2.7?
> > > >
> > > >
> > > > On Fri, Nov 2, 2018 at 6:31 AM Andrey Kuznetsov <st...@gmail.com>
> > > wrote:
> > > >
> > > > > Great news!
> > > > >
> > > > > Future release is about to contain mission critical Ignite workers
> > > > liveness
> > > > > monitoring, introduced in IGNITE-6587.
> > > > >
> > > > >
> > > > > пт, 2 нояб. 2018 г. в 13:23, Nikolay Izhikov <nizhikov@apache.org
> >:
> > > > >
> > > > > > Hello, Guys.
> > > > > >
> > > > > > Good news! We have 2 final tickets for 2.7.
> > > > > > So release date is very near!
> > > > > >
> > > > > > Let's collect new features and improvements of Ignite 2.7 and
> > > includes
> > > > it
> > > > > > to release notes and other documents.
> > > > > >
> > > > > > Can you answer and describe your contributions?
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Andrey Kuznetsov.
> > > > >
> > > >
> > >
> >
>
--
--
Maxim Muzafarov
Re: IGNITE-2.7. New Features
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Igniters,
Any more comments on release notes?
On Tue, Nov 6, 2018 at 9:01 AM Nikolay Izhikov <ni...@apache.org> wrote:
> Vladimir, Thank you so much for this list!
>
> пн, 5 нояб. 2018 г., 23:25 Vladimir Ozerov vozerov@gridgain.com:
>
> > Igniters,
> >
> > I assembled information about new features I am aware of. Please find the
> > list below and review it, so that we do not miss anything important. Note
> > that normally we do not include information about every single fix or
> minor
> > feature, as it will make release notes too big. Instead, we strive to
> keep
> > only the most prominent features/fixes in release notes. Minor things are
> > grouped into generic points such as "Stability and performance
> > improvements". Also we typically try to place the most important features
> > first, to catch user's attention as quickly as possible.
> >
> > Ignite:
> > * Added experimental support for multi-version concurrency control with
> > snapshot isolation
> > - available for both cache API and SQL
> > - use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
> > - not production ready, data consistency is not guaranteed in case of
> > node failures
> > * Implemented Transparent Data Encryption based on JKS certificates
> > * Implemented Node.JS Thin Client
> > * Implemented Python Thin Client
> > * Implemented PHP Thin Client
> > * Ignite start scripts now support Java 9 and higher
> > * Added ability to set WAL history size in bytes
> > * Added SslContextFactory.protocols and SslContextFactory.cipherSuites
> > properties to control which SSL encryption algorithms can be used
> > * Added JCache 1.1 compliance
> > * Added IgniteCompute.withNoResultCache method with semantics similar to
> > ComputeTaskNoResultCache annotation
> > * Spring Data 2.0 is now supported in separate module
> > 'ignite-spring-data_2.0'
> > * Added monitoring of critical system workers
> > * Added ability to provide custom implementations of ExceptionListener
> for
> > JmsStreamer
> > * Ignite KafkaStreamer was upgraded to use new KafkaConsmer configuration
> > * S3 IP Finder now supports subfolder usage instead of bucket root
> > * Improved dynamic cache start speed
> > * Improved checkpoint performance by decreasing mark duration.
> > * Added ability to manage compression level for compressed WAL archives.
> > * Added metrics for Entry Processor invocations.
> > * Added JMX metrics: ClusterMetricsMXBean.getTotalBaselineNodes and
> > ClusterMetricsMXBean.getActiveBaselineNodes
> > * Node uptime metric now includes days count
> > * Exposed info about thin client connections through JMX
> > * Inrtoduced new system property IGNITE_REUSE_MEMORY_ON_DEACTIVATE to
> > enable reuse of allocated memory on node deactivation (disabled by
> default)
> > * Optimistic transaction now will be properly rolled back if waiting too
> > long for new topology on remap
> > * ScanQuery with setLocal flag now checks if partition is actually
> present
> > on local node
> > * Improved cluster behavior when left node does not cause partition
> > affinity assignment changes
> > * Interrupting user thread during partition initialization will no longer
> > cause node to stop
> > * Fixed problem when partition lost event were not triggered if multiple
> > nodes left cluster
> > * Fixed massive node drop from cluster on temporary network issues
> > * Fixed service redeployment on cluster reactivation
> > * Fixed client node stability under ZooKeeper discovery
> > * Massive performance and stability improvements
> >
> > Ignite .Net:
> > * Add .NET Core 2.1 support
> > * Added thin client connection failover
> >
> > Ignite C++:
> > * Implemented Thin Client with base cache operations
> > * Implemented smart affinity routing for Thin Client to send requests
> > directly to nodes containing data when possible
> > * Added Clang compiler support
> >
> > SQL:
> > * Added experimental support for fully ACID transactional SQL with
> snapshot
> > isolation:
> > - use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
> > - transaction can be started through native API (IgniteTransactions),
> > thin JDBC driver or ODBC driver
> > - not production ready, data consistency is not guaranteed in case of
> > node failures
> > * Added a set of system views located in "IGNITE" schema to view cluster
> > information (NODES, NODE_ATTRIBUTES, NODE_METRICS, BASELINE_NODES)
> > * Added ability to create predefined SQL schemas
> > * Added GROUP_CONCAT function support
> > * Added string length constraint
> > * Custom Java objects are now inlined into primary and secondary indexes
> > what may significantly improve performance when AFFINITY_KEY is used
> > * Added timeout to fail query execution in case it cannot be mapped to
> > topology
> > * Restricted number of cores allocated for CREATE INDEX by default to 4
> to
> > avoid contention on index tree
> > Fixed transaction hanging during runtime error on commit.
> > * Fixed possible memory leak when result set size is multiple of page
> size
> > * Fixed situation when data may be returned from cache partitoins in LOST
> > state even when PartitionLossPolicy doesn't permit it
> > * Fixed "Caches have distinct sets of data nodes" during SQL JOIN query
> > execution between REPLICATED and PARTITIONED caches
> > * Fixed wrong result for sql queries when item size exceeds page size
> > * Fixed error during SQL query from client node with local flag set to
> > "true"
> > * Fixed handling UUID as column type
> >
> > JDBC:
> > * Implemented DataSource interface for thin driver
> >
> > ODBC:
> > * Added streaming mode support
> > * Fixed crash in Linux when there is more than 1023 open file descriptors
> > * Fixed bug that prevented cursors on server from being closed
> > * Fixed segmentation fault when reusing a closed connection
> >
> > Web Console:
> > * Added new metrics: WAL and Data size on disk
> > * Added support for "collocated" query mode on Query screen
> > * Added support for Java 9+ for Web Agent.
> > * Added ability to show/hide password field value
> > * Implemented execution of selected part of SQL query
> > * Implemented explain of selected part of SQL query
> > * Implemented connection to secured cluster
> > * Implemented responsive full screen layout
> > * Split "Sign In" page to three separate pages
> > * UI updated to modern look and feel
> > * Improved backend stability
> > * Fixed fail when working with web sockets
> >
> > REST:
> > * Added option IGNITE_REST_GETALL_AS_ARRAY for array format in "getAll"
> > call
> >
> > Visor:
> > * Added output of node "Consistent ID"
> > * Visor now collects information about cache groups instead of separate
> > caches to reduce memory consumption
> > * Improved help for "start" command
> > * Fixed output of cache metrics
> >
> > Control utility:
> > * Added information about transaction start time
> > * Added command to collect information about distribution of partitions
> > * Added command to reset lost partitions
> > * Added support for empty label (control.sh --tx label null)
> > * Added atomicity mode to utility output.
> > * Added orphaned local and remote transactions and ability to rollback
> them
> > * Added "--dump" flag to dump current partition state to file.
> > * Renamed command argument '--force' to '--yes'
> > * Removed "initOrder" and "loc keys" from info
> > * Fixed control utility hanging when connected to a joining node with PME
> >
> > ML:
> > * Added TensorFlow integration
> > * Added Estimator API support to TensorFlow cluster on top of Apache
> Ignite
> > * Added ANN algorithm based on ACD concept
> > * Added Random Forest algorithm
> > * Added OneHotEncoder for categorical features
> > * Added model estimation
> > * Added K-fold cross validation for ML models
> > * Added splitter for splitting dataset on test and train subsets
> > * Added ability of filtering data during datasets creation
> > * Added encoding categorical features with One-of-K Encoder
> > * Added MinMax scaler preprocessor
> > * Added gradient boosting for trees
> > * Added indexing for decision trees
> > * Added GDB convergence by error support
> > * Added ability to build pipeline of data preprocessing and model
> training
> > * Added ability to start and maintain TensorFlow cluster on top of Apache
> > Ignite
> > * Added support of Multi-Class for Logistic Regression
> > * Implemented distributed binary logistic regression
> >
> > De[endency updates:
> > * Apache Camel updated to 2.22.0
> > * Apache Commons Beanutils updated to 1.9.3
> > * Apache Hadoop Yarn updated to 2.7.7
> > * Apache Kafka updated to 1.1.0
> > * Apache Lucene updated to 7.4.0
> > * Apache Mesos updated to 1.5.0
> > * Apache Tomcat updated to 9.0.10
> > * Apache Zookeeper updated to 3.4.13
> > * Guava updated to 25.1-jre
> > * Jackson Databind updated to 2.9.6
> > * Jackson 1 usages replaced with Jackson 2
> > * JCraft updated to 0.1.54
> > * H2 version updated to 1.4.197
> > * Log4j 2.x updated to 2.11.0
> > * Netty updated to 4.1.27.Final
> > * RocketMQ updated to 4.3.0
> > * Scala 2.10.x was updated to 2.10.7
> > * Scala 2.11.x updated to 2.11.12
> >
> > On Fri, Nov 2, 2018 at 6:47 PM David Harvey <sy...@gmail.com>
> wrote:
> >
> > > IGNITE-9365 Force backups to different AWS availability zones using
> only
> > > Spring XML
> > >
> > > This deserves documentation enhancements:
> > > https://apacheignite.readme.io/docs/affinity-collocation
> > >
> > > What is the mechanism for proposing an update for 2.7?
> > >
> > >
> > > On Fri, Nov 2, 2018 at 6:31 AM Andrey Kuznetsov <st...@gmail.com>
> > wrote:
> > >
> > > > Great news!
> > > >
> > > > Future release is about to contain mission critical Ignite workers
> > > liveness
> > > > monitoring, introduced in IGNITE-6587.
> > > >
> > > >
> > > > пт, 2 нояб. 2018 г. в 13:23, Nikolay Izhikov <ni...@apache.org>:
> > > >
> > > > > Hello, Guys.
> > > > >
> > > > > Good news! We have 2 final tickets for 2.7.
> > > > > So release date is very near!
> > > > >
> > > > > Let's collect new features and improvements of Ignite 2.7 and
> > includes
> > > it
> > > > > to release notes and other documents.
> > > > >
> > > > > Can you answer and describe your contributions?
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrey Kuznetsov.
> > > >
> > >
> >
>
Re: IGNITE-2.7. New Features
Posted by Nikolay Izhikov <ni...@apache.org>.
Vladimir, Thank you so much for this list!
пн, 5 нояб. 2018 г., 23:25 Vladimir Ozerov vozerov@gridgain.com:
> Igniters,
>
> I assembled information about new features I am aware of. Please find the
> list below and review it, so that we do not miss anything important. Note
> that normally we do not include information about every single fix or minor
> feature, as it will make release notes too big. Instead, we strive to keep
> only the most prominent features/fixes in release notes. Minor things are
> grouped into generic points such as "Stability and performance
> improvements". Also we typically try to place the most important features
> first, to catch user's attention as quickly as possible.
>
> Ignite:
> * Added experimental support for multi-version concurrency control with
> snapshot isolation
> - available for both cache API and SQL
> - use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
> - not production ready, data consistency is not guaranteed in case of
> node failures
> * Implemented Transparent Data Encryption based on JKS certificates
> * Implemented Node.JS Thin Client
> * Implemented Python Thin Client
> * Implemented PHP Thin Client
> * Ignite start scripts now support Java 9 and higher
> * Added ability to set WAL history size in bytes
> * Added SslContextFactory.protocols and SslContextFactory.cipherSuites
> properties to control which SSL encryption algorithms can be used
> * Added JCache 1.1 compliance
> * Added IgniteCompute.withNoResultCache method with semantics similar to
> ComputeTaskNoResultCache annotation
> * Spring Data 2.0 is now supported in separate module
> 'ignite-spring-data_2.0'
> * Added monitoring of critical system workers
> * Added ability to provide custom implementations of ExceptionListener for
> JmsStreamer
> * Ignite KafkaStreamer was upgraded to use new KafkaConsmer configuration
> * S3 IP Finder now supports subfolder usage instead of bucket root
> * Improved dynamic cache start speed
> * Improved checkpoint performance by decreasing mark duration.
> * Added ability to manage compression level for compressed WAL archives.
> * Added metrics for Entry Processor invocations.
> * Added JMX metrics: ClusterMetricsMXBean.getTotalBaselineNodes and
> ClusterMetricsMXBean.getActiveBaselineNodes
> * Node uptime metric now includes days count
> * Exposed info about thin client connections through JMX
> * Inrtoduced new system property IGNITE_REUSE_MEMORY_ON_DEACTIVATE to
> enable reuse of allocated memory on node deactivation (disabled by default)
> * Optimistic transaction now will be properly rolled back if waiting too
> long for new topology on remap
> * ScanQuery with setLocal flag now checks if partition is actually present
> on local node
> * Improved cluster behavior when left node does not cause partition
> affinity assignment changes
> * Interrupting user thread during partition initialization will no longer
> cause node to stop
> * Fixed problem when partition lost event were not triggered if multiple
> nodes left cluster
> * Fixed massive node drop from cluster on temporary network issues
> * Fixed service redeployment on cluster reactivation
> * Fixed client node stability under ZooKeeper discovery
> * Massive performance and stability improvements
>
> Ignite .Net:
> * Add .NET Core 2.1 support
> * Added thin client connection failover
>
> Ignite C++:
> * Implemented Thin Client with base cache operations
> * Implemented smart affinity routing for Thin Client to send requests
> directly to nodes containing data when possible
> * Added Clang compiler support
>
> SQL:
> * Added experimental support for fully ACID transactional SQL with snapshot
> isolation:
> - use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
> - transaction can be started through native API (IgniteTransactions),
> thin JDBC driver or ODBC driver
> - not production ready, data consistency is not guaranteed in case of
> node failures
> * Added a set of system views located in "IGNITE" schema to view cluster
> information (NODES, NODE_ATTRIBUTES, NODE_METRICS, BASELINE_NODES)
> * Added ability to create predefined SQL schemas
> * Added GROUP_CONCAT function support
> * Added string length constraint
> * Custom Java objects are now inlined into primary and secondary indexes
> what may significantly improve performance when AFFINITY_KEY is used
> * Added timeout to fail query execution in case it cannot be mapped to
> topology
> * Restricted number of cores allocated for CREATE INDEX by default to 4 to
> avoid contention on index tree
> Fixed transaction hanging during runtime error on commit.
> * Fixed possible memory leak when result set size is multiple of page size
> * Fixed situation when data may be returned from cache partitoins in LOST
> state even when PartitionLossPolicy doesn't permit it
> * Fixed "Caches have distinct sets of data nodes" during SQL JOIN query
> execution between REPLICATED and PARTITIONED caches
> * Fixed wrong result for sql queries when item size exceeds page size
> * Fixed error during SQL query from client node with local flag set to
> "true"
> * Fixed handling UUID as column type
>
> JDBC:
> * Implemented DataSource interface for thin driver
>
> ODBC:
> * Added streaming mode support
> * Fixed crash in Linux when there is more than 1023 open file descriptors
> * Fixed bug that prevented cursors on server from being closed
> * Fixed segmentation fault when reusing a closed connection
>
> Web Console:
> * Added new metrics: WAL and Data size on disk
> * Added support for "collocated" query mode on Query screen
> * Added support for Java 9+ for Web Agent.
> * Added ability to show/hide password field value
> * Implemented execution of selected part of SQL query
> * Implemented explain of selected part of SQL query
> * Implemented connection to secured cluster
> * Implemented responsive full screen layout
> * Split "Sign In" page to three separate pages
> * UI updated to modern look and feel
> * Improved backend stability
> * Fixed fail when working with web sockets
>
> REST:
> * Added option IGNITE_REST_GETALL_AS_ARRAY for array format in "getAll"
> call
>
> Visor:
> * Added output of node "Consistent ID"
> * Visor now collects information about cache groups instead of separate
> caches to reduce memory consumption
> * Improved help for "start" command
> * Fixed output of cache metrics
>
> Control utility:
> * Added information about transaction start time
> * Added command to collect information about distribution of partitions
> * Added command to reset lost partitions
> * Added support for empty label (control.sh --tx label null)
> * Added atomicity mode to utility output.
> * Added orphaned local and remote transactions and ability to rollback them
> * Added "--dump" flag to dump current partition state to file.
> * Renamed command argument '--force' to '--yes'
> * Removed "initOrder" and "loc keys" from info
> * Fixed control utility hanging when connected to a joining node with PME
>
> ML:
> * Added TensorFlow integration
> * Added Estimator API support to TensorFlow cluster on top of Apache Ignite
> * Added ANN algorithm based on ACD concept
> * Added Random Forest algorithm
> * Added OneHotEncoder for categorical features
> * Added model estimation
> * Added K-fold cross validation for ML models
> * Added splitter for splitting dataset on test and train subsets
> * Added ability of filtering data during datasets creation
> * Added encoding categorical features with One-of-K Encoder
> * Added MinMax scaler preprocessor
> * Added gradient boosting for trees
> * Added indexing for decision trees
> * Added GDB convergence by error support
> * Added ability to build pipeline of data preprocessing and model training
> * Added ability to start and maintain TensorFlow cluster on top of Apache
> Ignite
> * Added support of Multi-Class for Logistic Regression
> * Implemented distributed binary logistic regression
>
> De[endency updates:
> * Apache Camel updated to 2.22.0
> * Apache Commons Beanutils updated to 1.9.3
> * Apache Hadoop Yarn updated to 2.7.7
> * Apache Kafka updated to 1.1.0
> * Apache Lucene updated to 7.4.0
> * Apache Mesos updated to 1.5.0
> * Apache Tomcat updated to 9.0.10
> * Apache Zookeeper updated to 3.4.13
> * Guava updated to 25.1-jre
> * Jackson Databind updated to 2.9.6
> * Jackson 1 usages replaced with Jackson 2
> * JCraft updated to 0.1.54
> * H2 version updated to 1.4.197
> * Log4j 2.x updated to 2.11.0
> * Netty updated to 4.1.27.Final
> * RocketMQ updated to 4.3.0
> * Scala 2.10.x was updated to 2.10.7
> * Scala 2.11.x updated to 2.11.12
>
> On Fri, Nov 2, 2018 at 6:47 PM David Harvey <sy...@gmail.com> wrote:
>
> > IGNITE-9365 Force backups to different AWS availability zones using only
> > Spring XML
> >
> > This deserves documentation enhancements:
> > https://apacheignite.readme.io/docs/affinity-collocation
> >
> > What is the mechanism for proposing an update for 2.7?
> >
> >
> > On Fri, Nov 2, 2018 at 6:31 AM Andrey Kuznetsov <st...@gmail.com>
> wrote:
> >
> > > Great news!
> > >
> > > Future release is about to contain mission critical Ignite workers
> > liveness
> > > monitoring, introduced in IGNITE-6587.
> > >
> > >
> > > пт, 2 нояб. 2018 г. в 13:23, Nikolay Izhikov <ni...@apache.org>:
> > >
> > > > Hello, Guys.
> > > >
> > > > Good news! We have 2 final tickets for 2.7.
> > > > So release date is very near!
> > > >
> > > > Let's collect new features and improvements of Ignite 2.7 and
> includes
> > it
> > > > to release notes and other documents.
> > > >
> > > > Can you answer and describe your contributions?
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrey Kuznetsov.
> > >
> >
>
Re: IGNITE-2.7. New Features
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Igniters,
I assembled information about new features I am aware of. Please find the
list below and review it, so that we do not miss anything important. Note
that normally we do not include information about every single fix or minor
feature, as it will make release notes too big. Instead, we strive to keep
only the most prominent features/fixes in release notes. Minor things are
grouped into generic points such as "Stability and performance
improvements". Also we typically try to place the most important features
first, to catch user's attention as quickly as possible.
Ignite:
* Added experimental support for multi-version concurrency control with
snapshot isolation
- available for both cache API and SQL
- use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
- not production ready, data consistency is not guaranteed in case of
node failures
* Implemented Transparent Data Encryption based on JKS certificates
* Implemented Node.JS Thin Client
* Implemented Python Thin Client
* Implemented PHP Thin Client
* Ignite start scripts now support Java 9 and higher
* Added ability to set WAL history size in bytes
* Added SslContextFactory.protocols and SslContextFactory.cipherSuites
properties to control which SSL encryption algorithms can be used
* Added JCache 1.1 compliance
* Added IgniteCompute.withNoResultCache method with semantics similar to
ComputeTaskNoResultCache annotation
* Spring Data 2.0 is now supported in separate module
'ignite-spring-data_2.0'
* Added monitoring of critical system workers
* Added ability to provide custom implementations of ExceptionListener for
JmsStreamer
* Ignite KafkaStreamer was upgraded to use new KafkaConsmer configuration
* S3 IP Finder now supports subfolder usage instead of bucket root
* Improved dynamic cache start speed
* Improved checkpoint performance by decreasing mark duration.
* Added ability to manage compression level for compressed WAL archives.
* Added metrics for Entry Processor invocations.
* Added JMX metrics: ClusterMetricsMXBean.getTotalBaselineNodes and
ClusterMetricsMXBean.getActiveBaselineNodes
* Node uptime metric now includes days count
* Exposed info about thin client connections through JMX
* Inrtoduced new system property IGNITE_REUSE_MEMORY_ON_DEACTIVATE to
enable reuse of allocated memory on node deactivation (disabled by default)
* Optimistic transaction now will be properly rolled back if waiting too
long for new topology on remap
* ScanQuery with setLocal flag now checks if partition is actually present
on local node
* Improved cluster behavior when left node does not cause partition
affinity assignment changes
* Interrupting user thread during partition initialization will no longer
cause node to stop
* Fixed problem when partition lost event were not triggered if multiple
nodes left cluster
* Fixed massive node drop from cluster on temporary network issues
* Fixed service redeployment on cluster reactivation
* Fixed client node stability under ZooKeeper discovery
* Massive performance and stability improvements
Ignite .Net:
* Add .NET Core 2.1 support
* Added thin client connection failover
Ignite C++:
* Implemented Thin Client with base cache operations
* Implemented smart affinity routing for Thin Client to send requests
directly to nodes containing data when possible
* Added Clang compiler support
SQL:
* Added experimental support for fully ACID transactional SQL with snapshot
isolation:
- use CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT to enable it
- transaction can be started through native API (IgniteTransactions),
thin JDBC driver or ODBC driver
- not production ready, data consistency is not guaranteed in case of
node failures
* Added a set of system views located in "IGNITE" schema to view cluster
information (NODES, NODE_ATTRIBUTES, NODE_METRICS, BASELINE_NODES)
* Added ability to create predefined SQL schemas
* Added GROUP_CONCAT function support
* Added string length constraint
* Custom Java objects are now inlined into primary and secondary indexes
what may significantly improve performance when AFFINITY_KEY is used
* Added timeout to fail query execution in case it cannot be mapped to
topology
* Restricted number of cores allocated for CREATE INDEX by default to 4 to
avoid contention on index tree
Fixed transaction hanging during runtime error on commit.
* Fixed possible memory leak when result set size is multiple of page size
* Fixed situation when data may be returned from cache partitoins in LOST
state even when PartitionLossPolicy doesn't permit it
* Fixed "Caches have distinct sets of data nodes" during SQL JOIN query
execution between REPLICATED and PARTITIONED caches
* Fixed wrong result for sql queries when item size exceeds page size
* Fixed error during SQL query from client node with local flag set to
"true"
* Fixed handling UUID as column type
JDBC:
* Implemented DataSource interface for thin driver
ODBC:
* Added streaming mode support
* Fixed crash in Linux when there is more than 1023 open file descriptors
* Fixed bug that prevented cursors on server from being closed
* Fixed segmentation fault when reusing a closed connection
Web Console:
* Added new metrics: WAL and Data size on disk
* Added support for "collocated" query mode on Query screen
* Added support for Java 9+ for Web Agent.
* Added ability to show/hide password field value
* Implemented execution of selected part of SQL query
* Implemented explain of selected part of SQL query
* Implemented connection to secured cluster
* Implemented responsive full screen layout
* Split "Sign In" page to three separate pages
* UI updated to modern look and feel
* Improved backend stability
* Fixed fail when working with web sockets
REST:
* Added option IGNITE_REST_GETALL_AS_ARRAY for array format in "getAll" call
Visor:
* Added output of node "Consistent ID"
* Visor now collects information about cache groups instead of separate
caches to reduce memory consumption
* Improved help for "start" command
* Fixed output of cache metrics
Control utility:
* Added information about transaction start time
* Added command to collect information about distribution of partitions
* Added command to reset lost partitions
* Added support for empty label (control.sh --tx label null)
* Added atomicity mode to utility output.
* Added orphaned local and remote transactions and ability to rollback them
* Added "--dump" flag to dump current partition state to file.
* Renamed command argument '--force' to '--yes'
* Removed "initOrder" and "loc keys" from info
* Fixed control utility hanging when connected to a joining node with PME
ML:
* Added TensorFlow integration
* Added Estimator API support to TensorFlow cluster on top of Apache Ignite
* Added ANN algorithm based on ACD concept
* Added Random Forest algorithm
* Added OneHotEncoder for categorical features
* Added model estimation
* Added K-fold cross validation for ML models
* Added splitter for splitting dataset on test and train subsets
* Added ability of filtering data during datasets creation
* Added encoding categorical features with One-of-K Encoder
* Added MinMax scaler preprocessor
* Added gradient boosting for trees
* Added indexing for decision trees
* Added GDB convergence by error support
* Added ability to build pipeline of data preprocessing and model training
* Added ability to start and maintain TensorFlow cluster on top of Apache
Ignite
* Added support of Multi-Class for Logistic Regression
* Implemented distributed binary logistic regression
De[endency updates:
* Apache Camel updated to 2.22.0
* Apache Commons Beanutils updated to 1.9.3
* Apache Hadoop Yarn updated to 2.7.7
* Apache Kafka updated to 1.1.0
* Apache Lucene updated to 7.4.0
* Apache Mesos updated to 1.5.0
* Apache Tomcat updated to 9.0.10
* Apache Zookeeper updated to 3.4.13
* Guava updated to 25.1-jre
* Jackson Databind updated to 2.9.6
* Jackson 1 usages replaced with Jackson 2
* JCraft updated to 0.1.54
* H2 version updated to 1.4.197
* Log4j 2.x updated to 2.11.0
* Netty updated to 4.1.27.Final
* RocketMQ updated to 4.3.0
* Scala 2.10.x was updated to 2.10.7
* Scala 2.11.x updated to 2.11.12
On Fri, Nov 2, 2018 at 6:47 PM David Harvey <sy...@gmail.com> wrote:
> IGNITE-9365 Force backups to different AWS availability zones using only
> Spring XML
>
> This deserves documentation enhancements:
> https://apacheignite.readme.io/docs/affinity-collocation
>
> What is the mechanism for proposing an update for 2.7?
>
>
> On Fri, Nov 2, 2018 at 6:31 AM Andrey Kuznetsov <st...@gmail.com> wrote:
>
> > Great news!
> >
> > Future release is about to contain mission critical Ignite workers
> liveness
> > monitoring, introduced in IGNITE-6587.
> >
> >
> > пт, 2 нояб. 2018 г. в 13:23, Nikolay Izhikov <ni...@apache.org>:
> >
> > > Hello, Guys.
> > >
> > > Good news! We have 2 final tickets for 2.7.
> > > So release date is very near!
> > >
> > > Let's collect new features and improvements of Ignite 2.7 and includes
> it
> > > to release notes and other documents.
> > >
> > > Can you answer and describe your contributions?
> > >
> >
> >
> > --
> > Best regards,
> > Andrey Kuznetsov.
> >
>
Re: IGNITE-2.7. New Features
Posted by David Harvey <sy...@gmail.com>.
IGNITE-9365 Force backups to different AWS availability zones using only
Spring XML
This deserves documentation enhancements:
https://apacheignite.readme.io/docs/affinity-collocation
What is the mechanism for proposing an update for 2.7?
On Fri, Nov 2, 2018 at 6:31 AM Andrey Kuznetsov <st...@gmail.com> wrote:
> Great news!
>
> Future release is about to contain mission critical Ignite workers liveness
> monitoring, introduced in IGNITE-6587.
>
>
> пт, 2 нояб. 2018 г. в 13:23, Nikolay Izhikov <ni...@apache.org>:
>
> > Hello, Guys.
> >
> > Good news! We have 2 final tickets for 2.7.
> > So release date is very near!
> >
> > Let's collect new features and improvements of Ignite 2.7 and includes it
> > to release notes and other documents.
> >
> > Can you answer and describe your contributions?
> >
>
>
> --
> Best regards,
> Andrey Kuznetsov.
>
Re: IGNITE-2.7. New Features
Posted by Andrey Kuznetsov <st...@gmail.com>.
Great news!
Future release is about to contain mission critical Ignite workers liveness
monitoring, introduced in IGNITE-6587.
пт, 2 нояб. 2018 г. в 13:23, Nikolay Izhikov <ni...@apache.org>:
> Hello, Guys.
>
> Good news! We have 2 final tickets for 2.7.
> So release date is very near!
>
> Let's collect new features and improvements of Ignite 2.7 and includes it
> to release notes and other documents.
>
> Can you answer and describe your contributions?
>
--
Best regards,
Andrey Kuznetsov.