You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by arnab phani <ph...@gmail.com> on 2021/06/30 14:08:08 UTC

Release notes for SystemDS 2.1

Hi All,

Please find the proposed release notes for release 2.1.
Feel free to add to it.

SystemDS 2.1 is a minor release. This release contains major improvements
to existing features, a large number of fixes, and some experimental
features to better support the end-to-end data science lifecycle.
In addition to that, this is the last release with Java 8 and Spark 2.x.

The major changes (compared to SystemDS 2.0) include

- Federated Backend: Extended operations and built-ins to support various
workflows and algorithms. Extended parameter server for federated training
of Neural Networks.
- A wealth of new and improved built-in functions including support for
string operations, time-series analysis, data cleaning, grid search
hyperparameter tuning, multi-threaded feature transformations.
- Better support for Eval and list data structure.
- Various performance improvements in SystemDS' local, distributed, and
native runtimes.
- Python API: Source functionality to allow arbitrary DML script,
auto-generated Python API for all built-in functions,  list support
allowing Pytonic access and manipulation of lazily evaluated DML elements,
multi return continuation, allowing continued execution past calls to
builtin functions.
- GPU Backend: Various performance improvements and fixes, cuda code
refactor, cuda code generation [experimental].
- Lineage Tracing and Reuse: Initial support for federated and GPU
backends, new hybrid lineage deduplication and reuse of compressed lineage
DAGs, lineage cache benefits estimator, improved cache eviction logic.
- Usability: Better Log4j integration, debug flag to reduce error logging,
support for remote debugging.
- Miscellaneous Improvements: Improved documentation, better testing,
streamlined release process.