This library provides a modular toolkit for data processing, allowing for efficient and flexible handling of data.
MDP (Modular toolkit for Data Processing) is a powerful software library that provides a wide range of data processing algorithms that can be easily combined to build highly complex data processing software. The software library offers a collection of supervised and unsupervised learning algorithms, among other data processing units (nodes) that can be combined into data processing sequences (flows) and more complex feed-forward network architectures.
MDP takes care of training or executing all nodes in the network, given a set of input data. This, in turn, allows the user to specify complex algorithms as simpler data processing steps in a natural way. MDP offers a steadily increasing base of available algorithms that includes, among others, Principal Component Analysis, several Independent Component Analysis algorithms, Slow Feature Analysis, Gaussian Classifiers, Restricted Boltzmann Machine, and Locally Linear Embedding.
Efficiency is highly considered in MDP's computations in terms of speed and memory. It is possible to perform learning using batches of data, and define the internal parameters of nodes to be single precision. The 'parallel' subpackage offers a parallel implementation of the basic nodes and flows, which drastically reduces computation time.
From the developer's perspective, MDP is a highly flexible and customizable framework that makes the implementation of new supervised and unsupervised learning algorithms easy and straightforward. The basic class, 'Node', takes care of tedious tasks such as numerical type and dimensionality checking, leaving the developer free to concentrate on the implementation of the learning and execution phases. The node then automatically integrates with the rest of the library, making it highly reusable. A node can have multiple training phases and even an undetermined number of phases, which allows for the implementation of algorithms that need to collect some statistics on the whole input before proceeding with the actual training.
MDP has been designed to be helpful in any context where trainable data processing algorithms are used. Though it was initially created as a tool for theoretical research in neuroscience, its simplicity on the user side together with the reusability of the implemented nodes make it also a valid educational tool. Additionally, the software offers an optional feature for crash recovery, which saves the current state of the flow in case of failure for later inspection.
MDP takes care of training or executing all nodes in the network, given a set of input data. This, in turn, allows the user to specify complex algorithms as simpler data processing steps in a natural way. MDP offers a steadily increasing base of available algorithms that includes, among others, Principal Component Analysis, several Independent Component Analysis algorithms, Slow Feature Analysis, Gaussian Classifiers, Restricted Boltzmann Machine, and Locally Linear Embedding.
Efficiency is highly considered in MDP's computations in terms of speed and memory. It is possible to perform learning using batches of data, and define the internal parameters of nodes to be single precision. The 'parallel' subpackage offers a parallel implementation of the basic nodes and flows, which drastically reduces computation time.
From the developer's perspective, MDP is a highly flexible and customizable framework that makes the implementation of new supervised and unsupervised learning algorithms easy and straightforward. The basic class, 'Node', takes care of tedious tasks such as numerical type and dimensionality checking, leaving the developer free to concentrate on the implementation of the learning and execution phases. The node then automatically integrates with the rest of the library, making it highly reusable. A node can have multiple training phases and even an undetermined number of phases, which allows for the implementation of algorithms that need to collect some statistics on the whole input before proceeding with the actual training.
MDP has been designed to be helpful in any context where trainable data processing algorithms are used. Though it was initially created as a tool for theoretical research in neuroscience, its simplicity on the user side together with the reusability of the implemented nodes make it also a valid educational tool. Additionally, the software offers an optional feature for crash recovery, which saves the current state of the flow in case of failure for later inspection.
What's New
Version 2.5: N/A