Karmasphere DP is a parallel language that is designed to deliver high-performance data processing capabilities. This non-blocking programming language is built to help users process large volumes of data efficiently.
Karmasphere DP language is a software that offers high-performance non-blocking parallel language for data processing. This project provides users with total control over the usage of system resources, such as CPU cores and disk I/O time, without requiring developers to add code to handle these resources. The original purpose was to collect attributes for URLs and domain names to use in an anti-spam system, but it has since turned into a comprehensive parallel programming language with many general-purpose operators.
The DP language is available as a standalone library that can be used in any Java 1.5 environment. Additionally, it can fully take advantage of multiprocessor systems like SMP or NUMA and can be scaled sideways. This is due to the interpreter and environment being stateless. Consequently, an entire cluster of machines can run the interpreter in parallel without requiring synchronization.
Traditional, sequential programs have to execute each instruction in order. In case an instruction requires CPU, disk, or network resources, it has to wait until the resources are available. This can create terrible performance issues, especially for network latency, that can be unpredictable. While it is possible to optimize resource usage in complex sequential programs, the DP language solves this problem by making parallel programming very easy.
In the DP language, programs are workflows that can be graphically represented with boxes and arrows notation. Every operation executes concurrently when the necessary resources are available. This means that operations that slow down traditional sequential programs don’t affect DP programs. The source language is almost identical to GraphViz, and debugging output from the interpreter is also in GraphViz format.
The core language comes with basic operators for processing and network operations, but it also supports developing additional operators using the provided framework. The entire interpreter is available as an API and can be executed on the command line, embedded in a query server, job server, RPC daemon, or any useful location. Additionally, it only uses resources specified when active and doesn’t consume when idle.
Furthermore, the DP language allows explicit typing of each arrow, using any Java type. Programs can be type-checked at compile-time if types are specified. Although the DP interpreter does not have to be aware of every type in the system, it can make inferences about types if they are registered.
The DP language is available as a standalone library that can be used in any Java 1.5 environment. Additionally, it can fully take advantage of multiprocessor systems like SMP or NUMA and can be scaled sideways. This is due to the interpreter and environment being stateless. Consequently, an entire cluster of machines can run the interpreter in parallel without requiring synchronization.
Traditional, sequential programs have to execute each instruction in order. In case an instruction requires CPU, disk, or network resources, it has to wait until the resources are available. This can create terrible performance issues, especially for network latency, that can be unpredictable. While it is possible to optimize resource usage in complex sequential programs, the DP language solves this problem by making parallel programming very easy.
In the DP language, programs are workflows that can be graphically represented with boxes and arrows notation. Every operation executes concurrently when the necessary resources are available. This means that operations that slow down traditional sequential programs don’t affect DP programs. The source language is almost identical to GraphViz, and debugging output from the interpreter is also in GraphViz format.
The core language comes with basic operators for processing and network operations, but it also supports developing additional operators using the provided framework. The entire interpreter is available as an API and can be executed on the command line, embedded in a query server, job server, RPC daemon, or any useful location. Additionally, it only uses resources specified when active and doesn’t consume when idle.
Furthermore, the DP language allows explicit typing of each arrow, using any Java type. Programs can be type-checked at compile-time if types are specified. Although the DP interpreter does not have to be aware of every type in the system, it can make inferences about types if they are registered.
What's New
Version 3.0.2: N/A