The cexec software facilitates the creation of ad-hoc clusters with ease.
cexec is a software designed for building ad-hoc clusters, which means that you can use it to create a cluster with a group of general-purpose UNIX-like machines. To install cexec, you will first need to build it by typing "make". After building cexec, you can copy the relevant executables into /usr/local/bin or any other place that is in your path. It would also be helpful to copy the manual pages (*.1) into your $MANPATH. Do keep in mind that the cexec on-wire protocol is still evolving in incompatible ways, which means that it's not a good idea to use the cluster tools across administrative domains.
A cexec "cluster" consists of one or several applications, an announcement address, a group of general-purpose UNIX-like machines, and a keypair that identifies applications and mutually authenticates clients and servers. The announcement address can either be a broadcast address, multicast address or even a unicast address, provided that it allows for other servers. The default value for the announcement address should suffice for most people (255.255.255.255), and this value should be stored in the $GROUP environment variable.
To build a cexec cluster, you need to decide on your applications, choose an announcement address, and have computers to run it. As an example, you can build a cluster-enabled version of "oggenc". To build this cluster, you will first need to build the keypair using "ckeygen". Then, you will need to distribute the "distributed_ogg" key to all of the worker machines and the "distributed_ogg.pub" key to all of the client machines. Afterward, you can start the service on all your workers and also start a logger service on any worker or client. Finally, to encode something, you can use any application (not just "oggenc") with this cluster. You can make this cluster as big as you want (with multicast tunnels) and cross as many networks as you want (with cproxy).
When "cexec" starts up, it locates the "best" copy of "cservice" on the network by broadcasting announcements. One of the "cservice" machines attempts to "connect back" to the cexec after a delay that's proportional to the systems' load. The first machine to "reach back" and perform the various challenges regarding the keypair is the winner. At this point, cexec multiplexes the local file descriptors over the work-channel, and cservice does the reverse on the other side. "cservice" uses pipes where possible, but will use socketpair() to emulate readwrite devices like terminals and sockets. When "cservice" is done, it sends its exit code back to "cexec". If "cexec" didn't like any part of the protocol exchange, it "complains". If everything went okay, it announces the exit code in the same way. These "alerts" are received by a "crat" running on the network.
The latest release of cexec doesn't regenerate parity, which should help acquire loaded hosts (above runq length 10.0) faster.
A cexec "cluster" consists of one or several applications, an announcement address, a group of general-purpose UNIX-like machines, and a keypair that identifies applications and mutually authenticates clients and servers. The announcement address can either be a broadcast address, multicast address or even a unicast address, provided that it allows for other servers. The default value for the announcement address should suffice for most people (255.255.255.255), and this value should be stored in the $GROUP environment variable.
To build a cexec cluster, you need to decide on your applications, choose an announcement address, and have computers to run it. As an example, you can build a cluster-enabled version of "oggenc". To build this cluster, you will first need to build the keypair using "ckeygen". Then, you will need to distribute the "distributed_ogg" key to all of the worker machines and the "distributed_ogg.pub" key to all of the client machines. Afterward, you can start the service on all your workers and also start a logger service on any worker or client. Finally, to encode something, you can use any application (not just "oggenc") with this cluster. You can make this cluster as big as you want (with multicast tunnels) and cross as many networks as you want (with cproxy).
When "cexec" starts up, it locates the "best" copy of "cservice" on the network by broadcasting announcements. One of the "cservice" machines attempts to "connect back" to the cexec after a delay that's proportional to the systems' load. The first machine to "reach back" and perform the various challenges regarding the keypair is the winner. At this point, cexec multiplexes the local file descriptors over the work-channel, and cservice does the reverse on the other side. "cservice" uses pipes where possible, but will use socketpair() to emulate readwrite devices like terminals and sockets. When "cservice" is done, it sends its exit code back to "cexec". If "cexec" didn't like any part of the protocol exchange, it "complains". If everything went okay, it announces the exit code in the same way. These "alerts" are received by a "crat" running on the network.
The latest release of cexec doesn't regenerate parity, which should help acquire loaded hosts (above runq length 10.0) faster.
What's New
Version 1.26: N/A