What is Distributed Programming?

Distributed programming is one form of parallel programming or parallel computing. Parallel programming involves computers and computing units within computers working concurrently on a particular problem, such as predicting tomorrow's weather. The computing units can be very closely located and coordinated or can be located apart. When the computing units are apart, it is called distributed programming. In such a scenario, very often the computing units differ from one another, and the operating system and network setup differs too, making programming of the computing activity particularly challenging.

When solving a problem in a distributed manner, the program needs to be split up so that parts of the program can run on the different computing units; these parts are often called "processes." The processes run simultaneously but need to communicate inputs and results to one another. If the processes are running on different hardware, such as one part running on Intel and another running on SUN, then the programs have to be compiled and optimized differently.

One way to solve a sufficiently difficult problem is to break up the input parts, and have the different computing units work on the different parts using the same algorithm, the set of rules or steps for problem solving. For example, to crack a genome of 10,000 pairs, the first 1,000 pairs could be assigned to the first computing unit, the second 1,000 pairs assigned to the second computing unit and so on, all using the same algorithm. With distributed programming, one advantage is that the different computing units could run different algorithms to solve the same problem, thus leading to a significantly better solution. This is akin to solving a jigsaw puzzle with some people piecing together the border while others put together pieces of a particular color.

Coordination of the distributed computing processes can be a particularly difficult task. Some computing units may fail or may get interrupted to handle other work. Messages containing the inputs or the results of the computation may fail to reach their destinations. If the programs are written in a naïve manner, then the loss of a computing unit or some messages can cause the entire set of computers to hang.

In distributed programming, one process could be the controlling process, essentially getting work done by the other processes, or, all processes could work in a peer-to-peer manner with no process being the "master." Some examples of problems attempted with distributed programming include analyzing geological data for resources such as petroleum, the modeling of proteins and biological molecules, the cracking of coded messages, and military simulations. The SETI project to search for intelligent extra-terrestrial life from the radio messages received by Earth is perhaps one of the best-known examples.