In Search of Clusters: : The Coming Battle in Lowly Parallel Computing

Citation

Galletly, J. (1998), "In Search of Clusters: : The Coming Battle in Lowly Parallel Computing", Kybernetes, Vol. 27 No. 4, pp. 453-455. https://doi.org/10.1108/k.1998.27.4.453.2

This book describes Pfister’s crusade for lowly (the author’s word) parallel computer systems. In the author’s words, the book “discusses both the hardware and the software of lowly parallel computers, the everyday, practical work gangs of computing: symmetric multiprocessors (SMPs), and in particular, clusters of computers”. At first, I was confused. The way that SMPs and clusters were both referred to by the author, I thought both were being championed. But, no, further reading reveals that the SMP is, in fact, the chief competitor to the cluster. The author’s objective is to show this.

I must admit that I was rather sceptical when I first started reading this book. I thought that I would find the author’s rather “hip”, jokey style too off‐putting. But no, on the contrary, after a few pages, I found it to be infectious, and I read on. As well as a wealth of technical information, the book is full of interesting anecdotes and digs at the computing establishment. Pfister is no respecter of reputations. As he says, “… many are also going to find something to be annoyed about … The humour‐ and irony‐impaired will also have a bit of trouble here and there”.

The book is in four parts. Part 1 (Chapters 1‐4) is entitled “What is a cluster, and why use them?”; Parts 2 (Chapters 5‐6) and 3 (Chapters 7‐10) deal with “Hardware” and “Software”, respectively; while Part 4 (Chapters 11‐13) concludes the book with an examination of “Systems”.

Part 1 is what the title says it is: a description of clusters and their uses. However, Pfister makes the reader wait until Chapter 4 before he gives a definition of a cluster:

A cluster is a type of parallel or distributed system that consists of a collection of interconnected whole computers and is utilized as a single, unified computing resource.

In the preceding three chapters, the reader is carefully introduced to the concept of clusters, with examples which illustrate their main characteristics, and is presented with arguments to justify their usefulness.

The thrust of Part 2 is a more detailed discussion of the hardware of clusters and their competitor, the SMP. Like all good architecture designers, Pfister first describes a broad classification of cluster hardware organization in terms of their memory and I/O subsystems. This gives rise to four categories (one of which has no current examples!): I/O‐attached message‐based and shared storage systems; and memory‐attached message‐based and shared storage systems. The second chapter of Part 2 is given over wholly to the SMP, albeit to detail their shortcomings. Here, too, a definition of an SMP is provided. Pfister examines various architectural issues. The various problems of cache memories on multiprocessor systems are discussed at length.

Part 3 deals with the software concerns surrounding SMPs and clusters. While the previous part dealt mainly with cache memory problems and their solutions, this part weighs in, thick and fast, with a number of parallel programming issues. It bolsters the stark message that “Programs written to exploit SMP parallelism will not work on clusters” and vice versa. The shared memory model of SMPs and the message‐passing model of clusters are reviewed, and their pitfalls and intricacies examined by way of examples such as the programming of a (simplified!) wave equation. After the theory, another chapter deals with commercial programming models, and touches on problems such as distributed I/O access.

Finally, in Part 4, the author presents a comparison of SMPs versus clusters, based, again, on local memory and I/O. Scaling and performance issues such as memory bandwidth and memory contention are examined. Not surprisingly, clusters win, but not overwhelmingly so. The disproportionate costs of large SMP systems is a telling factor.

Pfister’s book is very readable; it has a serious message which is presented in a light‐hearted, sometimes irreverent, manner. The author explains technical concepts simply, aided with clearly illustrated diagrams. The book is accessible to anyone with a general background knowledge of modern computer systems, although some exposure to parallel systems would help. It is definitely not an academic textbook, but then it does not purport to be, although some highly technical details are very neatly explained.