How the Apple Qmaster System Distributes Batches
The Apple Qmaster cluster controller determines the most efficient use of the cluster
resources. It makes this determination based on the availability of each service node and
the number of separable parts (described next) of the batch.
Because Apple Qmaster subdivides individual batches across different service nodes, the
work is shared and completed more quickly. And, because this method uses all the service
nodes as much as possible, you avoid under-utilizing your resources.
352
Chapter 29
Apple Qmaster and Distributed Processing
Batches can be distributed to a cluster by the cluster controller in one or both of the
following ways. (Apple Qmaster determines which way is the most efficient for specific
batches, depending on the circumstances.)
• The batch is subdivided into data segments: For example, for a render batch, the cluster
controller could divide the frames into groups (segments). Each segment would be
processed in parallel on the service nodes in the cluster.
• The batch is subdivided into tasks: For example, for a render batch, the cluster controller
could subdivide the rendering work into different processing tasks. Different tasks
would be run on different service nodes.
Rather than actually moving segments, Apple Qmaster tells the service nodes which
segments to read via the network, where to find them, and what to do with them. Below
is an example of how one batch could be processed in an Apple Qmaster system.
Batch submitted by
Compressor or
Apple Qmaster (job request
for frames 1–30)
Cluster controller
divides and distributes
job to available
service nodes
Service node 1
Service node 2
Service node 3
Instructions specifying
locations of source files
and frames 11–20
Instructions specifying
locations of source files
and frames 21–30
Instructions specifying
locations of source files
and frames 01–10
Processed file (frames 1-30)
placed in specified
destination
In distributing batches, Apple Qmaster uses the technology built in to Mac OS X to locate
services in a cluster on the same IP subnet and to dynamically share and receive
information. Because the computers can continually transmit their current processing
availability status, Apple Qmaster can distribute (load-balance) the workload evenly across
the cluster.
353
Chapter 29
Apple Qmaster and Distributed Processing