Compressor 4 - Other Possible Components of a Distributed Processing Network

background image

Other Possible Components of a Distributed Processing Network

A distributed processing network can consist of as few as one or two computers, whereas
a high-volume network may include many computers, an Xserve system and Xserve
cluster nodes in a rack, and high-speed networking infrastructures. You can scale up a
distributed processing system as your workload demands by adding features and devices
to the network that supports it.

There are many ways to expand the capacity of a distributed processing network. You
could include any of the following.

High-speed switch and cables: A 100Base-T or Gigabit Ethernet switch and compatible

cables to allow your data to move over the LAN at maximum speed.

Multiple clients: Multiple client computers can use the services of the same cluster. And,

you can have multiple client applications on the same client computer, using the same
cluster.

Multiple clusters: Depending on how extensive your network is and how many clients

it needs to serve, you may want to divide up available computers and create more than
one cluster to serve various clients. (Users select the cluster they want to send a batch
to when they submit the batch.)

Multiple service nodes: In general, more service nodes mean more processing power.

In deciding how many service nodes to have in a cluster, consider the ratio of data
movement time to computing time. If the processing demand is greater than the
network demand required to move job segments throughout the cluster, as is the case
with rendering, more service nodes are a good idea. If the computing load, per job, is
closer to the network load, having a smaller number of service nodes per cluster may
be more efficient. If you are using the Apple Qmaster distributed processing system
with applications other than Shake or Compressor, consult the application’s user manual
on how to optimize the number of service nodes.

Storage device: A storage device, such as a remote disk or group of disk arrays, can be

used as cluster scratch storage, which is a place for short-term storage of temporary
data generated by the cluster controller, clients, and service nodes. (You set the scratch
storage location in the Apple Qmaster Sharing window of Compressor. See

About

Advanced Settings in the Apple Qmaster Sharing Window

and

Using Cluster Storage

for more information.) Alternatively, a storage device can be used as a final destination
for the files after they are processed.

For more information, see

Examples of a Distributed Processing Network

.