parallel processing
Home > Data Center Definitions - Parallel processing
SearchDataCenter.com Definitions (Powered by WhatIs.com)
EMAIL THIS
LOOK UP TECH TERMS Powered by: WhatIs.com
Search listings for thousands of IT terms:
Browse tech terms alphabetically:
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z #

parallel processing


Show me everything on Cloud Computing, grid and utility computing

DEFINITION - In computers, parallel processing is the processing of program instructions by dividing them among multiple processors with the objective of running a program in less time. In the earliest computers, only one program ran at a time. A computation-intensive program that took one hour to run and a tape copying program that took one hour to run would take a total of two hours to run. An early form of parallel processing allowed the interleaved execution of both programs together. The computer would start an I/O operation, and while it was waiting for the operation to complete, it would execute the processor-intensive program. The total execution time for the two jobs would be a little over one hour.

The next improvement was multiprogramming. In a multiprogramming system, multiple programs submitted by users were each allowed to use the processor for a short time. To users it appeared that all of the programs were executing at the same time. Problems of resource contention first arose in these systems. Explicit requests for resources led to the problem of the deadlock. Competition for resources on machines with no tie-breaking instructions lead to the critical section routine.

Vector processing was another attempt to increase performance by doing more than one thing at a time. In this case, capabilities were added to machines to allow a single instruction to add (or subtract, or multiply, or otherwise manipulate) two arrays of numbers. This was valuable in certain engineering applications where data naturally occurred in the form of vectors or matrices. In applications with less well-formed data, vector processing was not so valuable.

The next step in parallel processing was the introduction of multiprocessing. In these systems, two or more processors shared the work to be done. The earliest versions had a master/slave configuration. One processor (the master) was programmed to be responsible for all of the work in the system; the other (the slave) performed only those tasks it was assigned by the master. This arrangement was necessary because it was not then understood how to program the machines so they could cooperate in managing the resources of the system.

Solving these problems led to the symmetric multiprocessing system (SMP). In an SMP system, each processor is equally capable and responsible for managing the flow of work through the system. Initially, the goal was to make SMP systems appear to programmers to be exactly the same as single processor, multiprogramming systems. (This standard of behavior is known as sequential consistency). However, engineers found that system performance could be increased by someplace in the range of 10-20% by executing some instructions out of order and requiring programmers to deal with the increased complexity. (The problem can become visible only when two or more programs simultaneously read and write the same operands; thus the burden of dealing with the increased complexity falls on only a very few programmers and then only in very specialized circumstances.) The question of how SMP machines should behave on shared data is not yet resolved.

As the number of processors in SMP systems increases, the time it takes for data to propagate from one part of the system to all other parts grows also. When the number of processors is somewhere in the range of several dozen, the performance benefit of adding more processors to the system is too small to justify the additional expense. To get around the problem of long propagation times, message passing systems were created. In these systems, programs that share data send messages to each other to announce that particular operands have been assigned a new value. Instead of a broadcast of an operand's new value to all parts of a system, the new value is communicated only to those programs that need to know the new value. Instead of a shared memory, there is a network to support the transfer of messages between programs. This simplification allows hundreds, even thousands, of processors to work together efficiently in one system. (In the vernacular of systems architecture, these systems "scale well.") Hence such systems have been given the name of massively parallel processing (MPP) systems.

The most successful MPP applications have been for problems that can be broken down into many separate, independent operations on vast quantities of data. In data mining, there is a need to perform multiple searches of a static database. In artificial intelligence, there is the need to analyze multiple alternatives, as in a chess game. Often MPP systems are structured as clusters of processors. Within each cluster the processors interact as in a SMP system. It is only between the clusters that messages are passed. Because operands may be addressed either via messages or via memory addresses, some MPP systems are called NUMA machines, for Non-Uniform Memory Addressing.

SMP machines are relatively simple to program; MPP machines are not. SMP machines do well on all types of problems, providing the amount of data involved is not too large. For certain problems, such as data mining of vast data bases, only MPP systems will serve.

Learn more about Cloud Computing, grid and utility computing
Learning Guide: Grid computing: This learning guide puts all grid computing related news, advice, whitepapers, and other resources at your fingertips.
High performance computing fast guide: In this guide to implementing high performance computing (HPC), find news, tips and advice on data center applications for supercomputing.
PNNL seeks supercomputer to battle radioactive plume: Pacific Northwest National Labs is replacing its four-year-old supercomputer. Its replacement will be used to tackle environmental remediation and sustainable energy.
Does the mainframe play a role in grid computing?: A grid computing effort that includes the mainframe can focus on improving grid robustness and introducing mission-critical applications to the grid.
High performance computing fast guide: In this guide to implementing high performance computing (HPC), find news, tips and advice on data center applications for supercomputing.
A grid glossary: Top 30 terms: This page puts the top 30 grid-related terms into one easy glossary.
Learning Guide: Grid computing: This learning guide puts all grid computing related news, advice, whitepapers, and other resources at your fingertips.
Grid computing: Our latest featured topic directs you to a learning guide we've created that puts the latest grid computing related news, advice, whitepapers, and other content at your fingertips.
Quiz: What's your grid grade?: Are you a grid groupie or just in a grid lock? Take this quiz to assess your knowledge of grid computing and all its related verbiage. This quiz is a tough one with lots of terms that have similar ...

LAST UPDATED: 27 Mar 2007

Do you have something to add to this definition? Let us know.
Send your comments to techterms@whatis.com

More resources from around the web:
- Carnegie-Mellon University hosts a listing of supercomputing and parallel processing research terms and links.
- At the University of Wisconsin, Doug Burger and Mark Hill have created The WWW Computer Architecture Home Page .
- David A. Bader provides an IEEE listing of parallel computing sites .





FILE EXTENSION AND FILE FORMAT LIST
File Extension and File Format List:
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z #


RELATED CONTENT
The hidden costs of scale-out supercomputing
Since the mid-1990s, supercomputer users have moved from scale-up big-box servers to scale-out clusters. But some say beware the hidden costs.
Liquid Web opens 'cloud'-enabling data center
A Web hoster will run Storm, it own cloud computing platform. at its new data center in Michigan. Storm will compete with Amazon Elastic Compute...
Application performance monitoring firm targets cloud computing
Nimsoft's new application performance monitoring software extends monitoring to the cloud and offers an alternative to the costly Big Four systems...

RELATED GLOSSARY TERMS
Terms from Whatis.com − the technology online dictionary
Blue Gene  (SearchDataCenter.com)
Blue Gene is an experimental parallel processing supercomputer developed by IBM that employs thousands of processors, each of which demands minimal...
cluster computing  (SearchDataCenter.com)
In computers, clustering is the use of multiple computers, typically PCs or UNIX workstations, multiple storage devices, and redundant...




Data Center Decisions: Lights-Out or Managed Data Center?
Find parallel processing Solutions for your Data Center
HomeNewsTopicsITKnowledge ExchangeTipsBlogsMultimediaWhite PapersEvents
About Us  |  Contact Us  |  For Advertisers  |  For Business Partners  |  Site Index  |  RSS
SEARCH 
TechTarget provides technology professionals with the information they need to perform their jobs - from developing strategy, to making cost-effective purchase decisions and managing their organizations' technology projects - with its network of technology-specific websites, events and online magazines.

TechTarget Corporate Web Site  |  Media Kits  |  Site Map




All Rights Reserved, Copyright 2005 - 2009, TechTarget | Read our Privacy Policy
  TechTarget - The IT Media ROI Experts