Distributed Programming: Two Traditional Communication Mechanism

shared memory vs message passing

Shared Memory

Key Abstraction:

• Every task can access any location in an application’s distributed memory space. as is the case with threads in a single process, where all threads share the process address space

Feature

• Tasks exchange data implicitly via sharing and not by explicitly sending and receiving messages.

• The shared-memory model, consequently, supports synchronisation mechanisms that distributed programs must use to control the order in which various tasks can perform read/write operations. This goal typically can be achieved using semaphores, locks, and/or barriers.

Synchronisation Mechanism

1. Semaphore

A semaphore is a point-to-point synchronization mechanism that involves two parallel/distributed tasks. Semaphores use two operations: post and wait. The post operation acts like depositing a token, signaling that data has been produced. The wait operation blocks until signaled by the post operation that it can proceed to consume data.

2. Locks

Locks protect critical sections, which are regions that only one task can access (typically write) at a time. Locks involve two operations, lock and unlock, for acquiring and releasing a lock associated with a critical section. A lock can be held by only one task at a time, and other tasks cannot acquire it until released.

3. Barrier

Last, a barrier defines a point beyond which a task is not allowed to proceed until every other task reaches that point.

Example

• DSM-OpenMP

The Message-Passing Programming Model

Key Abstraction:

• Resembles processes that, unlike threads, each maintain a private memory space.

Feature

• To send and receive data via explicit messages, this model incurs communication overheads
• No explicit synchronization is needed
• The interacting tasks require no illusion of a single, shared address space.

Example:

• Message passing interface (MPI)
• Implementation: MPICH

Comparison