CMPUT329 - Fall 2003

TopicJ:

The Concept of Pipelining

José Nelson Amaral

What is Pipelining?

Pipelining is a key implementation technique used

to build fast processors. It allows the execution of

multiple instructions to overlap in time.

A pipeline within a processor is similar to a car

assembly line. Each assembly station is called a

pipe stage or a pipe segment.

The throughput of an instruction pipeline is

the measure of how often an instruction exits the

pipeline.

Pipelining: Its Natural!

Laundry Example

Ann, Brian, Cathy, Dave
each have one load of clothes
to wash, dry, and fold

Washer takes 30 minutes

Dryer takes 40 minutes

“Folder” takes 20 minutes

A

B

C

D

Sequential Laundry

Sequential laundry takes 6 hours for 4 loads

If they learned pipelining, how long would laundry take?

A

B

C

D

30

40

20

30

40

20

30

40

20

30

40

20

6 PM

7

8

9

10

11

Midnight

T

a

s

k

O

r

d

e

r

Time

Pipelined Laundry
Start work ASAP

Pipelined laundry takes 3.5 hours for 4 loads

A

B

C

D

6 PM

7

8

9

10

11

Midnight

T

a

s

k

O

r

d

e

r

Time

30

40

40

40

40

20

What is preventing

them from doing it

faster?

How could we eliminate

this limiting factor?

Pipelined Laundry
Start work ASAP

A

B

C

D

6 PM

7

8

9

10

11

Midnight

T

a

s

k

O

r

d

e

r

Time

30

40

40

40

40

20

Pipeline throughput:

How many loads are

completed per unit

of time?

Pipeline Latency:

Once a load starts, how

long does it takes to

complete it?

Steps to Execute Each
Instruction Type

Pipeline Stages

We can divide the execution of an instruction

into the following stages:

IF: Instruction Fetch

ID: Instruction Decode

EX: Execution

MEM: Memory Access

WB: Write Back

Pipeline Throughput and Latency

IF

ID

EX

MEM

WB

5 ns

4 ns

5 ns

10 ns

4 ns

Consider the pipeline above with the indicated

delays. We want to know what is the pipeline

throughput and the pipeline latency.

Pipeline throughput: instructions completed per second.

Pipeline latency: how long does it take to execute a

single instruction in the pipeline.

Pipeline Throughput and Latency

IF

ID

EX

MEM

WB

5 ns

4 ns

5 ns

10 ns

4 ns

Pipeline throughput: how often an instruction is completed.

Pipeline latency: how long does it take to execute an

instruction in the pipeline.

Is this right?

Pipeline Throughput and Latency

IF

ID

EX

MEM

WB

5 ns

4 ns

5 ns

10 ns

4 ns

Simply adding the latencies to compute the pipeline

latency, only would work for an isolated instruction

IF

MEM

ID

I1

L(I1) = 28ns

EX

WB

MEM

ID

IF

I2

L(I2) = 33ns

EX

WB

MEM

ID

IF

I3

L(I3) = 38ns

EX

WB

MEM

ID

IF

I4

L(I5) = 43ns

EX

WB

We are in trouble! The latency is not constant.

This happens because this is an unbalanced

pipeline. The solution is to make every state

the same length as the longest one.

Pipeline Throughput and Latency

IF

ID

EX

MEM

WB

5 ns

4 ns

5 ns

10 ns

4 ns

Simply adding the latencies to compute the pipeline

latency, only would work for an isolated instruction

IF

MEM

ID

I1

L(I1) = L(I2) = L(I3) = L(I4) = 50ns

EX

WB

IF

MEM

ID

I2

L(I2) = 50ns

EX

WB

IF

MEM

ID

EX

WB

IF

MEM

ID

EX

0

10

20

30

40

50

60

I3

I4

Pipeline Throughput and Latency

IF

ID

EX

MEM

WB

5 ns

4 ns

5 ns

10 ns

4 ns

How long does it take to execute 20000 instructions

in this pipeline? (disregard bubles caused by

branches, cache misses, and hazards)

How long would it take using the same modules

without pipelining?

Pipeline Throughput and Latency

IF

ID

EX

MEM

WB

5 ns

4 ns

5 ns

10 ns

4 ns

Thus the speedup that we got from the pipeline is:

How can we improve this pipeline design?

We need to reduce the unbalance to increase

the clock speed.

Pipeline Throughput and Latency

IF

ID

EX

MEM1

WB

5 ns

4 ns

5 ns

5 ns

4 ns

Now we have one more pipeline stage, but the

maximum latency of a single stage is reduced in half.

MEM2

5 ns

The new latency for a single instruction is:

Pipeline Throughput and Latency

IF

ID

EX

MEM1

WB

5 ns

4 ns

5 ns

5 ns

4 ns

MEM2

5 ns

IF

MEM1

ID

I1

EX

WB

MEM1

IF

MEM1

ID

I2

EX

WB

MEM1

IF

MEM1

ID

I3

EX

WB

MEM1

IF

MEM1

ID

I4

EX

WB

MEM1

IF

MEM1

ID

I5

EX

WB

MEM1

IF

MEM1

ID

I6

EX

WB

MEM1

IF

MEM1

ID

I7

EX

WB

MEM1

Pipeline Throughput and Latency

IF

ID

EX

MEM1

WB

5 ns

4 ns

5 ns

5 ns

4 ns

MEM2

5 ns

How long does it take to execute 20000 instructions

in this pipeline? (disregard bubles caused by

branches, cache misses, etc, for now)

Thus the speedup that we get from the pipeline is:

Pipeline Throughput and Latency

IF

ID

EX

MEM1

WB

5 ns

4 ns

5 ns

5 ns

4 ns

MEM2

5 ns

What have we learned from this example?

1. It is important to balance the delays in the

stages of the pipeline

2. The throughput of a pipeline is 1/max(delay).

3. The latency is Nmax(delay), where N is the

number of stages in the pipeline.

Pipelining Lessons

A

B

C

D

6 PM

7

8

9

T

a

s

k

O

r

d

e

r

Time

30

40

40

40

40

20

Pipelining doesn’t help

latency of single task,

it helps throughput

of entire workload

Pipeline rate limited by

slowest pipeline stage

Multiple tasks operating

simultaneously

Pipeline rate limited by

slowest pipeline stage

Potential speedup =

Number pipe stages

Unbalanced lengths of

pipe stages reduces speedup

Time to “fill” and “drain”

pipeline reduces speedup

Computer Pipelines

Execute billions of instructions, so throughput is what matters

DLX desirable features: all instructions same length, registers located in same place in instruction format, memory operands only in loads or stores

5 Steps of DLX Datapath

Pipelined DLX Datapath

Data stationary control

local decode for each instruction phase / pipeline stage

Memory

Access

Write

Back

Instruction

Fetch

Instr. Decode

Reg. Fetch

Execute

Addr. Calc.

Visualizing Pipelining

I

n

s

t

r.

O

r

d

e

r

Time (clock cycles)

Its Not That Easy for Computers

Limits to pipelining: Hazards prevent next instruction from executing during its designated clock cycle

Structural hazards: HW cannot support this combination of instructions (single person to fold and put clothes away)

Data hazards: Instruction depends on result of prior instruction still in the pipeline (missing sock)

Control hazards: Pipelining of branches & other instructions stall the pipeline until the hazard “bubbles” in the pipeline

One Memory Port/Structural Hazards

I

n

s

t

r.

O

r

d

e

r

Time (clock cycles)

Load

Instr 1

Instr 2

Instr 3

Instr 4

One Memory Port/Structural Hazards

I

n

s

t

r.

O

r

d

e

r

Time (clock cycles)

Load

Instr 1

Instr 2

stall

Instr 3

Data Hazard on R1

I

n

s

t

r.

O

r

d

e

r

Time (clock cycles)

add r1,r2,r3

sub r4,r1,r3

and r6,r1,r7

or r8,r1,r9

xor r10,r1,r11

CC1

CC2

CC3

CC4

CC5

CC6

Three Generic Data Hazards

InstrI followed by InstrJ

Read After Write (RAW)
InstrJ tries to read operand before InstrI writes it

Three Generic Data Hazards

InstrI followed by InstrJ

Write After Read (WAR)
InstrJ tries to write operand before InstrI reads i

Gets wrong operand

Can’t happen in DLX 5 stage pipeline because:

All instructions take 5 stages, and

Reads are always in stage 2, and

Writes are always in stage 5

Three Generic Data Hazards

InstrI followed by InstrJ

Write After Write (WAW)
InstrJ tries to write operand before InstrI writes it

Leaves wrong result ( InstrI not InstrJ )

Can’t happen in DLX 5 stage pipeline because:

All instructions take 5 stages, and

Writes are always in stage 5

Forwarding to Avoid Data Hazard

I

n

s

t

r.

O

r

d

e

r

Time (clock cycles)

add r1,r2,r3

sub r4,r1,r3

and r6,r1,r7

or r8,r1,r9

xor r10,r1,r11

HW Change for Forwarding
Figure 3.20, Page 161

I

n

s

t

r.

O

r

d

e

r

Time (clock cycles)

lw r1, 0(r2)

sub r4,r1,r6

and r6,r1,r7

or r8,r1,r9

Data Hazard Even with Forwarding
Figure 3.12, Page 153

Data Hazard Even with Forwarding
Figure 3.13, Page 154

I

n

s

t

r.

O

r

d

e

r

Time (clock cycles)

lw r1, 0(r2)

sub r4,r1,r6

and r6,r1,r7

or r8,r1,r9

MIPS actutally didn’t interlecok: MPU without Interlocked Pipelined Stages

CMPUT329 - Fall 2000

CMPUT329

Jose Nelson Amaral

1998-02-10T08:49:30

Trial User