take Θ( f(n) ) runtime algorithm
divide into k subproblems
recombine with Θ( g(n) ) method, where g(n) = o(f(n))
result: o( f(n) ) algorithm
(a bi)(c di) = (ac-bd) (bc + ad)i 4 mult, 2 add/sub x: ac y: bd z: (a+b)(c+d) (a bi)(c di) = (x-y) (z-x-y)i 3 mult, 5 add/sub
def bitlength(n): return len(bin(n))-2 def leftright(x,n): # x > 1 left = x >> n/2 right = x - (left << n/2) return left, right def fastmul(x,y): # x,y > 0 n = bitlength(x) if n <= 4: return x*y xl, xr = leftright(x,n) yl, yr = leftright(y,n) pl = fastmul(xl,yl) pr = fastmul(xr,yr) pm = fastmul(xl+xr,yl+yr) return (pl << 2*(n/2)) + ((pm - pl - pr) << n/2) + pr for j in range(400): for k in range(400): assert( fastmul(j,k) == j*k)
- assume 7 digit inputs - shift <- 7/2 = 3 - split inputs: aaaa bbb cccc ddd - a' b' c' d' - L <- a' * c' - R <- b' * d' - S <- (a' + b') * (c' + d') - return L * base^(2*shift) + (S-L-R) * base^shift + R - correct ? aaaa bbb * cccc ddd -------- llllllll mmmmmmm rrrrrr a'c' a'd'+b'c' b'd' - want L = a'c' check :) - want R = b'd' check :) - want M = S-L-R = b'c' + a'd' ??? S-L-R = a'c' + b'c' + a'd' + b'd - a'c' - b'd' = b'c' + a'd' check :)
return x*y if x <= 10 or y <= 10 (1234,5678) (12,56) (1,5) 5 (2,6) 12 1+2,5+6 (3,11) 33 5 33-5-12 12 500 + 160 + 12 672 (34,78) (3,7) 21 (4,8) 32 3+4,7+8 (7,15) 105 21 105-21-32 32 2100 + 520 + 32 2652 12+34,56+78 (46,134) (4,13) 52 (6,4) 24 4+6,13+4 (10,17) 170 52 170-52-24 24 5200 + 940 + 24 6164 672 6164-672-2652 2652 6720000 + 284000 + 2652 7006652
T(n) = 3T(n/2) + O(n)
will see T(n) = O(nlg 3) = O(n1.58..)
O(n2) school, peasant
O(nlg 3) Karatsuba
O(nlog35) Toom-Cook 3-way
O(nlog n log log n) Schonhage-Strassen
O(nlog n 2O(log * n)) Furer
in practice, faster algorithms not worth it unless n large
python uses school for small n, Karatsuba for large n
integers a>0, b>1, real d≥0, T(n) = a T(n/b) + O(nd),
if bd > a ( d > lgb a ) then T(n) ∈ O(nd)
if bd = a ( d = lgb a ) then T(n) ∈ O(nd lg n)
if bd < a ( d < lgb a ) then T(n) ∈ O(nlgba)
assume for
assume
assume
let
if
if
if
if
T(n) = 3 T(n/2) + O(n)
a=3 b=2 d=1 T(n) ∈ O(nlg 3) = O(n1.58..)
key a list item
usual performance measure: count key comparisons
e.g. binary search, key comparisons
master theorem: a=1, b=2, d=0, so
to accurately measure binary search runtime, need to know
how big are numbers in list ?
how is division by 2 implemented ?
how is key comparison implemented ?
e.g. each list item: {1,…,m}, div2: shift, compare: subtract
then
def ms(x): # based on: #http://stackoverflow.com/questions/18761766/mergesort-python result = [] if len(x) < 2: return x mid = len(x)/2 ; y = ms(x[:mid]) ; z = ms(x[mid:]) i,j = 0,0 while i < len(y) and j < len(z): if y[i] > z[j]: result.append(z[j]) ; j += 1 else: result.append(y[i]) ; i += 1 result += y[i:] + z[j:] return result
def select(L,k): A = []; B = []; C = [] v = L[random.randint(0,len(L)-1)] for j in L: if j<v: A.append(j) elif j>v: C.append(j) else: B.append(j) if k <= len(A): return select(A,k) if k <= len(A) + len(B): return v return select(C,k-len(A)-len(B))
worst case: so
best case:
average case:
prove by induction: , so
# partition, qs:http://hetland.org/coding/python/quicksort.html def partition(list, start, end): pivot = list[end] # partition around the last value bottom = start-1 # initially outside sublist to be partitioned top = end # " " done = 0 while not done: # until all elements partitioned... while not done: # until we find an out of place element... bottom = bottom+1 # move the bottom up. if bottom == top: # if hit top... done = 1 # ... we are done. break if list[bottom] > pivot: # bottom out of place? list[top] = list[bottom] # put it at top... break # ... resume search from top while not done: # until we find an out of place element... top = top-1 # move top down. if top == bottom: # if hit bottom... done = 1 # ... we are done. break if list[top] < pivot: # top out of place? list[bottom] = list[top] # put it at bottom... break # ... resume search from bottom list[top] = pivot # replace pivot return top # return split index
def quicksort(list, start, end): if start < end: # two or more elements? split = partition(list, start, end) quicksort(list, start, split-1) quicksort(list, split+1, end) return
can show , where
so
improvements
pick splitter to be median of L[1], L[n/2], L[n]
recursively sort smaller subproblem (so stack depth),
eliminate remaining tail recursion
only operation allowed on keys: <
e.g. allowed L[19] < L[27] ?
e.g. not allowed L[19] > (L[27] + L[92]) ?
consider decision tree: after 1st comparison, branch left or right
number tree leaves must be at least number of possible inputs = n!
after t comparisons, at most 2t tree leaves
so for m = n! tree leaves we need at least lg (n!) comparisons
lg (n!) ∈ Θ(n lg n)
so any comparison-based sort of n keys takes Ω(n lg n) time
can we comparison-sort 6 keys with at most 9 key comparisons?
no 6!= 720, and 29 = 512 < 720, so lg 720 > 9