CS入门学习笔记11-MIT 6.00.1x- Lecture 9

Lecture 9 Efficiency and orders of growth

他人笔记参考：https://www.cnblogs.com/HuZihu/p/7614460.html

Goals in designing programs

1. It returns the correct answer on all illegal inputs;
2. It performs the computation efficiently

1. Computational complexity

• How much time will it take a program to run
• How much memory will it need to run

need to balance minimizing computational complexity with conceptual complexity

2. How to measure

程序运行的时间会受电脑性能、python环境与版本、input值的大小等，所以要测算的是每一个step可能会花费的时间。

Basic measuring steps

• Random access machine(RAM)
• in terms of size of input

几种measure中的可能情况

• Best case
• Worst case
• Average case
  此时我们需要测量worst case，将它作为一个程序运行时间的上限

当我们需要有一个循环重复1亿次，每次里面有8steps时，与之相比的是30次循环，每次有10steps，在这种对比中重要的是循环的次数，而非每次循环包括的step数。
——在比较时要学会抓住重点，抓住大头。

两个measure的原则：

作业题中错误：

1. 判断step数目

    for i in range(1000):
        total += i

此式应该有3000步，因为for包含对i的一次赋值，total += i 包括一次加法运算和一次赋值。

def program3(L):
    totalSum = 0
    highestFound = None
    for x in L:
        totalSum += x

    for x in L:
        if highestFound == None:
            highestFound = x
        elif x > highestFound:
            highestFound = x

    return (totalSum, highestFound)

steps: 7 * n +2

explanation:
In the worst case scenario, L is a list with its elements sorted in increasing order (eg, [1, 3, 5, 7, …]). In this case we execute the first two assignment statements (2 steps). Next we execute the first loop n times. This first loop has three steps (one for the assignment of x each time through the loop, as well as two for the += operation), adding 3*n steps.

Finally we execute the second loop n times. The first time we execute this loop, we perform 3 steps - one for the assignment of x; then we run the check if highestFound == None, and finding it to be True, we execute the assignment highestFound = x.

The next (n-1) times we execute this loop, we perform 4 steps: one for the assignment of x, then we run the check if highestFound == None, and finding it to be False, we run the check elif x > highestFound. Since this is always True (the list is sorted in increasing order), we execute the assignment highestFound = x. Therefore in the second loop we execute 3 + (n-1)4 = 3 + 4n - 4 = 4*n - 1 steps.

Finally we execute the return statement, which is one more step.

Pulling this all together, we can see that in the worst case we execute 2 + 3n + 4n - 1 + 1= 7*n + 2 steps.

3. Asymptotic Notation 渐进记法

从上例可以看出，当x不断增大时，x平方的影响愈来愈大，以至于前面的加数和x平方前面的系数都可以忽略。因此：

• 最重要的永远是时间增长率最快的那些步骤
• 可以省去乘积中那些常数系数
• 大O记号法（big O）
  对于规模为 n的输入，考量算法在最坏情况下的运行情况，也就是当n增大时，运行这个函数所增加的运行时间的上界。

作业题中错误：

def program1(L):
    multiples = []
    for x in L:
        for y in L:
            multiples.append(x*y)
    return multiples

step: 3*n^2 + n + 2
explanation:
In the worst case scenario, L is a long list. In this case we go through the loop for x in L n times. Every time through this loop, we execute an assignment of a value to x, and then the inner loop for y in L. The assignment takes 1 step on each iteration; how many steps does the inner loop take on each iteration?

The inner loop has three operations (assignment of a value to y, xy, and list appending). So the inner loop executes 3n times on each iteration of the outer loop. Thus the nested loop structure is executed n * (3n + 1) = 3n**2 + n times!

Adding in two steps (for the first assignment statement, and the return statement) we see that in the worst case, this program executes 3*n**2 + n + 2 steps.

def program2(L):
    squares = []
    for x in L:
        for y in L:
            if x == y:
                squares.append(x*y)
    return squares

step: 4*n^2 + n + 2
explanation:
In the worst case scenario, L is a long list of one repeated number (ie [2, 2, 2, 2, …]. In this case we go through the loop for x in L n times. Every time through this loop, we perform one assigment of a value to the variable x, then we execute the inner loop for y in L n times.

The inner loop performs one assigment of a value to the variable y. It then has one operation that is checked every time (if x == y). Since the WORST case is when the list is composed of identical elements, this check is always true - so the third and fourth operations (xy, and list appending) are always performed. So the inner loop executes 4n times on each iteration of the outer loop. Thus the nested loop structure is executed n * (4n + 1) = 4n**2 + n times!

Adding in two steps (for the first assignment statement, and the return statement) we see that in the worst case, this program executes 4*n**2 + n + 2 steps.

def program3(L1, L2):
    intersection = []
    for elt in L1:
        if elt in L2:
            intersection.append(elt)
    return intersection

step: n^2 + 2*n + 2
explanation:
In the worst case scenario, every element of L1 is the same as the very last element of L2. In this case we go through the loop for elt in L1 n times. Every time through this loop, we perform one assigment of a value to the variable elt, then we execute the check if elt in L2.

How long does it take to execute this check? Well in the worst case, elt is the very LAST element of L2. In order to check if elt in L2, Python iterates over the elements of L2 until it either finds the one you’re looking for, or L2 runs out of elements. Thus in the worst case, the check if elt in L2 takes n steps. After this, we perform one append operation. So the conditional plus the append takes n + 1 steps on each iteration of the loop. Thus the for loop takes n * (n + 2) = n**2 + 2*n steps!

Adding in two steps (for the first assignment statement, and the return statement) we see that in the worst case, this program executes n**2 + 2*n + 2 steps.

4. Complexity classes

从上往下，复杂度不断增加。

往往是在递归函数中出现不止一个递归指令时会出现此种指数复杂度。

下面的例子为O[2ⁿ]

# 列举出一个list中包含的所有subset
def genSubsets(L):
    res = []
    if len(L) == 0:
        return [[]]
    smaller =genSubsets(L[:-1])
    print 'smaller: '
    print smaller
    extra = L[-1:]
    print 'extra: ' 
    print extra
    new = []
    for small in smaller:
        new.append(small + extra)
    return smaller + new

从逻辑层面来思考的话，当list中有k个元素时，包含的subset会有2^k个。所以每做一次迭代都会计算2^k次。

logarithmic grows quite slow，有些时候比constant还好用：