Sequencing Problem

Preface

Yesterday, I had learned lesson one about Introduction to Algorithms from open.163.com . Thanks for NetEase And MIT.

---

Insertion Sort

We start with insertion sort , which is an efficient algorithm for sorting a small number of elements.

pseudocode:

INSERTION-SORT(A)

for j <- 2 to length[A]
    do key <- A[j]
       ↓Insert A[j] into the sorted sequence A[1...j-1]
       i <- j-1
       while i>0 and A[i]>key
           do A[i+1] <- A[i]
               i <- i-1
        A[i+1] <- key

Loop invariants and the correctness of insertion sort

We state these properties of A[1…j-1]( the elements in A[1...j-1] had been sorted A[1…j-1] are the elements originally in positions 1 through j-1 ,but now in sorted order ) formally as a loop invariant
We use loop invariants to help us understand why an algorithm is correct. We must know three things about a loop variant:
+Initialization: It is true prior to the first iteration of the loop.
+Maintenance: If it is true before an iteration of the loop, it remains true before next iteration.
+Termination: When the loop terminates, the invariant gives us a useful property that helps us show that the algorithm is correct.

——INTRODUCTION TO ALGORITHMS(Second Edition)

$$Time(n) =\sum_{i=2}^n \theta(j)$$

Maybe you do not know what $\theta$ ( theta ) means. There is a BIG IDEA OF ALGORITHM，to be able to take what is apparently a really messy, complicated situation and reduce it to being able to do some mathematics. And that idea is called asymptotic analysis. It is to ignore machine-dependent constants and instead of the actual running time, look at the growth of the running time.

Asymptotic notation

theta notation

$\theta$ ( weak notation )

Describtion: from a formula, drop low order terms and ignore leading constants.
ex: $3n^3+90n^2-5n+6046$ —> $\theta(n^3)$


Now, we can talk about this formula.

$$Time(n) =\sum_{i=2}^n \theta(j)$$
i <- j-1 through this loop from 1 ~ i, and it is doing just a constant amount of stuff for each step of the value of i. (get a value from list[i] is a constant amout of stuff.)
$j = c*n$ ——– c: constant / n:times
ex:
if $i = 4$

 while i>0 and A[i]>key
          do A[i+1] <- A[i]
               i <- i-1

do A[i+1] <- A[i] should be done four times, each time cost a constant amout of stuff.

$$(four-times) * (constant) ---->j=c*n$$
$$↓$$
$$Time(n) =\sum_{i=2}^n \theta(j) ------ j=c*n$$
$$↓$$
$$Time(n) =\sum_{i=2}^n \theta(j)=\theta(n^2)$$

Actually, this is a Arithmetic Series.

In mathematics, an arithmetic progression (AP) or arithmetic sequence is a sequence of numbers such that the difference between the consecutive terms is constant. For instance, the sequence 5, 7, 9, 11, 13, 15 … is an arithmetic progression with common difference of 2.

WikiPedia

We can get some information from this diagram(sorry, ugly, but just see).
While $n=n_0$ , $\theta(n^3)$ and $\theta(n^2)$ use the same amount of time. But when $n>n_0$， it is obvious that $\theta(n^2)$ is faster.
This is the reason that why asymptotic analysis is a BIG IDEA OF ALGORITHM

There is a BIG IDEA OF ALGORITHM，to be able to take what is apparently a really messy, complicated situation and reduce it to being able to do some mathematics.
It is to ignore machine-dependent constants and instead of the actual running time, look at the growth of the running time.

Asymptotic analysis will throughout this blog.

—————————

Insertion Sort (Python)

def insertion_sort(list):
    for i in range(2,len(list)):# for(i=2;i<len(list);i++): wrong !
        list[0] = list[i]# save current value
        if(list[i] < list[i-1]):# list has been sorted befor i 
            for j in range(1,i):
                if(list[0] < list[j]):# find the right index to insert
                    break
            for k in range(j,i)[::-1]:# move back the value where index[j,i] one size
                                      # range(j,i)[::-1] can sort [1,2,3,4] to [4,3,2,1]
                list[k+1] = list[k]
            list[j] = list[0]# change value(in order to sort list)
    return list
A = [0,2,8,2,6,4,3,9,1,3,5,6,2,3,4,6,8,2,3,1]# demo list
print insertion_sort(A)

---

Merge Sort

$\theta(1)$	1. If n=1, done
$2T(n)$	2. Recursively sort -> A[1,n/2],A[n/2+1,n]
$\theta(n)$	3. Merge 2 sorted arrays

$\theta(1)$— a constant amout of stuff

——————————

20 12
13 11
7 9
2 1
choose the smaller one
1,
1,2,
1,2,7, -> new array

——————————

every step here is some fixed number of operations that is independent of the size of the arrays at each step

From above computation, it uses the time that can be express as $\theta(n)$.(The time actually go through this and merge two arrays is order n.)(call this linear time.)

Recurrence:

$$T(n)= \left\{ \begin{array}{c} \theta(1),if(n=1)(usually-omit)\\ 2T(n/2)+\theta(n),if(n>1) \end{array} \right.$$

then, we use the recursion tree technique to compute.

$$T(n)=2T(n/2)+cn$$

$\theta(n)$ just like a $constant * n$ | C is constant

h(high) = the number of n reduce by half until to 1 is lgn || leaves = n

—————————

$$\begin{align} T(n) &=cn*\lg n+\theta(n)\\\ &=\theta(n\lg n) \end{align}$$

$\theta(n\lg n)$ is asymptotically faster than $\theta(n^2)$. We can find some $n = n_0$ to accomplish(just like a diagram above which compare $\theta(n^2)$ to $\theta(n^3)$).

Merge Sort

#include <stdio.h>

#define LEN 8
int a[LEN] = { 5, 2, 4, 7, 1, 3, 2, 6 };

void merge(int start, int mid, int end)
{
    int n1 = mid - start + 1;
    int n2 = end - mid;
    int left[n1], right[n2];
    int i, j, k;

    for (i = 0; i < n1; i++) /* left holds a[start..mid] */
        left[i] = a[start+i];
    for (j = 0; j < n2; j++) /* right holds a[mid+1..end] */
        right[j] = a[mid+1+j];

    i = j = 0;
    k = start;
    while (i < n1 && j < n2)
        if (left[i] < right[j])
            a[k++] = left[i++];
        else
            a[k++] = right[j++];

    while (i < n1) /* left[] is not exhausted */
        a[k++] = left[i++];
    while (j < n2) /* right[] is not exhausted */
        a[k++] = right[j++];
}

void sort(int start, int end)
{
    int mid;
    if (start < end) {
        mid = (start + end) / 2;
        printf("sort (%d-%d, %d-%d) %d %d %d %d %d %d %d %d\n", 
               start, mid, mid+1, end, 
               a[0], a[1], a[2], a[3], a[4], a[5], a[6], a[7]);
        sort(start, mid);
        sort(mid+1, end);
        merge(start, mid, end);
        printf("merge (%d-%d, %d-%d) to %d %d %d %d %d %d %d %d\n", 
               start, mid, mid+1, end, 
               a[0], a[1], a[2], a[3], a[4], a[5], a[6], a[7]);
    }
}

int main(void)
{
    sort(0, LEN-1);
    return 0;
}

via tsinghua.mergesort

UpdateLog

2015/12/21

—————————

Update : Insertion Sort & Merge Sort