计算理论笔记（一）DFA NFA Regex

Problems and languages

An alphabet: a finite set of symbols

• Σ={0,1}\Sigma=\{0,1\}Σ={0,1}
• Σ={a,b,c,…,z}\Sigma=\{a,b,c,\dots,z\}Σ={a,b,c,…,z}

A string: a finite sequence of symbols from some alphabet $\Sigma$

Length of a string $w$ is #symbols in $w$, noted as $|w|$

• If $|w|=0$, $w$ is an empty string $e$

${\Sigma }^i$: the set of all strings of length $i$ over $\Sigma$

• Σ={0,1}\Sigma=\{0,1\}Σ={0,1}
• Σ0={e}\Sigma^0=\{e\}Σ0={e}
• Σ1={0,1}\Sigma^1=\{0,1\}Σ1={0,1}
• Σ2={00,01,10,11}\Sigma^2=\{00,01,10,11\}Σ2={00,01,10,11}
• $\cdots$

${\Sigma }^{+}$: the set of all non-empty strings over $\Sigma$

${\Sigma }^{\ast }$: the set of all strings over $\Sigma$

• Σ∗=Σ+∪{e}\Sigma^*=\Sigma^+\cup\{e\}Σ∗=Σ+∪{e}

Concatenation: $vw$

String exponentiation: $w^{i+1}=w{w}^i,w^0=e$, defined by induction

Reversal: $w=a_1{a}_2\cdots a_n,w^R=a_n{a}_{n-1}\cdots a_1$

• $|w|=0,w=w^R=e$
• $|w|\ge 1,w=ua(u\in {\Sigma }^{\ast },a\in \Sigma ),w^R=a{u}^R$

A set of strings over some alphabet $\Sigma$ is called a language

Any decision problem $⟺$ Problem about certain language

A Deterministic Finite Automata(DFA): a 5-tuple $(K,\Sigma ,\delta ,s,F)$

• $K$: a finite set of states
• $\Sigma$: a finite set of input symbols
• $\delta$: transition function, $K\times \Sigma \to K$
• $s\in K$: initial state
• $F\subseteq K$: a set of final states

A configuration of a DFA $M=(K,\Sigma ,\delta ,s,F)$: an element of $K\times {\Sigma }^{\ast }$

Yields in one step: $(q,w){⊢}_M(q^{\prime },w^{\prime })$, if $w=a{w}^{\prime }(a\in \Sigma )$ and $\delta (q,a)=q^{\prime }$

• $(q,w){⊢}_M^{\ast }(q^{\prime },w^{\prime })$

$M$ accepts $w\in {\Sigma }^{\ast }$ if $(s,w){⊢}_M^{\ast }(q,e)$, for some $q\in F$

$L(M)$: a set of all $w\in {\Sigma }^{\ast }$ accepted by $M$, uniquely

$M$ accepts a language $L$ iff

• $L$ contains every string accepted by $M$
• every string in $L$ is accepted by $M$

A language is regular if it is accepted by some DFA

Regular operation

• Union: A∪B={w:w∈A∨w∈B}A\cup B=\{w:w\in A\lor w\in B\}A∪B={w:w∈A∨w∈B}
• Intersection: A∩B={w:w∈A∧w∈B}A\cap B=\{w:w\in A\land w\in B\}A∩B={w:w∈A∧w∈B}
• Complement: A‾={w:w∈Σ∗−A}\overline{A}=\{w:w\in\Sigma^*-A\}A={w:w∈Σ∗−A}
• Concatenation: A∘B={ab:a∈A∧b∈B}A \circ B=\{ab:a\in A\land b\in B\}A∘B={ab:a∈A∧b∈B}
• Star: A∗={w1w2…wn:wi∈A,n∈N}A^*=\{w_1w_2\dots w_n:w_i\in A,n\in N\}A∗={w1​w2​…wn​:wi​∈A,n∈N}, $e\in A^{\ast }$

Theorem: If $A$ and $B$ are regular, so is $A\cup B$ (by DFA)

• Idea: $\exists M_1(K_1,\Sigma ,{\delta }_1,s_1,F_1),M_2(K_2,\Sigma ,{\delta }_2,s_2,F_2)$ accept $A,B$, respectively, construct $M_3(K_3,\Sigma ,{\delta }_3,s_3,F_3)$ accepting $A\cup B$
• $K_3=K_1\times K_2$
• ${\delta }_3:K_3\times \Sigma \to \Sigma$, with constraint ${\delta }_3((q_1,q_2),w)=({\delta }_1(q_1,w),{\delta }_2(q_2,w))=(q_1^{\prime },q_2^{\prime })$
• $s_3=(s_1,s_2)$
• F3={(q1,q2)∈K1×K2:q1∈F1∨q2∈F2}F_3=\{(q_1,q_2)\in K_1\times K_2:q_1\in F_1\lor q_2\in F_2\}F3​={(q1​,q2​)∈K1​×K2​:q1​∈F1​∨q2​∈F2​}

Non-deterministic Finite Automata(NFA)

• several choices for the next state
• may switch states without reading any input symbols

Definition of NFA: a 5-tuple $(K,\Sigma ,\Delta ,s,F)$

• $K$: a finite set of states
• $\Sigma$: a finite set of input symbols
• $\Delta$: transition relation, a subset of K×(Σ∪{e})×KK\times(\Sigma\cup\{e\})\times KK×(Σ∪{e})×K
• $s\in K$: initial state
• $F\subseteq K$: a set of final states

A configuration of a NFA $N=(K,\Sigma ,\Delta ,s,F)$: an element of $K\times {\Sigma }^{\ast }$

Yields in one step: $(q,w){⊢}_M(q^{\prime },w^{\prime })$, if w=aw′(a∈(Σ∪{e}))w=aw'(a\in(\Sigma\cup\{e\}))w=aw′(a∈(Σ∪{e})) and $(q,a,q^{\prime })\in \Delta$

• $(q,w){⊢}_M^{\ast }(q^{\prime },w^{\prime })$

$N$ accepts $w\in {\Sigma }^{\ast }$ if $(s,w){⊢}_M^{\ast }(q,e)$, for some $q\in F$

$L(N)$: a set of all $w\in {\Sigma }^{\ast }$ accepted by $N$, uniquely

DFA is a special case of NFA: function $(q,a)\to \delta (q,a)$ can be converted to relation $(q,a,\delta (q,a))$

Theorem: For any NFA $N=(K,\Sigma ,\Delta ,s,F)$, there always exists DFA $M=(K^{\prime },\Sigma ,\delta ,s^{\prime },F^{\prime })$ s.t. $L(N)=L(M)$

• $K^{\prime }=\mathcal{P}(K)$
• ∀q∈K,E(q)={p∈K:(q,e)⊢M∗(p,e)}\forall q\in K, E(q)=\{p\in K:(q,e)\vdash_M^*(p,e)\}∀q∈K,E(q)={p∈K:(q,e)⊢M∗​(p,e)}
• $\forall Q\subseteq K,E(Q)=\bigcup _{q\in Q}E(q)$
• $\delta :K^{\prime }\times \Sigma \to K^{\prime }$, with constraint δ(Q,a)=⋃q∈QE({p∈K:(q,a,p)∈Δ})\delta(Q,a)=\displaystyle\bigcup_{q\in Q}{E(\{p\in K:(q,a,p)\in\Delta\})}δ(Q,a)=q∈Q⋃​E({p∈K:(q,a,p)∈Δ})
• $s^{\prime }=E(s)$
• F′={Q⊆K:Q∩F≠∅}F'=\{Q\subseteq K:Q\cap F\neq\varnothing\}F′={Q⊆K:Q∩F​=∅}
• claim: $\forall p,q\in K$, $\forall w\in \Sigma$, $(p,w){⊢}_M^{\ast }(q,e)$ iff $(E(p),w){⊢}_M^{\ast }(Q,e)$ for some $Q$ containing $q$
• $M$ accepts $w⟺N$ accepts $w$:
  • $(s,w){⊢}_M^{\ast }(q,e)$ for some $q\in F$
  • $(E(s),w){⊢}_M^{\ast }(Q,e)$ with $q\in Q$
  • note: $q\in Q\iff Q\cap F\ne \varnothing \iff Q\in F^{\prime }$

Theorem: If $A$ and $B$ are regular, so is $A\circ B$

• Idea: $\exists N_1(K_1,\Sigma ,{\Delta }_1,s_1,F_1),N_2(K_2,\Sigma ,{\Delta }_2,s_2,F_2)$ accept $A,B$, respectively, construct $N_3(K_3,\Sigma ,{\Delta }_3,s_3,F_3)$ accepting $A\circ B$
• $K_3=K_1\cup K_2$
• Δ3=Δ1∪Δ2∪{(q,e,s2):q∈F1}\Delta_3=\Delta_1\cup\Delta_2\cup\{(q,e,s_2):q\in F_1\}Δ3​=Δ1​∪Δ2​∪{(q,e,s2​):q∈F1​}
• $s_3=s_1$
• $F_3=F_2$

Theorem: If $A$ is regular, so is $A^{\ast }$

• Idea: $\exists N(K,\Sigma ,\Delta ,s,F)$ accept $A$, construct $N^{\prime }(K^{\prime },\Sigma ,{\Delta }^{\prime },s^{\prime },F^{\prime })$ accepting $A^{\ast }$
• K′=K∪{s′}K'=K\cup\{s'\}K′=K∪{s′}
• Δ′=Δ∪{(s′,e,s)}∪{(q,e,s):q∈F}\Delta'=\Delta\cup\{(s',e,s)\}\cup\{(q,e,s):q\in F\}Δ′=Δ∪{(s′,e,s)}∪{(q,e,s):q∈F}
• F′=F∪{s′}F'=F\cup\{s'\}F′=F∪{s′}

Theorem: If $A$ and $B$ are regular, so is $A\cup B$ (by NFA)

• Idea: $\exists N_1(K_1,\Sigma ,{\Delta }_1,s_1,F_1),N_2(K_2,\Sigma ,{\Delta }_2,s_2,F_2)$ accept $A,B$, respectively, construct $N_3(K_3,\Sigma ,{\Delta }_3,s_3,F_3)$ accepting $A\cup B$
• K3=K1∪K2∪{s3}K_3=K_1\cup K_2\cup\{s_3\}K3​=K1​∪K2​∪{s3​}
• Δ3=Δ1∪Δ2∪{(s3,e,s1)}∪{(s3,e,s2)}\Delta_3=\Delta_1\cup\Delta_2\cup\{(s_3,e,s_1)\}\cup\{(s_3,e,s_2)\}Δ3​=Δ1​∪Δ2​∪{(s3​,e,s1​)}∪{(s3​,e,s2​)}
• $F_3=F_1\cup F_2$

Regex: defined inductively on an alphabet $\Sigma$

• Base case:
  • $\varnothing$ is a regex, $L(\varnothing )=\varnothing$
  • any symbol $a$ in $\Sigma$ is a regex, L({a})={a}L(\{a\})=\{a\}L({a})={a}
• Inductive case:
  • If $R_1$ and $R_2$ are regex, so is $R_1\cup R_2$, $L(R_1\cup R_2)=L(R_1)\cup L(R_2)$
  • If $R_1$ and $R_2$ are regex, so is $R_1\circ R_2$, $L(R_1\circ R_2)=L(R_1)\circ L(R_2)$
  • If $R$ is a regex, so is $R^{\ast }$, $L(R^{\ast })=L(R{)}^{\ast }$
• Precedence: $\ast >\circ >\cup$
• Example:
  • L({e})=∅∗L(\{e\})=\varnothing^*L({e})=∅∗
  • L({w∈{a,b}∗:wL(\{w\in\{a,b\}^*:wL({w∈{a,b}∗:w starts with $a$ and end with $b\})=a(a\cup b{)}^{\ast }b$
  • L({w∈{0,1}∗:wL(\{w\in\{0,1\}^*:wL({w∈{0,1}∗:w has at least two occurence of $0\})=(0\cup 1{)}^{\ast }0(0\cup 1{)}^{\ast }0(0\cup 1{)}^{\ast }$

Theorem: A language is regular iff it is described by some regex

• Idea
  • Regex $\to$ NFA, composition of operations above
  • NFA $\to$ Regex, elimination of states
• Given a NFA $N$
  • Convert $N$ into an equivalent NFA $N^{\prime }$, such that
    • $N^{\prime }$ has no arc entering initial state, by adding a new initial state with an only arc labeled $e$ to the original initial state
    • $N^{\prime }$ has only one final state and there is no arc leaving this final state, by adding a new final state with arcs labeled $e$ from original final states
  • Eliminate, one by one, all the states except the initial state and final state

Let $N=(K,\Sigma ,\Delta ,s,F)$ be a NFA. WLOG, assume that

• K={q1,q2,…,qn},s=qn−1,F={qn}K=\{q_1,q_2,\dots,q_n\},s=q_{n-1},F=\{q_n\}K={q1​,q2​,…,qn​},s=qn−1​,F={qn​}
• (p,a,qn−1)∉Δ,∀p∈K,∀a∈Σ∪{e}(p,a,q_{n-1})\notin\Delta,\forall p\in K,\forall a\in\Sigma\cup\{e\}(p,a,qn−1​)∈/​Δ,∀p∈K,∀a∈Σ∪{e}
• (qn,a,p)∈Δ,∀p∈K,∀a∈Σ∪{e}(q_n,a,p)\in\Delta,\forall p\in K,\forall a\in\Sigma\cup\{e\}(qn​,a,p)∈Δ,∀p∈K,∀a∈Σ∪{e}
• Subproblem: $\forall i,j\in [1,n]$ and $\forall k\in [0,n]$, define
  • Lijk={w∈Σ∗:wL_{ij}^k=\{w\in\Sigma^*:wLijk​={w∈Σ∗:w drives $N$ from $q_i$ to $q_j$ without passing any intermediate state having index greater than $k\}$
  • $R_{ij}^k\to L_{ij}^k$
• Goal: $L(N)=L_{(n-1)n}^{n-2}$
• Base case: $k=0$, Lij0={{a:(qi,a,qj)∈Δ},i≠j{a:(qi,a,qj)∈Δ}∪{e},i=jL_{ij}^0=\begin{cases}\{a:(q_i,a,q_j)\in\Delta\}&,i\neq j \\ \{a:(q_i,a,q_j)\in\Delta\}\cup\{e\}&,i=j\end{cases}Lij0​={{a:(qi​,a,qj​)∈Δ}{a:(qi​,a,qj​)∈Δ}∪{e}​,i​=j,i=j​
• Recurrence: $k\ge 1$, $L_{ij}^k=L_{ij}^{k-1}\cup \left(L_{ik}^{k-1}\circ {\left(L_{kk}^{k-1}\right)}^{\ast }\circ L_{kj}^{k-1}\right)$

To proof regular:

• By definition: DFA, NFA, Regex
• By closure property: union, intersection, complement, concatenation, star

Pumping Theorem

• DFA defines finite number of states to distinguish finite number of distinct patterns
• DFA may accept infinite strings
• Let $L$ be a regular language. There exitsts an integer $p\ge 1$ s.t. any string $w\in L$ with $|w|\ge p$ can be divided into three pieces $w=xyz$, satisfying the following conditions:
  • $\forall i\ge 0,x{y}^{i}z\in L$
  • $|y|>0$
  • $|xy|\le p\Rightarrow |y|\le p$
• Proof: Let $M$ be a DFA that accepts $L$, $p$ be # distinct states of $M$
  • If every string in $L$ has length less than $p$, i.e. $L$ is finite
  • Otherwise, let $w$ be any string in $L$ with $|w|\ge p$, $w=a_1{a}_2\dots a_n(n\ge p)$, which is accepted by a DFA with $n+1$ states $(q_0,q_1,\dots ,q_n)$.
  • According to Penguin Hole Theorem, there must exist $0\le i<j\le p$ such that $q_i=q_j$
  • Such DFA satisfying:
    • $\forall i\ge 0,x{y}^{i}z\in L$
    • $|y|=j-i>0$
    • $|xy|=j\le p$
• regular $\to$ pumping, not pumping $\to$ not regular

Example: Show that L={aibi:i≥0}L=\{a^ib^i:i\ge 0\}L={aibi:i≥0} is not regular

• Assume for the sake of contradiction, $L$ is regular
• Let $p$ be the pumping length
• Let $w=a^p{b}^p\in L$, then $w$ can be written as $w=xyz$ such that:
  • $\forall i\ge 0,x{y}^{i}z\in L$
  • $|y|>0$
  • $|xy|\le p$
• For $a^p{b}^p$, $\exists k\ge 1$, s.t. $y=a^k$, therefore $x{y}^{0}z=a^{p-k}{b}^p\notin L$, contradicting with assumption

Example: Show that L={w∈{a,b}∗:wL=\{w\in\{a,b\}^*:wL={w∈{a,b}∗:w has equal number of $a$’s and $b$’s $\}$ is not regular

• Proof: Assume that $L$ is regular, therefore L∩a∗b∗={aibi:i≥0}L\cap a^*b^*=\{a^ib^i:i\ge 0\}L∩a∗b∗={aibi:i≥0} is regular, contradicts.