Optimal BST

最新推荐文章于 2020-11-19 10:59:39 发布
转载 最新推荐文章于 2020-11-19 10:59:39 发布 · 3.5k 阅读 · 0
文章标签：

#algorithm #tree #search #pair #list #access

Optimal BST


Assume:
We have a list of n items: a1, a2, ..., an
Key(ak) = ak
Probability of accessing item ak is known in advance and is P(ak)
The list is ordered by keys, a1 <= a2 <= ... an

How to produce the BST that has the least search cost given the access probability for each key?


Optimal BST
Greedy Algorithm fails

Example:
list = a, b, c, d
P(a) = .1, P(b) = .2, P(c) = .3, P(d) = .4 Average Cost = 1*0.4 + 2*0.3 + 3*0.2 + 4*0.1 =  2.0
Optimal BST
Average Cost = 1*0.3 + 2*0.4 + 2*0.2 + 3*0.1 =  1.8
Finding the Optimal BST

Let A[j, k] = minimum average search time for a binary search tree with items aj <= aj+1 <= ... ak  
How to find A[1,n]?Set p = 1, 2, ..., n




j\k012345
10P(a1)    
200P(a2) A[2,4] 
3000P(a3)  
40000P(a4) 
500000P(a5)
6000000


Time Complexity for finding Optimal BST Theta(n 3)

Dynamic Programming


"Dynamic programming algorithm stores the results for small subproblems and looks them up, rather than recomputing them, when it needs them later to solve larger subproblems"

Baase


"Dynamic programming algorithm is very useful approach to many optimization problems. ... Usually, we can find a tailor-made algorithm which is more efficient than the straight-forward algorithm based on dynamic programming"


T. C. Hu

Steps in developing a dynamic programming algorithm


Characterize the structure of an optimal solution

Recursively define the value of an optimal solution

Compute the value of an optimal solution in a bottom-up fashion

Construct an optimal solution from computed information
Matrix-chain Multiplication
The Problem

Let A be a 10 * 100 matrix,
B be a 100 * 5 matrix C be a 5 * 50 matrix
Then
A * B is a 10 * 5 matrix B * C is a 100 * 50 matrix


Computing
A * B takes 10 * 100 * 5 = 5,000 multiplications (A * B) * C takes 10 * 5 * 50 = 2,500 additional mult. So (A * B) * C requires total of 7,500 multiplications


Computing
B * C takes 100 * 5 * 50 = 25,000 multiplications A * (B * C) takes 10 * 100 * 50 = 50,000 additional mult. So A * (B * C) requires total of 75,000 multiplications


But (A * B) * C = A * (B * C)
Matrix-chain Multiplication
The Problem

Given a chain <A1, A2, ..., An> matrices, where matrix Ak has dimension pk-1 * pk fully parenthesize the product A1* A2*... *An, in a way that minimizes the number of scalar multiplications

Characterize the structure of an optimal solution


Optimal solution is of the form:
(A1* ... *Ak) * (Ak+1* ... *An)

not showing the parentheses in (A1* ... *Ak) or in (Ak+1* ... *An)


We must use the optimal parenthesization of A1* ... *Ak and the optimal parenthesization of Ak+1* ... *An


Thus the optimal solution to an instance of the matrix-chain multiplication problem contains within it optimal solutions to subproblem instances

That is it is recursive
Recursively define the value of an optimal solution

Recall matrix Ak has dimension pk-1 * pk for k = 1,..., n



Let m[k, w] be the minimum the number of scalar multiplications needed to compute Ak* ... *Aw


If (Ak* ... *Az) * (Az+1* ... *Aw) is the optimal solution then
m[k, w] = m[k, z] + m[z+1, w] + pk-1 * pz * pk


since (Ak* ... *Az) is a pk-1 * pz matrix 

and (Az+1* ... *Aw) is a pz * pw matrix



But what is z?

Compute the optimal solution in a bottom-up fashion



Example 
Matrix		dimension		r		pr
A1		10*20			0		10	
A2		20*3			1		20
A3		3*5			2		3
A4		5*30			3		5
					4		30
 1234
106007501950
2 03002250
3  0450
4   0


Construct an optimal solution
from computed information

Example 
Matrix		dimension		r		pr
A1		10*20			0		10	
A2		20*3			1		20
A3		3*5			2		3
A4		5*30			3		5
					4		30
 1234
1A1A1A2(A1A2)A3(A1A2)(A3A4)
2 A2A2A3A2(A3A4)
3  A3A3A4
4   A4


Time Complexity of building the OBST?

We have a list of n items: a1, a2, ..., an


Probability of accessing item ak is P(ak) 


Let A[j, k] = minimum average search time for a binary search tree with items aj <= aj+1 <= ... ak  






Let root[j,k] = p that gave the minimum value for A[j, k]

That is root[j,k] = root of OBST for items aj, a2, ..., ak



Let w[j,k] = P(aj) + P(aj+1) + ... + P(ak) 
Constructing the OBST

for k = 1 to n do
A[k, k] = P(ak) A[k, k-1] = 0 root[k,k] = k w[k, k] = P(ak)
end
A[n+1, n] = 0


for diagonal = 1 to n -1 do
for j = 1 to n - diagonal do
k = j + diagonal w[j, k] = w[j, k - 1] + P(ak) let p, j <= p <= k, be the value minimizes:  root[j,k] = p A[j, k] = A[j, p-1] + A[p+1, k] + w[j, k]
end for
end for

Example 1 
k	1	2	3	4	5	6
ak	a	b	c	d	e	f
P(ak)'s =   0.4	0.05	0.15	0.05	0.1	0.25

root 
1	1	1	1	1	3
0	2	3	3	3	5
0	0	3	3	3	5
0	0	0	4	5	6
0	0	0	0	5	6
0	0	0	0	0	6

A 
0	0.4	0.5	0.85	1	1.35	2.1
0	0	0.05	0.25	0.35	0.6	1.2
0	0	0	0.15	0.25	0.5	1.05
0	0	0	0	0.05	0.2	0.6
0	0	0	0	0	0.1	0.45
0	0	0	0	0	0	0.25
0	0	0	0	0	0	0
Example 2

P(ak)'s = (0.15 0.025 .05 .025 .05 .125 .025 .075 0.075 .05 .15 .075 .05 .025 .05)

Root 
1	1	1	1	1	3	3	6	6	6	6	6	6	6	6
0	2	3	3	3	5	6	6	6	6	6	9	9	9	11
0	0	3	3	3	5	6	6	6	6	9	9	9	9	11
0	0	0	4	5	6	6	6	6	6	9	9	9	9	11
0	0	0	0	5	6	6	6	6	8	9	9	9	11	11
0	0	0	0	0	6	6	6	8	8	9	9	11	11	11
0	0	0	0	0	0	7	8	8	9	9	11	11	11	11
0	0	0	0	0	0	0	8	8	9	9	11	11	11	11
0	0	0	0	0	0	0	0	9	9	11	11	11	11	11
0	0	0	0	0	0	0	0	0	10	11	11	11	11	11
0	0	0	0	0	0	0	0	0	0	11	11	11	11	11
0	0	0	0	0	0	0	0	0	0	0	12	12	12	13
0	0	0	0	0	0	0	0	0	0	0	0	13	13	13
0	0	0	0	0	0	0	0	0	0	0	0	0	14	15
0	0	0	0	0	0	0	0	0	0	0	0	0	0	15

A[1,15] = 2.925
Modified Algorithm

for diagonal = 1 to n -1 do
for j = 1 to n - diagonal do
k = j + diagonal w[j, k] = w[j, k - 1] + P(ak) let p,  root[j,k-1] <= p <= root[j+1,k], be the value minimizes:  root[j,k] = p A[j, k] = A[j, p-1] + A[p+1, k] + w[j, k]
end for
end for

Time Complexity

General Theorem

Let H(i, j) be a real number for 1 <= i < j <= n

Let c(i, j) be defined by:
c(i, i) = 0 c(i, j) = H(i, j) + 
Let K(i, j) = largest k, i <= k <=j, that minimizes c(i, k-1) + c(k, j)

H(i, j) is  monotone with respect to set inclusion of intervals if 
H(j, k) <= H(x, y) if j <= x < y <= k

H(i, j) satisfies the  quadrangle inequality (QI) if
H(j, k) + H(x, y) <= H(x, k) + H(j, y) if j <= x < k <= y

Theorem. If H(i, j) satisfies QI and is monotone then c(i, j) can be computed in time O(n 2)

Lemma. If H(i, j) satisfies QI and is monotone then c(i, j) satisfies QI
Lemma. If c(i, j) satisfies QI then we have
K(i, j) <= K(i, j+ 1) <= K(i+1, j+1)
Splay vs. OBST

Recall in OBST we have:
We have a list of n items: a1, a2, ..., an and leaves b0, b1, ..., bn Probability of accessing item ak is P(ak) =  Alphak Let  Betak be the probability of accessing a key that is between ak and ak+1, that is leaf bk

In splay tree we have  Betak = 0, and  Alphak = q(i)/m

Theorem 9: Let Popt be the be the weighted path length of optimum BST. We have:
where H = H( Beta0 , Alpha1 , Beta1 ,  Alpha2,  Beta2,... ,  Alphan,  Betan) is entropy

Thus we have:
 for some x
Since Popt is the average path length the total time spent to perform m accesses is m*Popt.
Now:

So m*H = 

Thus: m*Popt <= m*H + 1 =   + m

Nearly Optimal BST


We have a list of n items: a1, a2, ..., an

Probability of accessing item ak is P(ak) 

k 1 2 3 4 5 6
ak a b c d e f
P(ak)'s = 0.4 0.05 0.15 0.05 0.1 0.25
Method 1

Average Access Cost = 2.2
Nearly Optimal BST
k 1 2 3 4 5 6
ak a b c d e f
P(ak)'s = 0.4 0.05 0.15 0.05 0.1 0.25
Method 2


Average Access Cost = 2.1

Optimal Alphabetic Tree[1]


An alphabetic tree is a binary search tree in which all data is in the leaves. Internal nodes are used in search for the data





Let V1, V2,... Vn be the order of the leaves

Let wk be the weight, or frequency of access, of leaf Vk

Combining Vk and Vp, denote their parent node by Vkp and it weight wkp = wk+ wp

All leaves are square nodes, all parents are round nodes
Optimal Alphabetic Tree: Definitions


Two nodes are a compatible pair if they are adjacent or if all nodes between them are round nodes


The weight of a pair is the weight of the parent of the two nodes


A pair with minimum weight over all pairs is a minimum pair


Minimum compatible pair is the compatible pair with the least weight over all compatible pairs

Ties are broken by taking the pair with the left most left node If two compatible pairs have the same left node and the same weight, then pick the pair with the leftmost right node
Hu-Tucker Algorithm

1. Construction
Find the minimum compatible pair Replace the left node of the pair by the pair's parent Remove right node of the pair Repeat n-1 times Call the resultant tree T'

2. Level Assignment
Determine the level number Lk of every leaf Vk in T'
3. Reconstruction
We have the level numbers L1, L2,... Ln of all leaves Find the leftmost maximum level number, say Lk = q Then Lk+1 = q Replace Lk and Lk+1 with a parent node with level q - 1 Repeat n - 1 times

Theorem. The Hu-Tucker algorithm can be implemented to produce the optimal alphabetic tree with N leaves in O(Nlg(N)) time and O(N) space
How Well Does OBST and NOBST Perform?

We have a list of n items: a1, a2, ..., an and leaves b0, b1, ..., bn

Probability of accessing item ak is known in advance and is P(ak) =  Alphak

Let  Betak be the probability of accessing a key that is between ak and ak+1, that is leaf bk

The list is ordered by keys, b0< a1 <b1< a2 < ... < an < bn


( Beta0 , Alpha1 , Beta1 ,  Alpha2,  Beta2,... ,  Alphan,  Betan) is the access distribution

Let 

Let L(ak) be the level of the node ak


Let 

P is the weighted path length of a tree



Let ( Gamma1 , Gamma2,... ,  Gamman) be a discrete probability distribution, i.e.

Gammak >= 0 and  SigmaGammak =1


H( Gamma1 , Gamma2,... ,  Gamman) = 

is the  entropy of the distribution. ( 0*log 0 = 0)

Let TBB be the tree resulting from the nearly optimal BST algorithm 1


Theorem 7 [2]: Let L(ak) be the depth of node ak and let L(bk) be the depth of leaf bk in tree TBB Then
bk <= floor (log 1/bk) + 2 ak <= floor (log 1/ak)

Theorem 8 [3]: Let PBB be the weighted path length of the tree TBB. Then





Theorem 9 [4]: Let PBB be the weighted path length of tree TBB. Let Popt be the be the weighted path length of optimum BST. We have:
where H = H( Beta0 , Alpha1 , Beta1 ,  Alpha2,  Beta2,... ,  Alphan,  Betan) and 


This gives us 
Example.

In English, the probability of occurrence of the i-th most frequent word is approximately [5]


This yields


H( Alpha1 , Alpha2, ... ) = 10.2

The weighted path length of an optimum binary search tree for  all English words is no larger than 11.2.
Nearly Optimal BST

We have a list of n items: a1, a2, ..., an

Probability of accessing item ak is P(ak) 

k 1 2 3 4 5 6
ak a b c d e f
P(ak)'s = 0.4 0.05 0.15 0.05 0.1 0.25
Method 1

Problem:

k 1 2 3
ak a b c
P(ak)'s = 0.45 0.1 0.45

Algorithm for Nearly Optimal Lexicographic Tree[6]



Given the ordered set {a} of names, such that a1 <= a2 <= ... an , and two parameters, F and N0 


(1) If N <= N0 , use dynamic programming.

(2) If N> N0, let W[k, l] be the weight of the subtree with frequencies
Betak , Alphak+1 , Betak+1 , ... ,  Alphal,  Betal
F a parameter and AC the centroid. Form the ordered set of names {AF} = {AL} union AC, where the members of the set {AL} satisfy
|W[O,L-1] - W[L,N]| < W[O,N]/F, 1 <= F <= W[O,N]


(3) Find an index, max, such that  Alphamax = maximum  Alphai, where ai is in {AF}.
(4) If in the set {AF} there is at least one name preceding or equal to AC with associated frequency  Alphamax, let p be the index such that ap with  Alphap =  Alphamax, is lexicographically closest to AC. 
If there is no such p, let {AQ} be the null set and go to Step 6.

(5) If ap is the first member of {AF} and  Alphap-1 >  Alphap, form the set {AQ} = {ap-1 , ap-2, ., au}, where  Alphap-j-1 >  Alphap-j , j = 0, ... ,p-u-1 and  Alphau-1 <=  Alphau or u-p = floor(lg N); 
if ap is not the first member of {AF}, let {AQ} be the null set.

(6) If in the set {AF} there is at least one name following or equal to AC with associated frequency  Alphamax let r be the index such that ar with  Alphar =  Alphamax is lexicographically closest to AC; 
if there is no such r, let {AS} be the null set and go to Step 8.

(7) If ar is the last member of {AF} and  Alphar< Alphar+1, form the set {AS}= {ar+1, ar+2,...,av}, where  Alphar+j< Alphar+j+1, j=0, 1, ... ,v-r-1, and  Alphav>= Alphav+1or v-r = floor(lg N). 
If ar is not the last member of {AF}, let {AS} be the null set.
(8) Find an index, root, such that  Alpharoot = maximum  Alphai, where ai is in {AQ} union {AF}union{AS} and |W[O,root-1] - W[root,N]| is minimized; choose aroot as the root of the tree.


(9) Go to Step 1 and repeat the algorithm for the subtrees a1, a2, ... aroot-1 and aroot+1, ... aN, where N is root-I and N-root for the two cases.

Picking N0 and F

if  Beta/ Alpha is small (<= 3) than use N0= 15

if  Beta/ Alpha is large (> 3) than use N0= 25 or 30

if  Beta/ Alpha is large (> 2) than use F = 6

if  Beta/ Alpha ~ 1 than use F = 4

if  Beta/ Alpha < 1 than use F = 4

The tree takes O(Nlog(N)) to construct

Average search time for NOBST is within 2% of the average search time of the OBST
Average Search Length

N0= 15, F = 5

NAlpha freqBeta freqOBSTNOBST
519,846982,4973.41143.4114
1542,653959,6904.28644.2864
2560,087942,2565.06385.1033
5092,117910,2266.04836.1461
100138,975863,3687.00077.0437
150173,157829,1867.48857.5503
200200,412801,931 7.8795
500305,266697,077 8.9606
1000401,288601,055 9.6490
3000561,956440,387 10.6220
6000655,538346,805 11.1177
12000740,022262,321 11.1592
 
最优二叉搜索树(Optimal BST)-算法导论
taotao 的大学墓志
07-29 4936
问题描述：　维基百科定义: https://en.wikipedia.org/wiki/Optimal_binary_search_tree 　In the static optimality problem as defined by Knuth we are given a set of n{\displaystyle n} ordered elements and a set of 2
最优二叉查找树（optimal BST）
小威的专栏
11-26 5149
/** 最优二叉查找树：一棵有n个结点的二叉查找树，已知每个结点的查找概率Pi（且∑Pi=1），要使查找操作的平均比较次数最小。（这里讨论的是成功查找，不讨论不成功的查找） 动态规划： c[i][j]表示由结点i~j组成的BST成功查找的最小平均查找次数。 r[i][j]表示由结点i~j构成最优二叉查找树时的树根结点。 转换公式： c[i][j] = Min[i<=
算法导论实践——OptimalBST（最优搜索二叉树）
不定期更新系统使用、编程小技巧，哔哩哔哩账号：https://space.bilibili.com/399455629，欢迎关注
03-31 1171
算法导论实践——OptimalBST（最优搜索二叉树）——这波是脑壳痛理论C#实践OptimalBinarySearchTree.csProgram.cs 动态规划的思想，很巧妙（话又说回来，什么东西不巧妙呢） 理论 思路就是，取得Kr∈{Ki...Kj}，以Kr为根节点构造两颗最优BST， {Ki...Kr-1}和{Kr+1...Kj} 1.设E[i,j]代表{Ki...Kj}和{Di-1....
最优二叉查找树-optimal-BST--C++实现2
weixin_30587927的博客
01-06 165
我发现之前有这么一篇，竟然又写了一遍……不过，这个比较简单，没用class，姑且命名为2 经常看到有人写动态规划程序就用dp命名数组，发觉挺好，因为想一个合适的名字太费劲了，而且便于重用（与问题无关） 最优二叉查找树的思想： （建议还是看书吧，算法导论上有，大多数算法书上应该都有，挑一个容易读懂的看，有些算法算法导论上讲得很复杂（也很严密） https://class.coursera.o...
最优二叉查找树-optimal-BST--C++实现
weixin_30924087的博客
11-27 249
算法参考《算法导论》第15章--动态规划 //最优二叉查找树 #include <cstdio> #include <cstring> #include <limits> class optimalBST{ private: float *p, *q; int n; float **e;//(n+2)*(n+1)//e[1.....
optimal_BST.rar_optimal search tree
09-22
**最优二叉搜索树（Optimal Binary Search Tree）** 最优二叉搜索树，也被称为最优点搜索树，是一种特殊的二叉查找树，它的设计目的是为了在执行搜索操作时尽可能地减少平均查找时间。这种树的构造是基于给定的一组...
Construct Optimal Binary Search Tree by Using Greedy Algorithm
02-21
1. 二叉搜索树（Binary Search Tree，简称BST）： 二叉搜索树是一种特殊的二叉树，其左子树上所有节点的值均小于它的根节点的值，右子树上所有节点的值均大于它的根节点的值。这种性质使得二叉搜索树在搜索、插入和...
Optimal Binary Search Tree
10-06
### 最优二叉查找树（Optimal Binary Search Tree） #### 概述 最优二叉查找树（Optimal Binary Search Tree, OBST）是计算机科学领域内一种高效的搜索数据结构，其设计目标是在已知各节点访问概率的情况下，构建...
O(n^2)-time Algorithm to OPTIMAL-BST Question
Pandoras Box
07-27 2047
Knuth [184] has shown that there are always roots of optimal subtrees such that root[i, j - 1] ≤ root[i, j] ≤ root[i + 1, j] for all 1 ≤ i The original O(n^3)-time algorithm is: To
最优BST的动态规划算法Java实现
ljsspace的专栏
05-08 1687
//输入p: 访问概率数组； 0,1,....n是排好序的Key值 //输出A=new float[n][n]: 最优时间矩阵 //输出R=new int[n][n]: 根结点矩阵 static void optimalBST(float[] p,float[][] A,int[][] R){ int n = p.length; for(int i=0;in-1) break; float min=-1; int
BST最优二叉检索树(java版本)
gg_gogoing的专栏
04-10 1181
这个内容是去年暑假讲的,但是一直没有实现, 其实说白了就是区间dp,求一个序列构成二叉树,中序遍历有序. 核心和其他区间dp一样,枚举中间值.然后枚举出来后再将整个区间的概率累加,因为相当于加深了一层. JAVA代码,附测试数据 import java.util.Arrays; import java.util.Scanner; public class Main { /*
Algorithm Analysis & Design: Dynamic Programming - Optimal BST
BenBenBenBenBenB的博客
02-24 360
Hi peers, In this essay, I will introduce a dynamic programming algorithm that can construct an optimal binary search tree. Before we talk about the algorithm, let’s first understand what an optimal B...
optimizer（代码分析）
weixin_30497527的博客
03-03 186
torch.optim是里面是和优化算法相关的类。比如使用SGD算法用 optimizer = optim.SGD(net.parameters(),lr=1e-3) 这个地方要注意的是传入的第一个参数是网络的parameters。 这个类里面有param_group,是一个字典，里面包括： params: 网路可学习权重 lr: 学习率 weight_decay:权...
构造最优二叉查找树的时间复杂度分析
学海无涯
09-02 5831
<br />概念<br /><br />       二叉查找树（BST，Binary Search Tree） ，又名二叉搜索树或二叉检索树，是一颗满足如下条件的树： <br />       1、每个节点包含一个键值 <br />       2、每个节点有最多两个孩子 <br />       3、对于任意两个节点x和y，它们满足下述搜索性质: <br />             a、如果y在x的左子树里，则key[y] <= key[x] <br />             b、如
【算法学习】最优二叉查找树（动态规划）
热门推荐
计算机专业同学的充电站。
10-19 8万+
一、什么是最优二叉查找树 最优二叉查找树： 给定n个互异的关键字组成的序列K=，且关键字有序（k1 图一显示了给定上面的概率分布pi、qi，生成的两个二叉查找树的例子。图二就是在这种情况下一棵最优二叉查找树。 概率分布： i 0 1 2 3 4 5 p
【算法导论】最优二叉搜索树
编程小霸王的Blog
12-29 1万+
最优二叉搜索树       假定设定一个程序，实现英语文本到法语的翻译。对英语文本中出现的灭个单词，我们需要查找对应的法语单词。为了实现这些查找槽，我们可以创建一棵二搜索叉树，将n个英语单词作为关键词，对应的法语单词作为关联数据。由于文本中的每个单词都要进行搜索，我们希望花费在搜索上的总时间尽量减少。通过红黑树或其他平衡搜索树结构，我们可以假定每次搜索时间为O(lgn)，但是，单词出现的频率是不
算法导论—二叉搜索树(BST)
掉下个小石头
09-09 1508
华电北风吹 天津大学认知计算与应用重点实验室 日期：2015/9/9 与散列表一样，搜索树数据结构也支持动态集合操作，包含插入，查询，删除，最小值，最大值，前驱，后继等。一、二叉搜索树： 二叉搜索树节点：关键字key，卫星数据，左孩子指针，右孩子指针，父节点指针，其他特殊类型(红黑树的节点颜色，AVL树的树高等)。 二叉搜索树性质：x是二叉搜索树中的任意一个节点。若y是x左子树中任意一个节点有
算法导论第三版 第15章习题答案
DavidWang9527的专栏
11-19 2万+
15 Dynamic Programming
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值