《集体智慧编程》笔记（弃坑）

ychinata

已于 2022-04-21 09:39:27 修改

阅读量1k

点赞数

文章标签：人工智能 python

于 2016-10-24 17:41:55 首次发布

本文链接：https://blog.youkuaiyun.com/ychinata/article/details/52913760

版权

2022.4.21弃坑

第4章：搜索与排名

基于内容的排名

#查询
import searchengine
e = searchengine.searcher('searchindex.db')
e.query('functional programming')

#无评价函数
select w0.urlid,w0.location,w1.location from wordlocation w0,wordlocation w1 where w0.wordid=145 and w0.urlid=w1.urlid and w1.wordid=19
0.000000        http://kiwitobes.com/wiki/XSLT.html
0.000000        http://kiwitobes.com/wiki/XQuery.html
0.000000        http://kiwitobes.com/wiki/Unified_Modeling_Language.html
0.000000        http://kiwitobes.com/wiki/SNOBOL.html
0.000000        http://kiwitobes.com/wiki/Procedural_programming.html
0.000000        http://kiwitobes.com/wiki/Miranda_programming_language.html
0.000000        http://kiwitobes.com/wiki/ISWIM.html
0.000000        http://kiwitobes.com/wiki/Smalltalk_programming_language.html
0.000000        http://kiwitobes.com/wiki/Self_programming_language.html
0.000000        http://kiwitobes.com/wiki/MOO_programming_language.html
Out[5]: ([145, 19], [437, 436, 419, 389, 373, 372, 370, 365, 364, 361])

#评价函数为单词频度
select w0.urlid,w0.location,w1.location from wordlocation w0,wordlocation w1 where w0.wordid=145 and w0.urlid=w1.urlid and w1.wordid=19
1.000000        http://kiwitobes.com/wiki/Functional_programming.html
0.262476        http://kiwitobes.com/wiki/Categorical_list_of_programming_languages.html
0.062310        http://kiwitobes.com/wiki/Programming_language.html
0.043976        http://kiwitobes.com/wiki/Lisp_programming_language.html
0.036394        http://kiwitobes.com/wiki/Programming_paradigm.html
0.030880        http://kiwitobes.com/wiki/Multi-paradigm_programming_language.html
0.027295        http://kiwitobes.com/wiki/Perl.html
0.022057        http://kiwitobes.com/wiki/Declarative_programming.html
0.020265        http://kiwitobes.com/wiki/Generic_programming.html
0.019024        http://kiwitobes.com/wiki/Object-oriented_programming.html
Out[9]: ([145, 19], [220, 1, 2, 227, 289, 288, 171, 196, 293, 296])

#评价函数为文档位置
select w0.urlid,w0.location,w1.location from wordlocation w0,wordlocation w1 where w0.wordid=145 and w0.urlid=w1.urlid and w1.wordid=19
1.000000        http://kiwitobes.com/wiki/Functional_programming.html
0.150183        http://kiwitobes.com/wiki/Haskell_programming_language.html
0.149635        http://kiwitobes.com/wiki/Opal_programming_language.html
0.149091        http://kiwitobes.com/wiki/Miranda_programming_language.html
0.149091        http://kiwitobes.com/wiki/Joy_programming_language.html
0.149091        http://kiwitobes.com/wiki/Dylan_programming_language.html
0.149091        http://kiwitobes.com/wiki/Charity_programming_language.html
0.149091        http://kiwitobes.com/wiki/Curry_programming_language.html
0.149091        http://kiwitobes.com/wiki/Scheme_programming_language.html
0.148551        http://kiwitobes.com/wiki/Logo_programming_language.html
Out[11]: ([145, 19], [220, 225, 232, 372, 226, 223, 221, 141, 125, 253])

#评价函数为单词频度+文档位置
#weights=[(1.0,self.frequencyscore(rows)),
#         (1.5,self.locationscore(rows))]
select w0.urlid,w0.location,w1.location from wordlocation w0,wordlocation w1 where w0.wordid=145 and w0.urlid=w1.urlid and w1.wordid=19
2.500000        http://kiwitobes.com/wiki/Functional_programming.html
0.438190        http://kiwitobes.com/wiki/Categorical_list_of_programming_languages.html
0.265997        http://kiwitobes.com/wiki/Lisp_programming_language.html
0.242920        http://kiwitobes.com/wiki/Haskell_programming_language.html
0.239490        http://kiwitobes.com/wiki/Scheme_programming_language.html
0.235922        http://kiwitobes.com/wiki/Logo_programming_language.html
0.235216        http://kiwitobes.com/wiki/Miranda_programming_language.html
0.232735        http://kiwitobes.com/wiki/Dylan_programming_language.html
0.231218        http://kiwitobes.com/wiki/Curry_programming_language.html
0.230408        http://kiwitobes.com/wiki/ML_programming_language.html
Out[13]: ([145, 19], [220, 1, 227, 225, 125, 253, 372, 223, 141, 230])

#评价函数为单词距离
select w0.urlid,w0.location,w1.location from wordlocation w0,wordlocation w1 where w0.wordid=145 and w0.urlid=w1.urlid and w1.wordid=19
1.000000        http://kiwitobes.com/wiki/XSLT.html
1.000000        http://kiwitobes.com/wiki/XQuery.html
1.000000        http://kiwitobes.com/wiki/Procedural_programming.html
1.000000        http://kiwitobes.com/wiki/Miranda_programming_language.html
1.000000        http://kiwitobes.com/wiki/ISWIM.html
1.000000        http://kiwitobes.com/wiki/Smalltalk_programming_language.html
1.000000        http://kiwitobes.com/wiki/MOO_programming_language.html
1.000000        http://kiwitobes.com/wiki/SuperCollider.html
1.000000        http://kiwitobes.com/wiki/Smalltalk.html
1.000000        http://kiwitobes.com/wiki/Sather_programming_language.html
Out[15]: ([145, 19], [437, 436, 373, 372, 370, 365, 361, 352, 345, 342])

利用外部回指链接

#The PageRank Algorithm
import searchengine
crawler = searchengine.crawler('searchindex.db')
crawler.calculatepagerank()
cur = crawler.con.execute('select * from pagerank order by score desc')
values = cur.fetchall()
for i in range(3): print(values[i])
(438, 2.528516)
(2, 1.161464)
(543, 1.064252)

#查询
import searchengine
e = searchengine.searcher('searchindex.db')
e.query('functional programming')

#评价函数加上排名  
#weights=[(1.0,self.distancescore(rows)),
#         (1.0,self.frequencyscore(rows)),
#         (1.0,self.pagerankscore(rows))]
select w0.urlid,w0.location,w1.location from wordlocation w0,wordlocation w1 where w0.wordid=145 and w0.urlid=w1.urlid and w1.wordid=19
2.318146        http://kiwitobes.com/wiki/Functional_programming.html
2.062310        http://kiwitobes.com/wiki/Programming_language.html
1.400490        http://kiwitobes.com/wiki/Categorical_list_of_programming_languages.html
1.387165        http://kiwitobes.com/wiki/Object-oriented_programming.html
1.314186        http://kiwitobes.com/wiki/Programming_paradigm.html
1.278803        http://kiwitobes.com/wiki/Lisp_programming_language.html
1.257997        http://kiwitobes.com/wiki/Smalltalk.html
1.249883        http://kiwitobes.com/wiki/Haskell_programming_language.html
1.247156        http://kiwitobes.com/wiki/Python_programming_language.html
1.241643        http://kiwitobes.com/wiki/Perl.html
Out[3]: ([145, 19], [220, 2, 1, 296, 289, 227, 345, 225, 249, 171])

#评价函数再加上链接
#weights=[(1.0,self.distancescore(rows)),
#         (1.0,self.frequencyscore(rows)),
#         (1.0,self.pagerankscore(rows)),
#         (1.0,self.linktextscore(rows,wordids))] 
select w0.urlid,w0.location,w1.location from wordlocation w0,wordlocation w1 where w0.wordid=145 and w0.urlid=w1.urlid and w1.wordid=19
3.121124        http://kiwitobes.com/wiki/Functional_programming.html
3.062310        http://kiwitobes.com/wiki/Programming_language.html
1.699063        http://kiwitobes.com/wiki/Object-oriented_programming.html
1.496661        http://kiwitobes.com/wiki/Programming_paradigm.html
1.447028        http://kiwitobes.com/wiki/Categorical_list_of_programming_languages.html
1.353089        http://kiwitobes.com/wiki/Procedural_programming.html
1.352060        http://kiwitobes.com/wiki/Logic_programming.html
1.331260        http://kiwitobes.com/wiki/Imperative_programming.html
1.315676        http://kiwitobes.com/wiki/Lisp_programming_language.html
1.294123        http://kiwitobes.com/wiki/Generic_programming.html
Out[4]: ([145, 19], [220, 2, 296, 289, 1, 373, 258, 197, 227, 293])

从点击行为中学习

import nn
mynet = nn.searchnet('nn.db')
mynet.maketables()
wWorld, wRiver, wBank = 101,102,103
uWorldBank, uRiver, uEarth = 201,202,203
mynet.generatehiddennode([wWorld,wBank],[uWorldBank,uRiver,uEarth])
for c in mynet.con.execute('select * from wordhidden'): print (c):
(101, 1, 0.5)
(103, 1, 0.5)
for c in mynet.con.execute('select * from hiddenurl'): print (c)
(1, 201, 0.1)
(1, 202, 0.1)
(1, 203, 0.1)

#前馈法
mynet = nn.searchnet('nn.db')
mynet.getresult([wWorld,wBank],[uWorldBank,uRiver,uEarth])
Out[13]: [0.07601250837541615, 0.07601250837541615, 0.07601250837541615]

第5章：优化

描述题解

import optimization
s = [1,4,3,2,7,3,6,3,2,4,5,3]
optimization.printschedule(s)

#output
   Seymour       BOS  8:04-10:11 $ 95 12:08-14:05 $142
    Franny       DAL 12:19-15:25 $342 10:51-14:16 $256
     Zooey       CAK 10:53-13:36 $189  9:58-12:56 $249
      Walt       MIA  9:15-12:29 $225 16:50-19:26 $304
     Buddy       ORD 16:43-19:00 $246 10:33-13:11 $132
       Les       OMA 11:08-13:07 $175 15:07-17:21 $129

成本函数

In [75]: optimization.schedulecost(s)
Out[75]: 5285

随机搜索

domain = [(0,9)]*(len(optimization.people)*2)   #[(0,9)]*12，每人有10个航班可选择
s = optimization.randomoptimize(domain,optimization.schedulecost)
optimization.schedulecost(s)
Out[36]: 4542
optimization.printschedule(s)
Out[37]:
   Seymour       BOS 17:11-18:30 $108 15:25-16:58 $ 62
    Franny       DAL 15:44-18:55 $382  9:49-13:51 $229
     Zooey       CAK  9:15-12:14 $247 10:32-13:16 $139
      Walt       MIA 11:28-14:40 $248 16:50-19:26 $304
     Buddy       ORD 16:43-19:00 $246 15:04-17:23 $189
       Les       OMA 15:03-16:42 $135 11:07-13:24 $171