fastdb相关备忘

最新推荐文章于 2022-10-09 20:10:07 发布

转载最新推荐文章于 2022-10-09 20:10:07 发布 · 532 阅读

文章标签：

#fastdb

ROS 同时被 2 个专栏收录

59 篇文章

订阅专栏

数据库

32 篇文章

订阅专栏

选择fastdb主要有两个原因，
1.内存数据库，体量小，速度快
2.其数据结构与交易系统非常类似（很大可能性是交易系统fld那套东西借鉴了fastdb，猜的：）

部署&安装

下载
http://www.garret.ru/fastdb.html

ok@x1:~/下载$ ll fastdb-3.76.tar.gz
-rw-rw-r-- 1 ok ok 3287628 7月 10 10:29 fastdb-3.76.tar.gz

```
编译安装
```

ok@x1:~/下载$ ll fastdb-3.76.tar.gz
-rw-rw-r-- 1 ok ok 3287628 7月 10 10:29 fastdb-3.76.tar.gz
ok@x1:~/workspace/fastdb$ ./configure
ok@x1:~/workspace/fastdb$ make
ok@x1:~/workspace/fastdb$ sudo make install

修改源码

宏冲突
TYPE_DESCRIPTOR,fastdb也使用了这个宏，或者说交易系统也用了这个宏，连名字都不改一下。。。
纠结了一下，是该交易系统的代码呢还是还fastd的呢？
当然，哪个好改改哪个，fastdb走起。
有一处定义，和20几处的使用，当然不能手动改了，脚步继续走起：

find -name "*.h" | xargs sed -i 's/TYPE_DESCRIPTOR/FAST_TYPE_DESCRIPTOR/g'

FAST_TYPE_DESCRIPTOR是目标

config.h

frank@frank-ThinkPad-13:~/workspace/adapter/DatToText/bin$ ./parsedata -d trade20140221.dat
Incompatibility between headers and library: 6 vs. 4

注释掉下面的宏

// LARGE_DATABASE_SUPPORT - support databases with size larger than 4Gb.
// If this macro is defined FastDB maximal database size is limited by 1 terrabyte.
// Defnining this macro increase iniitial database size and object index size (because
// 64-bit offsetss are used instead fo 4-byte offsets). Also FastDB library compiled with
// LARGE_DATABASE_SUPPOR will not be able to work with database created by library
// built without LARGE_DATABASE_SUPPOR and visa versa.
#if !defined(_WIN32) || defined(_WIN64) // most unixes are now 64-bit, while 32-bit windows is still quite popular
#define LARGE_DATABASE_SUPPORT
#endif

重新编译安装fastdb

跑一下看看

只做了几个表先跑一下试试，生成数据文件dce_db.fdb，先用客户端试试：

frank@frank-ThinkPad-13:~/workspace/adapter/DatToText/bin$ subsql
SubSQL interactive utility for FastDB v. 3.75
Type 'help' for more information
>> help
SubSQL commands:
open 'database-name' ( 'database-file-name' ) ';'
select ('*') from <table-name> where <condition> ';'
update <table-name> set <field-name> '=' <expression> {',' <field-name> '=' <expression>} where <condition> ';'
create table <table-name> '('<field-name> <field-type> {',' <field-name> <field-type>}')' ';'
alter table <table-name> '('<field-name> <field-type> {',' <field-name> <field-type>}')' ';'
rename <table-name> '.' <old-field-name> 'to' <new-field-name> ';'
delete from <table-name>
drop table <table-name>
drop index <table-name> {'.' <field-name>} ';'
create index on <table-name> {'.' <field-name>} ';'
drop hash <table-name> {'.' <field-name>};
create hash on <table-name> {'.' <field-name>}field> ';'
insert into <table-name> values '(' <value>{',' <value>} ')' ';'
backup [compactify] 'file-name'
start server URL number-of-threads
stop server URL
start http server URL
stop http server
describe <table-name>
import 'xml-file-name'
export 'xml-file-name'
show
profile
commit
rollback
autocommit (on|off)
exit
help
>> open 'dce_db';
>> show
FastDB version : 3.75
Database version : 3.75
Database file size: 1048576 Kb
Object index size : 1048576 handles
Used part of index: 811734 handles
Number of users : 1
Number of readers : 1
Number of writers : 0
Number of blocked readers : 0
Number of blocked writers : 0
TABLES:
OID FixedSize Fields Columns TableName
---------------------------------------------------------
0x00201c 40 3 3 FTtCreditPreFrozen
0x002003 52 5 5 FTtMemberClearAccount
0x002002 76 8 8 FTtClient
0x000001 52 17 8 Metatable
>> describe FTtMemberClearAccount;
OID=0x002003, TableName=FTtMemberClearAccount
No Index FieldType RefTableName FieldName InverseFieldName Flg
--------------------------------------------------------------------------------
0 H+T String (null) MemberId (null) 3
1 - String (null) ClearAccountNo (null) 0
2 - String (null) Status (null) 0
3 - String (null) RegDate (null) 0
4 - String (null) LogoutDate (null) 0
>>
>> select * from FTtMemberClearAccount where rowno<10
;
--------------------------------------------^
Field not found
>> select * from FTtMemberClearAccount ;
MemberId ClearAccountNo Status RegDate LogoutDate
('0011', '0011', '0', '19930101', ''),
('0012', '0012', '0', '19930101', ''),
('0014', '0014', '0', '19930101', ''),
('0015', '0015', '0', '19930101', ''),
('0016', '0016', '0', '19930101', ''),
('0018', '0018', '0', '19990813', ''),
('0019', '0019', '0', '20000516', ''),
('0020', '0020', '0', '19930101', ''),
('0021', '0021', '0', '19930101', ''),
('0022', '0022', '0', '19990813', ''),
('0023', '0023', '0', '19990120', ''),
('0025', '0025', '0', '19990122', ''),
('0026', '0026', '0', '20001025', ''),
('0030', '0030', '0', '19930101', ''),
('0033', '0033', '0', '19930101', ''),
('0035', '0035', '0', '20010514', ''),
('0036', '0036', '0', '19930101', ''),
('0037', '0037', '0', '20001120', ''),
('0038', '0038', '0', '19930101', ''),
('0039', '0039', '0', '19930101', ''),
('0040', '0040', '0', '20000712', ''),
('0042', '0042', '0', '19991026', ''),
('0043', '0043', '0', '19990603', ''),
('0046', '0046', '0', '19930101', ''),
('0048', '0048', '0', '20000717', ''),
('0049', '0049', '0', '19990308', ''),
('0050', '0050', '0', '19930101', ''),
('0051', '0051', '0', '19930101', ''),
('0052', '0052', '0', '19930101', ''),
('0053', '0053', '0', '19990118', ''),
('0056', '0056', '0', '19991215', ''),
('0057', '0057', '0', '20001225', ''),
('0058', '0058', '0', '19930101', ''),
('0060', '0060', '0', '20021111', ''),
('0062', '0062', '0', '19990531', ''),
('0063', '0063', '0', '19990603', ''),
('0065', '0065', '0', '19930101', ''),
('0066', '0066', '0', '19990813', ''),
('0067', '0067', '0', '19991026', ''),
('0069', '0069', '0', '20001127', ''),
('0070', '0070', '0', '19930101', ''),
('0071', '0071', '0', '19991026', ''),
('0072', '0072', '0', '19991215', ''),
('0074', '0074', '0', '19930101', ''),
('0075', '0075', '0', '19930101', ''),
('0078', '0078', '0', '19930101', ''),
('0080', '0080', '0', '19930101', ''),
('0081', '0081', '0', '19930101', ''),
('0082', '0082', '0', '19990813', ''),
('0084', '0084', '0', '19990302', ''),
('0085', '0085', '0', '19930101', ''),
('0088', '0088', '0', '19930101', ''),
('0089', '0089', '0', '19930101', ''),
('0092', '0092', '0', '19990813', ''),
('0093', '0093', '0', '19990603', ''),
('0095', '0095', '0', '19930101', ''),
('0096', '0096', '0', '20011107', ''),
('0097', '0097', '0', '19930101', ''),
('0098', '0098', '0', '20090720', ''),
('0099', '0099', '0', '19930101', ''),
('0100', '0100', '0', '20000817', ''),
('0101', '0101', '0', '20001207', ''),
('0102', '0102', '0', '19930101', ''),
('0103', '0103', '0', '19930101', ''),
('0104', '0104', '0', '20010807', ''),
('0105', '0105', '0', '19930101', ''),
('0106', '0106', '0', '19990813', ''),
('0107', '0107', '0', '19930101', ''),
('0108', '0108', '0', '20000809', ''),
('0109', '0109', '0', '19930101', ''),
('0110', '0110', '0', '19930101', ''),
('0111', '0111', '0', '19991026', ''),
('0112', '0112', '0', '20000919', ''),
('0113', '0113', '0', '20001221', ''),
('0114', '0114', '0', '20010727', ''),
('0115', '0115', '0', '19930101', ''),
('0117', '0117', '0', '19930101', ''),
('0118', '0118', '0', '19930101', ''),
('0120', '0120', '0', '19930101', ''),
('0121', '0121', '0', '19930101', ''),
('0122', '0122', '0', '19930101', ''),
('0123', '0123', '0', '19930101', ''),
('0125', '0125', '0', '19930101', ''),
('0126', '0126', '0', '19990813', ''),
('0129', '0129', '0', '20001218', ''),
('0133', '0133', '0', '19930101', ''),
('0134', '0134', '0', '20010625', ''),
('0135', '0135', '0', '19930101', ''),
('0136', '0136', '0', '19930101', ''),
('0138', '0138', '0', '19930101', ''),
('0139', '0139', '0', '20021223', ''),
('0141', '0141', '0', '20010216', ''),
('0143', '0143', '0', '20030403', ''),
('0145', '0145', '0', '19930101', ''),
('0147', '0147', '0', '20010607', ''),
('0148', '0148', '0', '20000801', ''),
('0149', '0149', '0', '19930101', ''),
('0150', '0150', '0', '19990813', ''),
('0151', '0151', '0', '19930101', ''),
('0152', '0152', '0', '19930101', ''),
('0153', '0153', '0', '19930101', ''),
('0155', '0155', '0', '20010424', ''),
('0156', '0156', '0', '19930101', ''),
('0157', '0157', '0', '19930101', ''),
('0158', '0158', '0', '19930101', ''),
('0160', '0160', '0', '19930101', ''),
('0161', '0161', '0', '19930101', ''),
('0164', '0164', '0', '20010711', ''),
('0165', '0165', '0', '19930101', ''),
('0166', '0166', '0', '19991215', ''),
('0167', '0167', '0', '19930101', ''),
('0168', '0168', '0', '19930101', ''),
('0169', '0169', '0', '19930101', ''),
('0170', '0170', '0', '19930101', ''),
('0171', '0171', '0', '19930101', ''),
('0172', '0172', '0', '20011224', ''),
('0173', '0173', '0', '19991215', ''),
('0175', '0175', '0', '19930101', ''),
('0176', '0176', '0', '19930101', ''),
('0177', '0177', '0', '19990813', ''),
('0178', '0178', '0', '20000712', ''),
('0180', '0180', '0', '19930101', ''),
('0181', '0181', '0', '19930101', ''),
('0182', '0182', '0', '20000920', ''),
('0183', '0183', '0', '20060328', ''),
('0184', '0184', '0', '20010530', ''),
('0185', '0185', '0', '19930101', ''),
('0186', '0186', '0', '20000318', ''),
('0187', '0187', '0', '19981221', ''),
('0188', '0188', '0', '20010205', ''),
('0189', '0189', '0', '20000804', ''),
('0190', '0190', '0', '20000428', ''),
('0192', '0192', '0', '20000422', ''),
('0194', '0194', '0', '20011024', ''),
('0195', '0195', '0', '20000824', ''),
('0196', '0196', '0', '20000301', ''),
('0200', '0200', '0', '20010326', ''),
('0201', '0201', '0', '20010808', ''),
('0202', '0202', '0', '20011016', ''),
('0203', '0203', '0', '20020404', ''),
('0205', '0205', '0', '20011025', ''),
('0206', '0206', '0', '20010716', ''),
('0207', '0207', '0', '20011121', ''),
('0208', '0208', '0', '20011015', ''),
('0209', '0209', '0', '20010824', ''),
('0210', '0210', '0', '20020129', ''),
('0211', '0211', '0', '20011228', ''),
('0212', '0212', '0', '20011214', ''),
('0213', '0213', '0', '20020418', ''),
('0215', '0215', '0', '20020419', ''),
('0216', '0216', '0', '20020312', ''),
('0218', '0218', '0', '20020603', ''),
('0219', '0219', '0', '20020110', ''),
('0221', '0221', '0', '20030701', ''),
('0222', '0222', '0', '20030818', ''),
('0223', '0223', '0', '20030729', ''),
('0225', '0225', '0', '20031016', ''),
('0226', '0226', '0', '20030919', ''),
('0227', '0227', '0', '20031226', ''),
('0228', '0228', '0', '20040701', ''),
('0229', '0229', '0', '20040812', ''),
('0230', '0230', '0', '20040909', ''),
('0233', '0233', '0', '20050113', ''),
('0235', '0235', '0', '20050221', ''),
('0236', '0236', '0', '20051012', ''),
('0238', '0238', '0', '20061222', ''),
('0239', '0239', '0', '20070425', ''),
('0240', '0240', '0', '20071217', ''),
('0241', '0241', '0', '20080505', ''),
('0242', '0242', '0', '20080508', ''),
('0243', '0243', '0', '20080721', ''),
('0244', '0244', '0', '20081219', ''),
('0245', '0245', '0', '20090717', ''),
('0246', '0246', '0', '20100311', ''),
('0247', '0247', '0', '20100715', ''),
('0901', '0901', '0', '20100826', ''),
('0902', '0902', '0', '20100331', ''),
('0903', '0903', '0', '20110906', ''),
('0904', '0904', '0', '20100917', ''),
('0905', '0905', '0', '20110321', ''),
('0906', '0906', '0', '20110314', ''),
('0907', '0907', '0', '20110524', ''),
('0908', '0908', '0', '20110708', ''),
('0910', '0910', '0', '20130917', '')
184 records selected
>>

后面的工作

接下来就是要吧所有表都实现，当然，懒人一定不会手写代码的，生成代码脚本走起；脚本写完再回来～

$grep REGISTER MemDefs.h | awk -F ')' '{print $1 ");"}' > register.cpp
$grep class MemDefs.h.v0 | awk -F '\r' '{print $1 $2 ";"}' > class.cpp
$grep class MemDefs.h.v0 |awk -F '\r' '{print $1}'| awk '{print "void Insert" $2 "(" $2 " *fld);"}'> method.cpp
$sed -i 's/InsertCTt/Insert/g' method.cpp

数据文件最大两个G，大了就core。调试了一天，改了一大堆源码，设置各种选项，未果…………
注释里面很明显给出了参数，可以设置64位的支持。

/* define database offset bit size */
#define dbDatabaseOffsetBits 32

默认值是32，之前也尝试修改该参数，正常理解我改成过64，启动程序就core。这下晕了，继续看代码，里面给的例子是38，代码里面还是40的影子。更晕。
最后发现这个值得意义是2**32=2G，那理论上改大应该没问题，比如64，但是，看了代码之后发现他会先检查内存，因为fastdb认为所有数据都可以放在内存里面，如果设大了那么它会认为内存不过，所以最后设置为33，即4G。
* 这个问题算是解决了*

链接地址： http://blog.leanote.com/post/frank.x@aliyun.com/dat%E6%96%87%E4%BB%B6%E8%BD%ACfastdb