The challenges are collecting precise coverage, optimizing fuzzing performance, and analyzing root causes.
将DBMS分为Library-level和enterprise-level
传统灰盒fuzz的不足(以Squirrel为例):
However, the performance of coverage-guided DBMS fuzzing on libraries does not match with that of enterpriselevel DBMSs. For example, Squirrel discovered 51 new bugs on SQLite, but it failed to discover any bugs on PostgreSQL.
三点原因:
First, the coverage collected by conventional fuzzers is low in quality.
Next, the SQL generated by existing fuzzers cannot be accepted by enterprise-level DBMSs.
Even if a fuzzer can trigger a bug, the root cause analysis is still a tough task.
Ratel针对这三点提供的解决方案:
- improve the feedback precision on large-scale distributed systems with inter-binary coverage linkage and bijective block mapping;
2)enhance SQL generation with robustness-oriented strategy;
3)investigate the root cause of anomalies with on-line analysis and deduplication.
成果:
We use RATEL to test GaussDB continuously and discovered 32 previously-unknown bugs in GaussDB. For proper evaluation, we further extended our evaluation with Bloomberg LP’s distributed DBMS Comdb2 and the original PostgreSQL where GaussDB derived. We also discovered 42 bugs of Comdb2 and 5 bugs of PostgreSQL from their codebases.
提升语句多样性的方案:
1、放松语法检查
传统的语句突变生成(Squirrel为例):在AST上解析和突变;将AST转化为SQL查询;重新解析查询以保证语法正确性。期间发生任何错误都导致生成失败,种子被丢弃。
放松语法检查:只要是interesting test case中的合法AST,即使生成的statement语法不通过也给予保留。
2、定制字典
生成能够涵盖DBMS所有特性的AST不太现实,但制作包含所有关键字的词典是可行的。突变过程中使用该词典,尽管会导致很多语法错误,但增加了语句多样性。