我们如何使用Unity的代码覆盖率？

Previously we have written about our Runtime Tests,  Performance Tests and Unity Test Tools. In this blog post I will tell you about how we use code coverage.

以前，我们已经编写了有关运行时测试 ， 性能测试和Unity测试工具的文章 。 在这篇博客中，我将向您介绍我们如何使用代码覆盖率。

First and foremost code coverage is an ambitious thing and it should be used with caution. It would be an error to state that it measures quality of the test suites. You can easily write a test suite that doesn’t contain a single assertion, but produces 100% coverage.  Code coverage gives us insight into the thoroughness of our test suites. In conjunction with other metrics this is extremely useful and valuable information.

首先也是最重要的一点是代码覆盖率很高，应该谨慎使用。 声明它衡量测试套件的质量将是一个错误。 您可以轻松地编写一个不包含单个断言而是产生100％覆盖率的测试套件。 代码覆盖率使我们能够深入了解测试套件的完整性。 结合其他指标，这是非常有用和有价值的信息。

Our first goal has been to see how much C++ code is exercised by our current test suites. We evaluated a lot of different C++ code coverage tools and eventually chose BullseyeCoverage. Reasons:

我们的第一个目标是查看当前测试套件可以执行多少C ++代码。 我们评估了许多不同的C ++代码覆盖率工具，最终选择了BullseyeCoverage。 原因：

• We love what it measures (see below).

  我们喜欢它可以测量的东西(见下文)。

• Easy to integrate into our build system

  易于集成到我们的构建系统中

• Wide platform support

  广泛的平台支持

• Licensing and pricing model that fits our needs

  符合我们需求的许可和定价模型

• Good support quality

  良好的支持质量

我们衡量什么 (What do we measure)

声明范围 (Statement coverage)

The most widely known code coverage metric is  probably statement coverage. We don’t use it, but a small example is still in place:

最广为人知的代码覆盖率度量标准可能是语句覆盖率。 我们不使用它，但是仍然有一个小例子：

int* p = NULL;

int * p = NULL;

if (condition)  {

如果(条件){

      p = &variable;

p =＆variable;

}

}

*p = 123;

* p = 123;

 If “condition” is true we have 100% statement coverage meaning that each statement will be executed.  But, is 100 % statement coverage really enough to test this code?  How DO we thoroughly test it? The obvious answer is that we need at least two test cases, one for when the condition is true, another for when it is false.

如果“ condition”为true，则我们具有100％的语句覆盖率，这意味着将执行每个语句。 但是，100％的语句覆盖率真的足以测试此代码吗？ 我们如何彻底测试它？ 显而易见的答案是，我们至少需要两个测试用例，一个用于条件为真时，另一个用于条件为假时。

决策范围 (Decision coverage)

If the condition in the example above is evaluated to both ‘true’ and ‘false’ we tell that  we have 100% decision coverage.  Boolean expressions can, however, be more complex than a single variable. Decision coverage does not take this into consideration. This leads us to condition coverage.

如果上面示例中的条件被评估为“真”和“假”，则表明我们拥有100％的决策覆盖率。 但是，布尔表达式可能比单个变量更复杂。 决策范围未考虑到这一点。 这导致我们需要条件覆盖。

条件覆盖 (Condition coverage)

Consider the following boolean expression:

考虑以下布尔表达式：

If (c1 && c2 && c3) {

如果(c1 && c2 && c3){

      statement 1;

陈述1；

} else {

}其他{

      statement 2;

陈述2;

}

}

This metric yields 100% condition coverage if all 2^3 combinations of c1, c2 and c3 have been evaluated to true and false. c1 , c2, c3 are conditions, while (c1 && c2 && c3) is the decision.

如果将c1，c2和c3的所有2 ^ 3组合都评估为true和false，则此度量标准将产生100％的条件覆盖率。 c1，c2，c3是条件，而(c1 && c2 && c3)是决策。

However, it’s important to note that 100% condition does not imply 100% decision coverage, and vice versa, as the following example shows:

但是，请务必注意，100％条件并不意味着100％决策覆盖，反之亦然，如以下示例所示：

If (c1 && c2) {

如果(c1 && c2){

}

}

In this case setting c1=true, c2=false and c1=false, c2=true will satisfy condition coverage, but not decision coverage.

在这种情况下，设置c1 = true，c2 = false和c1 = false，c2 = true将满足条件覆盖范围，但不满足决策覆盖范围。

条件/决定范围 (Condition/Decision Coverage)

BullseyeCoverage measures a combination of condition and decision coverage (abbreviated C/D coverage in the following sections).

BullseyeCoverage衡量条件和决策范围的组合(以下几节中的C / D范围的缩写)。

If (c1 && c2 && c2) {

如果(c1 && c2 && c2){

      statement 1;

陈述1；

} else {

}其他{

      statement 2;

陈述2;

}

}

For a given segment of code 100% coverage is reported when:

对于给定的代码段，在以下情况下报告为100％的覆盖率：

• Each condition c1, c2, c3 was evaluated to true and false.

  每个条件c1，c2，c3都被评估为是和否。

• Decision (c1 && c2 && c3) was evaluated to true or false.

  决策(c1 && c2 && c3)被评估为是或否。

A more detailed explanation can be found here: Code Coverage Analysis

可以在这里找到更详细的解释： 代码覆盖率分析

功能范围 (Function Coverage)

We are also measuring function coverage which simply tells if each function has been invoked.

我们还在测量功能覆盖率，该覆盖率只是告诉每个功能是否已被调用。

我们如何使用它 (How do we use it)

跟踪一段时间内的覆盖率 (Track coverage over time)

Having C/D and function coverage setup for C++, we track how they are changing over time, for each test suite and for all test suites combined. An example is shown below:

有了C ++的C / D和功能覆盖设置，我们就可以跟踪每个测试套件和所有组合测试套件的变化情况。 一个例子如下所示：

On X-axis revision dates, on Y-axis – percentage.

在X轴修订日期上，在Y轴上-百分比。

 

We will notice ups and downs in coverage immediately. The reasons why it can go down are:

我们会立即注意到覆盖范围的起伏。 它可以下降的原因是：

• New code without tests has been added

  添加了没有测试的新代码

• Test suites have been changed (test cases moved to other suites, deleted, etc.)

  测试套件已更改(测试用例已移至其他套件，已删除等)

识别丢失的保险 (To identify missing coverage)

When we have the coverage data it is very easy to see which parts of the Unity codebase are covered with tests:

当我们获得覆盖率数据时，很容易看到测试覆盖了Unity代码库的哪些部分：

This report gives nice coverage overview per area. It is available for each suite and for summary aggregated report, which we call ‘overall’ suite.

该报告可以很好地涵盖每个区域的概况。 它适用于每个套件和汇总汇总报告，我们称之为“整体”套件。

在不失去覆盖范围的情况下优化测试套件 (To optimize test suites without losing coverage)

We have various test suites:

我们有各种测试套件：

• Native

  本机

• Integration

  积分

• Graphics

  图形

• Runtime

  运行

• others

  其他

One of our goals moving forward is to build a minimal and fast subset of tests from each suite preserving high coverage. For native tests this will be trivial (they are practically unit tests), but for other suites we need to be smart about it and here code coverage is an extremely valuable input, as it gives us insight into exactly which code paths a test is exercising – and what we lose by cutting it.

我们前进的目标之一是从每个套件中构建最小且快速的测试子集，以保持较高的覆盖率。 对于本机测试，这是微不足道的(实际上是单元测试)，但是对于其他套件，我们需要对此保持精明，这里的代码覆盖率是非常有价值的输入，因为它可以让我们深入了解测试正在执行的代码路径–以及削减它所造成的损失。

将慢速集成测试转换为快速本机测试时 (When converting slow integration tests to fast native tests)

When converting selected high level tests to native C++ unit test, we can rely on coverage data to ensure that we exercise the same code before and after conversion.

将选定的高级测试转换为本地C ++单元测试时，我们可以依靠coverage数据来确保在转换前后执行相同的代码。

改善测试和代码审查 (To improve test- and code reviews)

Developers and testers can use coverage data as one of the inputs when reviewing code and tests. It may give some unexpected insight. Our code coverage solution can be used when doing manual testing as well, to see exactly what parts of the codebase have been exercised.

在检查代码和测试时，开发人员和测试人员可以将覆盖率数据用作输入之一。 它可能会提供一些意想不到的见解。 在进行手动测试时，也可以使用我们的代码覆盖解决方案，以准确查看代码库的哪些部分已被执行。

未来步骤 (Future steps)

Moving ahead, an interesting challenge for us is to build a correspondence between test and code exercised by test. This will give us exciting possibilities:

前进，对我们来说，一个有趣的挑战是在测试和测试所执行的代码之间建立对应关系。 这将为我们提供令人兴奋的可能性：

• Find tests which exercise the same code. Such tests are definitely subjects for review and possibly for elimination

  查找执行相同代码的测试。 此类测试绝对是需要复查和可能消除的主题

• Incremental coverage – how much coverage newly added code has

  增量覆盖率–新添加的代码有多少覆盖率

• Analyze changes and run only tests which are related to these changes.

  分析更改并仅运行与这些更改相关的测试。

• Track coverage for managed code.

  跟踪托管代码的覆盖范围。

We’re sure that having a solid code coverage solution in place will give us some very interesting insights and possibilities. We will share our findings as we move along.

我们确信拥有一个可靠的代码覆盖解决方案将为我们提供一些非常有趣的见解和可能性。 我们将在前进的过程中分享我们的发现。

翻译自: https://blogs.unity3d.com/2014/02/27/how-do-we-use-code-coverage-for-unity/