自动化测试知识体系（ABOK）

作为一名专业的自动化测试工程师，不应该仅仅局限于对工具的掌握和使用，应该建立测试的自动化知识体系（ ABOK ）： http://www.automatedtestinginstitute.com/home/index.php?option=com_content&view=category&id=69&Itemid=95

 

 

Automation Body of Knowledge (ABOK) 自动化知识体系

The Automation Body of Knowledge is a tool-independent skill set designed to help software test automation professionals address automation challenges that are present in the world of software testing. （ABOK是独立于工具的技巧集，设计的目的是帮助软件测试自动化工程师处理现实世界中的软件测试自动化的挑战。）Automated Software Testing is a discipline that is separate from Manual Software Testing, and must be treated as such.（软件自动化测试应该作为一个独立于手工测试的学科出现。） The ABOK acknowledges this, and provides engineers with a way to assess, improve, and better market their test automation skills more effectively than tool-specific benchmarks can do alone. This body of knowledge may also be used by organizations as specific criteria for more effectively assessing resources and establishing career development tracks. Not every automation engineer is required to be proficient in every one of the ABOK skill categories, but knowledge of the different skill categories is essential in the continued improvement, growth and development. It is recommended that each test automation effort, however, have a team that represents most, if not all of these skills, whether the team is composed of one or many resources.

The Skill categories are listed below, and may be clicked on to get a summary of each category. Skill categories 1 through 7 (macroscopic) include skill concentrations for automated test leads, while skills 8 through 12 (microscopic) include skill concentrations for automated test engineers. Given the state of many software projects, the Test Lead depends on the Automation Engineer for answers regarding automation planning, therefore the Automation Engineer may also want to consider a concentration in skills 1 through 7 as well.  

 

Skill Category 1: Automation's Role in the STLC 自动化在软件测试生命周期（ STLC ）中的角色

Being successful in test automation first requires an understanding of what test automation is, and where it fits into the overall testing lifecycle.  

Difference between software testing and software test automation 软件测试自动化与软件测试之间的区别

Software Testing is a discipline in which test logic is designed to be implemented by a human being. As opposed to, Software Test Automation designs script logic that allows that manual test logic to be implemented by a tool. An expansion of this concept asserts that software test automation involves tool support for all aspects of a test project, not just automation of test execution. With that being said, as a test automator, you should be aware of all phases of the testing lifecycle, along with tools that support those phases.  

Test Tool Acquisition and Integration 测试工具选购与整合

The Test Tool Acquisition is the process of identifying the tool goals and objectives, making an effective business case and narrowing down a group of candidate tools to the selection(s) that best meet(s) the organizational needs. Test Tool Integration is, conversely, the process of gradually addressing tool considerations and widening the tool’s acceptance, use and benefits.The various sources provide extensive directions on how to conduct the tool selection and acquisition process. Most sources will agree, however, that this process should at a minimum include the following:

• Making a well-informed decision to automate
• Conducting a test tool evaluation
• Addressing tool implementation considerations
• Piloting the tool implementation

Often, organizations are pushed into making a decision about the introduction or expansion of test automation based on the ability to buy a specific automated test tool. It’s important to expand your vision to be able to identify several options – both commercial and non-commercial – and to be knowledgeable on how to perform an effective evaluation of these options so that a more informed decision can be made about test automation implementation.   

Automation Benefits and Misconceptions 自动化的益处与误解

Behind every decision to introduce test automation is a wide variety of stated or implied goals and expectations for what the automation must accomplish. Without a proper understanding of reasonable test automation goals, the automation effort will not survive, and will be destined to sit on the proverbial shelf of your test organization. Understanding the impact of test automation means having a firm grasp on the benefits of test automation, accompanied by an understanding of the common misconceptions surrounding test automation. Being equipped with this knowledge helps a Test Automator more effectively implement a test automation effort, “sell” the automation to the powers-that-be by presenting and tracking a business case, and manage effort-ending misconceptions and unrealistic expectations held by the organization decision makers.  

Automation Return-on-Investment (ROI) 自动化的 ROI 计算

Software test automation is an investment in time and money, and like any investment, the stakeholders – normally managers or customers – are expecting a positive return from that investment. If their expectations are not sufficiently met, stakeholders will probably discontinue the investing in automation. The investment is justified by identifying the potential return-on-investment (ROI), a ratio of benefits to costs. There are several approaches for calculating ROI including:

• Simple ROI Calculation – A percentage calculated in terms of the monetary savings that test automation provides.
• Efficiency ROI Calculation – Examines test automation benefits in terms of time that is saved on certain portions of the testing effort.
• Risk Reduction ROI Calculation – A percentage calculated in terms of the increased quality that results from the increased test coverage provided test automation.

 

Skill Category 2: Test Automation Types and Interfaces 测试自动化的类型和接口

Test Automation Types 测试自动化的类型

For simplicity the different types of test automation – functional (regression), unit, integration, performance, etc. – are often lumped into one ‘test automation’ category. It’s important to understand some of the basic differences among these different types, even if you don’t specialize in all of them, so that you can speak intelligently about them when questioned by members of the organization. Since all automators are often “painted with the same brush”, being totally oblivious to basic concepts in each type of automation can negatively impact all automation efforts in the organization. In addition, being able to display minimal knowledge in various types of automation may provide management with enough confidence in you to allow you to pilot a new automation effort, which would help you to ultimately expand your skills.  

Test Automation Interfaces 测试自动化的接口

The main interfaces made available for application automation are:

• Command Line Interface 命令行接口
• Application Programming Interface 应用程序编程接口
• Graphical User Interface 图形用户界面

The Graphical User Interface is probably the most popular interface for automation implementation, due to the fact if more closely simulates how a user might use the tool. The Command Line Interface and Application Programming Interface, however, are advantages given the fact that they make it simpler to implement test automation.

 

Skill Category 3: Automation Tools 自动化工具

A test automation professional also must be prepared to provide insight into tools that support all aspects of the testing lifecycle. This requires an understanding of the different types and categories of tools that support the testing lifecycle, as well as the criteria for assessing these different types of tools.

These different types of tools include:

• Software Configuration Management Tools
• Business/System Modeling Tools
• Requirements Management Tools
• Unit Testing Tools
• Test Management Tools
• Defect Tracking Tools
• Code Coverage Analyzer Tools
• Functional System Test Automation Tools
• Performance System Test Automation Tools

 

Skill Category 4: Test Automation Frameworks 测试自动化框架

Automation Scope 自动化的范围

Total automation cost is composed of both development costs and maintenance costs.  （自动化的成本由开发成本和维护成本组成） As the automation framework becomes more defined, scripting increases in complexity. While this causes development costs to increase, it also causes maintenance costs to decrease. As the scope widens, maintenance becomes increasingly important. （随着测试范围的扩大，维护成本变得重要起来） Also, as maintenance becomes increasingly important, the automation framework must be more advanced in order to reduce total automation costs. Therefore, before making a determination of the framework that will be used for test automation, an evaluation must be conducted on the implied and/or stated scope of the organization’s automation effort.

 

Roles and Responsibilities 角色和职责

Each framework type will have one or more of the following roles for creation and implementation of the framework:

• Team Lead 测试主管
• Test Engineer 测试工程师
• Lead Automation Architect 自动化测试架构师
• Cross Coverage Coordinator 组织协调人员
• Automation Engineer (Automator) 自动化测试工程师

Very often, one person may hold multiple roles.

 

Framework Generations 框架的发展

Over the years, automated testing has evolved, becoming increasingly defined and sophisticated with each new evolutionary phase.  This evolution may be discussed in terms of three generations:

·         First Generation – This generation is what this book refers to as the Common Approach.  It is primarily driven by record and playback. 第一代 – 录制回放

• Second Generation – The second-generation learns from the problems of the first generation.  It involves creating a framework in which to automate tests. It also incorporates the concepts of functional decomposition, which simply means creating modular functions for executing fundamental actions that occur in the execution of the tests in a given test bed.  These actions may then be called upon and executed independent of one another.  This generation also more greatly separates test data from automation code, through greater use of parameterization. 第二代 – 功能分解

·         Third Generation – The third-generation of automation builds upon the concepts of the second generation.  It strongly depends on functional decomposition, but evaluates the functions at a much higher level of abstraction, so that code may be reused by tests within the same application, and by tests in different applications.  This is accomplished by separating test data and specific application functionality from automation code. 第三代 – 数据与代码分离

During the design phase a determination must be made about which generation will define your automated test framework.

 

Skill Category 5: Automation Framework Design 自动化框架设计

The process of designing an automated test framework is not an exact science. It is possible to definitively identify the different types of frameworks, but the process used to select and implement a particular framework is a little more difficult to pinpoint in such a way that most of the industry experts agree. The most important thing however, at this point in automation history is to ensure that a well thought out approach based on common, successfully implemented industry practices is used and honed within a given organization. This skill category identifies an approach, from a high enough level that it will fit with where the IT industry currently is relative to test automation, while still remaining low-level enough to be useful for implementation. This approach involves the following steps:

1. Selecting a Framework Type 选择框架类型
2. Identifying Framework Components 确定框架组成部分
3. Identifying Framework Directory Structure 确定框架目录结构
4. Developing Implementation Standards 建立实施标准
5. Develop Automated Tests 开发自动化测试

 

Skill Category 6: Automated Test Script Concepts 自动化测试脚本思想

Test Selection 测试用例选择

Choosing what and when to automate involves an analysis of the following:

• Automate items that are tedious for manual execution, but relatively easy to automate. 对于那些手工测试比较繁琐而又容易实现自动化的优先考虑自动化实现。
• Items that need to be executed over and over again. 对于那些需要重复又重复执行的测试用例实现自动化。
• Items that will increase coverage beyond what will realistically be done manually 对于那些能有效提高测试覆盖率的测试用例实现自动化。

In addition, the decision depends on the goals of the organization such as cost goals, efficiency goals, and risk mitigation goals.  

Automated Test Design and Development 自动化测试的设计和开发

Automated test design and development are tied directly to the interface in which the tests will be created, the way quality attributes are applied at the test level, the elements that are common to automated tests, and the standards used for creation of tests.  

Automated Test Execution, Analysis and Reporting 自动化测试的执行、分析和报告

The biggest items to address regarding test execution are:

• How many machines will be used for automated test execution
• Error handling
• How the results are reported and analyzed

 

Skill Category 7: Quality Attribute Optimization 质量优化

This category defines various quality attributes of the automated test suite and identifies ways of addressing these attributes based on priorities and constraints surrounding each.

The following are some Quality Attributes that may be considered.

• Maintainability
• Portability
• Flexibility
• Robustness
• Scalability
• Reliability
• Usability
• Performance

 

Skill Category 8: Programming Concepts 编程思想

Whether you use a tool with a scripting language, tree structure and/or keywords, fundamental programming concepts remain paramount in effectively automating tests, and increasing the distance the test can cover through increased system coverage and test flexibility. Concepts such as variables, control flow statements (if..then..else, for..next, etc.), and modularity are discussed in this category.

 

Skill Category 9: Automation Objects 自动化对象

Recognizing Application Objects 识别应用程序对象

Once upon a time, functional software test automation primarily used an “analog” approach that relied on specific coordinate locations of a screen or application window for performing mouse and keyboard operations that simulated user activities on the application. This was slightly unreliable, and difficult to maintain on a large scale, because when the screen, window or application components moved ever so slightly, the automated test would fail because it was trying to operate on an element in a position in which it no longer existed. Modern test automation conversely takes a different approach that locates the objects on the screen based on properties that define that object and then performs the desired operations once the object is found.  

Object Maps 对象映射

One of the simplest ways to boost the maintainability and robustness of an automated test suite is through the introduction of Object Maps. Object Maps reduce the amount of information that is being maintained in an automated test by storing object properties in an external file and referencing objects according to variable names (also known as Logical Names) that are then tied to the properties. 

Object Models 对象模型

Many software applications are composed of several interrelated objects, and an object model is an abstract representation of the hierarchical group of related objects that define an application and work together to complete a set of application functions. The advantages offered by object models include: 

·   Increased application understanding

·   Increased scripting power

Dynamic Object Behavior 动态对象行为

One of the biggest challenges faced by automated test engineers is that of dynamic object behavior. People process information about the application based on visual inspection, while automated tests use object properties. People can, therefore, easily make adjustments when there is a slight change from a visual standpoint, and can ignore many property changes. The computer, however, cannot adjust as easily. The necessary adjustments have to be anticipated up front and programmed.

 

Skill Category 10: Debugging Techniques 调试技巧

Types of Errors 错误类型

Regardless of how well automated tests are designed and created, problems will occur. Sometimes the problems are related to the scripts, and sometimes they are related to the application, but the root cause is not always simple to find. The inability to effectively debug scripts can severely delay schedules, and can even bring automation to a screeching halt.  The first step in script debugging is in understanding the types of errors that may be encountered. There are 4 main generic types of errors that may be encountered upon script execution:

• Syntax Errors 语法错误
• Run-time Errors 运行时错误
• Logic Errors 逻辑错误
• Application Errors 应用程序错误

Debugging Techniques 调试技巧

The trickiest part of debugging is finding out the true source of an error. This involves ensuring the error is reproducible, and then the process of finding the main source of the error must begin. Very often what seems to be the source of the error is actually just a symptom of the true error, so there are several techniques that may be employed to find the source of an error. At that point a determination may be made on whether or not the error is due to an application failure or a script issue. Then solutions for fixing the error may be addressed. Effective debugging typically mirrors the following process: 

• Identifying error existence 识别错误
• Reproducing the error 重现错误
• Localizing the error 定位错误
• Fixing the error 修正错误

 

Skill Category 11: Error Handling 错误处理

Error Handling Implementation 错误处理的实现

Error handling is implemented in a variety of ways within automated test scripts, most of which can be summarized in the following three categories:

• Step Implementation
• Component Implementation
• Run Implementation

Error Handling Development 错误处理脚本的开发

Error handling routines are critical for test automation, particularly for effectively resolving runtime errors and other unexpected system events, because excessive runtime errors are the enemy of test automation robustness. The figure below illustrates an error handling development process that may be used for implementing a successful error handling approach. These steps include:

• Diagnose Potential Errors 诊断出潜在的错误
• Define Error Capture Mechanism  定义错误捕获机制
• Create Error Log Data 建立出错日志
• Create Error Handler Routine 创建错误处理函数

 

 

 

Skill Category 12: Automated Test Reporting 自动化测试报告

Test reporting and analysis is an extremely repetitive process, yet can be extremely time consuming. This category addresses what types of reports could be generated and how to effectively produce these reports, so that time may be saved in the analysis and reporting. The categories of reports that may be generated by an automated test framework include:

·   High-level (Suites/Tests) reports   高层（测试集/测试脚本）报告

·   Low-level (Verification Points) reports  底层（验证点）报告