Using Intelligent Agent for Automated Testing of Computer Games

Abstract

Given the increasing complexity and the rising popularity of games, the importance of testing increases as well. Currently, a lot of the testing is done by alpha testers because of their flexibility in handling changes in level designs and their ability to find bugs in a lot of different types of games. Using automation to perform part of this testing could reduce development costs and increase the speed at which bugs are discovered. Agent-based testing approaches offer flexility to deal quickly with changes in the environment and the possibility to define general strategies to reach a goal in a random environment.

During this research, the iv4XR framework is used to implement an agent-based tester to automatically play and test NetHack, a procedurally generated game. The effectiveness of this agent is assessed using code coverage, mutation testing, and monitoring of properties defined by LTL formulas.

The agent achieves low code coverage (14.2%) and a mutation score of (56.8%). Monitoring properties using LTL does show invalid state transitions within the game can be detected automatically. Improvements to the iv4XR framework are found for future use of the framework. Some of these improvements have been implemented.

This research concludes effective play testing using the agent has not been achieved. To increase effectivity of the agent, an improved strategy must be defined. Using LTL formulas to test properties in the game could greatly assist alpha testers by automatically monitoring a play through. Their potential will need to be investigated in future research.