Testing software has little in the way of an underpinning theory. Over time this has lead to ad hoc approaches. You have a problem; you think of a solution; you do some experiments to see how well it works. You have another problem, you find another solution.
Shannon's Information Theory is a simple concept that has proven itself to have wide and disparate application over many different fields: in communications, sociology, physics, biology, financial markets, machine learning, big data, statistics etc. etc. Is it possible that it might have some useful applications in software engineering?
A software program can be thought of as a collection of communication channels. At the macroscopic level the entire program can be seen as an input output channel, irrespective of its syntax and semantics. Finer grained communication channels include every function, class, path, assignment, and expression in the program. I argue that this view of a program provides us with a very powerful and flexible conceptual toolbox with which to solve problems and to explain solutions.
As examples, I first examine the work of Alshahwan and Harman on Output Diversity for test suite coverage and show that information theory explains their outstandingly good results. Then I use my own work on testability to show how information theory can offer solutions to coincidental correctness of test inputs.
Dr Clark is a Senior Lecturer in the Department of Computer Science at UCL and a member of Centre for Research into Evolution Search and Testing (CREST), the Software Systems Engineering Research Group (SSE), the Information Security Research Group (InfoSec), The UCL Academic Centre of Excellence in Cyber Security (SCR-CS), and the GCHQ Research Institute for Program Analysis and Verification for Cyber Security (RIPAV-CS).
His research interests are in applications of information theory and other statistical methods to problems in software security and software testing via formal methods and program analysis.
In my talk I will discuss the use of planning as means for generating test suites. Planning in Artificial Intelligence is considered as method for generating a sequence of actions to be applied in order to move from the given initial state to the wanted goal state. Such a plan, which is the sequence of actions, can be easily interpreted as a test case to be applied to check the system’s behavior. The question immediately arising is regarding the origin of actions in case of software or system testing. Here we discuss two solutions. One solution for object-oriented languages maps methods to actions but requires pre and post conditions to be specified. The other solution to be developed for security testing captures the behavior of an attacker. Actions usually used by an attacker to penetrate the system under test are formalized in a way such that a planner can generate potentially vulnerable sequences of actions. Besides the foundation of planning, the introduction of the two different applications of planning for testing, I also discuss previous research and future challenges.
Franz Wotawa received a M.Sc. in Computer Science (1994) and a PhD in 1996 both from the Vienna University of Technology. He is currently professor of software engineering at the Graz University of Technology and Dean of the Computer Science Faculty. Since the founding of the Institute for Software Technology in 2003 to the year 2009 Franz Wotawa had been the head of the institute. His research interests include model-based and qualitative reasoning, theorem proving, mobile robots, verification and validation, and software testing and debugging. Beside theoretical foundations he has always been interested in closing the gap between research and practice. For this purposes he founded Softnet Austria in 2006, which is a non-profit organization carrying out applied research projects together with companies.
During his career Franz Wotawa has written more than 280 papers for journals, books, conferences, and workshops. He supervised 64 master and 27 PhD students. Franz Wotawa has been member of a various number of program committees and organized several workshops and special issues of journals. He is a member of the Academia Europaea, the IEEE Computer Society, ACM the Austrian Computer Society (OCG), the Austrian Society for Artificial Intelligence, and Senior Member of the AAAI.