(e.g. due to more extensive functionality) and the application scenarios (e.g. with regard to the number of systems and users involved) that are controlled in use cases make it increasingly difficult and costly to protect these systems against security-relevant weaknesses in implementation. At the same time, the greater integration of development and operation of systems, known as DevOps, and shorter release cycles reduce the time available for adequate security tests. A continuous automation of security tests from the identification of test objectives and the derivation of test cases to test evaluation is therefore necessary.
Since no complex, networked system is completely secure, vulnerabilities and attacks that exploit them are unavoidable. The project therefore develops an automated procedure to derive a test case from a successful attack against an IoT application, which represents the actual core of a successful attack. This test case can support the creation of a patch against the security hole. Techniques of sensor-based attack detection using data analysis techniques and techniques for test case reduction such as delta debugging will be used.
One technique often used for such purposes, the measurement of code coverage is based on instrumentation and makes the recompilation of source code necessary. This is not always possible, for example, if a component is only available in compiled form. In addition, code coverage as the sole criterion cannot satisfy the needs of security testing with regard to the detection of vulnerabilities. Therefore the sensors in the project are not introduced into the IoT application to be tested, but observe the input and output data streams of the microservices that make up the IoT application. In this way, the behavior of the IoT application is comprehensively recorded. The collected data is analyzed, clustered and evaluated using modern classification methods such as machine learning. The process thus has a broad technological basis and leads to a universally applicable test evaluation function. This sensor-based approach reduces the effort that would otherwise be required by weak-point-specific detection mechanisms, and any necessary instrumentation and recompilation of the code is eliminated. In addition, this increases the detection rate of the system, as the number of false-positive and false-negative findings is reduced, thus increasing the number of true positive findings.
Even if a completely secure system is not technically feasible or not feasible in a meaningful way, the goal during development and maintenance of applications is to detect and close as many security gaps before delivery or before their active exploitation. Methods such as fuzzing use extensive knowledge of the communication protocol and security gaps to efficiently uncover weaknesses. In addition, fuzzing procedures usually work in such a way that they attempt to cover the entire input data space achievable with fuzzing heuristics. With the help of protocol models this can be limited. Nevertheless, fuzzing remains a test technique that requires the execution of a very large number of test cases. In addition, the information generated by the test object at runtime is usually hardly ever used to generate further test cases. Typically, only code coverage metrics are used. Search-based methods use a quality function that converts the algorithm to one or more (global or local) optimal solutions efficiently. This means that you can efficiently find a solution even without specific knowledge about a system or weaknesses. Genetic methods in particular imitate natural evolution with operators such as selection, mutation and recombination using a quality function called fitness function, and develop a solution step by step over several iterations, so-called generations. In combination with fuzzing techniques for mutating test cases, a considerable reduction of the search space can be achieved, since the fitness function considerably limits the search space and thus the application of fuzzing heuristics. Since the fitness function is the test evaluation, which evaluates the system behavior as reactions to a security test case, the search-based fuzzing is achieved without the need for further modifications to the IoT application or its source code. Usually an attack-based test case is used as a starting point. Taking into account the coupling effect, similar stored vulnerabilities can be found. Search-based techniques such as restarting (e.g. starting from functional test cases) and larger amplitudes of mutation allow the search-based method to detect vulnerabilities in other parts of an IoT application. Functional test cases can also serve as a starting point if no attacks have been recorded yet.