AN EXPERIMENTAL STACK ATTACKS DETECTION AND RECOVERY FRAMEWORK USING AGENTS, CHECKPOINTS AND ROLLBACK

Abstract

Stack based attacks are on the increase. This work generally studied stack-based vulnerabilities and attacks and focused on attacks which   employ the modification of return addresses used by control stacks. A control stack keeps track of the point in which a function returns control to after its execution. We proposed a framework that mitigates control stack attacks which utilizes kernel-controlled agent, checkpoints and rollback mechanisms. In the framework once a function is called the same return address (RA) is pushed to the control stack and also passed to the kernel-controlled agent. When a function call terminates the RA in the control stack is popped and passed to the kernel protected agent for comparison and if there is any disparity in the values of the RAs then there is an attack. In such cases the kernel protected agent directs execution of the process to stack at the latest checkpoint. The framework was implemented using Java Netbeans 7.2.1. Experimental results of the framework indicated successful detection of attacks and rollbacks in case of the attacks. Rollback indicated recovery from the attacks.