How to detect possible / potential stack overflow problems in a c / c++ program?

On Windows a stack overflow exception will be generated.

The following windows code illustrates this:

#include <stdio.h>#include <windows.h>void StackOverFlow(){  CONTEXT context;  // we are interested control registers  context.ContextFlags = CONTEXT_CONTROL;  // get the details  GetThreadContext(GetCurrentThread(), &context);  // print the stack pointer  printf("Esp: %X\n", context.Esp);  // this will eventually overflow the stack  StackOverFlow();}DWORD ExceptionFilter(EXCEPTION_POINTERS *pointers, DWORD dwException){  return EXCEPTION_EXECUTE_HANDLER;}void main(){  CONTEXT context;  // we are interested control registers  context.ContextFlags = CONTEXT_CONTROL;  // get the details  GetThreadContext(GetCurrentThread(), &context);  // print the stack pointer  printf("Esp: %X\n", context.Esp);  __try  {    // cause a stack overflow    StackOverFlow();  }  __except(ExceptionFilter(GetExceptionInformation(), GetExceptionCode()))  {    printf("\n****** ExceptionFilter fired ******\n");  }}

When this exe is run the following output is generated:

Esp: 12FC4CEsp: 12F96CEsp: 12F68C.....Esp: 33D8CEsp: 33AACEsp: 337CC****** ExceptionFilter fired ******

On Linux you get a segmentation fault if your code tries to write past the stack.

The size of the stack is a property inherited between processes. If you can read or modify it in the the shell using commands like ulimit -s (in sh, ksh, zsh) or limit stacksize (tcsh, zsh).

From a program, the size of the stack can be read using

#include <sys/resource.h>#include <stdio.h>int main() {    struct rlimit l;    getrlimit(RLIMIT_STACK, &l);    printf("stack_size = %ld\n", l.rlim_cur);    return 0;}

I don't know of a standard way to get the size of the available stack.

The stack starts with argc followed by the contents of argv and a copy of the environment, and then your variables. However because the kernel can randomize the location of the start of the stack, and there can be some dummy values above argc, it would be wrong to assume that you have l.rlim_cur bytes available below &argc.

One way to retrieve the exact location of the stack is to look at the file /proc/1234/maps (where 1234 is the process ID of your program). Once you know these bounds you can compute how much of your stack is used by looking at the address of the latest local variable.

gcc places an extra block of memory between the return address and the normal variables in "unsafe" function calls, like (in this example the function is void test() {char a[10]; b[20]}:

call stack:-----------return addressdummychar b[10]char a[20]

If the function write 36 bytes in the pointer 'a', the overflow will 'corrupt' the return address (possible security breach). But it will also change the value of the 'dummy', that is between the pointer and the return address, so the program will crash with a warning (you can disable this with a -fno-stack-protector)