How can I measure thread stack depth?

You can get an idea of the stack depth with something like an aspect that can be woven to your code (load time weaver to allow advising all loaded code except system class loader). The aspect would work around all executed code and would be able to note when you are calling a method and when you return. You can use this to capture most of your stack usage (you'll miss anything loaded from the system class loader, e.g. java.*). While not perfect, it avoids having to change your code to gather StackTraceElement[] at sample points and also gets you into non-jdk code that you might not have written.

For example (aspectj):

public aspect CallStackAdvice {   pointcut allMethods() : execution(* *(..)) && !within(CallStackLog);   Object around(): allMethods(){       String called = thisJoinPoint.getSignature ().toLongString ();       CallStackLog.calling ( called );       try {           return proceed();       } finally {           CallStackLog.exiting ( called );       }   }}public class CallStackLog {    private CallStackLog () {}    private static ThreadLocal<ArrayDeque<String>> curStack =         new ThreadLocal<ArrayDeque<String>> () {        @Override        protected ArrayDeque<String> initialValue () {            return new ArrayDeque<String> ();        }    };    private static ThreadLocal<Boolean> ascending =         new ThreadLocal<Boolean> () {        @Override        protected Boolean initialValue () {            return true;        }    };    private static ConcurrentHashMap<Integer, ArrayDeque<String>> stacks =          new ConcurrentHashMap<Integer, ArrayDeque<String>> ();    public static void calling ( String signature ) {        ascending.set ( true );        curStack.get ().push ( signature.intern () );    }    public static void exiting ( String signature ) {        ArrayDeque<String> cur = curStack.get ();        if ( ascending.get () ) {            ArrayDeque<String> clon = cur.clone ();            stacks.put ( hash ( clon ), clon );        }        cur.pop ();        ascending.set ( false );    }    public static Integer hash ( ArrayDeque<String> a ) {        //simplistic and wrong but ok for example        int h = 0;        for ( String s : a ) {            h += ( 31 * s.hashCode () );        }        return h;    }    public static void dumpStacks(){        //implement something to print or retrieve or use stacks    }}

And a sample stack might be like:

net.sourceforge.jtds.jdbc.TdsCore net.sourceforge.jtds.jdbc.JtdsStatement.getTds()public boolean net.sourceforge.jtds.jdbc.JtdsResultSet.next()public void net.sourceforge.jtds.jdbc.JtdsResultSet.close()public java.sql.Connection net.sourceforge.jtds.jdbc.Driver.connect(java.lang.String, java.util.Properties)public void phil.RandomStackGen.MyRunnable.run()

Very slow and has its own memory issues but can be workable to get you the stack information you need.

You can then use the max_stack and max_locals for each method in your stack traces to compute a frame size (see class file format) for the method. Based on the vm spec I believe this should be (max_stack+max_locals)*4bytes for the max frame size for a method (long/double occupy two entries on the operand stack/local vars and is accounted for in max_stack and max_locals).

You can easily javap the classes of interest and see the frame values if you don't have that much in your call stacks. And something like asm provides you with some easy tools to use to do this on a larger scale.

Once you have this computed, you need to estimate additional stack frames for JDK classes that might be called by you at your max stack points and add that to your stack sizes. It wont be perfect but it ought to get you a decent starting point for -Xss tuning without hacking around the JVM/JDK.

One other note: I don't know what JIT/OSR does to frame sizes or stack requirements so do be aware that you may have different impacts from -Xss tuning on a cold vs. warm JVM.

EDIT had a few hours of down time and threw together another approach. This is a java agent that will instrument methods to keep track of a max stack frame size and stack depth. This will be able to instrument most of the jdk classes along with your other code and libraries, giving you better results than the aspect weaver. You need asm v4 for this to work. It was more for the fun of it so file this under plinking java for fun, not profit.

First, make something to track the stack frame size and depth:

package phil.agent;public class MaxStackLog {    private static ThreadLocal<Integer> curStackSize =         new ThreadLocal<Integer> () {        @Override        protected Integer initialValue () {            return 0;        }    };    private static ThreadLocal<Integer> curStackDepth =         new ThreadLocal<Integer> () {        @Override        protected Integer initialValue () {            return 0;        }    };    private static ThreadLocal<Boolean> ascending =         new ThreadLocal<Boolean> () {        @Override        protected Boolean initialValue () {            return true;        }    };    private static ConcurrentHashMap<Long, Integer> maxSizes =         new ConcurrentHashMap<Long, Integer> ();    private static ConcurrentHashMap<Long, Integer> maxDepth =         new ConcurrentHashMap<Long, Integer> ();    private MaxStackLog () { }    public static void enter ( int frameSize ) {        ascending.set ( true );        curStackSize.set ( curStackSize.get () + frameSize );        curStackDepth.set ( curStackDepth.get () + 1 );    }    public static void exit ( int frameSize ) {        int cur = curStackSize.get ();        int curDepth = curStackDepth.get ();        if ( ascending.get () ) {            long id = Thread.currentThread ().getId ();            Integer max = maxSizes.get ( id );            if ( max == null || cur > max ) {                maxSizes.put ( id, cur );            }            max = maxDepth.get ( id );            if ( max == null || curDepth > max ) {                maxDepth.put ( id, curDepth );            }        }        ascending.set ( false );        curStackSize.set ( cur - frameSize );        curStackDepth.set ( curDepth - 1 );    }    public static void dumpMax () {        int max = 0;        for ( int i : maxSizes.values () ) {            max = Math.max ( i, max );        }        System.out.println ( "Max stack frame size accummulated: " + max );        max = 0;        for ( int i : maxDepth.values () ) {            max = Math.max ( i, max );        }        System.out.println ( "Max stack depth: " + max );    }}

Next, make the java agent:

package phil.agent;public class Agent {    public static void premain ( String agentArguments, Instrumentation ins ) {        try {            ins.appendToBootstrapClassLoaderSearch (                 new JarFile (                     new File ( "path/to/Agent.jar" ) ) );        } catch ( IOException e ) {            e.printStackTrace ();        }        ins.addTransformer ( new Transformer (), true );        Class<?>[] classes = ins.getAllLoadedClasses ();        int len = classes.length;        for ( int i = 0; i < len; i++ ) {            Class<?> clazz = classes[i];            String name = clazz != null ? clazz.getCanonicalName () : null;            try {                if ( name != null && !clazz.isArray () && !clazz.isPrimitive ()                        && !clazz.isInterface ()                         && !name.equals ( "java.lang.Long" )                        && !name.equals ( "java.lang.Boolean" )                        && !name.equals ( "java.lang.Integer" )                        && !name.equals ( "java.lang.Double" )                         && !name.equals ( "java.lang.Float" )                        && !name.equals ( "java.lang.Number" )                         && !name.equals ( "java.lang.Class" )                        && !name.equals ( "java.lang.Byte" )                         && !name.equals ( "java.lang.Void" )                        && !name.equals ( "java.lang.Short" )                         && !name.equals ( "java.lang.System" )                        && !name.equals ( "java.lang.Runtime" )                        && !name.equals ( "java.lang.Compiler" )                        && !name.equals ( "java.lang.StackTraceElement" )                        && !name.startsWith ( "java.lang.ThreadLocal" )                        && !name.startsWith ( "sun." )                         && !name.startsWith ( "java.security." )                        && !name.startsWith ( "java.lang.ref." )                        && !name.startsWith ( "java.lang.ClassLoader" )                        && !name.startsWith ( "java.util.concurrent.atomic" )                        && !name.startsWith ( "java.util.concurrent.ConcurrentHashMap" )                        && !name.startsWith ( "java.util.concurrent.locks." )                        && !name.startsWith ( "phil.agent." ) ) {                    ins.retransformClasses ( clazz );                }            } catch ( Throwable e ) {                System.err.println ( "Cant modify: " + name );            }        }        Runtime.getRuntime ().addShutdownHook ( new Thread () {            @Override            public void run () {                MaxStackLog.dumpMax ();            }        } );    }}

The agent class has the premain hook for instrumentation. In that hook, it adds a class transformer that instruments in the stack frame size tracking. It also adds the agent to the boot class loader so that it can process jdk classes, too. To do that, we need to retransform anything that might be loaded already, like String.class. But, we have to exclude a variety of things that are used by the agent or the stack logging which lead to infinite loops or other problems (some of that was found by trial and error). Finally, the agent adds a shutdown hook to dump the results to stdout.

public class Transformer implements ClassFileTransformer {    @Override    public byte[] transform ( ClassLoader loader,         String className, Class<?> classBeingRedefined,            ProtectionDomain protectionDomain, byte[] classfileBuffer )            throws IllegalClassFormatException {        if ( className.startsWith ( "phil/agent" ) ) {            return classfileBuffer;        }        byte[] result = classfileBuffer;        ClassReader reader = new ClassReader ( classfileBuffer );        MaxStackClassVisitor maxCv = new MaxStackClassVisitor ( null );        reader.accept ( maxCv, ClassReader.SKIP_DEBUG );        ClassWriter writer = new ClassWriter ( ClassWriter.COMPUTE_FRAMES );        ClassVisitor visitor =             new CallStackClassVisitor ( writer, maxCv.frameMap, className );        reader.accept ( visitor, ClassReader.SKIP_DEBUG );        result = writer.toByteArray ();        return result;    }}

The transformer drives two separate transformations - one to figure out the max stack frame size for each method and one to instrument the method for recording. It might be doable in a single pass but I didn't want to use the ASM tree API or spend more time figuring it out.

public class MaxStackClassVisitor extends ClassVisitor {    Map<String, Integer> frameMap = new HashMap<String, Integer> ();    public MaxStackClassVisitor ( ClassVisitor v ) {        super ( Opcodes.ASM4, v );    }    @Override    public MethodVisitor visitMethod ( int access, String name,         String desc, String signature,            String[] exceptions ) {        return new MaxStackMethodVisitor (             super.visitMethod ( access, name, desc, signature, exceptions ),             this, ( access + name + desc + signature ) );    }}public class MaxStackMethodVisitor extends MethodVisitor {    final MaxStackClassVisitor cv;    final String name;    public MaxStackMethodVisitor ( MethodVisitor mv,         MaxStackClassVisitor cv, String name ) {        super ( Opcodes.ASM4, mv );        this.cv = cv;        this.name = name;    }    @Override    public void visitMaxs ( int maxStack, int maxLocals ) {        cv.frameMap.put ( name, ( maxStack + maxLocals ) * 4 );        super.visitMaxs ( maxStack, maxLocals );    }}

The MaxStack*Visitor classes handle figuring out the max stack frame size.

public class CallStackClassVisitor extends ClassVisitor {    final Map<String, Integer> frameSizes;    final String className;    public CallStackClassVisitor ( ClassVisitor v,         Map<String, Integer> frameSizes, String className ) {        super ( Opcodes.ASM4, v );        this.frameSizes = frameSizes;        this.className = className;    }    @Override    public MethodVisitor visitMethod ( int access, String name,         String desc, String signature, String[] exceptions ) {        MethodVisitor m = super.visitMethod ( access, name, desc,                              signature, exceptions );        return new CallStackMethodVisitor ( m,                  frameSizes.get ( access + name + desc + signature ) );    }}public class CallStackMethodVisitor extends MethodVisitor {    final int size;    public CallStackMethodVisitor ( MethodVisitor mv, int size ) {        super ( Opcodes.ASM4, mv );        this.size = size;    }    @Override    public void visitCode () {        visitIntInsn ( Opcodes.SIPUSH, size );        visitMethodInsn ( Opcodes.INVOKESTATIC, "phil/agent/MaxStackLog",                "enter", "(I)V" );        super.visitCode ();    }    @Override    public void visitInsn ( int inst ) {        switch ( inst ) {            case Opcodes.ARETURN:            case Opcodes.DRETURN:            case Opcodes.FRETURN:            case Opcodes.IRETURN:            case Opcodes.LRETURN:            case Opcodes.RETURN:            case Opcodes.ATHROW:                visitIntInsn ( Opcodes.SIPUSH, size );                visitMethodInsn ( Opcodes.INVOKESTATIC,                        "phil/agent/MaxStackLog", "exit", "(I)V" );                break;            default:                break;        }        super.visitInsn ( inst );    }}

The CallStack*Visitor classes handle instrumenting methods with code to call the stack frame logging.

And then you need a MANIFEST.MF for the Agent.jar:

Manifest-Version: 1.0Premain-Class: phil.agent.AgentBoot-Class-Path: asm-all-4.0.jarCan-Retransform-Classes: true

Finally, add the following to your java command line for the program you want to instrument:

-javaagent:path/to/Agent.jar

You will also need to have the asm-all-4.0.jar in the same directory as the Agent.jar (or change Boot-Class-Path in the manifest to reference the location).

A sample output might be:

Max stack frame size accummulated: 44140Max stack depth: 1004

This is all a bit crude but works for me to get going.

Note: the stack frame size isn't a total stack size (still don't really know how to get that one). In practice, there are a variety of overheads for the thread stack. I found that I usually needed between 2 and 3 times the reported stack max frame size as a -Xss value. Oh, and be sure to do the -Xss tuning without the agent loaded as it adds to your stack size requirements.

I would reduce the -Xss setting in a test environment until you see a problem. Then add some head room.

Reducing your heap size would give your application more space for thread stacks.

Just switching to a 64-bit OS could give your application more memory as most 32-bit OSes only allow about 1.5 GB for each application, however a 32-bit application on a 64-bit OS can use up to 3-3.5 GB depending on the OS.

There is no readily usable tooling in the Java VM to query the stack depth in bytes. But you can get there. Here are some pointers:

• Exceptions contain arrays of stack frames which gives you the methods which were called.

• For each method, you can find the Code attribute in the .class file. This attribute contains the frame size per method in the field max_stack.

So what you need is a tool that compiles a HashMap which contains method name + file name + line number as keys and the value max_stack as values. Create an Throwable, fetch the stack frames from it with getStackTrace() and then iterate over the StackTraceElements.

Note:

Each entry on the operand stack can hold a value of any Java virtual machine type, including a value of type long or type double.

So each stack entry is probably 64bits, so you need to multiply max_stack with 8 to get bytes.