How to use MDC with thread pools? How to use MDC with thread pools? java java

How to use MDC with thread pools?


Yes, this is a common problem I've run into as well. There are a few workarounds (like manually setting it, as described), but ideally you want a solution that

  • Sets the MDC consistently;
  • Avoids tacit bugs where the MDC is incorrect but you don't know it; and
  • Minimizes changes to how you use thread pools (e.g. subclassing Callable with MyCallable everywhere, or similar ugliness).

Here's a solution that I use that meets these three needs. Code should be self-explanatory.

(As a side note, this executor can be created and fed to Guava's MoreExecutors.listeningDecorator(), ifyou use Guava's ListanableFuture.)

import org.slf4j.MDC;import java.util.Map;import java.util.concurrent.*;/** * A SLF4J MDC-compatible {@link ThreadPoolExecutor}. * <p/> * In general, MDC is used to store diagnostic information (e.g. a user's session id) in per-thread variables, to facilitate * logging. However, although MDC data is passed to thread children, this doesn't work when threads are reused in a * thread pool. This is a drop-in replacement for {@link ThreadPoolExecutor} sets MDC data before each task appropriately. * <p/> * Created by jlevy. * Date: 6/14/13 */public class MdcThreadPoolExecutor extends ThreadPoolExecutor {    final private boolean useFixedContext;    final private Map<String, Object> fixedContext;    /**     * Pool where task threads take MDC from the submitting thread.     */    public static MdcThreadPoolExecutor newWithInheritedMdc(int corePoolSize, int maximumPoolSize, long keepAliveTime,                                                            TimeUnit unit, BlockingQueue<Runnable> workQueue) {        return new MdcThreadPoolExecutor(null, corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);    }    /**     * Pool where task threads take fixed MDC from the thread that creates the pool.     */    @SuppressWarnings("unchecked")    public static MdcThreadPoolExecutor newWithCurrentMdc(int corePoolSize, int maximumPoolSize, long keepAliveTime,                                                          TimeUnit unit, BlockingQueue<Runnable> workQueue) {        return new MdcThreadPoolExecutor(MDC.getCopyOfContextMap(), corePoolSize, maximumPoolSize, keepAliveTime, unit,                workQueue);    }    /**     * Pool where task threads always have a specified, fixed MDC.     */    public static MdcThreadPoolExecutor newWithFixedMdc(Map<String, Object> fixedContext, int corePoolSize,                                                        int maximumPoolSize, long keepAliveTime, TimeUnit unit,                                                        BlockingQueue<Runnable> workQueue) {        return new MdcThreadPoolExecutor(fixedContext, corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);    }    private MdcThreadPoolExecutor(Map<String, Object> fixedContext, int corePoolSize, int maximumPoolSize,                                  long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);        this.fixedContext = fixedContext;        useFixedContext = (fixedContext != null);    }    @SuppressWarnings("unchecked")    private Map<String, Object> getContextForTask() {        return useFixedContext ? fixedContext : MDC.getCopyOfContextMap();    }    /**     * All executions will have MDC injected. {@code ThreadPoolExecutor}'s submission methods ({@code submit()} etc.)     * all delegate to this.     */    @Override    public void execute(Runnable command) {        super.execute(wrap(command, getContextForTask()));    }    public static Runnable wrap(final Runnable runnable, final Map<String, Object> context) {        return new Runnable() {            @Override            public void run() {                Map previous = MDC.getCopyOfContextMap();                if (context == null) {                    MDC.clear();                } else {                    MDC.setContextMap(context);                }                try {                    runnable.run();                } finally {                    if (previous == null) {                        MDC.clear();                    } else {                        MDC.setContextMap(previous);                    }                }            }        };    }}


We have run into a similar problem. You might want to extend ThreadPoolExecutor and override before/afterExecute methods to make the MDC calls you need before starting/stopping new threads.


IMHO the best solution is to:

  • use ThreadPoolTaskExecutor
  • implement your own TaskDecorator
  • use it: executor.setTaskDecorator(new LoggingTaskDecorator());

The decorator can look like this:

private final class LoggingTaskDecorator implements TaskDecorator {    @Override    public Runnable decorate(Runnable task) {        // web thread        Map<String, String> webThreadContext = MDC.getCopyOfContextMap();        return () -> {            // work thread            try {                // TODO: is this thread safe?                MDC.setContextMap(webThreadContext);                task.run();            } finally {                MDC.clear();            }        };    }}