API Logging in Json format

For request/response logging try Logbook and logstash-logback-encoder. You'll be able to use Markers to add structured contents.

For adding 'your own' fields in a JSON structure, I've written a code generator you might find interesting, it adds builder support: json-log-domain

You can use logback-contrib's JsonLayout inside any Logback appender. For example:

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">    <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">        <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">            <prettyPrint>false</prettyPrint>        </jsonFormatter>        <timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>        <appendLineSeparator>true</appendLineSeparator>    </layout></appender>

Given log statements like this ...

MDC.put("header1", "headerValue1");logger.info("hello!");logger.info("good bye!");

... the use of JsonLayout would result in Logback writing this:

{"timestamp":"2017-08-15 09:06:41.813","level":"INFO","thread":"main","mdc":{"header1":"headerValue1"},"logger":"com.stackoverflow.logback.LogbackTest","message":"hello!","context":"default"}{"timestamp":"2017-08-15 09:06:41.887","level":"INFO","thread":"main","mdc":{"header1":"headerValue1"},"logger":"com.stackoverflow.logback.LogbackTest","message":"good bye!","context":"default"}

I think this ticks the boxes of writing your log events as JSON documents whilst still retaining Logback's behaviour such as "control the logging level, time stamping log event, etc". There is some built-in support for changing the JSON format (e.g. you can include/exclude context, logger name etc) but the JsonLayout class provides an extension point which would allow you to change the attribute names in the resulting JSON by extending JsonLayout and overriding toJsonMap().

Edit 1: addressing your reply ("my question was more on how to achieve json subdocument/nesting as mentioned. MDC is limited to map of only") ... you could serialise your complex MDC values to JSON and add the serialised JSON representations to MDC. For example:

    Map<String, Object> complexMdcValue = new HashMap<>();    Map<String, Object> childMdcValue = new HashMap<>();    childMdcValue.put("name", "Joe");    childMdcValue.put("type", "Martian");    complexMdcValue.put("child", childMdcValue);    complexMdcValue.put("category", "etc");    MDC.put("complexNestedValue", objectMapper.writeValueAsString(complexMdcValue));    logger.info("hello!");

Would produce this output (in which the MVC logged key "complexNestedValue" is JSON containing a sub document):

{"timestamp":"2017-08-27 18:03:46.706","level":"INFO","thread":"main","mdc":{"complexNestedValue":"{\"category\":\"etc\",\"child\":{\"name\":\"Joe\",\"type\":\"Martian\"}}"},"logger":"com.stackoverflow.logback.LogbackTest","message":"hello!","context":"default"}