Json logging (for example for oidng) is broken

[baszoetekouw]

The json logging does not work correctly. The logstash appender that is for example used in in roles/oidcng/templates/logback.xml.j2 looks like this

 <appender name="JSON_SYSLOG" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
   <destination>host.docker.internal:514</destination>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder">
        <customFields>{"app":"oidcng"}</customFields>
        <includeCallerData>true</includeCallerData>
        <fieldNames>
            <thread>[ignore]</thread>
            <version>[ignore]</version>
            <levelValue>[ignore]</levelValue>
       </fieldNames>
     </encoder>
  </appender>

This correctly format the json, but sends a Logstash-compatible message to a syslog port. In particular, it send pure json, whereas syslog instead expects an RFC3164 or RFC5452-compatible message. Such a syslog message contains a header followed by a message:

<PRIO> VERSION TIMESTAMP HOSTNAME APPNAME PROCID MSGID STRUC MESSAGE

for example:

<132> 1 20260507T12:00:00.000000+0200 docker1.dev.openconext.org oidcngjson - - - {"app":"oidng","more":"jsonfields"}

Using the current appender, rsyslog randoml interprets some of the json fields as its metadata fields cause for example the "servicename" to be {"@timestamp":.

This makes it impossible to match log messages and redirect them to the correct files.

[baszoetekouw]

Now clause suggests this solution:

 <!-- RFC 5424 syslog framing over TCP; JSON is the syslog MSG field.
      %syslogStart{LOCAL0} emits the <PRI> byte derived from level + facility.
      Note: %message is not JSON-escaped; avoid log messages containing literal " or \. -->
 <appender name="JSON_SYSLOG" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
   <destination>host.docker.internal:514</destination>
   <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
     <layout class="ch.qos.logback.classic.PatternLayout">
       <pattern>%syslogStart{LOCAL0}1 %d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX} {{ inventory_hostname }} oidcngjson - - - {"@timestamp":"%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX}","message":"%message","logger_name":"%logger","level":"%level","app":"oidcng","caller_class_name":"%class","caller_method_name":"%method","caller_file_name":"%file","caller_line_number":"%line"}%n</pattern>
     </layout>
   </encoder>
  </appender>

This basically constructs its own json sturcturem which seems undesirable...

[baszoetekouw]

Of je leest de documentatie: https://github.com/logfellow/logstash-logback-encoder/tree/main#prefixsuffixseparator

[baszoetekouw]

oplossing wordt dan:

 <appender name="JSON_SYSLOG" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
   <destination>host.docker.internal:514</destination>
   <encoder class="net.logstash.logback.encoder.LogstashEncoder">
     <!-- RFC 5424 syslog header as prefix; LogstashEncoder provides the JSON body.
          Trailing space after the syslog header uses %mdc{x} trick to prevent whitespace trimming. -->
     <prefix class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
       <layout class="ch.qos.logback.classic.PatternLayout">
         <pattern>%syslogStart{LOCAL0} 1 %d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX} {{ inventory_hostname }} oidcng - - - %mdc{_}</pattern>
       </layout>
     </prefix>
     <customFields>{"app":"oidcng"}</customFields>
     <includeCallerData>true</includeCallerData>
     <fieldNames>
       <thread>[ignore]</thread>
       <version>[ignore]</version>
       <levelValue>[ignore]</levelValue>
     </fieldNames>
   </encoder>
  </appender>