MaskingJsonGeneratorDecorator masks only complete string

See original GitHub issue

When the configuration is like this:

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <jsonGeneratorDecorator class="net.logstash.logback.mask.MaskingJsonGeneratorDecorator">
                <defaultMask>****</defaultMask>
                <value>command</value>
            </jsonGeneratorDecorator>
        </encoder>
    </appender>

It would mask only the string which is exactly “command”. For example log {“message”:“command”} would be transformed into {“message”:“****”}.

But it doesn’t mask if word ‘command’ is part of the string. For example log {“message”:“Sending command bla”} would result in {“message”:“Sending command bla”} and I would expect {“message”:“Sending **** bla”}

I think it’s because while checking if value matches logs, matches() method is used on Matcher, but should be used find()

Could you please take a look into this.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:4
  • Comments:6

github_iconTop GitHub Comments

4reactions
philsttrcommented, Jun 25, 2021

After reviewing the current implementation, I believe my initial comment above was incorrect. The path matching support was intended to mask full values. The value matching support was intended to mask all matching substrings within a string field value.

I have changed the current implementation to mask all matching substrings, and clarified the documentation. I’ll call out this change in the release notes for the next version.

2reactions
michael-wirthcommented, Jun 21, 2021

Hi @bhavin9695

I didn’t find a way to define it in the logback-spring.xml.

I solved it by defining the sensitive pattern in an external file.

Here is my source code (I implemented it in Kotlin). Hope this helps.

class RegexFindValueMasker : ValueMasker {

    private val patterns: List<Regex>

    init {
        ClassPathResource(REGEX_PATTERN_FILE_LOCATION).run {
            patterns = if (isFile) {
                inputStream.reader().readLines().map(::Regex)
            } else listOf()
        }
    }

    override fun mask(context: JsonStreamContext, value: Any) =
        if (context.currentName == MESSAGE && value is String) {

            patterns.flatMap { it.findAll(value) }
                .map { it.groupValues[1.coerceAtMost(it.groupValues.size)] }
                .distinct()
                .fold(value) { newValue, matchedLabel -> newValue.replace(matchedLabel, MASK) }
        } else null

    companion object {
        private const val REGEX_PATTERN_FILE_LOCATION = "logstash/mask.patterns"
        private const val MESSAGE = "message"
        private const val MASK = "*****"
    }
}

logstash-spring.xml

<!-- mask values in the log message -->
<jsonGeneratorDecorator class="net.logstash.logback.mask.MaskingJsonGeneratorDecorator">
    <!-- custom value masker, replaces values matching the patterns in logstash/mask.patterns -->
    <valueMasker class="ch.migrosbank.eb.starter.web.logging.logstash.RegexFindValueMasker"/>
</jsonGeneratorDecorator>

logstash/mask.patterns

(?i)contractId=(.*)(?:,|\)|$)
Read more comments on GitHub >

github_iconTop Results From Across the Web

How can I combine PrettyPrintingJsonGeneratorDecorator ...
Anyhow when I combine these two together, it only does one thing, either pretty print or masked . <appender name="consoleAsJSON" class="ch.qos.
Read more >
Index (Logstash Logback Encoder 6.5 API) - Javadoc.io
Masks values of an absolute or partial path within a JSON stream. PathBasedFieldMasker(String, Object) - Constructor for class net.logstash.logback.mask.
Read more >
Mask sensitive data in logs - Dhaval Kolapkar - Medium
Let us take an example of masking email logs. At my company… ... public void setPatternsProperty(String patternsProperty) {
Read more >
Masking Sensitive Data with Logback - HowToDoInJava
Logback tutorial to create custom PatternLayout to mask the sensitive data and NPI information from logs using regex patterns in logback.xml.
Read more >
The 5 Best Reusable Face Masks of 2022 - The New York Times
The “best” cloth face mask is the one you'll wear (and not fuss with). Here's how to find a mask that fits, ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found