Home

Awesome

OptionsToString Incremental Source Generator

OptionToStringGenerator codecov

Problem: I have a configuration class for use with IOptions and I want to safely log out its values at runtime, masking any sensitive values.

Solution: Use an incremental source generator to generate an extension method to get a string with masked values for the properties.

The methods to mask the values can be used outside of the generated code, too. See below for details.

This package generates an OptionsToString extension method for a class. Using attributes you can control how the values are masked. You can use this to log out the values of your configuration at startup, or via a REST endpoint.

Quick Example

Edit the source of your configuration class and decorate it with attributes.

namespace Test;
using Seekatar.OptionToStringGenerator;

[OptionsToString]
internal class PropertySimple
{
    [OutputMask]
    public string Secret { get; set; } = "Secret";

    public int RetryLimit { get; set; } = 5;

    [OutputRegex(Regex = "User Id=([^;]+).*Password=([^;]+)")]
    public string ConnectionString { get; set; } = "Server=myServerAddress;Database=myDataBase;User Id=myUsername;Password=myPassword;";
}

// usage
_logger.LogInformation(new PropertySimple().OptionsToString());

Output:

Test.PropertySimple:
  Secret           : "******"
  RetryLimit       : 5
  ConnectionString : "Server=myServerAddress;Database=myDataBase;User Id=***;Password=***;"

Alternatively, if you don't have the code for PropertySimple this will produce the same output.

internal class PropertyConfig
{
    [OutputPropertyMask(nameof(IOptionsSimple.Secret))]
    [OutputPropertyRegex(nameof(IOptionsSimple.ConnectionString), Regex = "User Id=([^;]+).*Password=([^;]+)")]
    public PropertySimple? PropertySimple { get; set; }
}

// usage
_logger.LogInformation(new PropertyConfig().PropertySimple.OptionsToString());

Usage

  1. Add the OptionToStringGenerator NuGet package to your project.
  2. If you can update the class
    1. Decorate a class with the OptionsToString attribute.
    2. Optionally decorate properties with an Output* attribute to specify how you want them to be masked. If you don't decorate a property, its full text is dumped out.
  3. If you don't want to or can't update the class
    1. Add a property to your class of the Type you want to dump out.
    2. Decorate the property with multiple OutputProperty* attributes to control how the properties are masked.

Example of Editing a Class

Here's a larger sample class that uses all the different types of masking. Anything without an attribute has its value written out in the clear. The output follows.

namespace Test;
using Seekatar.OptionToStringGenerator;

[OptionsToString]
public class PublicOptions
{
    public class AClass
    {
        public string Name { get; set; } = "maybe this is secret";
        public override string ToString() => $"{nameof(AClass)}: {Name}";
    }

    public string PlainText { get; set; } = "hi mom";

    public char Why { get; set; } = 'Y';

    public int PlainInt { get; set; } = 42;

    public double PlainDouble { get; set; } = 3.141;

    public double PlainDecimal { get; set; } = 6.02;

    public DateTime PlainDateTime { get; set; } = new DateTime(2020, 1, 2, 3, 4, 5);

    public DateOnly PlainDatOnly { get; set; } = new DateOnly(2020, 1, 2);

    public TimeOnly PlainTimeOnly { get; set; } = new TimeOnly(12, 23, 2);

    public TimeSpan TimeSpan { get; set; } = new TimeSpan(1, 2, 3, 4, 5);

    public Guid UUID { get; set; } = Guid.Parse("6536b25c-3a45-48d8-8ea3-756e19f5bad1");

    public string? NullItem { get; set; }

    public AClass AnObject { get; set; } = new();

    [OutputRegex(Regex = @"AClass\:\s+(.*)")]
    public AClass AMaskedObject { get; set; } = new();

    [OutputMask]
    public string FullyMasked { get; set; } = "thisisasecret";

    [OutputMask(PrefixLen=3)]
    public string FirstThreeNotMasked { get; set; } = "abc1233435667";

    [OutputMask(SuffixLen=3)]
    public string LastThreeNotMasked { get; set; } = "abc1233435667";

    [OutputMask(PrefixLen = 3, SuffixLen=3)]
    public string FirstAndLastThreeNotMasked { get; set; } = "abc1233435667";

    [OutputMask(PrefixLen = 100)]
    public string NotMaskedSinceLongLength { get; set; } = "abc1233435667";

    [OutputLengthOnly]
    public string LengthOnly { get; set; } = "thisisasecretthatonlyshowsthelength";

    [OutputRegex(Regex="User Id=([^;]+).*Password=([^;]+)")]
    public string MaskUserAndPassword { get; set; } = "Server=server;Database=db;User Id=myUsername;Password=myPassword;";

    [OutputRegex(Regex="User Id=([^;]+).*Password=([^;]+)",IgnoreCase=true)]
    public string MaskUserAndPasswordIgnoreCase { get; set; } = "Server=server;Database=db;user Id=myUsername;Password=myPassword;";

    [OutputRegex(Regex = "User Id=([^;]+).*Password=([^;]+)")]
    public string RegexNotMatched { get; set; } = "Server=server;Database=db;user Id=myUsername;Password=myPassword;";

    public ConsoleColor Color { get; set; } = ConsoleColor.Red;

    [OutputIgnore]
    public string IgnoreMe { get; set; } = "abc1233435667";
}

// usage
var options = new PublicOptions();
_logger.LogInformation(options.OptionsToString());

The output has the class name (by default) followed by an indented list of all the properties' values masked as specified.

Test.PublicOptions:
  PlainText                     : "hi mom"
  Why                           : "Y"
  PlainInt                      : 42
  PlainDouble                   : 3.141
  PlainDecimal                  : 6.02
  PlainDateTime                 : 01/02/2020 03:04:05
  PlainDatOnly                  : 01/02/2020
  PlainTimeOnly                 : 12:23
  TimeSpan                      : 1.02:03:04.0050000
  UUID                          : 6536b25c-3a45-48d8-8ea3-756e19f5bad1
  NullItem                      : null
  AnObject                      : "AClass: maybe this is secret"
  AMaskedObject                 : "AClass: ***"
  FullyMasked                   : "*************"
  FirstThreeNotMasked           : "abc**********"
  LastThreeNotMasked            : "**********667"
  FirstAndLastThreeNotMasked    : "abc*******667"
  NotMaskedSinceLongLength      : "abc1233435667"
  LengthOnly                    : Len = 35
  MaskUserAndPassword           : "Server=server;Database=db;User Id=***;Password=***;"
  MaskUserAndPasswordIgnoreCase : "Server=server;Database=db;user Id=***;Password=***;"
  RegexNotMatched               : "***Regex no match***!"
  Color                         : Red

Example of Using a Property

Here's a similar example where you don't have the source for the class, or don't want to change it. In this case, you use multiple OutputProperty* attributes, one for each property you want to mask.

This is from the tests where PropertyPublicClass is identical to PublicOptions, so the output will be the same aside from the class name.

namespace Test;
using Seekatar.OptionToStringGenerator;

public class PropertyTestOptions
{
    public MyClass(IOption<PropertyPublicClass> options, ILogger<PropertyTestOptions> logger)
    {
        _options =options.Value;
        logger.LogInformation(options.OptionsToString());
    }

    [OutputPropertyRegex(nameof(PropertyPublicClass.AMaskedObject), Regex = @"AClass\:\s+(.*)")]
    [OutputPropertyMask(nameof(PropertyPublicClass.FullyMasked))]
    [OutputPropertyMask(nameof(PropertyPublicClass.FirstThreeNotMasked), PrefixLen = 3)]
    [OutputPropertyMask(nameof(PropertyPublicClass.LastThreeNotMasked), SuffixLen = 3)]
    [OutputPropertyMask(nameof(PropertyPublicClass.FirstAndLastThreeNotMasked), PrefixLen = 3, SuffixLen = 3)]
    [OutputPropertyMask(nameof(PropertyPublicClass.NotMaskedSinceLongLength), PrefixLen = 100)]
    [OutputPropertyLengthOnly(nameof(PropertyPublicClass.LengthOnly))]
    [OutputPropertyRegex(nameof(PropertyPublicClass.MaskUserAndPassword), Regex = "User Id=([^;]+).*Password=([^;]+)")]
    [OutputPropertyRegex(nameof(PropertyPublicClass.MaskUserAndPasswordIgnoreCase), Regex = "User Id=([^;]+).*Password=([^;]+)", IgnoreCase = true)]
    [OutputPropertyRegex(nameof(PropertyPublicClass.RegexNotMatched), Regex = "User Id=([^;]+).*Password=([^;]+)")]
    [OutputPropertyIgnore(nameof(PropertyPublicClass.IgnoreMe) )]
    public PropertyPublicClass? PublicClass { get; set; }
}

Notes

Formatting Options

There are properties on the OptionsToStringAttribute for classes and OutputPropertyFormat for properties to control how the output is generated.

NameDescriptionDefault
ExcludeParentsExclude parent class propertiesfalse
IndentThe indenting string" " (Two spaces)
JsonFormat the output as JSONfalse
SeparatorThe name-value separator":"
SortSort the propertiesfalse
TitleThe title to use for the output. See belowClass name

In addition to literal text, the Title parameter can include property names in braces. For example

Mask.NullLiteral can be used to use a different value for null. The default is null.

// for a class
[OptionsToString(Title = nameof(TitleOptions) + "_{StringProp}_{IntProp}")]
public class TitleOptions
{
    public int IntProp { get; set; } = 42;
    public string StringProp { get; set; } = "hi mom";
}

// for a property
internal class PropertyTestSimple
{
    [OutputPropertyFormat(Title = nameof(TitleOptions) + "_{StringProp}_{IntProp}")]
    public TitleOptions TitleOptions { get; set; } = new ();
}

Both will output

TitleOptions_hi mom_42:
  IntProp    : 42
  StringProp : "hi mom"

Per-Property Formatting Options

For types that take a format string to ToString() such as DateTime, numbers, etc., you can use the OutputFormatToString attribute. You can also supply a custom method to format a property. For example flattening an array and masking its values. The sample below shows a few examples:

    # comma separate thousands
    [OutputFormatToString("N0")]
    public int PlainInt { get; set; } = 423433;

    # two decimal places
    [OutputFormatToString("0.00")]
    public double PlainDouble { get; set; } = 3.141;

    # use the U format for DateTime
    [OutputFormatToString("R")]
    public DateTime PlainDateTime { get; set; } = new DateTime(2020, 1, 2, 3, 4, 5);

    [OutputFormatProvider(typeof(FormatOptions), nameof(MyFormatter))]
    public List<string> Secrets { get; set; } = new List<string> { "secret", "hushhush", "psssst" };

    # mask each string in the array showing only the first 3 characters
    public static string? MyFormatter(List<string> o)
    {
        if (o is null) return null;
        return string.Join(",", o.Select(s => Mask.MaskSuffix(s, 3)));
    }

Output:

  PlainInt      : 423,433
  PlainDouble   : 3.14
  PlainDateTime : Thu, 02 Jan 2020 03:04:05 GMT
  Secrets       : "sec***,hus*****,pss***"

IEnumerable and IDictionary Properties

If you have a property that is an IEnumerable of objects marked with the OptionsToString attribute, it will be formatted as a list of the objects. Similarly, if you have a property that is a IDictionary with values marked with the OptionsToString attribute, it will be formatted as a list of the key-value pairs.

For a class like this:

using Seekatar.OptionToStringGenerator;
using System.Collections.Generic;

namespace Test;

[OptionsToString]
public class ArrayAndDictionaryOfOptions
{
    [OptionsToString]
    public class NestedItem
    {
        public string Name { get; set; } = "";
    }

    public IEnumerable<NestedItem> List { get; set; } = new List<NestedItem>
            {
                new() { Name = "Name1" },
                new() { Name = "Name2" }
            };

    public IDictionary<string, NestedItem> Dictionary { get; set; } = new Dictionary<string, NestedItem>
        {
                { "A", new () { Name = "NameA" } },
                { "B", new() { Name = "NameB" } }
        };
}

The output will be:

Test.ArrayAndDictionaryOfOptions:
  List       :
    Count: 2
    Test.ArrayAndDictionaryOfOptions.NestedItem[0]:
      Name : "Name1"
    Test.ArrayAndDictionaryOfOptions.NestedItem[1]:
      Name : "Name2"

  Dictionary :
    Count: 2
    Test.ArrayAndDictionaryOfOptions.NestedItem["A"]:
      Name : "NameA"
    Test.ArrayAndDictionaryOfOptions.NestedItem["B"]:
      Name : "NameB"

Nested Classes

If you have an option class that contains another option class, do not set any attributes the property, and the OptionsToString will be called on the nested class, with extra indent of the parent class.

[OptionsToString] // use default indent of two spaces
class NestedOptions
{
    [OutputMask(PrefixLen = 3)]
    public string NestedSecret { get; set; } = "ChildSecret";
}

[OptionsToString]
class ParentOfNested
{
    [OutputMask(PrefixLen = 3)]
    public string Secret { get; set; } = "ParentSecret";

    public NestedOptions Nested { get; set; } = new();
}

Will produce this:

Test.ParentOfNested:
  Secret : "Par*********"
  Nested : Test.NestedOptions:
    NestedSecret : "Chi********" 👈 This line has 2x parent indent

Attributes

For a class use these attributes.

NameOnDescription
OptionsToStringClassMarker for the class, and has formatting options
OutputMaskMemberMask the value with asterisks, with optional prefix and suffix clear
OutputRegexMemberMask the value with a regex
OutputLengthOnlyMemberOnly output the length of the value
OutputIgnoreMemberIgnore the property
OutputFormatToStringMemberFormat the value using ToString() with a format string
OutputFormatProviderMemberFormat the value using a custom method

For a property, use these attributes on the property

NameDescription
OutputPropertyFormatOptional Formatting options
OutputPropertyMaskMask the value with asterisks, with optional prefix and suffix
OutputPropertyRegexMask the value with a regex
OutputPropertyLengthOnlyOnly output the length of the value
OutputPropertyIgnoreIgnore the property

Warnings and Errors

If attributes have invalid parameters you will get warnings or errors from the compiler. They are documented here.

Trouble Shooting

Error CS9057

You may get an error when compiling your code that uses this package.

##[error]#15 7.135 CSC : error CS9057: The analyzer assembly '/root/.nuget/packages/seekatar.optiontostringgenerator/0.1.4/analyzers/dotnet/cs/Seekatar.OptionToStringGenerator.dll' references version '4.6.0.0' of the compiler, which is newer than the currently running version '4.4.0.0'.

You must use the .NET SDK 6.0.416 or higher. You can check your version with dotnet --list-sdks.

Using Seekatar.Mask

The methods used by the generated code to mask a value are available when you include the source generator NuGet package. They are in the Seekatar.Mask namespace.

using static Seekatar.Mask;

...
MaskSuffix("abc123", 3) // returns "abc***"

Methods are as follows. Each of these corresponds to an attribute as described above. All take object? and return string?. Check each for parameters that control usage.

MethodDescription
MaskAllReturn a string of the same length as the input, with all characters masked
MaskLengthOnlyReturn Len <length>
MaskPrefixMask the prefix of the string, showing only a few suffix characters
MaskPrefixSuffixShow only a few prefix and suffix characters
MaskRegexMask capture groups of a regex
MaskSuffixMask the suffix of the string, showing only a few prefix characters

Implementation

Big shout out to Andrew Lock and his blog series on incremental source generators. I used that as a starting point for this project.

His blog tells his story of building a source generator and you learn better ways to do things as you progress through the blog.

In particular, in the last entry he breaks out the Attributes into their own assembly. In the initial generator, he injects the Attributes as code with these lines in the Initialize method of the generator, which is the typical method like this:

context.RegisterPostInitializationOutput(ctx => ctx.AddSource(
    "ClassExtensionsAttribute.g.cs",
    SourceText.From(SourceGenerationHelper.Attribute, Encoding.UTF8)));

He says this works fine unless someone uses InternalsVisibleTo to expose the internals of one assembly to another. He tried several things to solve this before coming up with a robust solution in part 8 of his series. There's quite a bit of advanced csproj editing that he covers to get it to work. I applied similar changes and everything but the unit tests worked. After viewing his repo, I found his original unit test helper methods to build the code on-the-fly for the unit tests was different. After picking up those changes, the unit tests worked.

Basic Logic of OptionsToStringGenerator.Initialize()

This has the implementation of IIncrementalGenerator.Initialize method. For this generator here's what I did:

  1. Look for classes with at least one attribute (predicate, which must be very fast)
  2. Look for ones with my OptionToStringAttribute (transform, which can be slower)
  3. Execute() generates the code
    1. Take the syntax and get the semantic model of the class, extracting the name, accessibility, and list of properties with a get
    2. Generate the code for the extension method

Branching Strategy

  1. Branch from main for new features
  2. Pushes will trigger a build and test run using GitHub Actions
  3. When ready, create a PR to main
  4. To push to the NuGet Gallery create a releases/vX.X.X branch and push to it.

Debugging and Testing

To debug the generator, the unit test project calls RunGeneratorsAndUpdateCompilation to run the generator and get the output. The unit test output will be the C# code for the extension method of the objects.

The integration test project runs the generator then calls the extension methods and gets the output from it.

In both cases, the output is written to files and the Verify package is used to compare the output to a snapshot file.

For integration tests, if you make changes to the generator, you often have to restart Visual Studio to get it to load the new one.

Links to Documentation

These are links to the MS documentation for the items I used in the generator.

ISymbol -- Base class for all semantic symbols

IPropertySymbol -- Semantic for the property

INamedTypeSymbol -- More specific semantic for the class

Links