Diagnosing memory leaks caused by compile-time code

An IDE that uses hundreds of megabytes on a large solution is normal, and so is its memory rising while you work. The symptom that indicates a leak is memory that keeps rising for as long as you keep editing and never stabilizes, and that a garbage collection doesn't bring back down. If you observe that on a solution that uses Metalama, the cause may be in your own compile-time code. This article explains why that happens and how to find the field responsible.

Why compile-time code can retain memory

When you build from the command line, the compiler handles one snapshot of the project and exits, so nothing that compile-time code keeps in memory matters.

The IDE works differently. The Roslyn analysis process stays alive for as long as the solution is open, and it produces a new snapshot of the project on essentially every keystroke. It releases the previous one as soon as every component that received it does. A single retained snapshot keeps alive every syntax tree of the project, the full text of every file, and the symbol tables built from it: tens of megabytes on a medium project.

Metalama doesn't run your compile-time code on every keystroke. It runs it once and reuses the result:

• A fabric runs once per pipeline configuration, and everything it registers is reused for every later version of the project, until you change compile-time code.
• An inheritable aspect instance, a reference validator, and an annotation are filed under the path of the document they belong to, and are reused for every later version in which that document didn't change.

This is a deliberate design: recompiling your compile-time code between two keystrokes would be far too slow. The consequence is that any object of yours held by one of those results outlives the project snapshot it was created from. If one of its fields holds a declaration, such as an INamedType, that whole version of the project can never be released.

This is why memory never stabilizes. One retained snapshot costs what is described above; a field that keeps accumulating declarations retains one more snapshot for every edit, so the total grows for as long as the session lasts.

The three cases that cause this are:

• a field of a fabric, or a variable captured by a lambda passed to Select, Where, or AddAspect, that accumulates the declarations the query visits;
• a static field of a compile-time class used as a cache;
• a field of an inheritable aspect marked [NonCompileTimeSerialized] that holds a declaration.

Investigating the memory leak

Build the project once from the command line, passing the MetalamaDiagnoseMemoryLeaks property:

dotnet build MyProject.csproj -p:MetalamaDiagnoseMemoryLeaks=true

Metalama then inspects the objects that your compile-time code registered and reports every reference that keeps a project snapshot alive.

This is a one-time investigation, so pass the property on the command line rather than setting it in your project file. The analysis walks the whole object graph of everything your compile-time code registered, which takes time and memory, and it would slow down every build of every developer on the project.

Reading the report

Each retention in your own code is reported as warning LAMA0085, which names the type that holds the reference, the type of the object being held, and the chain of fields that leads from a long-lived object to it:

warning LAMA0085: The compile-time type 'MyFabric' holds a reference to a 'SourceNamedType',
which pins the Roslyn compilation. (...) The chain of references is:
fabric contributor #0 (AspectQuerySource<IDeclaration>) -> _query -> Owner -> _fabricInstance
-> Driver -> Fabric -> _seen -> _items -> [0].

Read the chain from left to right. It starts at an object that Metalama keeps for the lifetime of the project and ends at the object that can't be released. The part to act on is the segment inside your own types, here _seen, a List<INamedType> field of MyFabric.

A summary is reported as warning LAMA0086:

warning LAMA0086: The analysis of the references retained by compile-time code found
3 retention(s) in code written by the user and 25 retention(s) in Metalama itself. (...)
The full report is in '%TEMP%\Metalama\FabricRetentionReports\MyProject-net8.0.txt'.

Findings are split in two:

• Retentions in code written by you, which is what LAMA0085 reports and what you can fix.
• Retentions in Metalama itself, which are only counted. You can report these to us through GitHub, but they aren't something you can act on, and a non-zero count is normal.

The report file named by LAMA0086 contains both categories, with the full chain for each, formatted one field per line.

Durable and non-durable references

An IRef identifies the same declaration across project snapshots, which is why the API recommends it for passing declarations between aspects. That recommendation concerns a single pipeline run, and it comes with a distinction that matters here.

A reference is either durable or not, as reported by IsDurable:

• A durable reference, of type IDurableRef or IDurableRef<T>, stores only a string identifier and holds nothing else. It's safe in an object of any lifetime, and it's what Metalama itself stores in its own long-lived objects. Obtain one with ToDurableRef() on a declaration, or with ToDurable() on a reference you already have.
• Any other reference, such as one returned by ToRef, holds the symbol and the project snapshot behind it. It's correct and fast within a run, and it retains a snapshot as soon as you store it in something that outlives the run.

The diagnostic reports the second kind and never the first.

Fixing a retention

Do not store declaration objects

Never keep an IDeclaration, an INamedType, an IType, a SemanticModel, or a Roslyn ISymbol in anything that outlives a single pipeline run: a field of a fabric, a field of an inheritable aspect, a static field, or a variable captured by a lambda you register. Each of them is bound to the project snapshot it came from and keeps that snapshot alive.

The same applies to a plain IRef<T> obtained from ToRef. A reference resolves in any project snapshot, which makes it the right way to pass a declaration between aspects, but it holds the symbol and the snapshot behind it, so it isn't the right way to store one.

Store a durable reference instead

Store the durable reference, and resolve it later with GetTarget( compilation ) exactly as you would resolve any other reference. Resolving one costs an identifier lookup, which is why references aren't durable by default.

Declare the field as IDurableRef<T>, not IRef<T>. The conversion is then required by the field type, so a later edit can't omit it, and the constraint is stated by the signature rather than by a comment.

This is the retention LAMA0085 reports:

using Metalama.Framework.Code;
using Metalama.Framework.Fabrics;

namespace Doc.RetainedDeclaration_Wrong;

public class Fabric : ProjectFabric
{
    // WRONG. An INamedType belongs to the project snapshot it came from, and the fabric outlives every
    // snapshot, so this field pins one whole version of the project for as long as the solution is open.
    private INamedType? _registry;

    public override void AmendProject( IProjectAmender amender )
    {
        if ( TypeFactory.TryGetType( "Doc.Model.EntityRegistry", out var registry ) )
        {
            this._registry = registry;
        }
    }
}

Changing the type of the field is the whole fix:

using Metalama.Framework.Code;
using Metalama.Framework.Fabrics;

namespace Doc.RetainedDeclaration_Good;

public class Fabric : ProjectFabric
{
    // A durable reference holds only a string identifier. Declaring the field as IDurableRef<INamedType>
    // also makes the conversion mandatory at every assignment, so a later edit cannot omit it.
    private IDurableRef<INamedType>? _registry;

    public override void AmendProject( IProjectAmender amender )
    {
        if ( TypeFactory.TryGetType( "Doc.Model.EntityRegistry", out var registry ) )
        {
            this._registry = registry.ToDurableRef();
        }
    }

    // Resolve against the compilation you are working on, never against a stored one.
    private INamedType? GetRegistry( ICompilation compilation )
        => this._registry?.GetTarget( compilation );
}

Generic types and cross-project references

• A durable reference is backed by a declaration identifier, so a constructed generic type such as Base<int> resolves back to Base<T> and its type arguments are lost silently. When the stored value can be a constructed generic type, store its full name instead and match on that.
• When the value has to cross a process or a project boundary, store the SerializableDeclarationId returned by ToSerializableId() and resolve it with compilation.Factory.GetDeclarationFromId( id ).

When you only need to recognize a declaration later rather than to use it, storing its full name or its file path is usually enough and is always safe.

Limitations

• Compile-time code that behaves differently in the IDE, by testing IExecutionScenario.IsDesignTime, isn't covered: a command-line build can't reach that branch.
• The analysis reads the static fields of your compile-time assemblies, which runs the static constructors of the types that declare them. This is why it's opt-in.
• A field of an inheritable aspect that is not marked [NonCompileTimeSerialized] is already checked, more strictly, by the compile-time serializer: it reports an error rather than a warning when the field holds a declaration. This diagnostic covers the fields that the serializer skips.