Before you refactor InvoiceService, you want a test that detects any change of its behavior. Feathers calls such a
test a characterization test: it describes what the code actually does, not what it should do. A common form of
characterization test is the snapshot test, also known as golden master test or approval test: the test
compares the output of the code with a file that was approved earlier.
In this step, the Test configuration replays the interactions that the Record configuration recorded, as
described in Legacy testing example, step 2: Recording real interactions. A snapshot test then pins the output of InvoiceService.
Recording and replaying
The project has two configurations:
- In the
Recordconfiguration, the interceptors record the calls tointeractions.json, as in the previous step. - In the
Testconfiguration, the interceptors do not call the dependencies. They return the recorded results.
The following project fabric chooses the template from the preprocessor symbols that each configuration defines:
1using Metalama.Extensions.Interceptors;
2using Metalama.Framework.Aspects;
3using Metalama.Framework.Code;
4using Metalama.Framework.Fabrics;
5using System.Net.Mail;
6
7namespace Metalama.Samples.LegacyTesting3.Recording;
8
9// Records the calls to the hard-wired dependencies of the legacy code in the Record configuration,
10// and replays them in the Test configuration.
11internal class RecordAndReplayFabric : ProjectFabric
12{
13 public override void AmendProject( IProjectAmender amender )
14 {
15 string template;
16
17 if ( amender.Project.PreprocessorSymbols.Contains( "RECORD_INTERACTIONS" ) )
18 {
19 template = nameof(this.Record);
20 }
21 else if ( amender.Project.PreprocessorSymbols.Contains( "REPLAY_INTERACTIONS" ) )
22 {
23 template = nameof(this.Replay);
24 }
25 else
26 {
27 return;
28 }
29
30 amender.InterceptMethods(
31 calls => calls
32 .Type( typeof(Thread) )
33 .Methods( nameof(Thread.Sleep) )
34 .RedirectToSynthesizedMethod( template ) );
35
36 amender.InterceptMethods(
37 calls => calls
38 .Type( typeof(File) )
39 .Methods( nameof(File.ReadAllText) )
40 .RedirectToSynthesizedMethod( template ) );
41
42 amender.InterceptMethods(
43 calls => calls
44 .Type( typeof(BillingDatabase) )
45 .AllMethods()
46 .RedirectToSynthesizedMethod( template ) );
47
48 amender.InterceptMethods(
49 calls => calls
50 .Type( typeof(SmtpClient) )
51 .Methods( nameof(SmtpClient.Send) )
52 .RedirectToSynthesizedMethod( template ) );
53 }
54
55 [Template]
56 public dynamic? Record()
57 {
58 var interceptedMethod = meta.Interception.Method.InterceptedMethod;
59
60 var arguments = new List<object?>();
61
62 foreach ( var parameter in meta.Interception.Method.ArgumentParameters )
63 {
64 arguments.Add( parameter.Value );
65 }
66
67 var result = meta.Proceed();
68
69 InteractionRecorder.Record(
70 $"{interceptedMethod.DeclaringType.Name}.{interceptedMethod.Name}",
71 arguments,
72 result );
73
74 return result;
75 }
76
77 [Template]
78 public dynamic? Replay()
79 {
80 var interceptedMethod = meta.Interception.Method.InterceptedMethod;
81
82 var arguments = new List<object?>();
83
84 foreach ( var parameter in meta.Interception.Method.ArgumentParameters )
85 {
86 arguments.Add( parameter.Value );
87 }
88
89 // The intercepted method is not called.
90 if ( interceptedMethod.ReturnType.Equals( SpecialType.Void ) )
91 {
92 InteractionReplayer.Replay(
93 $"{interceptedMethod.DeclaringType.Name}.{interceptedMethod.Name}",
94 arguments );
95
96 return null;
97 }
98 else
99 {
100 var result = InteractionReplayer.Replay(
101 $"{interceptedMethod.DeclaringType.Name}.{interceptedMethod.Name}",
102 arguments,
103 interceptedMethod.ReturnType.ToType() );
104
105 return meta.Cast( interceptedMethod.ReturnType, result );
106 }
107 }
108}
109
In the Replay template, meta.Proceed() is not called, so the dependency is never called. For a method that
returns a value, the template gives the return type of the intercepted method to the replayer as a Type, with
ToType(), and casts the result to this type with meta.Cast. For a method that returns void, it only gives the
call to the replayer. Metalama generates the following interceptors:
Source Code
1using Metalama.Samples.LegacyTesting3;
2using System.Globalization;
3using System.Net.Mail;
4using System.Text;
5
6namespace Metalama.Samples.LegacyTesting.Tests.ReplayInteractions;
7
8// Legacy code. It calls the database, the file system, the clock and the mail server directly.
9public class InvoiceService
10{
11 private const decimal _lateFee = 25.00m;
12
13 // Sends a payment reminder for each overdue invoice of a customer, and returns a summary of the reminders.
14 public string SendReminders( int customerId )
15 {
16 var email = BillingDatabase.GetCustomerEmail( customerId );
17 var invoices = BillingDatabase.GetOverdueInvoices( customerId );
18 var template = File.ReadAllText( "reminder-template.txt" );
19
20 using var smtp = new SmtpClient();
21
22 // In production, the messages go to an SMTP server. In this example, they go to the outbox directory.
23 smtp.DeliveryMethod = SmtpDeliveryMethod.SpecifiedPickupDirectory;
24 smtp.PickupDirectoryLocation = Path.GetFullPath( "outbox" );
25
26 var summary = new StringBuilder();
27
28 foreach ( var invoice in invoices )
29 {
30 // A late fee applies after 30 days.
31 var amountDue = invoice.DaysOverdue > 30 ? invoice.Amount + _lateFee : invoice.Amount;
32 var formattedAmount = amountDue.ToString( "F2", CultureInfo.InvariantCulture );
33 var daysOverdue = invoice.DaysOverdue.ToString( CultureInfo.InvariantCulture );
34
35 var body = template
36 .Replace( "{Number}", invoice.Number, StringComparison.Ordinal )
37 .Replace( "{DaysOverdue}", daysOverdue, StringComparison.Ordinal )
38 .Replace( "{Amount}", formattedAmount, StringComparison.Ordinal );
39
40 smtp.Send( "[email protected]", email, $"Payment reminder: invoice {invoice.Number}", body );
41
42 summary.AppendLine(
43 CultureInfo.InvariantCulture,
44 $"{invoice.Number}: {formattedAmount} EUR sent to {email}." );
45
46 // The SMTP server accepts at most one message per second.
47 Thread.Sleep( 1000 );
48 }
49
50 return summary.ToString();
51 }
52}
53
Transformed Code
1using Metalama.Samples.LegacyTesting3;
2using System.Globalization;
3using System.Net.Mail;
4using System.Text;
5
6namespace Metalama.Samples.LegacyTesting.Tests.ReplayInteractions;
7
8// Legacy code. It calls the database, the file system, the clock and the mail server directly.
9public class InvoiceService
10{
11 private const decimal _lateFee = 25.00m;
12
13 // Sends a payment reminder for each overdue invoice of a customer, and returns a summary of the reminders.
14 public string SendReminders(int customerId)
15 {
16 var email = MetalamaInterceptors.BillingDatabase_GetCustomerEmail_Interceptor(customerId: customerId);
17 var invoices = MetalamaInterceptors.BillingDatabase_GetOverdueInvoices_Interceptor(customerId: customerId);
18 var template = MetalamaInterceptors.File_ReadAllText_Interceptor(path: "reminder-template.txt");
19
20 using var smtp = new SmtpClient();
21
22 // In production, the messages go to an SMTP server. In this example, they go to the outbox directory.
23 smtp.DeliveryMethod = SmtpDeliveryMethod.SpecifiedPickupDirectory;
24 smtp.PickupDirectoryLocation = Path.GetFullPath("outbox");
25
26 var summary = new StringBuilder();
27
28 foreach (var invoice in invoices)
29 {
30 // A late fee applies after 30 days.
31 var amountDue = invoice.DaysOverdue > 30 ? invoice.Amount + _lateFee : invoice.Amount;
32 var formattedAmount = amountDue.ToString("F2", CultureInfo.InvariantCulture);
33 var daysOverdue = invoice.DaysOverdue.ToString(CultureInfo.InvariantCulture);
34
35 var body = template
36 .Replace("{Number}", invoice.Number, StringComparison.Ordinal)
37 .Replace("{DaysOverdue}", daysOverdue, StringComparison.Ordinal)
38 .Replace("{Amount}", formattedAmount, StringComparison.Ordinal);
39
40 MetalamaInterceptors.SmtpClient_Send_Interceptor(smtp, from: "[email protected]", recipients: email, subject: $"Payment reminder: invoice {invoice.Number}", body: body);
41
42 summary.AppendLine(
43 CultureInfo.InvariantCulture,
44 $"{invoice.Number}: {formattedAmount} EUR sent to {email}.");
45
46 // The SMTP server accepts at most one message per second.
47 MetalamaInterceptors.Thread_Sleep_Interceptor(millisecondsTimeout: 1000);
48 }
49
50 return summary.ToString();
51 }
52}
53
The replayer takes the recorded interactions in order. For each call, it checks that the code calls the recorded method with the recorded arguments, and it returns the recorded result. A difference means that the behavior of the code has changed, so the replayer throws an exception:
1using System.Text.Json;
2
3namespace Metalama.Samples.LegacyTesting3.Recording;
4
5// Returns the recorded results instead of calling the dependencies. It checks that the legacy code makes the
6// recorded calls, in the same order and with the same arguments.
7public static class InteractionReplayer
8{
9 private static readonly Queue<Interaction> _pendingInteractions = new();
10
11 public static int PendingInteractionCount => _pendingInteractions.Count;
12
13 public static void Load( string json )
14 {
15 _pendingInteractions.Clear();
16
17 foreach ( var interaction in JsonSerializer.Deserialize<List<Interaction>>( json ) ?? [] )
18 {
19 _pendingInteractions.Enqueue( interaction );
20 }
21 }
22
23 // Replays a call to a method that returns void.
24 public static void Replay( string method, IReadOnlyList<object?> arguments ) => Dequeue( method, arguments );
25
26 // Replays a call to a method that returns a value, and returns the recorded result.
27 public static object? Replay( string method, IReadOnlyList<object?> arguments, Type resultType )
28 => Dequeue( method, arguments ).Result?.Deserialize( resultType );
29
30 private static Interaction Dequeue( string method, IReadOnlyList<object?> arguments )
31 {
32 if ( !_pendingInteractions.TryDequeue( out var interaction ) )
33 {
34 throw new InvalidOperationException(
35 $"The code calls {method}, but the recording has no more interactions." );
36 }
37
38 if ( interaction.Method != method )
39 {
40 throw new InvalidOperationException(
41 $"The code calls {method}, but the recording expects a call to {interaction.Method}." );
42 }
43
44 for ( var i = 0; i < arguments.Count; i++ )
45 {
46 var actualArgument = JsonSerializer.SerializeToElement( arguments[i] );
47
48 if ( !JsonElement.DeepEquals( actualArgument, interaction.Arguments[i] ) )
49 {
50 throw new InvalidOperationException(
51 $"The code calls {method} with the argument {actualArgument.GetRawText()}, "
52 + $"but the recording has {interaction.Arguments[i].GetRawText()}." );
53 }
54 }
55
56 return interaction;
57 }
58}
59
The characterization test
To create the test, run the program in the Record configuration, and copy interactions.json to the Recordings
directory of the test project. The test loads the recording, calls InvoiceService, and checks that all the recorded
calls were made. It then compares the summary that InvoiceService returns with a snapshot file:
1using Metalama.Samples.LegacyTesting3.Recording;
2using Xunit;
3
4namespace Metalama.Samples.LegacyTesting3.UnitTests;
5
6// Pins the current behavior of InvoiceService, as it was recorded in production, before the code is refactored.
7public class InvoiceServiceCharacterizationTests
8{
9 [Fact]
10 public void SendReminders()
11 {
12 // The recording was made by running the program in the Record configuration.
13 InteractionReplayer.Load( File.ReadAllText( Path.Combine( "Recordings", "send-reminders-42.json" ) ) );
14
15 var summary = new InvoiceService().SendReminders( 42 );
16
17 // The replayer has checked each call and its arguments, including the messages. All of them must be made.
18 Assert.Equal( 0, InteractionReplayer.PendingInteractionCount );
19
20 Snapshot.Verify( summary, "SendReminders" );
21 }
22}
23
The test checks the following:
- Each call to the dependencies, including the content of each message that the code sends, because the replayer compares the arguments with the recording.
- The order of the calls, including the calls to
Thread.Sleep. - The value that
SendRemindersreturns, because the snapshot file contains it:
INV-1001: 120.00 EUR sent to [email protected].
INV-1007: 70.50 EUR sent to [email protected].
The snapshot helper is a minimal implementation of snapshot testing. When the value differs from the
.verified.txt file, it writes the value to a .received.txt file and fails. To accept a new behavior, replace the
.verified.txt file with the .received.txt file. Libraries such as Verify
provide the same workflow with diff tools.
1using System.Reflection;
2using Xunit;
3
4namespace Metalama.Samples.LegacyTesting3.UnitTests;
5
6// Compares a value with a file of the Snapshots directory. When they differ, the test fails, and the value is written
7// to a .received.txt file next to the .verified.txt file. To accept the new value, replace the .verified.txt file with
8// the .received.txt file.
9internal static class Snapshot
10{
11 // The directory of the project is stored in the assembly by the project file. [CallerFilePath] cannot be used
12 // because a deterministic build maps the source paths to '/_/'.
13 private static readonly string _directory = Path.Combine(
14 typeof(Snapshot).Assembly.GetCustomAttributes<AssemblyMetadataAttribute>().Single( a => a.Key == "ProjectDirectory" ).Value!,
15 "Snapshots" );
16
17 public static void Verify( string actual, string name )
18 {
19 var verifiedPath = Path.Combine( _directory, $"{name}.verified.txt" );
20 var receivedPath = Path.Combine( _directory, $"{name}.received.txt" );
21
22 // Line endings depend on the platform and on the Git settings, so they are not compared.
23 actual = NormalizeLineEndings( actual );
24 var verified = File.Exists( verifiedPath ) ? NormalizeLineEndings( File.ReadAllText( verifiedPath ) ) : null;
25
26 if ( actual == verified )
27 {
28 File.Delete( receivedPath );
29
30 return;
31 }
32
33 File.WriteAllText( receivedPath, actual );
34
35 Assert.Fail( $"The value differs from the snapshot '{verifiedPath}'. The value is in '{receivedPath}'." );
36 }
37
38 private static string NormalizeLineEndings( string text ) => text.ReplaceLineEndings( "\n" );
39}
40
For instance, when you change the late fee from 25.00 to 30.00 EUR, the test fails, because the second message differs from the recording:
System.InvalidOperationException : The code calls SmtpClient.Send with the argument "Dear customer,\n\nInvoice INV-1007 is 41 days overdue. The amount due is 75.50 EUR.\n\nBilling department\n", but the recording has "Dear customer,\n\nInvoice INV-1007 is 41 days overdue. The amount due is 70.50 EUR.\n\nBilling department\n".
Next step
The characterization test pins the behavior of InvoiceService, so you can now refactor it. Replace the calls to
the static methods with calls to injected abstractions, and replace the characterization test with unit tests that
use test doubles of these abstractions. Then delete the fabric, the recorder and the replayer. See the last section
of Example: getting legacy code under test without refactoring.