How to refactor code that has no tests using the Golden Master technique

The number one rule when refactoring is to always have tests in place before you start refactoring. Without tests, refactoring becomes a risky endeavor.

Code that doesn’t have tests is often not designed to be testable. In order to make the code testable, you have to refactor it. See how this seems like an impossible task?

Luckily we have a solution: the Golden Master technique. In this article I’ll show how to add a Golden Master test to untested code. The Golden Master test paves the way for being able to safely begin refactoring.

What is a Golden Master?

The Golden Master is a saved copy of the output a program generates.

To generate a Golden Master you have to run the program and capture its output. The simplest example of this, which I’ll be showing in this article, is a console application that writes messages to the console. You can capture these messages and save them somewhere (ex: a text file).

Once you have the Golden Master, you can create a test around it. Every time you run your program you can capture the output and compare it with the Golden Master to verify nothing changed.

When creating a Golden Master test, each program will have its own difficulties. Yes, this is a difficult process, but with some creativity and determination, you will be able to pull it off.

The untested code

First let’s take a look at some untested code that we want to refactor. This comes from the Trivia refactoring kata. This code is intentionally bad and untested. Our goal in this article is to create a Golden Master test that will cover the Game class.

Game class

using System;
using System.Collections.Generic;
using System.Linq;

namespace Trivia
{
    public class Game
    {
        private readonly List<string> _players = new List<string>();

        private readonly int[] _places = new int[6];
        private readonly int[] _purses = new int[6];

        private readonly bool[] _inPenaltyBox = new bool[6];

        private readonly LinkedList<string> _popQuestions = new LinkedList<string>();
        private readonly LinkedList<string> _scienceQuestions = new LinkedList<string>();
        private readonly LinkedList<string> _sportsQuestions = new LinkedList<string>();
        private readonly LinkedList<string> _rockQuestions = new LinkedList<string>();

        private int _currentPlayer;
        private bool _isGettingOutOfPenaltyBox;

        public Game()
        {
            for (var i = 0; i < 50; i++)
            {
                _popQuestions.AddLast("Pop Question " + i);
                _scienceQuestions.AddLast(("Science Question " + i));
                _sportsQuestions.AddLast(("Sports Question " + i));
                _rockQuestions.AddLast(CreateRockQuestion(i));
            }
        }

        public string CreateRockQuestion(int index)
        {
            return "Rock Question " + index;
        }

        public bool IsPlayable()
        {
            return (HowManyPlayers() >= 2);
        }

        public bool Add(string playerName)
        {
            _players.Add(playerName);
            _places[HowManyPlayers()] = 0;
            _purses[HowManyPlayers()] = 0;
            _inPenaltyBox[HowManyPlayers()] = false;

            Console.WriteLine(playerName + " was added");
            Console.WriteLine("They are player number " + _players.Count);
            return true;
        }

        public int HowManyPlayers()
        {
            return _players.Count;
        }

        public void Roll(int roll)
        {
            Console.WriteLine(_players[_currentPlayer] + " is the current player");
            Console.WriteLine("They have rolled a " + roll);

            if (_inPenaltyBox[_currentPlayer])
            {
                if (roll % 2 != 0)
                {
                    _isGettingOutOfPenaltyBox = true;

                    Console.WriteLine(_players[_currentPlayer] + " is getting out of the penalty box");
                    _places[_currentPlayer] = _places[_currentPlayer] + roll;
                    if (_places[_currentPlayer] > 11) _places[_currentPlayer] = _places[_currentPlayer] - 12;

                    Console.WriteLine(_players[_currentPlayer]
                            + "'s new location is "
                            + _places[_currentPlayer]);
                    Console.WriteLine("The category is " + CurrentCategory());
                    AskQuestion();
                }
                else
                {
                    Console.WriteLine(_players[_currentPlayer] + " is not getting out of the penalty box");
                    _isGettingOutOfPenaltyBox = false;
                }
            }
            else
            {
                _places[_currentPlayer] = _places[_currentPlayer] + roll;
                if (_places[_currentPlayer] > 11) _places[_currentPlayer] = _places[_currentPlayer] - 12;

                Console.WriteLine(_players[_currentPlayer]
                        + "'s new location is "
                        + _places[_currentPlayer]);
                Console.WriteLine("The category is " + CurrentCategory());
                AskQuestion();
            }
        }

        private void AskQuestion()
        {
            if (CurrentCategory() == "Pop")
            {
                Console.WriteLine(_popQuestions.First());
                _popQuestions.RemoveFirst();
            }
            if (CurrentCategory() == "Science")
            {
                Console.WriteLine(_scienceQuestions.First());
                _scienceQuestions.RemoveFirst();
            }
            if (CurrentCategory() == "Sports")
            {
                Console.WriteLine(_sportsQuestions.First());
                _sportsQuestions.RemoveFirst();
            }
            if (CurrentCategory() == "Rock")
            {
                Console.WriteLine(_rockQuestions.First());
                _rockQuestions.RemoveFirst();
            }
        }

        private string CurrentCategory()
        {
            if (_places[_currentPlayer] == 0) return "Pop";
            if (_places[_currentPlayer] == 4) return "Pop";
            if (_places[_currentPlayer] == 8) return "Pop";
            if (_places[_currentPlayer] == 1) return "Science";
            if (_places[_currentPlayer] == 5) return "Science";
            if (_places[_currentPlayer] == 9) return "Science";
            if (_places[_currentPlayer] == 2) return "Sports";
            if (_places[_currentPlayer] == 6) return "Sports";
            if (_places[_currentPlayer] == 10) return "Sports";
            return "Rock";
        }

        public bool WasCorrectlyAnswered()
        {
            if (_inPenaltyBox[_currentPlayer])
            {
                if (_isGettingOutOfPenaltyBox)
                {
                    Console.WriteLine("Answer was correct!!!!");
                    _purses[_currentPlayer]++;
                    Console.WriteLine(_players[_currentPlayer]
                            + " now has "
                            + _purses[_currentPlayer]
                            + " Gold Coins.");

                    var winner = DidPlayerWin();
                    _currentPlayer++;
                    if (_currentPlayer == _players.Count) _currentPlayer = 0;

                    return winner;
                }
                else
                {
                    _currentPlayer++;
                    if (_currentPlayer == _players.Count) _currentPlayer = 0;
                    return true;
                }
            }
            else
            {
                Console.WriteLine("Answer was corrent!!!!");
                _purses[_currentPlayer]++;
                Console.WriteLine(_players[_currentPlayer]
                        + " now has "
                        + _purses[_currentPlayer]
                        + " Gold Coins.");

                var winner = DidPlayerWin();
                _currentPlayer++;
                if (_currentPlayer == _players.Count) _currentPlayer = 0;

                return winner;
            }
        }

        public bool WrongAnswer()
        {
            Console.WriteLine("Question was incorrectly answered");
            Console.WriteLine(_players[_currentPlayer] + " was sent to the penalty box");
            _inPenaltyBox[_currentPlayer] = true;

            _currentPlayer++;
            if (_currentPlayer == _players.Count) _currentPlayer = 0;
            return true;
        }


        private bool DidPlayerWin()
        {
            return !(_purses[_currentPlayer] == 6);
        }
    }

}

GameRunner class

using System;

namespace Trivia
{
    public class GameRunner
    {
        private static bool _notAWinner;

        public static void Main(string[] args)
        {
            var aGame = new Game();

            aGame.Add("Chet");
            aGame.Add("Pat");
            aGame.Add("Sue");

            var rand = new Random();

            do
            {
                aGame.Roll(rand.Next(5) + 1);

                if (rand.Next(9) == 7)
                {
                    _notAWinner = aGame.WrongAnswer();
                }
                else
                {
                    _notAWinner = aGame.WasCorrectlyAnswered();
                }
            } while (_notAWinner);
        }
    }
}

Create a unit test and copy the code from GameRunner

We need a way to run the Game class so that we can capture its output. The GameRunner class is a good start.

Here’s what we need to do:

  1. Create a unit test project that references the Trivia project.
  2. Add a unit test.
  3. Copy the GameRunner code over to the unit test.
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Trivia;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Trivia.Tests
{
    [TestClass()]
    public class GameTests
    {
        private static bool _notAWinner;

        [TestMethod()]
        public void GameTest()
        {
            var aGame = new Game();

            aGame.Add("Chet");
            aGame.Add("Pat");
            aGame.Add("Sue");

            var rand = new Random();

            do
            {
                aGame.Roll(rand.Next(5) + 1);

                if (rand.Next(9) == 7)
                {
                    _notAWinner = aGame.WrongAnswer();
                }
                else
                {
                    _notAWinner = aGame.WasCorrectlyAnswered();
                }
            } while (_notAWinner);
        }
    }
}

Hardcode the input

The Golden Master test has to be deterministic. Given input X, we expect output Y every time we run it. The GameRunner code is passing in random numbers. We can’t do that. We need to hardcode the input so we can know for sure that the same values are being passed in every time we run the program.

We need to:

  1. Add a queue of integers – rollsSequence. This is what we’ll be using for input.
  2. Replace calls to Random.Next() with calls to rollsSequence.Dequeue().
[TestMethod()]
public void GameTest()
{
	var rollsSequence = new Queue<int>(new List<int>()
	{
		6,9,2,4,7,0,5,1,0,2,8,4,2,3,8,1,2,5,1,7,8,4,5,7,8,1,3,1,9,2,9,6,2,2,1,0,2,6,1,2,8,9,7,1,3,5,2,5,1,3,
	});

	aGame.Add("Chet");
	aGame.Add("Pat");
	aGame.Add("Sue");

	do
	{
		aGame.Roll(rollsSequence.Dequeue());

		if (rollsSequence.Dequeue() == 7)
		{
			_notAWinner = aGame.WrongAnswer();
		}
		else
		{
			_notAWinner = aGame.WasCorrectlyAnswered();
		}

	} while (_notAWinner);
}

Capture the output

Now we can run the program and capture the output.

  1. Run the GameTest unit test in the Test Explorer.
  2. Click on the GameTest to see the test details.
  3. Click on Open additional output for this result (Note: it says something different in previous versions of Visual Studio).
  4. Copy the text.
Test Explorer window showing Standard Output from running the test with Console.WriteLine calls

Add the output as a variable in the test class

Now that we have a copy of the output, we need to save it somewhere. The simplest option is to save it as a string in the test class.

        private static readonly string GoldenMaster = 
@"Chet was added
They are player number 1
Pat was added
They are player number 2
Sue was added
They are player number 3
Chet is the current player
They have rolled a 6
Chet's new location is 6
The category is Sports
Sports Question 0
Answer was corrent!!!!
Chet now has 1 Gold Coins.
Pat is the current player
They have rolled a 2
Pat's new location is 2
The category is Sports
Sports Question 1
Answer was corrent!!!!
Pat now has 1 Gold Coins.
Sue is the current player
They have rolled a 7
Sue's new location is 7
The category is Rock
Rock Question 0
Answer was corrent!!!!
Sue now has 1 Gold Coins.
Chet is the current player
They have rolled a 5
Chet's new location is 11
The category is Rock
Rock Question 1
Answer was corrent!!!!
Chet now has 2 Gold Coins.
Pat is the current player
They have rolled a 0
Pat's new location is 2
The category is Sports
Sports Question 2
Answer was corrent!!!!
Pat now has 2 Gold Coins.
Sue is the current player
They have rolled a 8
Sue's new location is 3
The category is Rock
Rock Question 2
Answer was corrent!!!!
Sue now has 2 Gold Coins.
Chet is the current player
They have rolled a 2
Chet's new location is 1
The category is Science
Science Question 0
Answer was corrent!!!!
Chet now has 3 Gold Coins.
Pat is the current player
They have rolled a 8
Pat's new location is 10
The category is Sports
Sports Question 3
Answer was corrent!!!!
Pat now has 3 Gold Coins.
Sue is the current player
They have rolled a 2
Sue's new location is 5
The category is Science
Science Question 1
Answer was corrent!!!!
Sue now has 3 Gold Coins.
Chet is the current player
They have rolled a 1
Chet's new location is 2
The category is Sports
Sports Question 4
Question was incorrectly answered
Chet was sent to the penalty box
Pat is the current player
They have rolled a 8
Pat's new location is 6
The category is Sports
Sports Question 5
Answer was corrent!!!!
Pat now has 4 Gold Coins.
Sue is the current player
They have rolled a 5
Sue's new location is 10
The category is Sports
Sports Question 6
Question was incorrectly answered
Sue was sent to the penalty box
Chet is the current player
They have rolled a 8
Chet is not getting out of the penalty box
Pat is the current player
They have rolled a 3
Pat's new location is 9
The category is Science
Science Question 2
Answer was corrent!!!!
Pat now has 5 Gold Coins.
Sue is the current player
They have rolled a 9
Sue is getting out of the penalty box
Sue's new location is 7
The category is Rock
Rock Question 3
Answer was correct!!!!
Sue now has 4 Gold Coins.
Chet is the current player
They have rolled a 9
Chet is getting out of the penalty box
Chet's new location is 11
The category is Rock
Rock Question 4
Answer was correct!!!!
Chet now has 4 Gold Coins.
Pat is the current player
They have rolled a 2
Pat's new location is 11
The category is Rock
Rock Question 5
Answer was corrent!!!!
Pat now has 6 Gold Coins.
";

Note: The output I copied didn’t have a newline at the end, so I had to add one. Watch out for missing newlines when you copy output.

Capture Console.WriteLine() output into a local variable

We need a way to compare the output from the program every time we run it. Since the Game class is calling Console.WriteLine() to output messages, we can redirect that output to a local string variable. This will allow us to compare it to the Golden Master.

The way to redirect output from the console is by calling Console.SetOut(), as seen below:

StringBuilder capturedOutput = new StringBuilder();
Console.SetOut(new StringWriter(capturedOutput));

Compare the actual output with the Golden Master

We need to verify that the capturedOutput string is exactly the same as the Golden Master.

The simple approach would be to check if the strings are equal. But if there’s a difference, we want to know exactly which line has a difference, so we need to loop through the lines and compare them.

This test code has three possible outcomes:

  1. The strings are exactly the same.
  2. The GoldenMaster has more lines than the capturedOutput. This is why we need to verify the lengths are the same after the loop.
  3. The capturedOutput has more lines than the GoldenMaster. In this case, we would get an index out of range exception.
//Diff the actual and expected
var actualLines = capturedOutput.ToString().Split(new string[] { Environment.NewLine }, StringSplitOptions.None);
var expectedLines = GoldenMaster.ToString().Split(new string[] { Environment.NewLine }, StringSplitOptions.None);
for (var i = 0; i < actualLines.Length; i++)
{
	Assert.AreEqual(expectedLines[i], actualLines[i], $"Line {i} diff");
}
Assert.AreEqual(expectedLines.Length, actualLines.Length, "Different lengths");

Verify the test works by making it fail intentionally

You should always make sure your tests fail at first, otherwise you can’t really tell if they are passing for the right reason. So to make sure our test fails, we can purposely change the first line by adding a space:

 private static readonly string GoldenMaster = 
@"C het was added

The test fails, so it detected the difference as expected.

Test Detail Summary showing that the test properly detected the diff and showed which line has the diff

Now undo the intentional bug and make sure the test passes.

Final results – the Golden Master test

The final result is a Golden Master test. This enables us to start refactoring safely. After each small refactoring step, we can re-run the Golden Master test to verify we didn’t break anything.

using Microsoft.VisualStudio.TestTools.UnitTesting;
using Trivia;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;

namespace Trivia.Tests
{
    [TestClass()]
    public class GameTests
    {
        private static bool _notAWinner;

        [TestMethod()]
        public void GameTest()
        {

            var rollsSequence = new Queue<int>(new List<int>()
            {
                6,9,2,4,7,0,5,1,0,2,8,4,2,3,8,1,2,5,1,7,8,4,5,7,8,1,3,1,9,2,9,6,2,2,1,0,2,6,1,2,8,9,7,1,3,5,2,5,1,3,
            });

            StringBuilder capturedOutput = new StringBuilder();
            Console.SetOut(new StringWriter(capturedOutput));
            var aGame = new Game();

            aGame.Add("Chet");
            aGame.Add("Pat");
            aGame.Add("Sue");

            do
            {
                aGame.Roll(rollsSequence.Dequeue());

                if (rollsSequence.Dequeue() == 7)
                {
                    _notAWinner = aGame.WrongAnswer();
                }
                else
                {
                    _notAWinner = aGame.WasCorrectlyAnswered();
                }

            } while (_notAWinner);


            //Diff the actual and expected
            var actualLines = capturedOutput.ToString().Split(new string[] { Environment.NewLine }, StringSplitOptions.None);
            var expectedLines = GoldenMaster.ToString().Split(new string[] { Environment.NewLine }, StringSplitOptions.None);
            for (var i = 0; i < actualLines.Length; i++)
            {
                Assert.AreEqual(expectedLines[i], actualLines[i], $"Line {i} diff");
            }
            Assert.AreEqual(expectedLines.Length, actualLines.Length, "Different lengths");
        }

        private static readonly string GoldenMaster = 
@"Chet was added
They are player number 1
Pat was added
They are player number 2
Sue was added
They are player number 3
Chet is the current player
They have rolled a 6
Chet's new location is 6
The category is Sports
Sports Question 0
Answer was corrent!!!!
Chet now has 1 Gold Coins.
Pat is the current player
They have rolled a 2
Pat's new location is 2
The category is Sports
Sports Question 1
Answer was corrent!!!!
Pat now has 1 Gold Coins.
Sue is the current player
They have rolled a 7
Sue's new location is 7
The category is Rock
Rock Question 0
Answer was corrent!!!!
Sue now has 1 Gold Coins.
Chet is the current player
They have rolled a 5
Chet's new location is 11
The category is Rock
Rock Question 1
Answer was corrent!!!!
Chet now has 2 Gold Coins.
Pat is the current player
They have rolled a 0
Pat's new location is 2
The category is Sports
Sports Question 2
Answer was corrent!!!!
Pat now has 2 Gold Coins.
Sue is the current player
They have rolled a 8
Sue's new location is 3
The category is Rock
Rock Question 2
Answer was corrent!!!!
Sue now has 2 Gold Coins.
Chet is the current player
They have rolled a 2
Chet's new location is 1
The category is Science
Science Question 0
Answer was corrent!!!!
Chet now has 3 Gold Coins.
Pat is the current player
They have rolled a 8
Pat's new location is 10
The category is Sports
Sports Question 3
Answer was corrent!!!!
Pat now has 3 Gold Coins.
Sue is the current player
They have rolled a 2
Sue's new location is 5
The category is Science
Science Question 1
Answer was corrent!!!!
Sue now has 3 Gold Coins.
Chet is the current player
They have rolled a 1
Chet's new location is 2
The category is Sports
Sports Question 4
Question was incorrectly answered
Chet was sent to the penalty box
Pat is the current player
They have rolled a 8
Pat's new location is 6
The category is Sports
Sports Question 5
Answer was corrent!!!!
Pat now has 4 Gold Coins.
Sue is the current player
They have rolled a 5
Sue's new location is 10
The category is Sports
Sports Question 6
Question was incorrectly answered
Sue was sent to the penalty box
Chet is the current player
They have rolled a 8
Chet is not getting out of the penalty box
Pat is the current player
They have rolled a 3
Pat's new location is 9
The category is Science
Science Question 2
Answer was corrent!!!!
Pat now has 5 Gold Coins.
Sue is the current player
They have rolled a 9
Sue is getting out of the penalty box
Sue's new location is 7
The category is Rock
Rock Question 3
Answer was correct!!!!
Sue now has 4 Gold Coins.
Chet is the current player
They have rolled a 9
Chet is getting out of the penalty box
Chet's new location is 11
The category is Rock
Rock Question 4
Answer was correct!!!!
Chet now has 4 Gold Coins.
Pat is the current player
They have rolled a 2
Pat's new location is 11
The category is Rock
Rock Question 5
Answer was corrent!!!!
Pat now has 6 Gold Coins.
";
    }
}

Leave a Comment