Benchmarking Enum.ToString() performance in Visual Studio Code

I have been recently working quite a lot in Visual Studio Code (VSC), writing some Go code. With the Go for Visual Studio Code extension – writing code is easy and simple experience.

For .Net however, I was using full version of Visual Studio. There were several reasons for that. Initially, VSC was designed to build ASP.Net code and I rarely do web applications only. Technically speaking, one could have used command line tools like msbuild or csc for building other types of .Net components, but the experience wasn’t acceptable. Especially, if you’d try to use NuGet packages in your code.

Recently I thought, why not to give another try, because new tools for .Net development were added to the VSC.

The scenario

Recently we were investigating performance issues related to one server side application. One thing that came into our attention was code like this:

1

return EnumTypeValue.ToString();

Alone this code wouldn’t be a problem, if not the fact that it is called like 1,000,000 times per second.
It is a problem that has been discussed on Stackoverflow and these days it is easy to find the code behind Enum.ToString() on github. For example, here the code spends time evaluating if there’s a flags attribute through reflection of course:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

private static String InternalFormat(RuntimeType eT, Object value) 
{ 
  Contract.Requires(eT != null); 
  Contract.Requires(value != null); 
  if (!eT.IsDefined(typeof(System.FlagsAttribute), false)) // Not marked with Flags attribute 
  { 
    // Try to see if its one of the enum values, then we return a String back else the value 
    String retval = GetName(eT, value); 
    if (retval == null) 
      return value.ToString(); 
    else 
      return retval; 
  }
  else // These are flags OR'ed together (We treat everything as unsigned types) 
  { 
    return InternalFlagsFormat(eT, value); 
  } 
} 

As a result, single-liner ToString() can result in a measurable performance hit under heavy workload.

Could I write benchmark for this case in VSC now?

The problem is obvious and solution is clear, the question for me was whether it is reasonably possible to write .Net code that would benchmark that scenario using Visual Studio Code.

On the machine I had only VSC installed, nothing else. Opened VSC and, just after creating first file with extension .cs I was greeted with the message suggesting to install recommended extensions.

Nice surprise after choosing “Show recommendations” was the list of two extensions: C# and Mono Debug.

I have decided to go with C# extension only. This resulted in yet another prompt that I need to install .Net CLI tools.

Note: I don’t have full version of Visual Studio installed on this particular machine, so I needed to download .NET Core SDK for Windows.

I did installed that (restart of the machine was required) and went back to VSC and Command Line.

First things first – setup new project in folder:

  dotnet new 

Guess what type of application I got by default? Console application! Back to the command line roots. Next, I’ve added some code in separate files:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

  public enum SomeEnum {     
    Black,     
    Red,     
    Blue,     
    Green,     
    Yellow,     
    Beige,     
    White 
  }

  public static class SomeEnumRepository 
  {     
    private static Dictionary<SomeEnum, string> _namesBySomeEnum;     
    
    static SomeEnumRepository()     
    {         
      _namesBySomeEnum = Enum.GetValues(typeof(SomeEnum))
        .Cast<SomeEnum>()
        .ToDictionary(k => k, k => k.ToString());     
    }     
    
    public static string ToStringFromDictionary(this SomeEnum se)     
    {         
      return _namesBySomeEnum[se];     
    } 
  }

  public class EnumToStringBenchmark     
  {        
    public string EnumToString(SomeEnum value)         
    {            
      return value.ToString();         
    }         
    public string EnumToStringCustom(SomeEnum value)         
    {             
      return value.ToStringFromDictionary();         
    }     
  }

Coding experience in C# has improved a lot since last time I have tried (thanks to the extension):

• Missing using statements can be added from warnings
• cw expands to Console.WriteLine (love it!)
• If something is missing – restore is suggested automatically and if executed, references to the packages are added to the project.json file
• Shift-Alt-F – formatted code

dotnet run –c RELEASE

– builds and runs the code.

So far so good, but that was only a half way. I wanted to run benchmark using BenchmarkDotNet, which meant that there are some NuGet packages involved in the process. Of course, I could do the timers, but that wouldn’t give me the easy way to look into GC and memory usage statistics. I added dependencies to the project.json file and was immediatelly greeted with suggestion to “Restore” dependencies, which too nearly 6 seconds and immediately failed, because diagnostics module requires .Net 4.x version. Because of that, reference to core clr had to be removed and .net 4.6.1 added to the project.json file. Resulting file looks like that:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

{   
  "version": "1.0.0-\*",   
  "buildOptions": {     
    "debugType": "portable",     
    "emitEntryPoint": true   
  },   
  "dependencies": {     
    "BenchmarkDotNet":"0.\*",     
    "BenchmarkDotNet.Diagnostics.Windows":"0.\*"  
  },   
  "frameworks": {     
    "net461": {}   
  } 
} 

It was easy to figure out that you need to type the NuGet package name in dependencies section. Another nice part, that Intellisense was suggesting package versions. It was slightly harder regarding framework version, because, at least on my machine, I cannot have both net461 and coreclr configured at the same time even without diagnostics module.

Note: I needed developer pack: https://www.microsoft.com/en-us/download/details.aspx?id=49978 to be installed on my machine.

The code was still building, so I needed to attach the benchmark to the existing code and see for the results.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

public class Program {     
  public static void Main(string[] args)     
  {         
    BenchmarkRunner.Run<EnumToStringBenchmark>(
      ManualConfig
        .Create(DefaultConfig.Instance)                 
        .With(new Job { LaunchCount = 1, WarmupCount = 2, TargetCount = 10, Runtime = Runtime.Clr})                 
        .With(new MemoryDiagnoser())                 
      );
  } 
} 

public class EnumToStringBenchmark {     
  private Random _rnd;     
  private SomeEnum[] _possibleValues;     
  private static int _count;         
  
  [Setup]     
  public void SetupData()     
  {         
    _possibleValues = Enum.GetValues(typeof(SomeEnum))             
      .Cast<SomeEnum>()             
      .ToArray();         
    _count = _possibleValues.Length;         
    _rnd = new Random();     
  }     
  
  [Benchmark]     
  public string EnumToString()     
  {         
    return _possibleValues[_rnd.Next(_count)].ToString();     
  }     
  
  \[Benchmark\]     
  public string EnumToStringCustom()     
  {         
    return _possibleValues[_rnd.Next(_count)].ToStringFromDictionary();     
  } 
}

These are the results on my machine:

The code is available on GitHub

The verdict

There’s still some mess, especially in the area of handling different frameworks and runtimes, however the state looks much better now than even a half a year ago. I definitely could start building some server-side .Net apps using Visual Studio Code and I like that. Also, we already know that project.json is almost gone before even making full appearance. That means, inevitable changes, issues and status “not yet there”.