Benchmarking Enum.ToString() performance in Visual Studio Code

I have been recently working quite a lot in Visual Studio Code (VSC), writing some Go code. With the Go for Visual Studio Code extension – writing code is easy and simple experience.

For .Net however, I was using full version of Visual Studio. There were several reasons for that. Initially, VSC was designed to build ASP.Net code and I rarely do web applications only. Technically speaking, one could have used command line tools like msbuild or csc for building other types of .Net components, but the experience wasn’t acceptable. Especially, if you’d try to use NuGet packages in your code.

Recently I thought, why not to give another try, because new tools for .Net development were added to the VSC.

The scenario

Recently we were investigating performance issues related to one server side application. One thing that came into our attention was code like this:

  
return EnumTypeValue.ToString();  

Alone this code wouldn’t be a problem, if not the fact that it is called like 1,000,000 times per second.
It is a problem that has been discussed on Stackoverflow and these days it is easy to find the code behind Enum.ToString() on github.
For example, here the code spends time evaluating if there’s a flags attribute through reflection of course:

 
private static String InternalFormat(RuntimeType eT, Object value) 
{ 
  Contract.Requires(eT != null); 
  Contract.Requires(value != null); 
  if (!eT.IsDefined(typeof(System.FlagsAttribute), false)) // Not marked with Flags attribute 
  { 
    // Try to see if its one of the enum values, then we return a String back else the value 
    String retval = GetName(eT, value); 
    if (retval == null) 
      return value.ToString(); 
    else 
      return retval; 
  } 
  else // These are flags OR'ed together (We treat everything as unsigned types) 
  { 
    return InternalFlagsFormat(eT, value); 
  } 
} 

As a result, single-liner ToString() can result in a measurable performance hit under heavy workload.

Could I write benchmark for this case in VSC now?

The problem is obvious and solution is clear, the question for me was whether it is reasonably possible to write .Net code that would benchmark that scenario using Visual Studio Code.

On the machine I had only VSC installed, nothing else. Opened VSC and, just after creating first file with extension .cs I was greeted with the message suggesting to install recommended extensions.

VSC-Recommendation-Prompt

Nice surprise after choosing “Show recommendations” was the list of two extensions: C# and Mono Debug.

VSC-Recommended-Extensions

I have decided to go with C# extension only. This resulted in yet another prompt that I need to install .Net CLI tools.

VSC-CLI-Required

Note: I don’t have full version of Visual Studio installed on this particular machine, so I needed to download .NET Core SDK for Windows.

I did installed that (restart of the machine was required) and went back to VSC and Command Line.

First things first – setup new project in folder:

 
dotnet new 

Guess what type of application I got by default? Console application! Back to the command line roots.
Next, I’ve added some code in separate files:

 
public enum SomeEnum
{
    Black,
    Red,
    Blue,
    Green,
    Yellow,
    Beige,
    White
}

public static class SomeEnumRepository {
    private static Dictionary<SomeEnum, string> _namesBySomeEnum;
 
    static SomeEnumRepository()
    {
        _namesBySomeEnum = Enum.GetValues(typeof(SomeEnum))
            .Cast<SomeEnum>()
            .ToDictionary(k => k, k => k.ToString());
    }
 
    public static string ToStringFromDictionary(this SomeEnum se)
    {
        return _namesBySomeEnum[se];
    }
}

public class EnumToStringBenchmark
    {
        public string EnumToString(SomeEnum value)
        {
            return value.ToString();
        }
 
        public string EnumToStringCustom(SomeEnum value)
        {
            return value.ToStringFromDictionary();
        }
    }

Coding experience in C# has improved a lot since last time I have tried (thanks to the extension):

  • Missing using statements can be added from warnings
  • cw expands to Console.WriteLine (love it!)
  • If something is missing – restore is suggested automatically and if executed, references to the packages are added to the project.json file
  • Shift-Alt-F – formatted code
dotnet run –c RELEASE

– builds and runs the code.

So far so good, but that was only a half way. I wanted to run benchmark using BenchmarkDotNet, which meant that there are some NuGet packages involved in the process. Of course, I could do the timers, but that wouldn’t give me the easy way to look into GC and memory usage statistics. I added dependencies to the project.json file and was immediatelly greeted with suggestion to “Restore” dependencies, which too nearly 6 seconds and immediately failed, because diagnostics module requires .Net 4.x version. Because of that, reference to core clr had to be removed and .net 4.6.1 added to the project.json file. Resulting file looks like that:

{
  "version": "1.0.0-*",
  "buildOptions": {
    "debugType": "portable",
    "emitEntryPoint": true
  },
  "dependencies": {
    "BenchmarkDotNet":"0.*",
    "BenchmarkDotNet.Diagnostics.Windows":"0.*"
  },
  "frameworks": {
    "net461": {}
  }
}

It was easy to figure out that you need to type the NuGet package name in dependencies section. Another nice part, that Intellisense was suggesting package versions.
It was slightly harder regarding framework version, because, at least on my machine, I cannot have both net461 and coreclr configured at the same time even without diagnostics module.

Note: I needed developer pack: https://www.microsoft.com/en-us/download/details.aspx?id=49978 to be installed on my machine.

The code was still building, so I needed to attach the benchmark to the existing code and see for the results.

public class Program
{
    public static void Main(string[] args)
    {
        BenchmarkRunner.Run<EnumToStringBenchmark>(
            ManualConfig
                .Create(DefaultConfig.Instance)
                .With(new Job { LaunchCount = 1, WarmupCount = 2, TargetCount = 10, Runtime = Runtime.Clr})
                .With(new MemoryDiagnoser())
                );
    }
}
 
public class EnumToStringBenchmark
{
    private Random _rnd;
    private SomeEnum[] _possibleValues;
    private static int _count;
    
    [Setup]
    public void SetupData()
    {
        _possibleValues = Enum.GetValues(typeof(SomeEnum))
            .Cast<SomeEnum>()
            .ToArray();
        _count = _possibleValues.Length;
        _rnd = new Random();
    }
 
    [Benchmark]
    public string EnumToString()
    {
        return _possibleValues[_rnd.Next(_count)].ToString();
    }
    [Benchmark]
    public string EnumToStringCustom()
    {
        return _possibleValues[_rnd.Next(_count)].ToStringFromDictionary();
    }
}

These are the results on my machine:

Method Median StdDev Gen 0 Bytes Alloc/Op
EnumToString 497.5330 ns 1.0337 ns 334.00 25,99
EnumToStringCustom 28.6341 ns 0.0456 ns 0,00

The code is available on GitHub

The verdict

There’s still some mess, especially in the area of handling different frameworks and runtimes, however the state looks much better now than even a half a year ago. I definitely could start building some server-side .Net apps using Visual Studio Code and I like that.
Also, we already know that project.json is almost gone before even making full appearance. That means, inevitable changes, issues and status “not yet there”.

Thoughts on Windows Azure Mobile Services

I was excited, when I saw the Windows Azure Mobile Services announcement. The idea itself is great – all those different services joined into a single, simple cohesive API. While SDK for other platforms (even Windows Phone) is currently not available, but that is not a big problem, because underneath it uses the same HTTP.

Mobile services provide several important features: data storage, authentication, authorization and push notifications. In fact, if you think about developing apps for the user that have to share the state (e.g. between devices) – that’s a perfect thing. Considering the fact that according the Windows Azure pricing site you can

Run up to 10 Mobile Services for free in a multitenant environment

This is great, isn’t it? And the provided ToDo List Sample application shows just like that. Everything is good, except one thing – mobile services don’t include SQL Server cost. And it does cost …

Suggested model is valid, if you’re developing multitenant application and you want to hold the data as part of your service – then you pay for the data storage and bandwidth.

Another story is, if you are really making yet another “to do” application (YATDA), then it is about the user to decide, whether they want to keep it locally or make available on all their devices and potentially pay for the storage and bandwidth usage.

So, even after introduction of mobile services, fact is still the same: there is still no place for per-user (not per-application) storage for application data that should be available on his multiple devices.

Windows 8 has an ability to sync settings, but that works only between Windows 8 machines and only if you have switched to Microsoft account. Really? Thanks, but no, thanks.

Storing application data on SkyDrive is also not an option – its “fair use” requirements prohibit storing anything else except documents and pictures.

Mobile services on the other hand requires an app developer to own, manage and pay for. And most probably he will want that money back either in form of payment or Ads. Can you imagine YATDA that has “Sign-up for the free trial” as it is shown in the sample? Me not.

So what’s missing in the story? IMO only one thing – this service should be also available as part other Windows Online services together with Outlook, SkyDrive, Calendar and others. Ideally, each Windows Online user could have allocated 100MB or 1GB database for free that he could use for all his applications he wants to make available on all devices.

I hope it will happen sometime in the near future, because this would make “three-screen” strategy possible and easy.

Parameterized queries, filtered indexes and LINQ

Sometimes I like to cite “Remember remember the fifth of November” in case, when I trip the mine that I teach others to avoid.

This time it happened, when reviewing some database structure and related code. So, here it goes.

The database had a table, which can be scripted as shown below. As you can probably note, we are talking about a table that holds queue of jobs that have to be performed.

CREATE TABLE [dbo].[Jobs](
    [Id] [int] IDENTITY(1,1) NOT NULL,
    [Priority] [tinyint] NOT NULL,
    [DateInserted] [datetime] NOT NULL,
    [DateAllocated] [datetime] NULL,
    [DateFetched] [datetime] NULL,
    [Status] [tinyint] NOT NULL,
    [SerializedData]  NULL,
    [ParentId] [int] NULL,
    [ReferenceID] [bigint] NULL,
    CONSTRAINT [PK_Jobs] PRIMARY KEY CLUSTERED ( [Id] ASC )
    )
GO

ALTER TABLE [dbo].[Jobs] ADD  CONSTRAINT 
[DF_Jobs_DateInserted]  DEFAULT (getdate()) FOR [DateInserted]
GO

ALTER TABLE [dbo].[Jobs] ADD  CONSTRAINT 
[DF_Jobs_Status]  DEFAULT (1) FOR [Status]
GO

The use case for the table was simple:

  • One part of the application inserts new jobs based on customer activities and by this sets the default date and status (1)
  • Then service fetches jobs, by taking Id’s of specified amount of new (status=1) jobs and setting appropriate date on DateFetched and updating status to (2)
  • Later at some point XML data will be retrieved using the Id’s retrieved previously

Initially, the code was LINQ 2 SQL in the program itself and was something like this:

var q = ctx.Jobs
            .Where(j => j.Status == 1)
            .Select(j => new { j.Id, j.Priority, j.DateInserted, j.Status })
            .Take(count);
For the testing purposes, I’ve loaded the table with 100k rows, out of which only 401 had status=1.
SELECT TOP (100) Id, [Priority], DateInserted FROM Jobs
WHERE Status=1
Running the query without any additional indexes, you will get the results, but there will be definitely a table scan with a little bit more of logical reads than you could expect.
00-InitialQueryPlan
(100 row(s) affected)Table 'Jobs'. Scan count 1, logical reads 648

 

Now, because the second step is looking for the new records (status=1), which normally should be small number of records comparing to the whole table size, so I immediately thought about creating non-clustered index for the status column, filter on the status=1 and include appropriate values (to have a small covering index). So I created one:
CREATE NONCLUSTERED INDEX IX_Jobs_NewJobs 
ON Jobs ([Status])
INCLUDE ([Priority], [DateInserted])
WHERE ([Status]=1)
This of course had a nice effect on the previous query:
01-AfterIndexCreated
(100 row(s) affected)Table 'Jobs'. Scan count 1, logical reads 2

For a moment I thought, that the rest should be fine, because query and LINQ statement were fairly straight forward with not much magic in them, but somehow, I’ve decided to see what query LINQ does produce.

The query was almost the same with one exception – parameterization, where the parameter value was of course equal to (1):
SELECT TOP (100) [t0].[Id], [t0].[Priority], [t0].[DateInserted], [t0].[Status]
FROM [dbo].[Jobs] AS [t0]
WHERE [t0].[Status] = @p0
However the shocking discovery for me at that point was the query plan:
00-InitialQueryPlan
Yes, SQL server was NOT using the non-clustered index I’ve created moments ago. WTF!
I’ve also checked the query actually shown by the SQL Server Profiler:
exec sp_executesql N'SELECT TOP (5) [t0].[Id], 
[t0].[Priority], [t0].[DateInserted], [t0].[Status]
FROM [dbo].[Jobs] AS [t0]
WHERE [t0].[Status] = @p0',N'@p0 int',@p0=1
Again, result was obviously the same.
Ok, it was obviously related to the parameterization of the query rather than some LINQ issue. It just happened that LINQ actually parameterizes (and for good) queries.
Quick Bing check resulted in http://msdn.microsoft.com/en-us/library/cc280372.aspx, which has clear statement on the subject: “In some cases, a parameterized query does not contain enough information at compile time for the query optimizer to choose a filtered index. It might be possible to rewrite the query to provide the missing information
And this was the moment, when I cited the famous phrase. Was I sleeping during my own sessions, where I was talking about query plans, their caching, etc., like described here?
The problem becomes obvious, when you think that there are other possible values for which query plan using the non-clustered index would become invalid.
Solution
In my case, because of this and some other issues, I’ve decided to use stored procedure, which basically does all the job (retrieving the data and updating the state in one nice transaction).
UPDATE TOP (@BatchSize) Jobs 
SET Status = 2, DateFetched = GETUTCDATE()
OUTPUT inserted.Id, inserted.[Priority], inserted.DateInserted
WHERE Status = 1

Works like a charm Smile

After party or summary of presentation @ Powered by MVP

Today I was giving presentation at “Powered by MVP” event in Lithuania.

The topic: what to do, when programs leak, crash and misbehave in many other ways.

The summary: there are tools that can help you finding the cause problem and solving them. They only require some time to learn and some more time to master, but otherwise – it is much better than automatic IIS pool recycling because of a faulty app.

I would suggest the following way of mastering this part of skills.

Familiarize yourself with the subject like memory management

Equip yourself with tools

  • Grab sysinternals suite here. Familiarize with tools. I mean, even if just your PC startup is too long, check autoruns – I bet you’ll be amazed. Process Explorer and Process Dump – are the two developers tools that you must be familiar with.
  • Windows SDK, where the WinDBG lies hidden Smile
  • Visual Studio
  • CLR Profiler
  • Any other third party tool like: Memory profiler from SciTech, Red Gates ANTS Memory profiler, JetBrains Profiler

Learn performance counters related to memory and performance. Don’t just say slow. It is not important. It is important to know why it is slow. Performance counters often can help answer that question.

Try some samples with those tools, read help materials and additional information on those, like:

Learn further

Read books like: “Windows Internals” and “Advanced Windows Debugging” (Recommended by @alejacma).

Start reading really interesting blogs on the subject, like:

I guess that’s it for todays session. I hope you liked it.

If you didn’t understood everything, well – don’t say that I didn’t warned you about session being @ 400 level Smile. Anyway, don’t worry – step by step you can learn those things fairly quickly. And sometimes, they can help a lot.

IIS responds with 403 using certificate based authentication

From time to time I have to deal with certificate based authentication, when developing WCF services and from time to time I’m falling into the same pit.

Today I was configuring WCF service to use certificates for authentication (via AD certificate mapping). After configuring the IIS and WCF I’ve tried to access the SVC help page/metadata, but was getting 403.7 Forbidden: Client certificate required from IIS. The IIS logs contained something like this:

<date time> W3SVC1 <IP> GET /site/service.svc – 443 – <IP> <Browser> 403 7 64

Bing came out with the support KB article on this issue, but all possible causes were dealt with: CA was trusted, certs were not revoked or expired. And then it hit me – “not expired”, yes, of course – how the IIS checks the revocation of the certificate? Simply by looking into the certificates CRL distribution points information (if it is present there) and it must be accessible and reachable from IIS, which hosts the service. To check if everything is ok, just copy CRL’s URL from the certificate and try to open it via browser on IIS hosting service.

In my situation that was the problem, which was fixed easily by entering appropriate DNS records.

Of course, it is possible to switch off the certificate revocation checking on IIS, but that’s completely NOT recommended.

Partner Conference: Windows Server AppFabric

I hope everyone had a great time yesterday during the Microsoft Partner Conference and enjoyed all the sessions you’ve attended and all the opportunities to talk to each other.

As I was giving a presentation on Windows Server AppFabric (not the cloudy one) to the developers. 45 minutes – is too small period of time to cover all the features and usage patterns good enough, so I have tried to bring the main ideas so that you can look after them later.

As a follow up, I’ve decided to put a list of reference materials available on the web so you can read those at your pace in comfort of whatever environment you like Smile

Windows Server AppFabric:

Other things I’ve mentioned during the presentation:

Yesterday, I’ve heard some comments that AppFabric is a complex beast and I must agree that it’s true in some way, but taking the other angle – it takes a lot of concerns regarding persistence, correlation, monitoring (incl. via SCOM) diagnostics, scalability, caching, management, tracking from developers head and makes it almost “out-of-the-box experience”. And from personal experience I know that developers like to talk about implementing business features, but hate even to think about “diagnostics” or “management”.

So, to sum up: consider Windows Server AppFabric as an “application server” that certainly requires attention and effort in understanding, setting up and configuration, but in return in can give a lot of “infrastructure” services like scale-out, persistence, management, monitoring, diagnostics, etc. for your services.

Stuff to read [8]

Posted by / October 1, 2010 / Posted in .Net

WCF WSDL in a single file

As always, there is something I need from time to time, but not frequently enough to have it close to me, so I have to search for it from time to time. I mean OK, bing is good, but still it takes some time, so I’m publishing this as a note to myself and maybe the others. One of such problems – to generate single WSDL file for WCF service. Mainly this is required if you have some interoperability issues.

Ok, so to have the single WSDL right way you need following things (assuming you’re hosting WCF in IIS):

So, its fairly easy to achieve in the end. And if you think what is else possible to do with custom behaviors, service hosts, etc. – you will realize how much WCF is rock’n’roll.

How to get files associated to specified work item from TFS

Today I’ve got a question on how to get files that are associated with specific work item. The requirement behind that was to grab configuration, data and similar files that were associated to a specific change request, which was entered as a work item.
After quick binging and browsing of the Team Foundation Server SDK, I’ve got the sample below.

static void Main(string[] args)
{
    using (TeamFoundationServer tfs = TeamFoundationServerFactory.GetServer(args[0],
        new UICredentialsProvider()))
    {
        tfs.EnsureAuthenticated();
        WorkItemStore wiStore = (WorkItemStore)tfs.GetService(typeof(WorkItemStore));
        VersionControlServer vcs = (VersionControlServer)tfs.GetService(typeof(VersionControlServer));

        int workItemId;
        if (int.TryParse(args[1], out workItemId))
        {
            WorkItem workItem = wiStore.GetWorkItem(workItemId);
            Console.WriteLine("Work item: {0}", workItem.Title);
            //We look for links associated with work item
            foreach (Link link in workItem.Links)
            {
                ExternalLink extLink = link as ExternalLink;
                if (extLink != null)
                {
                    ArtifactId artifact = LinkingUtilities.DecodeUri(extLink.LinkedArtifactUri);
                    //For this example I grab Changeset directly
                    //however in the real scenario you could grab other related workitems
                    //and this way parse entire tree up to the changeset
                    if (String.Equals(artifact.ArtifactType, "Changeset", StringComparison.Ordinal))
                    {
                        // Convert the artifact URI to Changeset object.
                        Changeset cs = vcs.ArtifactProvider.GetChangeset(new Uri(extLink.LinkedArtifactUri));
                        foreach (Change change in cs.Changes)
                        {
                            //We want to download files only
                            if (change.Item.ItemType == ItemType.File)
                            {
                                RetrieveFile(change);
                            }
                        }
                    }
                }
            }
            Console.ReadLine();
        }
    }
}

private static void RetrieveFile(Change change)
{
    //Technically we should process the information
    //i.e. you should create separate folders according to the server location
    //to avoid the file overwriting
    //if folders are not present - those should be created
    string fileName = change.Item.ServerItem.Split('/').Last();
    Console.WriteLine("Item name: {0}. Last operation {1}", fileName, change.ChangeType);
    change.Item.DownloadFile(@"C:MyProjectsVS10SolutionGetWIFilesttt" + fileName);
}

Here go the credits: http://blogs.msdn.com/buckh/archive/2006/08/12/artifact_uri_to_changeset.aspx
A quick reminder to myself (and the others): TFS is VERY extensible. You can extend the processes, work items, reports, functionality, … just need the right tools