.NET Core vs .NET Framework: Testing Performance

In recent years everyone in IT who is doing anything related to Microsoft .NET Framework knows, that there is tremendous pressure to move from the “old” complete .NET Framework to the new .NET Core Framework.

The pressure firstly came only from Microsoft but now also from the .NET community. You can feel that, especially when observing third party opensource projects. Slowly but steadily, they are slowing the development of full .NET Framework related code and are moving to the .NET Core framework.

A great example of such a third-party project is IdentityServer. Here, for example, the newest version (4.0) supports only .NET Core. If you want to use it in the full .NET Framework, then you must revert to the 3.0 release.

Trend from .NET Framework to .NET Core

So, the trend is clear. But what is the reason for that?

Ok, that is a very general question. But I don’t want to try to give the complete answer here. However, one of the biggest reasons I think is the performance gains the .NET Framework Core offers us. That is clear when comparing to the full .NET Framework.

As I will show later in this post, the gains can be very massive, ranging from 30%, up to 1200% in some situations. Such significant performance gains cannot be left unseen.

Here is where I’d like to share with my findings with you.

My accidental past discovery

Ok, so here is my story about this topic.

About a year or so, I was comparing the speed of hashing algorithms. I was to choose the fastest one for comparing application log files.

Algorithms

Usage had nothing to do with security, so I considered weaker algorithms such as MD5.

What I noticed was that the SHA256 algorithm was the fastest on my machine. It was much quicker than SHA1 or MD5. That result made little sense, as I know that from a computing perspective, that SHA256 is much harder than MD5.

Workstation

That comparison was one I made in a console application written in .NET Core 2.2. The base of my CPU in my workstation at the time was AMD Ryzen. Therefore the test was single-threaded with the CPU core number made irrelevant. I compared the results on an Intel-based workstation, with the results differing, entirely.

On an Intel-based CPU, the fastest algorithm was MD5. The next instance was SHA1, and then SHA256, which turned out to be the slowest. That totalled to being almost two or three times slower than MD5. That prompted me to start digging for the reason. What I found was that it was my workstation CPU.

All AMD processors based on Zen microarchitecture (Ryzen, ThreadRipper and EPYC) come with additional instructions. These are called IA SHA Extensions which most of the Intel-based processors simply lack. Ironically, Intel invented them.

.NET Core: developed with performance in mind

With my discovery in-mind, I did the same comparison using the full .NET Framework.

Results were consistent among all the CPUs, as SHA256 was slower than MD5, and SHA1 on both AMD and Intel-based systems. So, it was clear to me that the full .NET Framework was unable to take advantage of additional CPU features.

This was the first time I had discovered that .NET Core’s design must have been performance-centred. My suspicion then came true. I then started comparing the Frameworks having performance as a metric.

After this, I used the “BenchmarkDotNet” v12 library in a simple console application to test few compute-heavy operations between frameworks. I also carried this out between the different CPU architectures to find the differences. Thanks to that library, the statistics of the tests and reports were automatically generated.

Testing procedure

Here is the list of operations that were put into the test (the test was single-threaded):

1. EnumParse – parsing the string value to an enum value
2. LinqOrderBySkipFirst– ordering the reversed sequence of ten million integers, skipping 4 of them and then choosing the first element
3. Sha256– computing the SHA256 hash of the 100MB array
4. StringStartsWith– checking if the string “abcdefghijklmnopqrstuvwxyz” begins with a “abcdefghijklmnopqrstuvwxy-” value (repeated 100,000,000 times)
5. Deserialize– creating, serializing and deserializing a list of 100,000 objects (each containing two string properties holding an integer value) using BinaryFormatter

For testing, I used the following workstations:

1. AMD Ryzen 7 3700X (25GB RAM) – tests conducted in a virtual machine within VMWare ESXi 6.7U1 (build 13004448) hypervisor.
2. Intel Core i7-8665U (16GB RAM) – tests conducted on my current laptop.

All environments were using 64-bit Windows 10.0.18363. I compiled code in Release mode with enforced x64 architecture.

Test results

For more information, consult the results table (time measured in nanoseconds – lower values indicate better performance):

.NET Core vs .NET Framework: Conclusions

After analyzing the results and my experience with both frameworks, I came to the following conclusions:

The full .NET Framework is not able to leverage the full potential of the operating platform. It ignored additional available processor instructions, such as the IA SHA Extensions mentioned above.

.NET Core featured in all my tests much faster than the full .NET – sometimes 7 or even up to 13 times faster.

Choosing the right CPU architecture can dramatically change the behaviour of your application, so the results gathered from one architecture can be invalid on the other and vice versa. For example – if your .NET Core application performs much SHA256 hashing computations, then the best option would be to use AMD Zen-based servers instead of Intel-based ones.

So to sum up – if performance is a significant factor in your application, then you should opt for .NET Core vs .NET Framework.

Disclaimers

First disclaimer

Please note, that tests showed here focused only on Framework performance itself (there were almost no I/O operations involved like database access, file access or network access).

So, if your application is highly I/O dependent, then beware that you can see little to no gains from choosing the .NET Core vs .NET Framework.

Second disclaimer

The comparison between CPU architectures may contain some measurement error because I was comparing a desktop-grade CPU with a mobile grade CPU (simply because currently I have access only to those two CPUs).

The fairest comparison would be to compare, for example, the Ryzen 7 3800X with Core i7 9700k to minimize that error.

However, taking into account that the test was single-threaded, with both machines not have sporting background task running, the error should be relatively small with minimal effect on the bigger picture.