News

Developers Pit .NET Core 2.0 Performance Against Java, Go

• By David Ramel
• 08/22/2017

It didn't take long for developers to start putting Microsoft's new .NET Core. 2.0 release through its paces in benchmark tests against alternative languages.

Microsoft last week released .NET Core 2.0 (the open source, modular implementation of .NET) shortly after the companion .NET Standard 2.0 (for standardized API usage across all .NET implementations).

Also last week, two developers soon matched up .NET Core 2.0 against Java and Go in separate benchmarking tests to gauge its comparative performance in a variety of tasks.

Here are summations of each test.

.NET Core 2.0 vs Java -- Performance Notes
Anthony Lloyd, who blogs about F# development and other dev topics, has been working with the Computer Language Benchmarks Game, which has measured "C# .NET Core programs versus Java."

"When I first saw the C# .NET Core vs Java benchmarks the score was .NET Core 2.0 4 -- Java 6," he said in a blog post. "This surprised me as I was under the impression .NET Core 2.0 performance would be very good."

He looked at six benchmarking programs where Java was faster, made some changes to improve .NET Core 2.0 performance and related his notes and some conclusions. The testing details are too extensive for this article, but following are a couple conclusions.

"So now the score is .NET Core 2.0 6 -- Java 4," he said after running his improved benchmarking tests.

"My overall impression is that .NET Core 2.0 and Java perform about the same. Java possibly has a small edge on some calculations and array manipulation and .NET having better parallel libraries."

He also listed some caveats about his methodology and the effectiveness of such tests in general:

The test machine is a single 64-bit quad core machine. This may not represent current servers. I found optimizing for performance on a different machine to be very interesting. I tended to over optimize algorithm constants and need to look for more universal performance optimizations.

The performance benchmark testing process seems to be robust. There could be some bias in the configurations but I was not aware of any.

Fresh out of ideas about how to improve the tests further, he asked fellow developers to weigh in with any ideas on how to do that.

Go vs .NET Core in Terms of HTTP Performance
Self-taught developer Gerasimos Maropoulos had heard a lot about the performance of .NET Core -- especially on Web servers -- but he waited until the v. 2.0 release before conducting his tests against Go.

Using the Model-View-Controller (MVC) architecture, he measured the expected response and stability of the runtimes, so he used simple text messages rather than having to account for JSON or XML encoders. With few choices for Web frameworks for Go that support MVC, he settled on Iris. He described it as "A fast, simple and efficient micro Web framework for Go. It provides a beautifully expressive and easy to use foundation for your next Web site, API or distributed app."

Following are the metrics he used for his testing:

• Time to complete the 5,000,000 requests -- smaller is better.
• Reqs/sec. -- bigger is better.
• Latency -- smaller is better
• Throughput -- bigger is better.
• Memory usage -- smaller is better.
• LOC (Lines Of Code) -- smaller is better.

And here are his conclusions:

• .NET Core MVC Application, written using 86 lines of code, ran for 2 minutes and 8 seconds serving 39,311.56 requests per second within 3.19ms latency in average and 229.73ms max, the memory usage of all these was ~126MB (without the dotnet host).
• Iris MVC Application, written using 27 lines of code, ran for 47 seconds serving 105,643.71 requests per second within 1.18ms latency in average and 22.01ms max, the memory usage of all these was ~12MB.

Maropoulos also ran another benchmark test, this time incorporating HTML generated by templates via the view engine.

Here's his take on that test:

• .NET Core MVC with Templates Application ran for 1 minute and 20 seconds serving 11,738.60 requests per second with 89.03MB/s within 10.10ms latency in average and 1.97s max, the memory usage of all these was ~193MB (without the dotnet host).
• Iris MVC with Templates Application ran for 37 seconds serving 26,656.76 requests per second with 192.51MB/s within 1.18ms latency in average and 22.52ms max, the memory usage of all these was ~17MB.

Just yesterday, he announced he ran yet another test in view of developer requests to test .NET core's lower-level Kestrel, "a cross-platform HTTP server based on libuv, a cross-platform asynchronous I/O library."

You can see the complete results here, where he ran two different kinds of tests.

.NET Core 2.0 performance still lagged behind Iris, but not as much. That led Maropoulos to conclude: "We really have to admit that Kestrel runs pretty fast compared to its MVC side, congrats to the open source project's contributors and Microsoft for started and supporting that!"

Developers examining such tests should take Lloyd's advice to heart: "No toy benchmarks truly represent the performance of a large application," he said. However, regarding his .NET Core 2.0/Java testing, he said, "These benchmarks do look to solve larger real-world problems than most I have seen."