Micro-benchmarks

I was looking for a way to benchmark a piece of code… I came up with 5 libraries that make it very easy 🙂

I’m not going to write a tutorial on how to use each one because I would basically be rewriting their documentation sections. What I will do is show you how to get started. As an example I will write a simple benchmark that tests copy constructor of

std::string

. But first, the libraries:

1. Google Benchmark
2. Catch2
3. Hayai
4. Celero
5. Nonius

Catch2 and Nonius are header only libraries; be aware of long compile times 🙁 Google, Catch2, and Nonius automatically pick the number of runs and iterations for you, which is nice: no guessing how many times you need to run a function you want to benchmark to get a reasonable performance reading.

---

Google Benchmark:

#include 
#include 

using namespace std;

string short_string = "hello";
string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

void BM_StringCopyShort(benchmark::State& state)
{
	for (auto _ : state)
		string copy(short_string);
}
BENCHMARK(BM_StringCopyShort);

void BM_StringCopyLong(benchmark::State& state)
{
	for (auto _ : state)
		string copy(long_string);
}
BENCHMARK(BM_StringCopyLong);

BENCHMARK_MAIN();

---

---

Catch2:

#include 
#define CATCH_CONFIG_MAIN
#include 

using namespace std;

string short_string = "hello";
string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

TEST_CASE("string ops", "[benchmark]")
{
	BENCHMARK("StringCopyShort")
	{
		string copy(short_string);
	}

	BENCHMARK("StringCopyLong")
	{
		string copy(long_string);
	}
}

---

---

Hayai:

#include 
#include 

using namespace std;

string short_string = "hello";
string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

BENCHMARK(StringOps, StringCopyShort, 100, 1'000'000)
{
	string copy(short_string);
}

BENCHMARK(StringOps, StringCopyLong, 100, 100'000)
{
	string copy(long_string);
}

---

---

Celero:

#include 
#include 

using namespace std;

string short_string = "hello";
string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

BASELINE(StringCopyShort, Baseline10, 100, 100'000)
{
	string copy(short_string);
}

BENCHMARK(StringCopyShort, Baseline1000, 100, 1'000'000)
{
	string copy(short_string);
}

BASELINE(StringCopyLong, Baseline10, 100, 100'000)
{
	string copy(long_string);
}

BENCHMARK(StringCopyLong, Baseline1000, 100, 1'000'000)
{
	string copy(long_string);
}

CELERO_MAIN;

---

---

Nonius:

#include 
#define NONIUS_RUNNER
#include 
#include 

using namespace std;

string short_string = "hello";
string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

NONIUS_BENCHMARK("StringCopyShort", []
{
	string copy(short_string);
})

NONIUS_BENCHMARK("StringCopyLong", []
{
	string copy(long_string);
})

---

---