2019 | Vorbrodt's C++ Blog

Propagate exceptions across threads

Posted on March 24, 2019March 28, 2021 by Martin

What if you need to catch an exception in a worker thread and re-throw it in the main thread that’s waiting for the worker to finish? std::future works this way. If you spawn a future on a new thread using std::async(std::launch::async, ...); and that future’s worker throws an exception, when you later call get() on the future it will emit that exception.

You do it by wrapping the worker thread’s function in try { /* CODE */ } catch(...) {} and capturing the current exception pointer ( std::exception_ptr) using std::current_exception. You can then re-throw the captured exception using the pointer and std::rethrow_exception. Below is an example that illustrates this technique. Just remember, if you have multiple worker threads make sure to have multiple std::exception_ptr instances; one per worker thread.

exceptions.cpp:

#include <iostream>
#include <sstream>
#include <thread>
#include <exception>
#include <stdexcept>

using namespace std;

int main(int, char**)
{
	exception_ptr ep = nullptr;

	thread t([&]()
	{
		try
		{
			stringstream str;
			str << this_thread::get_id();
			throw runtime_error(str.str().c_str());
		}
		catch(...)
		{
			ep = current_exception();
		}
	});
	t.join();

	if(ep != nullptr)
	{
		try
		{
			rethrow_exception(ep);
		}
		catch(exception& e)
		{
			cout << "Thread " << this_thread::get_id() << " caught exception from thread " << e.what() << endl;
		}
	}
}

#include <iostream>

#include <sstream>

#include <thread>

#include <exception>

#include <stdexcept>

using namespace std;

int main(int, char**)

{

exception_ptr ep = nullptr;

thread t([&]()

{

try

{

stringstream str;

str << this_thread::get_id();

throw runtime_error(str.str().c_str());

}

catch(...)

{

ep = current_exception();

}

});

t.join();

if(ep != nullptr)

{

try

{

rethrow_exception(ep);

}

catch(exception& e)

{

cout << "Thread " << this_thread::get_id() << " caught exception from thread " << e.what() << endl;

}

Thread 0x1048d75c0 caught exception from thread 0x700001cf3000
Program output.

int main()

Posted on March 24, 2019March 28, 2021 by Martin

I have been spanked by certain commenter (who shall not remain anonymous 😉 ) on here and FB about my style of naming unused main arguments and unnecessary return 1; at the end of every main function.

I have though about and I… concede the point of his argument 🙂 From now on the style on this blog shall be as follows (if arguments to main are not needed):

int main()
{
    // CODE...
}

int main()

{

// CODE...

}

P.S. C++ standard allows for two valid signatures: int main() and int main(int argc, char** argv), see here.

XML-RPC

Posted on March 23, 2019March 23, 2019 by Martin

XML-RPC is yet another method of implementing remote procedure calls. It used XML over HTTP to transmit data. In my past live working at TLO I used XML-RPC-C library to implement communication between cluster nodes and a cluster management system. I thought the library was well designed and easy to use so I wanted to introduce you to it.

Below is a simple client and server implementation using the XML-RPC-C library. The server implements one RPC that accepts one string parameter and returns one string. The client makes the call to the server saying hello and prints the reply. The code is easy to read and does not need any further explanation 🙂

The client. xmlrpc_c.cpp:

#include <iostream>
#include <string>
#include <xmlrpc-c/girerr.hpp>
#include <xmlrpc-c/base.hpp>
#include <xmlrpc-c/client_simple.hpp>

using namespace std;

int main(int argc, char** argv)
{
	string serverUrl("http://localhost:8080/RPC2");
	string methodName("hello");

	xmlrpc_c::clientSimple client;
	xmlrpc_c::value result;

	client.call(serverUrl, methodName, "s", &result, "XMLRPC client says hello!");

	auto reply = xmlrpc_c::value_string(result);

	cout << static_cast<string>(reply) << endl;

	return 1;
}

#include <iostream>

#include <string>

#include <xmlrpc-c/girerr.hpp>

#include <xmlrpc-c/base.hpp>

#include <xmlrpc-c/client_simple.hpp>

using namespace std;

int main(int argc, char** argv)

{

string serverUrl("http://localhost:8080/RPC2");

string methodName("hello");

xmlrpc_c::clientSimple client;

xmlrpc_c::value result;

client.call(serverUrl, methodName, "s", &result, "XMLRPC client says hello!");

auto reply = xmlrpc_c::value_string(result);

cout << static_cast<string>(reply) << endl;

return 1;

}

The server. xmlrpc_s.cpp:

#include <iostream>
#include <string>
#include <stdexcept>
#include <xmlrpc-c/base.hpp>
#include <xmlrpc-c/registry.hpp>
#include <xmlrpc-c/server_abyss.hpp>

using namespace std;

class hello : public xmlrpc_c::method
{
public:
	void execute(const xmlrpc_c::paramList& params, xmlrpc_c::value* retval)
	{
		string msg(params.getString(0));
		params.verifyEnd(1);

		cout << msg << endl;

		*retval = xmlrpc_c::value_string("XMLRPC server says hello!");
	}
};

int main(int argc, char** argv)
{
	xmlrpc_c::registry registry;
	registry.addMethod("hello", new hello);

	xmlrpc_c::serverAbyss server(xmlrpc_c::serverAbyss::constrOpt().registryP(&registry).portNumber(8080));
	server.run();

	return 1;
}

#include <iostream>

#include <string>

#include <stdexcept>

#include <xmlrpc-c/base.hpp>

#include <xmlrpc-c/registry.hpp>

#include <xmlrpc-c/server_abyss.hpp>

using namespace std;

class hello : public xmlrpc_c::method

{

public:

void execute(const xmlrpc_c::paramList& params, xmlrpc_c::value* retval)

{

string msg(params.getString(0));

params.verifyEnd(1);

cout << msg << endl;

*retval = xmlrpc_c::value_string("XMLRPC server says hello!");

}

};

int main(int argc, char** argv)

{

xmlrpc_c::registry registry;

registry.addMethod("hello", new hello);

xmlrpc_c::serverAbyss server(xmlrpc_c::serverAbyss::constrOpt().registryP(&registry).portNumber(8080));

server.run();

return 1;

}

Base64 encoding

Posted on March 23, 2019March 23, 2019 by Martin

Base64 encoding: turning binary data into ASCII text for the purpose of saving it to text files like XML, or transmitting it over protocols like HTTP, or embedding it into web page files, and many other purposed. That’s the basic idea behind it. For every 3 bytes of input you get 4 bytes of output, so it’s not crazy inflated.

I was looking for a library that has built in, easy to use and clean base64 encoding and decoding functions but didn’t really find anything to my liking. So I looked for a reference implementation and found one at wikibooks.org. Their C++ implementation (released to public domain so I could freely use and modify it) was my starting point. I beautified the code and brought it closer to modern C++ 🙂 So now you have a header only, clean base64 encode and decode functions you can use in your projects: base64.hpp.

I wrote a program that encodes and decodes an input string, checks the original against the decoded one, and also checks the encoded base64 text against reference base64 string taken from wiki. The implementation checks out and produces correct encoded strings and decoded data 🙂 Below is the test program.

base64.cpp:

#include <iostream>
#include <string>
#include <vector>
#include <iterator>
#include "base64.hpp"
#include "ascii_escape_code.hpp"

using namespace std;
using namespace ascii_escape_code;

int main(int argc, char** argv)
{
	string input
	{
		"Man is distinguished, not only by his reason, but by this singular passion from other animals, "
		"which is a lust of the mind, that by a perseverance of delight in the continued and indefatigable "
		"generation of knowledge, exceeds the short vehemence of any carnal pleasure."
	};
	string reference
	{
		"TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyByZWFzb24sIGJ1dCBieSB0aGlz"
		"IHNpbmd1bGFyIHBhc3Npb24gZnJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2Yg"
		"dGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaWdodCBpbiB0aGUgY29udGlu"
		"dWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdlbmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRo"
		"ZSBzaG9ydCB2ZWhlbWVuY2Ugb2YgYW55IGNhcm5hbCBwbGVhc3VyZS4="
	};

	try
	{
		vector<base64::byte> data(begin(input), end(input));

		cout << bold << bright_red << "Input: " << reset << italic << input << reset << endl << endl;
		cout << bold << bright_red << "Reference: " << reset << reference << endl << endl;

		auto encoded = base64::encode(data);
		cout << bold << bright_red << "Encoded: " << reset << encoded << endl << endl;

		if(encoded != reference) throw runtime_error("Oh snap! Encoded data does not match reference!");
		else cout << bright_red << "Encoded data matches reference :o)" << reset << endl << endl;

		auto decoded = base64::decode(encoded);
		string s2(begin(decoded), end(decoded));

		cout << bold << bright_red << "Decoded: " << reset << italic << s2 << reset << endl << endl;

		if(data != decoded) throw runtime_error("Oh snap! Input data does not match decoded!");
		else cout << bright_red << "Decoded data matches original :o)" << reset << endl << endl;
	}
	catch(exception& e)
	{
		cerr << slow_blink << bright_red << e.what() << reset << endl << endl;
	}

	return 1;
}

#include <iostream>

#include <string>

#include <vector>

#include <iterator>

#include "base64.hpp"

#include "ascii_escape_code.hpp"

using namespace std;

using namespace ascii_escape_code;

int main(int argc, char** argv)

{

string input

{

"Man is distinguished, not only by his reason, but by this singular passion from other animals, "

"which is a lust of the mind, that by a perseverance of delight in the continued and indefatigable "

"generation of knowledge, exceeds the short vehemence of any carnal pleasure."

};

string reference

{

"TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyByZWFzb24sIGJ1dCBieSB0aGlz"

"IHNpbmd1bGFyIHBhc3Npb24gZnJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2Yg"

"dGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaWdodCBpbiB0aGUgY29udGlu"

"dWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdlbmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRo"

"ZSBzaG9ydCB2ZWhlbWVuY2Ugb2YgYW55IGNhcm5hbCBwbGVhc3VyZS4="

};

try

{

vector<base64::byte> data(begin(input), end(input));

cout << bold << bright_red << "Input: " << reset << italic << input << reset << endl << endl;

cout << bold << bright_red << "Reference: " << reset << reference << endl << endl;

auto encoded = base64::encode(data);

cout << bold << bright_red << "Encoded: " << reset << encoded << endl << endl;

if(encoded != reference) throw runtime_error("Oh snap! Encoded data does not match reference!");

else cout << bright_red << "Encoded data matches reference :o)" << reset << endl << endl;

auto decoded = base64::decode(encoded);

string s2(begin(decoded), end(decoded));

cout << bold << bright_red << "Decoded: " << reset << italic << s2 << reset << endl << endl;

if(data != decoded) throw runtime_error("Oh snap! Input data does not match decoded!");

else cout << bright_red << "Decoded data matches original :o)" << reset << endl << endl;

}

catch(exception& e)

{

cerr << slow_blink << bright_red << e.what() << reset << endl << endl;

}

return 1;

}

Input: Man is distinguished, not only by his reason, but by this singular passion from other animals, which is a lust of the mind, that by a perseverance of delight in the continued and indefatigable generation of knowledge, exceeds the short vehemence of any carnal pleasure.

Reference:
TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyByZWFzb24sIGJ 1dCBieSB0aGlzIHNpbmd1bGFyIHBhc3Npb24gZnJvbSBvdGhlciBhbmltYWxzLCB3 aGljaCBpcyBhIGx1c3Qgb2YgdGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmF uY2Ugb2YgZGVsaWdodCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWd hYmxlIGdlbmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9ydC B2ZWhlbWVuY2Ugb2YgYW55IGNhcm5hbCBwbGVhc3VyZS4=

Encoded:
TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyByZWFzb24sIGJ 1dCBieSB0aGlzIHNpbmd1bGFyIHBhc3Npb24gZnJvbSBvdGhlciBhbmltYWxzLCB3 aGljaCBpcyBhIGx1c3Qgb2YgdGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmF uY2Ugb2YgZGVsaWdodCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWd hYmxlIGdlbmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9ydC B2ZWhlbWVuY2Ugb2YgYW55IGNhcm5hbCBwbGVhc3VyZS4=

Encoded data matches reference :o)

Decoded: Man is distinguished, not only by his reason, but by this singular passion from other animals, which is a lust of the mind, that by a perseverance of delight in the continued and indefatigable generation of knowledge, exceeds the short vehemence of any carnal pleasure.

Decoded data matches original :o)
Program output.

And here is the encoder and decoder function implementation.

base64.hpp:

#pragma once

#include <string>
#include <vector>
#include <stdexcept>
#include <cstdint>

namespace base64
{
	inline static const char kEncodeLookup[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
	inline static const char kPadCharacter = '=';

	using byte = std::uint8_t;

	inline std::string encode(const std::vector<byte>& input)
	{
		std::string encoded;
		encoded.reserve(((input.size() / 3) + (input.size() % 3 > 0)) * 4);

		std::uint32_t temp{};
		auto it = input.begin();

		for(std::size_t i = 0; i < input.size() / 3; ++i)
		{
			temp  = (*it++) << 16;
			temp += (*it++) << 8;
			temp += (*it++);
			encoded.append(1, kEncodeLookup[(temp & 0x00FC0000) >> 18]);
			encoded.append(1, kEncodeLookup[(temp & 0x0003F000) >> 12]);
			encoded.append(1, kEncodeLookup[(temp & 0x00000FC0) >> 6 ]);
			encoded.append(1, kEncodeLookup[(temp & 0x0000003F)      ]);
		}

		switch(input.size() % 3)
		{
		case 1:
			temp = (*it++) << 16;
			encoded.append(1, kEncodeLookup[(temp & 0x00FC0000) >> 18]);
			encoded.append(1, kEncodeLookup[(temp & 0x0003F000) >> 12]);
			encoded.append(2, kPadCharacter);
			break;
		case 2:
			temp  = (*it++) << 16;
			temp += (*it++) << 8;
			encoded.append(1, kEncodeLookup[(temp & 0x00FC0000) >> 18]);
			encoded.append(1, kEncodeLookup[(temp & 0x0003F000) >> 12]);
			encoded.append(1, kEncodeLookup[(temp & 0x00000FC0) >> 6 ]);
			encoded.append(1, kPadCharacter);
			break;
		}

		return encoded;
	}

	std::vector<byte> decode(const std::string& input)
	{
		if(input.length() % 4)
			throw std::runtime_error("Invalid base64 length!");

		std::size_t padding{};

		if(input.length())
		{
			if(input[input.length() - 1] == kPadCharacter) padding++;
			if(input[input.length() - 2] == kPadCharacter) padding++;
		}

		std::vector<byte> decoded;
		decoded.reserve(((input.length() / 4) * 3) - padding);

		std::uint32_t temp{};
		auto it = input.begin();

		while(it < input.end())
		{
			for(std::size_t i = 0; i < 4; ++i)
			{
				temp <<= 6;
				if     (*it >= 0x41 && *it <= 0x5A) temp |= *it - 0x41;
				else if(*it >= 0x61 && *it <= 0x7A) temp |= *it - 0x47;
				else if(*it >= 0x30 && *it <= 0x39) temp |= *it + 0x04;
				else if(*it == 0x2B)                temp |= 0x3E;
				else if(*it == 0x2F)                temp |= 0x3F;
				else if(*it == kPadCharacter)
				{
					switch(input.end() - it)
					{
					case 1:
						decoded.push_back((temp >> 16) & 0x000000FF);
						decoded.push_back((temp >> 8 ) & 0x000000FF);
						return decoded;
					case 2:
						decoded.push_back((temp >> 10) & 0x000000FF);
						return decoded;
					default:
						throw std::runtime_error("Invalid padding in base64!");
					}
				}
				else throw std::runtime_error("Invalid character in base64!");

				++it;
			}

			decoded.push_back((temp >> 16) & 0x000000FF);
			decoded.push_back((temp >> 8 ) & 0x000000FF);
			decoded.push_back((temp      ) & 0x000000FF);
		}

		return decoded;
	}
}

100

101

102

103

104

105

106

107

108

109

110

111

#pragma once

#include <string>

#include <vector>

#include <stdexcept>

#include <cstdint>

namespace base64

{

inline static const char kEncodeLookup[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";

inline static const char kPadCharacter = '=';

using byte = std::uint8_t;

inline std::string encode(const std::vector<byte>& input)

{

std::string encoded;

encoded.reserve(((input.size() / 3) + (input.size() % 3 > 0)) * 4);

std::uint32_t temp{};

auto it = input.begin();

for(std::size_t i = 0; i < input.size() / 3; ++i)

{

temp = (*it++) << 16;

temp += (*it++) << 8;

temp += (*it++);

encoded.append(1, kEncodeLookup[(temp & 0x00FC0000) >> 18]);

encoded.append(1, kEncodeLookup[(temp & 0x0003F000) >> 12]);

encoded.append(1, kEncodeLookup[(temp & 0x00000FC0) >> 6 ]);

encoded.append(1, kEncodeLookup[(temp & 0x0000003F) ]);

}

switch(input.size() % 3)

{

case 1:

temp = (*it++) << 16;

encoded.append(1, kEncodeLookup[(temp & 0x00FC0000) >> 18]);

encoded.append(1, kEncodeLookup[(temp & 0x0003F000) >> 12]);

encoded.append(2, kPadCharacter);

break;

case 2:

temp = (*it++) << 16;

temp += (*it++) << 8;

encoded.append(1, kEncodeLookup[(temp & 0x00FC0000) >> 18]);

encoded.append(1, kEncodeLookup[(temp & 0x0003F000) >> 12]);

encoded.append(1, kEncodeLookup[(temp & 0x00000FC0) >> 6 ]);

encoded.append(1, kPadCharacter);

break;

}

return encoded;

}

std::vector<byte> decode(const std::string& input)

{

if(input.length() % 4)

throw std::runtime_error("Invalid base64 length!");

std::size_t padding{};

if(input.length())

{

if(input[input.length() - 1] == kPadCharacter) padding++;

if(input[input.length() - 2] == kPadCharacter) padding++;

}

std::vector<byte> decoded;

decoded.reserve(((input.length() / 4) * 3) - padding);

std::uint32_t temp{};

auto it = input.begin();

while(it < input.end())

{

for(std::size_t i = 0; i < 4; ++i)

{

temp <<= 6;

if (*it >= 0x41 && *it <= 0x5A) temp |= *it - 0x41;

else if(*it >= 0x61 && *it <= 0x7A) temp |= *it - 0x47;

else if(*it >= 0x30 && *it <= 0x39) temp |= *it + 0x04;

else if(*it == 0x2B) temp |= 0x3E;

else if(*it == 0x2F) temp |= 0x3F;

else if(*it == kPadCharacter)

{

switch(input.end() - it)

{

case 1:

decoded.push_back((temp >> 16) & 0x000000FF);

decoded.push_back((temp >> 8 ) & 0x000000FF);

return decoded;

case 2:

decoded.push_back((temp >> 10) & 0x000000FF);

return decoded;

default:

throw std::runtime_error("Invalid padding in base64!");

}

else throw std::runtime_error("Invalid character in base64!");

++it;

}

decoded.push_back((temp >> 16) & 0x000000FF);

decoded.push_back((temp >> 8 ) & 0x000000FF);

decoded.push_back((temp ) & 0x000000FF);

}

return decoded;

}

Extremely Fast Compression Algorithm

Posted on March 22, 2019March 22, 2019 by Martin

LZ4. GitHub repository here. It is open-source, available on pretty much every platform, and widely used in the industry.

It was extremely easy to get started with it. The C API could not possibly be any simpler (I’m looking at you zlib 😛 ); you pass in 4 parameters to the compression and decompression functions: input buffer, input length, output buffer, and max output length. They return either the number of bytes produced on the output, or an error code. Just be careful when compressing random data (which you should not be doing anyways!): the output is larger than the input!

Here’s a short example that compresses a vector of thousand characters:

compression.cpp:

#include <iostream>
#include <vector>
#include <lz4.h>

using namespace std;

using buffer = vector<char>;

void lz4_compress(const buffer& in, buffer& out)
{
	auto rv = LZ4_compress_default(in.data(), out.data(), in.size(), out.size());
	if(rv < 1) cerr << "Something went wrong!" << endl;
	else out.resize(rv);
}

void lz4_decompress(const buffer& in, buffer& out)
{
	auto rv = LZ4_decompress_safe(in.data(), out.data(), in.size(), out.size());
	if(rv < 1) cerr << "Something went wrong!" << endl;
	else out.resize(rv);
}

int main(int argc, char** argv)
{
	buffer data(1000, 'X');
	buffer compressed(data.size()), decompressed(data.size());

	lz4_compress(data, compressed);
	cout << "LZ4 compress, bytes in: " << data.size() << ", bytes out: " << compressed.size() << endl;

	lz4_decompress(compressed, decompressed);
	cout << "LZ4 decompress, bytes in: " << compressed.size() << ", bytes out: " << decompressed.size() << endl;

	if(data != decompressed) cerr << "Oh snap! Data mismatch!" << endl;
	else cout << "Decompressed data matches original :o)" << endl;

	return 1;
}

#include <iostream>

#include <vector>

#include <lz4.h>

using namespace std;

using buffer = vector<char>;

void lz4_compress(const buffer& in, buffer& out)

{

auto rv = LZ4_compress_default(in.data(), out.data(), in.size(), out.size());

if(rv < 1) cerr << "Something went wrong!" << endl;

else out.resize(rv);

}

void lz4_decompress(const buffer& in, buffer& out)

{

auto rv = LZ4_decompress_safe(in.data(), out.data(), in.size(), out.size());

if(rv < 1) cerr << "Something went wrong!" << endl;

else out.resize(rv);

}

int main(int argc, char** argv)

{

buffer data(1000, 'X');

buffer compressed(data.size()), decompressed(data.size());

lz4_compress(data, compressed);

cout << "LZ4 compress, bytes in: " << data.size() << ", bytes out: " << compressed.size() << endl;

lz4_decompress(compressed, decompressed);

cout << "LZ4 decompress, bytes in: " << compressed.size() << ", bytes out: " << decompressed.size() << endl;

if(data != decompressed) cerr << "Oh snap! Data mismatch!" << endl;

else cout << "Decompressed data matches original :o)" << endl;

return 1;

}

LZ4 compress, bytes in: 1000, bytes out: 14
LZ4 decompress, bytes in: 14, bytes out: 1000
Decompressed data matches original :o)
Program output.

Parsing command line options

Posted on March 21, 2019March 28, 2021 by Martin

In case you haven’t noticed, I love Boost 😛 so I’m going to introduce you to its Program Options library. It is a command line options parser; it can also read options from an INI file, and it works with STL containers. It supports showing a help message, setting default values for missing options, allows multiple instances of the same option, etc. For complete set of features I refer you to the documentation. This post will be a short introduction.

Let’s start with some code that defines what options our program will accept:

desc.add_options()
	("help", "produce help message")
	("int,i", value<int>(&v)->default_value(42), "int value")
	("float,f", value<float>(&f)->default_value(3.141f), "float value")
	("string,s", value<string>(&s)->default_value("Vorbrodt"), "string value")
	("int_list,a", value<vector<int>>(&vi), "list of int values")
	("string_list,b", value<vector<string>>(&vs), "list of string values")
;

desc.add_options()

("help", "produce help message")

("int,i", value<int>(&v)->default_value(42), "int value")

("float,f", value<float>(&f)->default_value(3.141f), "float value")

("string,s", value<string>(&s)->default_value("Vorbrodt"), "string value")

("int_list,a", value<vector<int>>(&vi), "list of int values")

("string_list,b", value<vector<string>>(&vs), "list of string values")

;

The first option invoked with --help will display a friendly description, like this:

Allowed options:
–help produce help message
-i [ –int ] arg (=42) int value
-f [ –float ] arg (=3.14100003) float value
-s [ –string ] arg (=Vorbrodt) string value
-a [ –int_list ] arg list of int values
-b [ –string_list ] arg list of string values
Options help message.

Next is an integer value option; the first string "int,i" means that you can specify it as either --int or -i on the command line. When specified its value will be pulled into variable v, and if not specified the default value will be 42. Next two options for float and string behave in the exact same way. The parser will throw an exception if you specify those options more than once.
Next are list options: they allow you to specify the same option multiple times and are returned through the vi and vs variables which are std::vector’s of int and string.

Here is the program invocation and the output it produces:

./bin/options -i 1 -f 3.141 -s “Martin” -a 10 -a 11 -a 12 -b “Vorbrodt’s” -b “Blog”
Int value was set to 1
Float value was set to 3.141
String value was set to “Martin”
List of ints value was set to 10
List of ints value was set to 11
List of ints value was set to 12
List of strings value was set to “Vorbrodt’s”
List of strings value was set to “Blog”
Program invocation and output.

Complete source code of the program below.

options.cpp:

#include <iostream>
#include <vector>
#include <string>
#include <boost/program_options.hpp>

using namespace std;
using namespace boost::program_options;

int main(int argc, char** argv)
{
	int v{};
	float f{};
	string s{};
	vector<int> vi{};
	vector<string> vs{};

	options_description desc("Allowed options");
	desc.add_options()
		("help", "produce help message")
		("int,i", value<int>(&v)->default_value(42), "int value")
		("float,f", value<float>(&f)->default_value(3.141f), "float value")
		("string,s", value<string>(&s)->default_value("Vorbrodt"), "string value")
		("int_list,a", value<vector<int>>(&vi), "list of int values")
		("string_list,b", value<vector<string>>(&vs), "list of string values");

	variables_map vm{};
	store(parse_command_line(argc, argv, desc), vm);
	notify(vm);

	if (vm.size() == 0 || vm.count("help"))
	{
		cout << desc << "\n";
		return 1;
	}

	if(vm.count("int")) cout << "Int value was set to " << v << endl;
	if(vm.count("float")) cout << "Float value was set to " << f << endl;
	if(vm.count("string")) cout << "String value was set to \"" << s << "\"" << endl;
	if(vm.count("int_list"))
		for(auto it : vi)
			cout << "List of ints value was set to " << it << endl;
	if(vm.count("string_list"))
		for(auto& it : vs)
			cout << "List of strings value was set to \"" << it << "\"" << endl;

	return 1;
}

#include <iostream>

#include <vector>

#include <string>

#include <boost/program_options.hpp>

using namespace std;

using namespace boost::program_options;

int main(int argc, char** argv)

{

int v{};

float f{};

string s{};

vector<int> vi{};

vector<string> vs{};

options_description desc("Allowed options");

desc.add_options()

("help", "produce help message")

("int,i", value<int>(&v)->default_value(42), "int value")

("float,f", value<float>(&f)->default_value(3.141f), "float value")

("string,s", value<string>(&s)->default_value("Vorbrodt"), "string value")

("int_list,a", value<vector<int>>(&vi), "list of int values")

("string_list,b", value<vector<string>>(&vs), "list of string values");

variables_map vm{};

store(parse_command_line(argc, argv, desc), vm);

notify(vm);

if (vm.size() == 0 || vm.count("help"))

{

cout << desc << "\n";

return 1;

}

if(vm.count("int")) cout << "Int value was set to " << v << endl;

if(vm.count("float")) cout << "Float value was set to " << f << endl;

if(vm.count("string")) cout << "String value was set to \"" << s << "\"" << endl;

if(vm.count("int_list"))

for(auto it : vi)

cout << "List of ints value was set to " << it << endl;

if(vm.count("string_list"))

for(auto& it : vs)

cout << "List of strings value was set to \"" << it << "\"" << endl;

return 1;

}

ANSI escape codes

Posted on March 21, 2019April 11, 2021 by Martin

ANSI escape codes are a way to do more than just plain text in the terminal (be it Windows cmd.exe or UNIX xterm). A picture is worth a thousand words so here’s what I was able to do with them:

All of the text appearance manipulation and coloring was done using a tiny library I wrote yesterday and a very intuitive C++ syntax. Here’s the code responsible for the screenshot above:

colors.cpp:

#include <iostream>
#include "ansi_escape_codes.hpp"

using namespace std;
using namespace ansi_escape_codes;

int main(int argc, char** argv)
{
	cout << bold << "BOLD" << reset << endl;
	cout << faint << "FAINT" << reset << endl;
	cout << italic << "ITALIC" << reset << endl;
	cout << underline << "UNDERLINE" << reset << endl;
	cout << slow_blink << "SLOW BLINK" << reset << endl;
	cout << inverse << "REVERSE" << reset << endl;
	cout << endl;

	for(int n = 0; n <= 255; ++n)
		cout << color_n(n) << "X";
	cout << reset << endl << endl;

	for(int n = 0; n <= 255; ++n)
		cout << color_bg_n(n) << "X";
	cout << reset << endl << endl;

	for(int r = 0; r <= 255; r+=32)
		for(int g = 0; g <= 255; g+=32)
			for(int b = 0; b <= 255; b+=32)
				cout << color_rgb(r, g, b) << "X";
	cout << reset <<endl << endl;

	for(int r = 0; r <= 255; r+=32)
		for(int g = 0; g <= 255; g+=32)
			for(int b = 0; b <= 255; b+=32)
				cout << color_bg_rgb(r, g, b) << "X";
	cout << reset <<endl << endl;

	return 1;
}

#include <iostream>

#include "ansi_escape_codes.hpp"

using namespace std;

using namespace ansi_escape_codes;

int main(int argc, char** argv)

{

cout << bold << "BOLD" << reset << endl;

cout << faint << "FAINT" << reset << endl;

cout << italic << "ITALIC" << reset << endl;

cout << underline << "UNDERLINE" << reset << endl;

cout << slow_blink << "SLOW BLINK" << reset << endl;

cout << inverse << "REVERSE" << reset << endl;

cout << endl;

for(int n = 0; n <= 255; ++n)

cout << color_n(n) << "X";

cout << reset << endl << endl;

for(int n = 0; n <= 255; ++n)

cout << color_bg_n(n) << "X";

cout << reset << endl << endl;

for(int r = 0; r <= 255; r+=32)

for(int g = 0; g <= 255; g+=32)

for(int b = 0; b <= 255; b+=32)

cout << color_rgb(r, g, b) << "X";

cout << reset <<endl << endl;

for(int r = 0; r <= 255; r+=32)

for(int g = 0; g <= 255; g+=32)

for(int b = 0; b <= 255; b+=32)

cout << color_bg_rgb(r, g, b) << "X";

cout << reset <<endl << endl;

return 1;

}

Pretty easy to follow I hope 🙂 The library defines a bunch of stream manipulators that inject the appropriate escape sequences. For example, cout << bold << "BOLD"; will print out, you guessed it, bolded text. color_n picks from the 8 bit color table. color_rgb let’s you define a 24 bit truecolor. The _bg_ version is for selecting the background color. Here’s the complete source code for the library, hope you will find it useful!

P.S. The color_rgb does not appear to work on Mac OS terminal. So far all the codes work only on Linux; haven’t tested on Windows 🙂

ansi_escape_code.hpp:

#pragma once

#include <ostream>

namespace ansi_escape_codes {

#define ENTRY(name, code) \
	static inline struct name##_opcode {} name; \
	inline std::ostream& operator << (std::ostream& os, const name##_opcode&) \
	{ os << "\x1b[" #code "m"; return os; }

ENTRY(reset, 0)
ENTRY(bold, 1)
ENTRY(faint, 2)
ENTRY(italic, 3)
ENTRY(underline, 4)
ENTRY(slow_blink, 5)
ENTRY(rapid_blink, 6)
ENTRY(inverse, 7)
ENTRY(conceal, 8)
ENTRY(crossed_out, 9)
ENTRY(default_font, 10)
ENTRY(alt_font_1, 11)
ENTRY(alt_font_2, 12)
ENTRY(alt_font_3, 13)
ENTRY(alt_font_4, 14)
ENTRY(alt_font_5, 15)
ENTRY(alt_font_6, 16)
ENTRY(alt_font_7, 17)
ENTRY(alt_font_8, 18)
ENTRY(alt_font_9, 19)
ENTRY(fraktur, 20)
ENTRY(doubly_underline, 21)
ENTRY(normal, 22)
ENTRY(not_italic_not_raktur, 23)
ENTRY(underline_off, 24)
ENTRY(blink_off, 25)
ENTRY(inverse_off, 27)
ENTRY(reveal, 28)
ENTRY(not_crossed_out, 29)
ENTRY(default_foreground_color, 39)
ENTRY(default_background_color, 49)
ENTRY(framed, 51)
ENTRY(encircled, 52)
ENTRY(overlined, 53)
ENTRY(not_framed_or_encircled, 54)
ENTRY(not_overlined, 55)

#define COLOR_ENTRY(name, code) \
	static inline struct name##_color {} name; \
	inline std::ostream& operator << (std::ostream& os, const name##_color&) \
	{ os << "\x1b[" #code "m"; return os; }

COLOR_ENTRY(black,   30)
COLOR_ENTRY(red,     31)
COLOR_ENTRY(green,   32)
COLOR_ENTRY(yellow,  33)
COLOR_ENTRY(blue,    34)
COLOR_ENTRY(magenta, 35)
COLOR_ENTRY(cyan,    36)
COLOR_ENTRY(white,   37)

COLOR_ENTRY(black_bg,   40)
COLOR_ENTRY(red_bg,     41)
COLOR_ENTRY(green_bg,   42)
COLOR_ENTRY(yellow_bg,  43)
COLOR_ENTRY(blue_bg,    44)
COLOR_ENTRY(magenta_bg, 45)
COLOR_ENTRY(cyan_bg,    46)
COLOR_ENTRY(white_bg,   47)

COLOR_ENTRY(bright_black,   90)
COLOR_ENTRY(bright_red,     91)
COLOR_ENTRY(bright_green,   92)
COLOR_ENTRY(bright_yellow,  93)
COLOR_ENTRY(bright_blue,    94)
COLOR_ENTRY(bright_magenta, 95)
COLOR_ENTRY(bright_cyan,    96)
COLOR_ENTRY(bright_white,   97)

COLOR_ENTRY(bright_black_bg,   100)
COLOR_ENTRY(bright_red_bg,     101)
COLOR_ENTRY(bright_green_bg,   102)
COLOR_ENTRY(bright_yellow_bg,  103)
COLOR_ENTRY(bright_blue_bg,    104)
COLOR_ENTRY(bright_magenta_bg, 105)
COLOR_ENTRY(bright_cyan_bg,    106)
COLOR_ENTRY(bright_white_bg,   107)

struct color_n
{
	color_n(unsigned char n) : m_n(n) {}
	friend std::ostream& operator << (std::ostream& os, const color_n& n);
private:
	unsigned char m_n;
};

struct color_bg_n
{
	color_bg_n(unsigned char n) : m_n(n) {}
	friend std::ostream& operator << (std::ostream& os, const color_bg_n& n);
private:
	unsigned char m_n;
};

inline std::ostream& operator << (std::ostream& os, const color_n& n)
{ os << "\x1b[38;5;" << (int)n.m_n << "m"; return os; }

inline std::ostream& operator << (std::ostream& os, const color_bg_n& n)
{ os << "\x1b[48;5;" << (int)n.m_n << "m"; return os; }

struct color_rgb
{
	color_rgb(unsigned char r, unsigned char g, unsigned char b) : m_r(r), m_g(g), m_b(b) {}
	friend std::ostream& operator << (std::ostream& os, const color_rgb& c);
private:
	unsigned char m_r, m_g, m_b;
};

struct color_bg_rgb
{
	color_bg_rgb(unsigned char r, unsigned char g, unsigned char b) : m_r(r), m_g(g), m_b(b) {}
	friend std::ostream& operator << (std::ostream& os, const color_bg_rgb& c);
private:
	unsigned char m_r, m_g, m_b;
};

inline std::ostream& operator << (std::ostream& os, const color_rgb& c)
{ os << "\x1b[38;2;" << (int)c.m_r << ";" << (int)c.m_g << ";" << (int)c.m_b << "m"; return os; }

inline std::ostream& operator << (std::ostream& os, const color_bg_rgb& c)
{ os << "\x1b[48;2;" << (int)c.m_r << ";" << (int)c.m_g << ";" << (int)c.m_b << "m"; return os; }

}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

#pragma once

#include <ostream>

namespace ansi_escape_codes {

#define ENTRY(name, code) \

static inline struct name##_opcode {} name; \

inline std::ostream& operator << (std::ostream& os, const name##_opcode&) \

{ os << "\x1b[" #code "m"; return os; }

ENTRY(reset, 0)

ENTRY(bold, 1)

ENTRY(faint, 2)

ENTRY(italic, 3)

ENTRY(underline, 4)

ENTRY(slow_blink, 5)

ENTRY(rapid_blink, 6)

ENTRY(inverse, 7)

ENTRY(conceal, 8)

ENTRY(crossed_out, 9)

ENTRY(default_font, 10)

ENTRY(alt_font_1, 11)

ENTRY(alt_font_2, 12)

ENTRY(alt_font_3, 13)

ENTRY(alt_font_4, 14)

ENTRY(alt_font_5, 15)

ENTRY(alt_font_6, 16)

ENTRY(alt_font_7, 17)

ENTRY(alt_font_8, 18)

ENTRY(alt_font_9, 19)

ENTRY(fraktur, 20)

ENTRY(doubly_underline, 21)

ENTRY(normal, 22)

ENTRY(not_italic_not_raktur, 23)

ENTRY(underline_off, 24)

ENTRY(blink_off, 25)

ENTRY(inverse_off, 27)

ENTRY(reveal, 28)

ENTRY(not_crossed_out, 29)

ENTRY(default_foreground_color, 39)

ENTRY(default_background_color, 49)

ENTRY(framed, 51)

ENTRY(encircled, 52)

ENTRY(overlined, 53)

ENTRY(not_framed_or_encircled, 54)

ENTRY(not_overlined, 55)

#define COLOR_ENTRY(name, code) \

static inline struct name##_color {} name; \

inline std::ostream& operator << (std::ostream& os, const name##_color&) \

{ os << "\x1b[" #code "m"; return os; }

COLOR_ENTRY(black, 30)

COLOR_ENTRY(red, 31)

COLOR_ENTRY(green, 32)

COLOR_ENTRY(yellow, 33)

COLOR_ENTRY(blue, 34)

COLOR_ENTRY(magenta, 35)

COLOR_ENTRY(cyan, 36)

COLOR_ENTRY(white, 37)

COLOR_ENTRY(black_bg, 40)

COLOR_ENTRY(red_bg, 41)

COLOR_ENTRY(green_bg, 42)

COLOR_ENTRY(yellow_bg, 43)

COLOR_ENTRY(blue_bg, 44)

COLOR_ENTRY(magenta_bg, 45)

COLOR_ENTRY(cyan_bg, 46)

COLOR_ENTRY(white_bg, 47)

COLOR_ENTRY(bright_black, 90)

COLOR_ENTRY(bright_red, 91)

COLOR_ENTRY(bright_green, 92)

COLOR_ENTRY(bright_yellow, 93)

COLOR_ENTRY(bright_blue, 94)

COLOR_ENTRY(bright_magenta, 95)

COLOR_ENTRY(bright_cyan, 96)

COLOR_ENTRY(bright_white, 97)

COLOR_ENTRY(bright_black_bg, 100)

COLOR_ENTRY(bright_red_bg, 101)

COLOR_ENTRY(bright_green_bg, 102)

COLOR_ENTRY(bright_yellow_bg, 103)

COLOR_ENTRY(bright_blue_bg, 104)

COLOR_ENTRY(bright_magenta_bg, 105)

COLOR_ENTRY(bright_cyan_bg, 106)

COLOR_ENTRY(bright_white_bg, 107)

struct color_n

{

color_n(unsigned char n) : m_n(n) {}

friend std::ostream& operator << (std::ostream& os, const color_n& n);

private:

unsigned char m_n;

};

struct color_bg_n

{

color_bg_n(unsigned char n) : m_n(n) {}

friend std::ostream& operator << (std::ostream& os, const color_bg_n& n);

private:

unsigned char m_n;

};

inline std::ostream& operator << (std::ostream& os, const color_n& n)

{ os << "\x1b[38;5;" << (int)n.m_n << "m"; return os; }

inline std::ostream& operator << (std::ostream& os, const color_bg_n& n)

{ os << "\x1b[48;5;" << (int)n.m_n << "m"; return os; }

struct color_rgb

{

color_rgb(unsigned char r, unsigned char g, unsigned char b) : m_r(r), m_g(g), m_b(b) {}

friend std::ostream& operator << (std::ostream& os, const color_rgb& c);

private:

unsigned char m_r, m_g, m_b;

};

struct color_bg_rgb

{

color_bg_rgb(unsigned char r, unsigned char g, unsigned char b) : m_r(r), m_g(g), m_b(b) {}

friend std::ostream& operator << (std::ostream& os, const color_bg_rgb& c);

private:

unsigned char m_r, m_g, m_b;

};

inline std::ostream& operator << (std::ostream& os, const color_rgb& c)

{ os << "\x1b[38;2;" << (int)c.m_r << ";" << (int)c.m_g << ";" << (int)c.m_b << "m"; return os; }

inline std::ostream& operator << (std::ostream& os, const color_bg_rgb& c)

{ os << "\x1b[48;2;" << (int)c.m_r << ";" << (int)c.m_g << ";" << (int)c.m_b << "m"; return os; }

}

Two-factor authentication

Posted on March 20, 2019March 28, 2021 by Martin

If you don’t know what multi-factor authentication is please read this before continuing. I am going to assume you understand the security concepts mentioned in this post…

In plain English: two-factor authentication is something you know (your password) and something you have (your token). I will focus on the token in this post. Apps like Google Authenticator and Authy generate one-time time-based tokens, or passwords. They generate them by hashing a shared secret combined with the current time. By default, the resulting token changes every 30 seconds giving the user a short window to authenticate to a service.
You can set it up with GitHub for example; in your user security settings you enable 2-factor authentication, GitHub then generates the shared secret for you which you import into the authenticator app. From then on when you login you must provide your password plus the generated token. Because both you and GitHub have access to the shared secret, both can generate the same token at the same time. If the user provided and the GitHub generated tokens match, the authentication succeeds and you’re logged in.

So what is this post about anyways? What I set-out to do today was to generate the one-time tokens programmatically from a C++ program. I wanted to test this by feeding the same shared secret to Authy and see that both my program and Authy generate the same tokens. With this working I, or you the reader, could add two-factor authentication to our applications, which is cool 🙂

Initially I started reading about the algorithm used to generate the tokens: Time-based One-time Password Algorithm. I sure as hell didn’t want to implement all of this from scratch, so I started looking for an OpenSSL implementation. During my search I came across a free (and available on Mac and Linux) framework to do what I wanted: OATH Toolkit. Once I started reading the documentation the rest fell in place very easily. I generated a dummy shared secret: 00112233445566778899 and fed it to Authy as well as my program (Google Authenticator requires it to be base32 encoded).

Below are screenshots of Authy and my program generating the same tokens. And of course the code!

#include <iostream>
#include <thread>
#include <cstdlib>
#include <liboath/oath.h>

using namespace std;

int main(int argc, char** argv)
{
	auto result = oath_init();
	if(result != OATH_OK)
	{
		cerr << oath_strerror(result) << endl;
		return -1;
	}

	const char secret[] = "00112233445566778899";
	const auto time_step = OATH_TOTP_DEFAULT_TIME_STEP_SIZE;
	const auto digits = 6;

	while(true)
	{
		auto t = time(NULL);
		auto left = time_step - (t % time_step);

		char otp[digits + 1] = {};
		result = oath_totp_generate(
			secret,
			sizeof(secret),
			t,
			time_step,
			OATH_TOTP_DEFAULT_START_TIME,
			digits,
			otp);

		if(result != OATH_OK)
		{
			cerr << oath_strerror(result) << endl;
			return -1;
		}

		cout << "OTP: " << otp << " (" << left << ") \r";
		cout.flush();

		this_thread::sleep_for(1s);
	}

	oath_done();

	return 1;
}

#include <iostream>

#include <thread>

#include <cstdlib>

#include <liboath/oath.h>

using namespace std;

int main(int argc, char** argv)

{

auto result = oath_init();

if(result != OATH_OK)

{

cerr << oath_strerror(result) << endl;

return -1;

}

const char secret[] = "00112233445566778899";

const auto time_step = OATH_TOTP_DEFAULT_TIME_STEP_SIZE;

const auto digits = 6;

while(true)

{

auto t = time(NULL);

auto left = time_step - (t % time_step);

char otp[digits + 1] = {};

result = oath_totp_generate(

secret,

sizeof(secret),

time_step,

OATH_TOTP_DEFAULT_START_TIME,

digits,

otp);

if(result != OATH_OK)

{

cerr << oath_strerror(result) << endl;

return -1;

}

cout << "OTP: " << otp << " (" << left << ") \r";

cout.flush();

this_thread::sleep_for(1s);

}

oath_done();

return 1;

}

Plugins: loading code at runtime

Posted on March 19, 2019March 28, 2021 by Martin

On windows we have the .dll files, .so files on Linux, and .dylib files on Mac. They all have one thing in common: they can be loaded at runtime and provide entry points to call into. One example is an online chat client that uses plugins to add support for more protocols (Google Chat, ICQ, IRC, etc). There are plenty more examples but you get the idea: drop a binary in plugins folder and restart the app and you have just added more functionality without having to recompile and redeploy your application.

So how do we do it? We could go the difficult route and use OS specific calls to find and load the plugin file, then do some more platform specific code to extract the entry point and call it. Or, we could make it easy on ourselves and use Boost DLL library 🙂

I am no expert on this library nor do I want to write a complete tutorial on it; the documentation is great. I have just started using it today in fact and wanted to see how easy it would be to get a basic plugin built and loaded by another program. It took all of 20 minutes to come up with a basic structure so I decided to make a short blog post about my experience so far. And so far it looks promising!

I started by creating a dynamic library which exports two function calls: one to get the name of the plugin and another to get its version. Below is all the code needed to create such a plugin with Boost:

#include <boost/dll/alias.hpp>

namespace plugin
{
	const char* plugin_name()
	{
		return "Vorbrodt's 1st Plugin";
	}

	const char* plugin_version()
	{
		return "1.0";
	}
}

BOOST_DLL_ALIAS(plugin::plugin_name, plugin_name);
BOOST_DLL_ALIAS(plugin::plugin_version, plugin_version);

#include <boost/dll/alias.hpp>

namespace plugin

{

const char* plugin_name()

{

return "Vorbrodt's 1st Plugin";

}

const char* plugin_version()

{

return "1.0";

}

BOOST_DLL_ALIAS(plugin::plugin_name, plugin_name);

BOOST_DLL_ALIAS(plugin::plugin_version, plugin_version);

Next I wrote a simple program which accepts the path to the plugin as a command line argument, loads the library, finally extracts and calls the two entry points. Here’s the code:

#include <iostream>
#include <boost/function.hpp>
#include <boost/dll/import.hpp>

using namespace std;
using namespace boost::dll;
using namespace boost::filesystem;

int main(int argc, char** argv)
{
	path lib_path(argv[1]);

	typedef const char* (plugin_name_t)();
	typedef const char* (plugin_version_t)();

	auto plugin_name = import_alias<plugin_name_t>(lib_path, "plugin_name", load_mode::append_decorations);
	auto plugin_version = import_alias<plugin_version_t>(lib_path, "plugin_version", load_mode::append_decorations);

	cout << "Plugin name    : " << plugin_name() << endl;
	cout << "Plugin version : " << plugin_version() << endl;

	return 1;
}

#include <iostream>

#include <boost/function.hpp>

#include <boost/dll/import.hpp>

using namespace std;

using namespace boost::dll;

using namespace boost::filesystem;

int main(int argc, char** argv)

{

path lib_path(argv[1]);

typedef const char* (plugin_name_t)();

typedef const char* (plugin_version_t)();

auto plugin_name = import_alias<plugin_name_t>(lib_path, "plugin_name", load_mode::append_decorations);

auto plugin_version = import_alias<plugin_version_t>(lib_path, "plugin_version", load_mode::append_decorations);

cout << "Plugin name : " << plugin_name() << endl;

cout << "Plugin version : " << plugin_version() << endl;

return 1;

}

Plugin name : Vorbrodt’s 1st Plugin
Plugin version : 1.0
Program output.

It works! You can take it from here and build a cool plugin engine. Oh I forgot to mention, the same code compiles and behaves the same on Windows, Linux, and Mac 🙂

Micro-benchmarks

Posted on March 18, 2019March 28, 2021 by Martin

I was looking for a way to benchmark a piece of code… I came up with 5 libraries that make it very easy 🙂

I’m not going to write a tutorial on how to use each one because I would basically be rewriting their documentation sections. What I will do is show you how to get started. As an example I will write a simple benchmark that tests copy constructor of std::string. But first, the libraries:

Catch2 and Nonius are header only libraries; be aware of long compile times 🙁 Google, Catch2, and Nonius automatically pick the number of runs and iterations for you, which is nice: no guessing how many times you need to run a function you want to benchmark to get a reasonable performance reading.

Google Benchmark:

#include <string>
#include <benchmark/benchmark.h>

using namespace std;

string short_string = "hello";
string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

void BM_StringCopyShort(benchmark::State& state)
{
	for (auto _ : state)
		string copy(short_string);
}
BENCHMARK(BM_StringCopyShort);

void BM_StringCopyLong(benchmark::State& state)
{
	for (auto _ : state)
		string copy(long_string);
}
BENCHMARK(BM_StringCopyLong);

BENCHMARK_MAIN();

#include <string>

#include <benchmark/benchmark.h>

using namespace std;

string short_string = "hello";

string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

void BM_StringCopyShort(benchmark::State& state)

{

for (auto _ : state)

string copy(short_string);

}

BENCHMARK(BM_StringCopyShort);

void BM_StringCopyLong(benchmark::State& state)

{

for (auto _ : state)

string copy(long_string);

}

BENCHMARK(BM_StringCopyLong);

BENCHMARK_MAIN();

Catch2:

#include <string>
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>

using namespace std;

string short_string = "hello";
string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

TEST_CASE("string ops", "[benchmark]")
{
	BENCHMARK("StringCopyShort")
	{
		string copy(short_string);
	}

	BENCHMARK("StringCopyLong")
	{
		string copy(long_string);
	}
}

#include <string>

#define CATCH_CONFIG_MAIN

#include <catch2/catch.hpp>

using namespace std;

string short_string = "hello";

string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

TEST_CASE("string ops", "[benchmark]")

{

BENCHMARK("StringCopyShort")

{

string copy(short_string);

}

BENCHMARK("StringCopyLong")

{

string copy(long_string);

}

Hayai:

#include <string>
#include <hayai/hayai.hpp>

using namespace std;

string short_string = "hello";
string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

BENCHMARK(StringOps, StringCopyShort, 100, 1'000'000)
{
	string copy(short_string);
}

BENCHMARK(StringOps, StringCopyLong, 100, 100'000)
{
	string copy(long_string);
}

#include <string>

#include <hayai/hayai.hpp>

using namespace std;

string short_string = "hello";

string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

BENCHMARK(StringOps, StringCopyShort, 100, 1'000'000)

{

string copy(short_string);

}

BENCHMARK(StringOps, StringCopyLong, 100, 100'000)

{

string copy(long_string);

}

Celero:

#include <string>
#include <celero/Celero.h>

using namespace std;

string short_string = "hello";
string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

BASELINE(StringCopyShort, Baseline10, 100, 100'000)
{
	string copy(short_string);
}

BENCHMARK(StringCopyShort, Baseline1000, 100, 1'000'000)
{
	string copy(short_string);
}

BASELINE(StringCopyLong, Baseline10, 100, 100'000)
{
	string copy(long_string);
}

BENCHMARK(StringCopyLong, Baseline1000, 100, 1'000'000)
{
	string copy(long_string);
}

CELERO_MAIN;

#include <string>

#include <celero/Celero.h>

using namespace std;

string short_string = "hello";

string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

BASELINE(StringCopyShort, Baseline10, 100, 100'000)

{

string copy(short_string);

}

BENCHMARK(StringCopyShort, Baseline1000, 100, 1'000'000)

{

string copy(short_string);

}

BASELINE(StringCopyLong, Baseline10, 100, 100'000)

{

string copy(long_string);

}

BENCHMARK(StringCopyLong, Baseline1000, 100, 1'000'000)

{

string copy(long_string);

}

CELERO_MAIN;

Nonius:

#include <string>
#define NONIUS_RUNNER
#include <nonius/nonius.h++>
#include <nonius/main.h++>

using namespace std;

string short_string = "hello";
string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

NONIUS_BENCHMARK("StringCopyShort", []
{
	string copy(short_string);
})

NONIUS_BENCHMARK("StringCopyLong", []
{
	string copy(long_string);
})

#include <string>

#define NONIUS_RUNNER

#include <nonius/nonius.h++>

#include <nonius/main.h++>

using namespace std;

string short_string = "hello";

string long_string("0123456789abcdefghijklmnopqrstuvwxyz");

NONIUS_BENCHMARK("StringCopyShort", []

{

string copy(short_string);

})

NONIUS_BENCHMARK("StringCopyLong", []

{

string copy(long_string);

})

SQL database access

Posted on March 18, 2019March 28, 2021 by Martin

When I first set out to make a post about database access from C++ I was going to write about MySQL Connector/C++. But for some reason that didn’t sit well with me. After sleeping on it I realized it didn’t appeal to me because it was 1) limited to only one database backend and 2) too low-level of an API. I wanted to write about a library that supports multiple database backends and abstracts the connection details as much as possible. Ideally I wanted a library that brings you closest to SQL syntax rather than deal in C++ mechanics. So in my quest for cool and portable C++ libraries I decided to keep looking…

And then I came across SOCI – The C++ Database Access Library 🙂 It has everything I was looking for: multiple backend support (DB2, Firebird, MySQL, ODBC, Oracle, PostgresSQL, and SQLite3), and a very natural way of issuing SQL queries thanks to operator overloading and template sorcery. Even their first example is purposely left without comments because it is that easy to read and understand.

Besides a very natural way of talking to a SQL backend what I like most about it is that it allows you, in a non-intrusive way (no existing code change needed), to store and retrieve your custom data structures to and from database tables.

So I installed MySQL server on my Mac using brew package manager and started coding. Within 30 minutes I had a working example that connected to my database server, created a database and a table, inserted rows into the table from my custom Person data structure, counted the rows, retrieved a table entry with a given ID, and finally cleaned up after itself.

The only code that requires explaining is the struct type_conversion<Person>; object. It is SOCI’s mechanism of converting to and from custom data structures and requires 2 methods: from_base which converts from a set of row values to a structure, and to_base which goes the other way. The rest is self explanatory! Here’s how you can get started:

#include <iostream>
#include <string>
#include <exception>
#include <soci/soci.h>
#include <soci/mysql/soci-mysql.h>

using namespace std;
using namespace soci;

struct Person
{
	int ID;
	string FirstName;
	string LastName;
	int DOB;
	string EMail;
};

namespace soci
{
	template<>
	struct type_conversion<Person>
	{
		typedef values base_type;

		static void from_base(const values& v, indicator, Person& p)
		{
			p.ID = v.get<int>("ID");
			p.FirstName = v.get<string>("FirstName");
			p.LastName = v.get<string>("LastName");
			p.DOB = v.get<int>("DOB");
			p.EMail = v.get<string>("EMail");
		}

		static void to_base(const Person& p, values& v, indicator& ind)
		{
			v.set("ID", p.ID);
			v.set("FirstName", p.FirstName);
			v.set("LastName", p.LastName);
			v.set("DOB", p.DOB);
			v.set("EMail", p.EMail);
			ind = i_ok;
		}
	};
}

int main()
{
	try
	{
		session sql(mysql, "host=localhost user=root password=''");

		sql << "CREATE DATABASE blog";
		sql << "USE blog";
		sql << "CREATE TABLE people (ID INT, FirstName TEXT, LastName TEXT, DOB INT, EMail TEXT)";

		Person him{1, "Martin", "Vorbrodt", 19800830, "martin@vorbrodt.blog"};
		Person her{2, "Dorota", "Vorbrodt", 19800127, "dorota@vorbrodt.blog"};
		sql << "INSERT INTO people (ID, FirstName, LastName, DOB, EMail) VALUES (:ID, :FirstName, :LastName, :DOB, :EMail)", use(him);
		sql << "INSERT INTO people (ID, FirstName, LastName, DOB, EMail) VALUES (:ID, :FirstName, :LastName, :DOB, :EMail)", use(her);

		int count{};
		sql << "SELECT COUNT(*) FROM people", into(count);
		cout << "Table 'people' has " << count << " row(s)" << endl;

		Person pOut{};
		sql << "SELECT * FROM people WHERE ID = 1", into(pOut);
		cout << pOut.FirstName << ", " << pOut.LastName << ", " << pOut.DOB << ", " << pOut.EMail << endl;

		sql << "DROP TABLE people";
		sql << "DROP DATABASE blog";
	}
	catch (exception& e)
	{
		cerr << e.what() << endl;
	}
}

#include <iostream>

#include <string>

#include <exception>

#include <soci/soci.h>

#include <soci/mysql/soci-mysql.h>

using namespace std;

using namespace soci;

struct Person

{

int ID;

string FirstName;

string LastName;

int DOB;

string EMail;

};

namespace soci

{

template<>

struct type_conversion<Person>

{

typedef values base_type;

static void from_base(const values& v, indicator, Person& p)

{

p.ID = v.get<int>("ID");

p.FirstName = v.get<string>("FirstName");

p.LastName = v.get<string>("LastName");

p.DOB = v.get<int>("DOB");

p.EMail = v.get<string>("EMail");

}

static void to_base(const Person& p, values& v, indicator& ind)

{

v.set("ID", p.ID);

v.set("FirstName", p.FirstName);

v.set("LastName", p.LastName);

v.set("DOB", p.DOB);

v.set("EMail", p.EMail);

ind = i_ok;

}

};

}

int main()

{

try

{

session sql(mysql, "host=localhost user=root password=''");

sql << "CREATE DATABASE blog";

sql << "USE blog";

sql << "CREATE TABLE people (ID INT, FirstName TEXT, LastName TEXT, DOB INT, EMail TEXT)";

Person him{1, "Martin", "Vorbrodt", 19800830, "[email protected]"};

Person her{2, "Dorota", "Vorbrodt", 19800127, "[email protected]"};

sql << "INSERT INTO people (ID, FirstName, LastName, DOB, EMail) VALUES (:ID, :FirstName, :LastName, :DOB, :EMail)", use(him);

sql << "INSERT INTO people (ID, FirstName, LastName, DOB, EMail) VALUES (:ID, :FirstName, :LastName, :DOB, :EMail)", use(her);

int count{};

sql << "SELECT COUNT(*) FROM people", into(count);

cout << "Table 'people' has " << count << " row(s)" << endl;

Person pOut{};

sql << "SELECT * FROM people WHERE ID = 1", into(pOut);

cout << pOut.FirstName << ", " << pOut.LastName << ", " << pOut.DOB << ", " << pOut.EMail << endl;

sql << "DROP TABLE people";

sql << "DROP DATABASE blog";

}

catch (exception& e)

{

cerr << e.what() << endl;

}

Table ‘people’ has 2 row(s)
Martin, Vorbrodt, 19800830, [email protected]
Program output.

Measuring CPU time

Posted on March 16, 2019March 28, 2021 by Martin

Measuring how long your program ran for is easy with std::chrono, but what if you need details about user and system space time? Easy! Use Boost CPU Timer library 🙂 It’s another simple library from Boost with only few classes: auto_cpu_timer is like a RAII object; it starts the clock in its constructor, stops it in its destructor, and prints the elapsed time to standard output. cpu_timer class is for manual starting and stopping of the clock; it also allows you to retrieve the elapsed times (wall, user, and system) in nanoseconds.

In the below example I create three threads: procUser spends 100% of its time in the user space calling std::sqrt function. procSystem spawns a lot of threads causing transitions into the kernel. And procTimer is just an illustration of cpu_timer usage.

#include <iostream>
#include <thread>
#include <cmath>
#include <cstdlib>
#include <boost/timer/timer.hpp>

using namespace std;
using namespace boost::timer;

int main()
{
	srand((unsigned int)time(NULL));

	auto_cpu_timer program_timer(3);

	auto procUser = [](long work)
	{
		for (long i = 0; i < work; ++i)
			sqrt(123.456L);
	};

	auto procSystem = [](long work)
	{
		for (long i = 0; i < work; ++i)
			thread([](){}).detach();
	};

	auto procTimer = [](long work)
	{
		cpu_timer timer;
		timer.start();

		for(long i = 0; i < work; ++i)
			rand();

		timer.stop();
		cout << "Thread timer:" << timer.format(3);
	};

	thread t1(procUser, 1000000000);
	thread t2(procSystem, 100000);
	thread t3(procTimer, 100000000);

	t1.join();
	t2.join();
	t3.join();

	cout << "Program timer:";

	return 1;
}

#include <iostream>

#include <thread>

#include <cmath>

#include <cstdlib>

#include <boost/timer/timer.hpp>

using namespace std;

using namespace boost::timer;

int main()

{

srand((unsigned int)time(NULL));

auto_cpu_timer program_timer(3);

auto procUser = [](long work)

{

for (long i = 0; i < work; ++i)

sqrt(123.456L);

};

auto procSystem = [](long work)

{

for (long i = 0; i < work; ++i)

thread([](){}).detach();

};

auto procTimer = [](long work)

{

cpu_timer timer;

timer.start();

for(long i = 0; i < work; ++i)

rand();

timer.stop();

cout << "Thread timer:" << timer.format(3);

};

thread t1(procUser, 1000000000);

thread t2(procSystem, 100000);

thread t3(procTimer, 100000000);

t1.join();

t2.join();

t3.join();

cout << "Program timer:";

return 1;

}

Thread timer: 0.750s wall, 1.790s user + 0.850s system = 2.640s CPU (352.1%)
Program timer: 3.171s wall, 5.080s user + 2.980s system = 8.060s CPU (254.2%)
Program output.

Printing stack traces

Posted on March 15, 2019March 15, 2019 by Martin

This one will be short 🙂

If you want to add more diagnostic information to your programs make sure to check out Boost.Stacktrace library. With it you can capture and print current stack traces. It’s especially useful when combined with exception handling; it allows you to know right away where the exception originated from.

#include <iostream>
#include <future>
#define BOOST_STACKTRACE_GNU_SOURCE_NOT_REQUIRED
#include <boost/stacktrace.hpp>

using namespace std;
using namespace boost::stacktrace;

void f1();
void f2();
void f3();

void f1() { f2(); }
void f2() { f3(); }
void f3() { cout << stacktrace() << endl << endl; }

int main(int argc, char** argv)
{
	f1();

	auto f = async(launch::async, []() { cout << stacktrace() << endl << endl; });
	f.get();

	return 1;
}

#include <iostream>

#include <future>

#define BOOST_STACKTRACE_GNU_SOURCE_NOT_REQUIRED

#include <boost/stacktrace.hpp>

using namespace std;

using namespace boost::stacktrace;

void f1();

void f2();

void f3();

void f1() { f2(); }

void f2() { f3(); }

void f3() { cout << stacktrace() << endl << endl; }

int main(int argc, char** argv)

{

f1();

auto f = async(launch::async, []() { cout << stacktrace() << endl << endl; });

f.get();

return 1;

}

0# f3() in /Users/martin/stacktrace
1# f2() in /Users/martin/stacktrace
2# f1() in /Users/martin/stacktrace
3# main in /Users/martin/stacktrace

0# main::$_0::operator()() const in /Users/martin/stacktrace
1# void std::__1::__async_func<main::$_0>::__execute<>(std::__1::__tuple_indices<>) in /Users/martin/stacktrace
2# std::__1::__async_func<main::$_0>::operator()() in /Users/martin/stacktrace
3# std::__1::__async_assoc_state<void, std::__1::__async_func<main::$_0> >::__execute() in /Users/martin/stacktrace
4# void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (std::__1::__async_assoc_state<void, std::__1::__async_func<main::$_0> >::*)(), std::__1::__async_assoc_state<void, std::__1::__async_func<main::$_0> >*> >(void*) in /Users/martin/stacktrace
5# _pthread_body in /usr/lib/system/libsystem_pthread.dylib
6# _pthread_start in /usr/lib/system/libsystem_pthread.dylib
Program output

Generating Unique IDs

Posted on March 14, 2019March 14, 2019 by Martin

I like simple and well written C++ libraries, and I’m especially fond of Boost, so I’m going to blog about it some more 🙂

Today I’ll show you how to generate unique IDs using Boost::UUID header only library. There really isn’t much to be said about it other than it’s simple, consists of only few hearer files, and allows you to generate, hash, and serialize unique IDs.

Refer to the documentation for a complete overview. Here’s a sample program that generates and prints a unique ID:

#include <iostream>
#include <boost/uuid/uuid.hpp>
#include <boost/uuid/uuid_io.hpp>
#include <boost/uuid/uuid_generators.hpp>

using namespace std;
using namespace boost::uuids;

int main(int argc, char** argv)
{
	auto gen = random_generator();
	uuid id = gen();

	cout << id << endl;

	return 1;
}

#include <iostream>

#include <boost/uuid/uuid.hpp>

#include <boost/uuid/uuid_io.hpp>

#include <boost/uuid/uuid_generators.hpp>

using namespace std;

using namespace boost::uuids;

int main(int argc, char** argv)

{

auto gen = random_generator();

uuid id = gen();

cout << id << endl;

return 1;

}

5023e2d9-8008-4142-b1c7-87cda7cb36b3
Program output.

RPC with Protocol Buffers

Posted on March 13, 2019March 28, 2021 by Martin

Just when I thought I covered the topic of RPC in C++ with Thrift, I came across a new and very promising RPC framework: Google’s gRPC. The more I read about it the more I like it: just like Thrift it supports many programing languages and internally uses Protocol Buffers to encode messages. So I decided to give it a try and see how easily I could create a simple client and serve programs…

Just like with Thrift, the first step is to define your RPC service. Here’s my definition of a simple service with one RPC method which accepts an input parameter and provides a return value:

syntax = "proto3";

message Input
{
	string input_msg = 1;
}

message Output
{
	string output_msg = 1;
}

service gRPCService
{
	rpc hello(Input) returns (Output) {}
}

syntax = "proto3";

message Input

{

string input_msg = 1;

}

message Output

{

string output_msg = 1;

}

service gRPCService

{

rpc hello(Input) returns (Output) {}

}

I saved it as grpc_service.proto and now was the time to generate the client and server stubs. Unlike Thrift, this is a two step process: 1) you must generate the protocol buffer files, and 2) generate the gRPC client/server stubs. Both actions are done with protoc compiler and a custom plugin that comes with gRPC:

protoc -I . –cpp_out=. grpc_service.proto
protoc -I . –grpc_out=. –plugin=protoc-gen-grpc=/usr/local/bin/grpc_cpp_plugin grpc_service.proto
gRPC code generation steps.

This will compile our service definition file and produce 4 output files: .h and .cc for protocol buffer messages, and .h and .cc for gRPC client/server stubs.

Alright, that was easy. Let’s see what it takes to implement a gRPC client program… after 15 minutes reading through the documentation I came up with the following:

#include <iostream>
#include <grpc/grpc.h>
#include <grpc++/channel.h>
#include <grpc++/client_context.h>
#include <grpc++/create_channel.h>
#include <grpc++/security/credentials.h>
#include "grpc_service.grpc.pb.h"

using namespace std;
using namespace grpc;

class gRPCClient
{
public:
	gRPCClient(shared_ptr<Channel> channel)
	: stub(gRPCService::NewStub(channel)) {}

	string hello(const std::string& msg)
	{
		Input input;
		Output output;
		ClientContext context;

		input.set_input_msg(msg);

		Status status = stub->hello(&context, input, &output);
		if (status.ok())
		{
			return output.output_msg();
		}
		else
		{
			cout << status.error_code() << ": " << status.error_message() << endl;
			return "RPC failed";
		}
	}

private:
	unique_ptr<gRPCService::Stub> stub;
};

int main(int argc, char** argv)
{
	gRPCClient client(grpc::CreateChannel("localhost:50051", grpc::InsecureChannelCredentials()));
	string reply = client.hello("gRPC client says hi!");
	cout << "Client received: " << reply << endl;

	return 1;
}

#include <iostream>

#include <grpc/grpc.h>

#include <grpc++/channel.h>

#include <grpc++/client_context.h>

#include <grpc++/create_channel.h>

#include <grpc++/security/credentials.h>

#include "grpc_service.grpc.pb.h"

using namespace std;

using namespace grpc;

class gRPCClient

{

public:

gRPCClient(shared_ptr<Channel> channel)

: stub(gRPCService::NewStub(channel)) {}

string hello(const std::string& msg)

{

Input input;

Output output;

ClientContext context;

input.set_input_msg(msg);

Status status = stub->hello(&context, input, &output);

if (status.ok())

{

return output.output_msg();

}

else

{

cout << status.error_code() << ": " << status.error_message() << endl;

return "RPC failed";

}

private:

unique_ptr<gRPCService::Stub> stub;

};

int main(int argc, char** argv)

{

gRPCClient client(grpc::CreateChannel("localhost:50051", grpc::InsecureChannelCredentials()));

string reply = client.hello("gRPC client says hi!");

cout << "Client received: " << reply << endl;

return 1;

}

Little more code needed to construct and invoke the RPC method as compared to Thrift, but not too bad. The extra lines are mostly around creating the protocol buffer message objects and setting their properties.

The corresponding gRPC server code is pretty much just as easy to implement as with Thrift. Here’s the simple server that prints out the message it receives and sends back a reply:

#include <iostream>
#include <grpc/grpc.h>
#include <grpc++/server.h>
#include <grpc++/server_builder.h>
#include <grpc++/server_context.h>
#include <grpc++/security/server_credentials.h>
#include "grpc_service.grpc.pb.h"

using namespace std;
using namespace grpc;

class gRPCServiceImpl final : public gRPCService::Service
{
public:
	Status hello(ServerContext* context, const Input* in, Output* out) override
	{
		cout << in->input_msg() << endl;
		out->set_output_msg("gRPC server says hi!");
		return Status::OK;
	}
};

int main(int argc, char** argv)
{
	gRPCServiceImpl service;
	ServerBuilder builder;
	builder.AddListeningPort("0.0.0.0:50051", grpc::InsecureServerCredentials());
	builder.RegisterService(&service);
	unique_ptr<Server> server(builder.BuildAndStart());
	cout << "Server starting..." << endl;
	server->Wait();

	return 1;
}

#include <iostream>

#include <grpc/grpc.h>

#include <grpc++/server.h>

#include <grpc++/server_builder.h>

#include <grpc++/server_context.h>

#include <grpc++/security/server_credentials.h>

#include "grpc_service.grpc.pb.h"

using namespace std;

using namespace grpc;

class gRPCServiceImpl final : public gRPCService::Service

{

public:

Status hello(ServerContext* context, const Input* in, Output* out) override

{

cout << in->input_msg() << endl;

out->set_output_msg("gRPC server says hi!");

return Status::OK;

}

};

int main(int argc, char** argv)

{

gRPCServiceImpl service;

ServerBuilder builder;

builder.AddListeningPort("0.0.0.0:50051", grpc::InsecureServerCredentials());

builder.RegisterService(&service);

unique_ptr<Server> server(builder.BuildAndStart());

cout << "Server starting..." << endl;

server->Wait();

return 1;

}

So there you have it folks! It is very easy to get started with this framework.

As far as which one you should prefer (Thrift vs gRPC) I can’t honestly say. I don’t know enough about them to claim one is better than the other. What I can say is that Thrift is a more mature framework that has been around for longer. You decide 🙂

P.S. On my Mac I was able to use Homebrew package manager to install the required headers/libraries/executables for gRPC. My Ubuntu 18.04 Linux has libgrpc available in its online repository. I also verified that Microsoft’s vcpkg has ports available for Windows.

Serialize data to XML

Posted on March 10, 2019March 28, 2021 by Martin

Protocol Buffers are not always an option and you just have to serialize your data to XML. Then what? Luckily there is a serialization library available from Boost and it makes that pretty easy for us. You don’t even have to modify your existing data structures to write them out as XML: the library is non-invasive.

Let’s say I want to serialize a list of people to a file, and read it back later. My data structures would be defined like this:

struct person
{
	string name;
	int dob;
	string email;
};

using people = vector<person>;

struct person

{

string name;

int dob;

string email;

};

using people = vector<person>;

And once a vector of person’s is serialized I want it to look something like this:

<people class_id="0" tracking_level="0" version="1">
	<count>2</count>
	<item_version>1</item_version>
	<item class_id="1" tracking_level="0" version="1">
		<person.name>Martin Vorbrodt</person.name>
		<person.dob>19800830</person.dob>
		<person.email>martin@vorbrodt.blog</person.email>
	</item>
	<item>
		<person.name>Dorota Vorbrodt</person.name>
		<person.dob>19810127</person.dob>
		<person.email>dorota@vorbrodt.blog</person.email>
	</item>
</people>

<item_version>1</item_version>

<person.name>Martin Vorbrodt</person.name>

<person.dob>19800830</person.dob>

<person.email>martin@vorbrodt.blog</person.email>

</item>

<item>

<person.name>Dorota Vorbrodt</person.name>

<person.dob>19810127</person.dob>

<person.email>dorota@vorbrodt.blog</person.email>

</item>

</people>

Easy! First you must define a generic serialize function for your data structure, then you instantiate an XML output archive with an ofstream object and pass it the data. Reading is done by instantiating an XML input archive with an ifstream object and loading the data into a variable. Like this:

#include <iostream>
#include <fstream>
#include <string>
#include <vector>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/serialization/vector.hpp>
using namespace std;
using namespace boost::archive;
using namespace boost::serialization;

struct person
{
	string name;
	int dob;
	string email;
};

using people = vector<person>;

BOOST_CLASS_VERSION(person, 1);
BOOST_CLASS_VERSION(people, 1);

template<typename Archive>
void serialize(Archive& ar, person& person, const unsigned int version)
{
	ar & BOOST_SERIALIZATION_NVP(person.name);
	ar & BOOST_SERIALIZATION_NVP(person.dob);
	ar & BOOST_SERIALIZATION_NVP(person.email);
}

int main(int argc, char** argv)
{
	person me{"Martin Vorbrodt", 19800830, "martin@vorbrodt.blog"};
	person her{"Dorota Vorbrodt", 19810127, "dorota@vorbrodt.blog"};
	people us{me, her};
	{
		ofstream ofs("data.xml");
		xml_oarchive oa(ofs, boost::archive::no_header);
		oa << make_nvp("people", us);
	}

	people us_too{};
	{
		ifstream ifs("data.xml");
		xml_iarchive ia(ifs, boost::archive::no_header);
		ia >> make_nvp("people", us_too);
	}

	for(auto& p : us_too)
	{
		cout << "Name  : " << p.name << endl;
		cout << "DOB   : " << p.dob << endl;
		cout << "EMail : " << p.email << endl << endl;
	}

	return 1;
}

#include <iostream>

#include <fstream>

#include <string>

#include <vector>

#include <boost/archive/xml_oarchive.hpp>

#include <boost/archive/xml_iarchive.hpp>

#include <boost/serialization/vector.hpp>

using namespace std;

using namespace boost::archive;

using namespace boost::serialization;

struct person

{

string name;

int dob;

string email;

};

using people = vector<person>;

BOOST_CLASS_VERSION(person, 1);

BOOST_CLASS_VERSION(people, 1);

template<typename Archive>

void serialize(Archive& ar, person& person, const unsigned int version)

{

ar & BOOST_SERIALIZATION_NVP(person.name);

ar & BOOST_SERIALIZATION_NVP(person.dob);

ar & BOOST_SERIALIZATION_NVP(person.email);

}

int main(int argc, char** argv)

{

person me{"Martin Vorbrodt", 19800830, "[email protected]"};

person her{"Dorota Vorbrodt", 19810127, "[email protected]"};

people us{me, her};

{

ofstream ofs("data.xml");

xml_oarchive oa(ofs, boost::archive::no_header);

oa << make_nvp("people", us);

}

people us_too{};

{

ifstream ifs("data.xml");

xml_iarchive ia(ifs, boost::archive::no_header);

ia >> make_nvp("people", us_too);

}

for(auto& p : us_too)

{

cout << "Name : " << p.name << endl;

cout << "DOB : " << p.dob << endl;

cout << "EMail : " << p.email << endl << endl;

}

return 1;

}

Name : Martin Vorbrodt
DOB : 19800830
EMail : [email protected]

Name : Dorota Vorbrodt
DOB : 19810127
EMail : [email protected]
Program output.

The library has built in support for STL containers. It can also write the data in many output formats, not just XML. Luckily you only have to define one serialization function per data type and it will work with all input and output archives. Heck, you could even define a serialization function for your protocol buffers data types 😉

Thrift: or how to RPC

Posted on March 10, 2019March 28, 2021 by Martin

Just like my previous Protocol Buffers post, this one is also meant as a brief introduction that will point you in the right direction rather than an exhaustive tutorial. Here we go…

Again we are in search of a portable library, this time not for serialization, but for a portable RPC mechanism. On Windows we have the WCF, but what if we want support for many platforms and programming languages? All that is answered by Thrift (also see here). Initially developed at Facebook it is now free and open source project.

Let’s start by creating a simple thrift file that defines a “service”, or a RPC server, with functions and parameters:

namespace cpp Service

service Service
{
	void ping(),
	void hello(1:string msg),
	oneway void async_call()
}

namespace cpp Service

service Service

{

void ping(),

void hello(1:string msg),

oneway void async_call()

}

Here in a file service.thrift we have defined a RPC server called Service with three functions (one asynchronous) and a string parameter msg. Next we need to compile it. Just like protocol buffers, thrift is a code generator. It will produce everything needed to instantiate both the server and the client:

thrift –gen cpp -out . service.thrift
Thrift basic usage.

The above command will produce several header and source files for us. Now we are ready to implement our C++ client that will connect to the RPC server and issue remote procedure calls. The code is straight forward and easy to read and understand:

#include <iostream>
#include <thrift/transport/TSocket.h>
#include <thrift/protocol/TBinaryProtocol.h>
#include <thrift/transport/TTransportUtils.h>
#include "Service.h"

using namespace std;
using namespace apache::thrift;
using namespace apache::thrift::protocol;
using namespace apache::thrift::transport;
using namespace Service;

int main(int argc, char** argv)
{
	auto socket = make_shared<TSocket>("localhost", 9090);
	auto transport = make_shared<TBufferedTransport>(socket);
	auto protocol = make_shared<TBinaryProtocol>(transport);

	ServiceClient client(protocol);

	try
	{
		transport->open();

		client.ping();
		client.hello("Martin says hi!");
		client.async_call();

		transport->close();
	}
	catch(TException& tx)
	{
		cout << "ERROR: " << tx.what() << endl;
	}

	return 1;
}

#include <iostream>

#include <thrift/transport/TSocket.h>

#include <thrift/protocol/TBinaryProtocol.h>

#include <thrift/transport/TTransportUtils.h>

#include "Service.h"

using namespace std;

using namespace apache::thrift;

using namespace apache::thrift::protocol;

using namespace apache::thrift::transport;

using namespace Service;

int main(int argc, char** argv)

{

auto socket = make_shared<TSocket>("localhost", 9090);

auto transport = make_shared<TBufferedTransport>(socket);

auto protocol = make_shared<TBinaryProtocol>(transport);

ServiceClient client(protocol);

try

{

transport->open();

client.ping();

client.hello("Martin says hi!");

client.async_call();

transport->close();

}

catch(TException& tx)

{

cout << "ERROR: " << tx.what() << endl;

}

return 1;

}

The server code is slightly more complicated, but not by much 🙂 In this post I’m using the most basic functions of thrift for illustration purposes. But know that it is quite capable of handling huge workloads and many connections. Here’s the corresponding server code:

#include <iostream>
#include <thrift/transport/TServerSocket.h>
#include <thrift/protocol/TBinaryProtocol.h>
#include <thrift/transport/TTransportUtils.h>
#include <thrift/server/TSimpleServer.h>
#include "Service.h"

using namespace std;
using namespace apache::thrift;
using namespace apache::thrift::protocol;
using namespace apache::thrift::transport;
using namespace apache::thrift::server;
using namespace Service;

class ServiceHandler : public ServiceIf
{
public:
	ServiceHandler() = default;

	void ping() override { cout << "ping()" << endl; }
	void hello(const string& msg) override { cout << msg << endl; }
	void async_call() override { cout << "async_call()" << endl; }
};

int main(int argc, char** argv)
{
	TSimpleServer server(
		std::make_shared<ServiceProcessor>(std::make_shared<ServiceHandler>()),
		std::make_shared<TServerSocket>(9090),
		std::make_shared<TBufferedTransportFactory>(),
		std::make_shared<TBinaryProtocolFactory>());

	cout << "Starting the server..." << endl;

	server.serve();

	return 1;
}

#include <iostream>

#include <thrift/transport/TServerSocket.h>

#include <thrift/protocol/TBinaryProtocol.h>

#include <thrift/transport/TTransportUtils.h>

#include <thrift/server/TSimpleServer.h>

#include "Service.h"

using namespace std;

using namespace apache::thrift;

using namespace apache::thrift::protocol;

using namespace apache::thrift::transport;

using namespace apache::thrift::server;

using namespace Service;

class ServiceHandler : public ServiceIf

{

public:

ServiceHandler() = default;

void ping() override { cout << "ping()" << endl; }

void hello(const string& msg) override { cout << msg << endl; }

void async_call() override { cout << "async_call()" << endl; }

};

int main(int argc, char** argv)

{

TSimpleServer server(

std::make_shared<ServiceProcessor>(std::make_shared<ServiceHandler>()),

std::make_shared<TServerSocket>(9090),

std::make_shared<TBufferedTransportFactory>(),

std::make_shared<TBinaryProtocolFactory>());

cout << "Starting the server..." << endl;

server.serve();

return 1;

}

The extra code here is the ServiceHandler class which will do the actual RPC work. Let’s put it all together now. After I start the server and execute the client program on my machine, I get the following output:

Starting the server…
ping()
Martin says hi!
async_call()
Thrift RPC server output.

It works! I hope you enjoyed this little introduction to thrift. Now go read all about it!

P.S. As always, complete source and build files available at my GitHub.

Protocol Buffers: or how to serialize data

Posted on March 10, 2019April 2, 2021 by Martin

This post is meant as a brief introduction that will point you in the right direction rather than an exhaustive tutorial. Here we go…

Have you ever had to write code that serialized structured data into an efficient binary format, to be later saved to disk or sent over the network? Do you remember how difficult and time consuming it was? Haven’t you wished there was a standard C++ library to do it instead of reinventing the wheel? Well, today is your lucky day 🙂

Say hello to Google’s Protocol Buffers! It is a highly portable (and free!) library that allows you to define near arbitrary (numerical, strings, structures, lists, vectors, maps, you name it) data formats and easily serialize and deserialize them into a platform portable binary format. Why not use XML you may ask? How about 3 to 10 times smaller output, and 20 to 100 times faster serialization and deserialization just to name a few 🙂

Let’s start by defining a protocol buffer file that defines a “Person” data structure:

syntax = "proto2";
  
package data;

message Person
{
  required string name = 1;
  required int32 dob = 2;
  optional string email = 3;
}

syntax = "proto2";

package data;

message Person

{

required string name = 1;

required int32 dob = 2;

optional string email = 3;

}

Save that to protobuf.proto file and let’s compile it using the protoc code generator. Yes, Protocol Buffers is a code generator; it takes as input a .proto file and spits out C++ classes (it can also produce code in C#, Java, JS, ObjC, PHP, Python, and Ruby).

protoc -I=. –cpp_out=. protobuf.proto
protoc basic usage.

The above command will produce 2 files:

#include <iostream>
#include <string>
#include "protobuf.pb.h"
using namespace std;

int main(int argc, char** argv)
{
        GOOGLE_PROTOBUF_VERIFY_VERSION;

        data::Person p;

        p.set_name("Martin Vorbrodt");
        p.set_dob(19800830);
        p.set_email("martin@vorbrodt.blog");

        string binary;
        p.SerializeToString(&binary);

        data::Person p2;
        p2.ParseFromString(binary);

        cout << "Name  = " << p2.name() << endl;
        cout << "DOB   = " << p2.dob() << endl;
        cout << "EMail = " << p2.email() << endl;
        
        google::protobuf::ShutdownProtobufLibrary();
        return 1;
}

#include <iostream>

#include <string>

#include "protobuf.pb.h"

using namespace std;

int main(int argc, char** argv)

{

GOOGLE_PROTOBUF_VERIFY_VERSION;

data::Person p;

p.set_name("Martin Vorbrodt");

p.set_dob(19800830);

p.set_email("[email protected]");

string binary;

p.SerializeToString(&binary);

data::Person p2;

p2.ParseFromString(binary);

cout << "Name = " << p2.name() << endl;

cout << "DOB = " << p2.dob() << endl;

cout << "EMail = " << p2.email() << endl;

google::protobuf::ShutdownProtobufLibrary();

return 1;

}

This code is pretty self explanatory. We create, set and serialize one data structure of type data::Person, then deserialize it into another. The output is what we would expect:

Name = Martin Vorbrodt
DOB = 19800830
EMail = [email protected]
Program output.

That’s enough of an introduction. I hope you will read up on Protocol Buffers and realize the enormous potential of this library. Happy coding!

P.S. As always, complete source and build files available at my GitHub.

Well, that was no fun :(

Posted on March 9, 2019March 14, 2019 by Martin

No, not blogging, that’s still fun 🙂 moving my website to bluehost over the last 24 hours! But it’s finally done and the Vorbrodt’s C++ Blog is smooth sailing once again… on a brand spanking new domain! But have no fear, the old one will kindly redirect.

So what have I learned through this exercise? For starters website migrations, no matter how trivial (it’s only a basic blog with 2 pages and 46 posts after-all), just don’t go smooth. 95% of it will work, but then the annoying 5% that didn’t migrate will eat up 23 1/2 hours of your life trying to chase it down and patch it up! I can’t imagine moving a huge corporate or banking website… but I guess that’s what staging environments are for.

So my blog uses a custom theme, has few WordPress and Jetpack widgets, and few custom modifications to the php files by yours truly. Non of that migrated! See, I did an export followed by an import operation, and that only moved the data (posts, pages, comments). So I had to hunt down and reinstall theme and plugins on the new site; then I re-implemented the few php changes to make the theme to my liking again.

But the most irritating part of the export / import process was the fact that the post excerpts vanished into thin air! So when I fired up the site with the new theme all I cloud see were post titles… no excerpts on the front page! Luckily the post content was preserved perfectly. I then looked for and tried four or five different excerpts plugins to no avail, until I found the one I needed: Excerpt Editor. Nothing about it was automatic, but at least it let me rebuild, one by one, my post excerpts. Ufff. That took a while.

Once I got the page up and running the way it was before I immediately purchased a Jetpack Personal plan for $39/year which offers automatic site backups and restorations. It backs up the database (pages, posts, comments) as well as themes, plugins, uploads, etc. Hopefully I’ll never have to use it, but you know what they say… it’s better to have it and not need it, then need it and not have it 🙂
Oh, and the site hosting is $10/month, another $30/year for domain name and privacy protection on it. Pretty low price to pay for a total peace of mind!

Finally a word about my website hosting setup up to yesterday: yes it was done from home and a constant source of headaches! For starters I’m on a residential 1Gbit line with 40Mbit upload. Great for streaming 4K TV but not so hot for uploading website content (plus the upload latency was bad). Then there’s the whole dynamic IP address thing… so I had to create a sub domain with my google domain, enable DDNS service on it through my OpenWrt router, and pray it doesn’t change too often. Of course I couldn’t run the web server directly on the router, so I pushed it behind the firewall onto my Qnap NAS. Getting WordPress to run on this thing smoothly was an issue and required plenty of workarounds (like a custom cron job to pull wp-cron.php file every minute from the web server or else the internal WordPress tasks would get delayed and bad things would happen). Just a mess overall. Oh and don’t get me started on using LetsEncrypt certificates for https access. I love that they provide them for free, but for 90 days at a time! Really?! And then there was the weekend I spent figuring out how to serve the intermediate certificate along with my domain certificate from Qnap’s custom build Apache server… so I could get an A rating from SSL Labs 🙂

Anyways, too much venting and not enough C++ in this post so I’ll stop now!

P.S. If you got this far, please re-subscribe to email notifications if you’ve done so in the past. Those sadly didn’t survive the export / import process 🙁

Parallel STL

Posted on March 7, 2019March 18, 2021 by Martin

C++17 standard introduced execution policies to the standard algorithms; those allow for parallel and SIMD optimizations. I wanted to see how much faster the Parallel STL can be on my quad core system but non of my compilers currently support it. Luckily Intel has implemented it and made it available to the world 🙂

On a side note, in this post’s example I will be using several frameworks: TBB needed to compile the Parallel STL. And Catch2 to create the test benchmark. All are freely available on GitHub. BTW, thanks to Benjamin from Thoughts on Coding for pointing me toward the Catch2 library. It’s great for creating unit tests and benchmarks.

Let’s benchmark the following operations using STL and PSTL: generating random numbers, sorting the generated random numbers, finally verifying if they’re sorted. The performance increase on my quad core 2012 MacBook Pro with i7 2.3GHz is about 5x! Nice!

benchmark name iters elapsed ns average
———————————————————–
STL 1 10623612832 10.6236 s
PSTL 1 1967239761 1.96724 s
Program output.

#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
#include <vector>
#include <random>
#include <algorithm>
#include <pstl/execution>
#include <pstl/algorithm>
using namespace std;
using namespace pstl;

const unsigned long long COUNT = 100'000'000;

TEST_CASE("STL vs PSTL", "[benchmark]")
{
	auto seed = random_device{}();

	vector<int> data(COUNT);

	BENCHMARK("STL")
	{
		generate(data.begin(), data.end(), mt19937{seed});
		sort(data.begin(), data.end());
		is_sorted(data.begin(), data.end());
	}

	BENCHMARK("PSTL")
	{
		generate(pstl::execution::par_unseq, data.begin(), data.end(), mt19937{seed});
		sort(pstl::execution::par_unseq, data.begin(), data.end());
		is_sorted(pstl::execution::par_unseq, data.begin(), data.end());
	}
}

#define CATCH_CONFIG_MAIN

#include <catch2/catch.hpp>

#include <vector>

#include <random>

#include <algorithm>

#include <pstl/execution>

#include <pstl/algorithm>

using namespace std;

using namespace pstl;

const unsigned long long COUNT = 100'000'000;

TEST_CASE("STL vs PSTL", "[benchmark]")

{

auto seed = random_device{}();

vector<int> data(COUNT);

BENCHMARK("STL")

{

generate(data.begin(), data.end(), mt19937{seed});

sort(data.begin(), data.end());

is_sorted(data.begin(), data.end());

}

BENCHMARK("PSTL")

{

generate(pstl::execution::par_unseq, data.begin(), data.end(), mt19937{seed});

sort(pstl::execution::par_unseq, data.begin(), data.end());

is_sorted(pstl::execution::par_unseq, data.begin(), data.end());

}

Practical Modern C++

Year: 2019

Propagate exceptions across threads

int main()

XML-RPC

Base64 encoding

Extremely Fast Compression Algorithm

Parsing command line options

ANSI escape codes

Two-factor authentication

Plugins: loading code at runtime

Micro-benchmarks

SQL database access

Measuring CPU time

Printing stack traces

Generating Unique IDs

RPC with Protocol Buffers

Serialize data to XML

Thrift: or how to RPC

Protocol Buffers: or how to serialize data

Well, that was no fun :(

Parallel STL