fun with dlsym
Prerequisites:
typedef int (*hook_fn)(void **); hook_fn f;
A problem that often occurs when using dlsym is that it returns a pointer to void when you need a function pointer. In ISO C and C++ it is forbidden to convert between both:
f = dlsym(h, "hook");
When compiling C gcc you'll get the following warning: ISO C forbids initialization between function pointer and ‘void *’. g++ makes this even an error: invalid conversion from ‘void*’ to ‘int (*)(void**)’. After adding the needed cast there still remains a warning: ISO C++ forbids casting between pointer-to-function and pointer-to-object.
A common workaround is to trick the compiler by casting the location of the pointer you want to assign to to void **:
*(void **)(&f) = dlsym(h, "hook");
This shuts up gcc but with g++ you'll eventually get the following warning: dereferencing type-punned pointer will break strict-aliasing rules. I don't want to be distracted by those warnings and I tried to think of another way to get rid of them. My solution now is to cast dlsym to return the function type I want:
typedef hook_fn (*hook_dlsym_t)(void *, const char *); f = ((hook_dlsym_t)(dlsym))(h, "hook");