smhk

Using gprof with POSIX FreeRTOS

TL;DR: To profile a C program with gprof, build and link with -pg, and ensure that the program terminates cleanly (e.g. handles SIGINT and SIGTERM). Then it will generate a gmon.out file that you can analyze with gprof. If running the program under systemd, be sure to set GMON_OUT_PREFIX and WorkingDirectory=, so that it writes gmon.out.<pid> to a known location.

A good option for profiling C code is The GNU Profiler, or as it’s more commonly known, gprof. The official documentation is excellent, but for my benefit, here is a summary.

There are three main steps to using gprof:

  1. Compile and link your program with profiling enabled.
  2. Execute your program to generate profile data, typically a file named gmon.out.
  3. Run gprof <program> gmon.out to analyze the profile data.

Compiling and linking with profiling enabled §

If using gcc as your compiler, enabling gprof support is as simple as adding the flag -pg to the compilation and linking stages1. See the official docs for more details.

Additionally, you can add the -g flag at the compilation stage to enable line-by-line profiling.

For example, if using SCons as your build system:

SConstruct
profile = int(ARGUMENTS.get('PROFILE', 0))

# ...

if profile:
    env.Append(CCFLAGS=[
        "-pg",  # Required for profiling
        "-g",   # (Optional) Enable line-by-line profiling
        ])
    env.Append(LINKFLAGS=[
        "-pg",  # Required for profiling
        ])

Then you would run scons PROFILE=1 to build with profiling.

Executing your program to generate profile data §

Run your program as normal. When it exits, it should create a file named gmon.out in its working directory.

If your program does not exit cleanly, e.g. due to an unhandled signal, then gmon.out will not be created!

This point is crucial when profiling POSIX FreeRTOS. Prior to this pull request, merged in 2021, the POSIX FreeRTOS demo program did not handle the SIGINT signal2, which is received when Ctrl-C is used to terminate a program. It also does not handle the SIGTERM signal, which is received when a program running under systemd is stopped by systemd stop <service>.3

Fortunately, it is simple to modify your C program to exit cleanly in both these scenarios:

#include <signal.h>

// ...

void exit_cleanly(int signal)
{
    exit(1);
}

int main(int argc, char *argv[])
{
    // ...

    // Handle SIGINT, so that Ctrl-C will exit cleanly.
    signal(SIGINT, exit_cleanly);

    // Handle SIGTERM, so that "systemd stop <service>" will exit cleanly.
    signal(SIGTERM, exit_cleanly);

    // ...
}

This will enable the gmon.out file to get generated when the program is terminated by Ctrl-C or systemd.

Execute your program under systemd §

In addition to handling SIGTERM for systemd support as described above, here are a couple of additional points.

  • Setting the environment variable GMON_OUT_PREFIX allows changing the name of the gmon.out file, but crucially also appends the PID of the process to the end of the filename4. If multiple instances of the same program are running under systemd, this makes it possible to easily profile them all simultaneously, and to match the profile data to the processes. I recommend setting GMON_OUT_PREFIX=gmon.out, and then you will end up with files named gmon.out.<pid>.
  • Since the gmon.out file is written to the program’s working directory, it may help to update the systemd service file to specify the working directory. This is as easy as adding WorkingDirectory=/var/log/ under the [Service] section.

With both of these points applied, after running the service under systemd and then stopping it, you should find file(s) at:

/var/log/gmon.out.<pid>

Run gprof to analyze the profile data §

Analyzing the data is outside the scope of these notes. See the manual for details.


  1. Supposedly, a bug in gcc versions 5 and 6 may mean that the flag -no-pie is required. At time of writing I am using gcc 11, so this is not an issue. Run gcc -dumpversion to check. ↩︎

  2. It’s unclear exactly which release this pull request first appeared in, but given the date I would guess it was included in FreeRTOSv202107.00 at the earliest. The changelog appears to be here, but it is not exhaustive, and the section titled “Changes between FreeRTOS 202104.00 and FreeRTOS 202107.00 released July 2021” only lists three points, none of which relate to the SIGINT handling. Nowhere in the changelog mentions SIGINT or signal handling. ↩︎

  3. Admittedly, it’s probably quite uncommon to run POSIX FreeRTOS as a systemd service, but there are legitimate reasons! ↩︎

  4. The GMON_OUT_PREFIX environment variable does not seem well documented. It is mentioned in this answer, though you actually need to set it at run time, not build time, as pointed out by this answer. The environment variable does not appear anywhere in this documentation, but it does appear here with the signal sentence: “Set the GMON_OUT_PREFIX environment variable; this name will be appended with the PID of the running program”. ↩︎