Open Source Summit + Embedded Linux Conference + OSPOCon 2021

For Linux there are many tools available to perform system profiling. But sometimes there are problems where counting hardware and software events does not provide a clear picture. Such problems are often related to scheduling or inter-process communication. For these types of problems it is usually necessary to investigate not only what happened, but more importantly, when: Tracing. There are several tracing solutions available for Linux. This talk focuses on four of them: ftrace, BPF, SystemTap, LTTng. For a specific problem scenario, each tracing solution is used, the results presented, and finally compared. The solutions are rated based on their usability, their ability to present collected data, their effects on system runtime, and their availability for various kernels and distributions. This talk is targeted for developers with no or limited experience tracing in Linux. However, it should also be interesting for developers who have much tracing experience, but are only familiar with one or two of the tracing solutions.