Clearing Linux File System cache

The Linux kernel has a file system cache which can significantly speed up repeated file operations made by applications on the same file, both in terms of file / directory stat() calls (for metadata) and reading the contents. This cache makes use of unused system memory (RAM) to store the metadata and contents of previously-accessed files.

However, when testing or benchmarking file I/O performance on Linux, this file system caching system can obfuscate performance of file operations because the first time a stat() or read operation is done on a file can take some time, and then repeated subsequent tries can then be almost instant due to the file system cache.

In this situation, clearing (or ‘dropping’) the file system cache is the solution, whereby this operation is done before every test run.

The command to do this and completely flush / drop the file system cache (using sudo) is:

echo 3 | sudo tee /proc/sys/vm/drop_caches

Valgrind Debugging common usage

Valgrind is a tool on Linux (it might work on some other UNIX platforms as well?) that allows validating and debugging memory accesses of applications (in addition to other things like profiling cache accesses and validating some thread locking correctness), which can be very useful for debugging issues in applications written in non-‘safe’ programming languages like C and C++. It can help identify issues like use-after-free, buffer overruns and other similar issues which conventional debuggers (gdb, lldb) might not always catch until is too late (i.e. you just see the segfault, not the original root cause event).

It can also track down memory leaks as well - although other more specialised applications like heaptrack and bytehound can in some cases be better tools to use for just profiling memory usage (as opposed to memory correctness as well), as these other applications just concentrate on that and are in some cases much faster when debugging your application compared to fully running through valgrind. However in some cases I have found both heaptrack and bytehound have produced non-existant (effectively not possible) stacktraces for memory allocations, whereas valgrind doesn’t seem to have this issue.

Additional args:

--track-origins=yes will track the origin of all memory allocations which can be helpful to work out what memory was causing any particular issue.

--leak-check=full will perform full memory leak checking.

--max-threads=2000 or similar is needed if the application being profiled/debugged uses more than the default of 500 threads valgrind can cope with.

--log-file=/tmp/valgrind_run1.txt will tell valgrind to write the output (which can be incredibly verbose) to a file instead of stdout which is often a good idea.