Changing Mac Unified Memory GPU limit

Apple Silicon Macs (M-series CPUs) have “unified memory”, where the the CPU and GPU can share the same core memory.

By default however (at least on MacOS 13 and 14), the amount of memory that can be “wired” or allocated to the GPU is only around 66-75% of the total system memory (the ratio depends on the amount of system memory), but it’s possible to change this and increase the amount of memory the GPU can access.

On MacOS 14 - Sonoma:

sudo sysctl iogpu.wired_limit_mb=27700

On MacOS 13 - Ventura:

sudo sysctl debug.iogpu.wired_limit=27700

Will allocate just over 27 GB of memory to the GPU, while still leaving a bit left dedicated to the CPU.

Format a number with thousand separator characters in Rust

The following code function snippet is an example of how to format a number (already in String/&str form) with thousand separator characters in Rust:

pub fn thousand_separate_string_number(string_val: &str) -> String {
    let mut s = String::new();
    let mut local_copy = string_val.to_string();
    
    let decimal_pos = local_copy.rfind('.');
    if let Some(pos) = decimal_pos {
        // deal with any decimal place digit and following digits first
        s.push_str(&string_val[pos..]);
        local_copy = string_val[..pos].to_string();
    }
    
    let chrs = local_copy.chars().rev().enumerate();
    for (idx, val) in chrs {
        if idx != 0 && idx % 3 == 0 {
            s.insert(0, ',');
        }
        s.insert(0, val);
    }
    s
}

Get length of longest String in a Vec in Rust

The following code snippet is an example of how to find out the length of the longest String in a Vec using Rust:

let string_vec = vec!["One", "Two", "Three", "Twenty"];
let max_string_length = string_vec.iter().map(|t| t.len()).max().unwrap();
eprintln!("Max string length: {}", max_string_length);

Check what the active network interface speed is on Linux

Linux command to print out the active speed (in MBit/sec) of the primary (IPv4) physical network interface (non-wireless).

ip -o -4 route show to default | awk '{print "/sys/class/net/"$5"/speed"}' | xargs cat

Note: this won’t work for wireless interfaces, and virtual interfaces (i.e. many VPSs) will incorrectly show -1 for this, it’s only reliable for reporting on physical network interfaces.

Deleting 'DS_Store' files recursively

MacOS uses additional hidden .DS_Store files in directories to store additional metadata about the directory which is only specific to MacOS.

Unfortunately, by default, many file synchronisation tools (rsync, syncthing) by default synchronise these files to other non-MacOS systems (although it is possible to manually configure them to ignore specific filename patterns), which can be annoying.

The following UNIX (should work on GNU/Linux, MacOS probably other platforms) find command will recursively find files with that filename starting in the current directory, and both print the name of the found files out to stdout and delete them:

find . -name ".DS_Store" -type f -print -delete

Plotting time-series data with matplotlib

Below is an example of how to plot time-series data (data values attached to actual dates) read from a .csv (comma-separated value) file, with matplotlib in Python.

Note: the code uses extra modules than is strictly necessary for convenience: pandas for parsing date formats, and seaborn for styling the plots.

import matplotlib as mpl
import matplotlib.pyplot as plt
import datetime
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.dates as mdates
from matplotlib.dates import DateFormatter
from pandas.plotting import register_matplotlib_converters

register_matplotlib_converters()

# read values from specified .csv file
input_file = "/tmp/path_to_input_file.csv"

date_values = []
plot_values = []

data_file = open(input_file, "r")
for line in data_file:
    # skip empty or commented lines
    if len(line) == 0:
        continue
    if line[0] == '#':
        continue

    # make sure there's at least one comma separator in the line
    if not ',' in line:
        continue

    # split first two columns, into date and value for that date
    pair = line.split(',')
    date_str = pair[0].strip()
    value_str = pair[1].strip()

    # parse the date string into an actual date type
    date_value = pd.to_datetime(date_str, format="%d/%m/%Y")

    date_values.append(date_value)
    # Note: for floating point values, this cast to int() will need to be changed
    plot_values.append(int(value_str))

sns.set()
sns.set_style('darkgrid', {'axes.linewidth': 0.5, 'axes.edgecolor':'black'})

fig, ax = plt.subplots(1, 1)

fig.tight_layout()
fig.set_figwidth(17)
fig.set_figheight(9)

ax.plot(date_values, plot_values)

# set up comma thousand-separator formatting for the Y-axis values
ax.get_yaxis().set_major_formatter(mpl.ticker.FuncFormatter(lambda x, p: format(int(x), ',')))
ax.get_yaxis().set_minor_locator(mpl.ticker.AutoMinorLocator())

ax.grid(visible=True, which='major', color='w', linewidth=0.8)
ax.grid(visible=True, which='minor', color='w', linewidth=0.4)

# plot minor X-axis gridlines every month
ax.xaxis.set_minor_locator(mdates.MonthLocator(interval=1))

# X-axis values start with the first date value and end with the last date value by default
x_axis_start_date = date_values[0]
x_axis_end_date = date_values[-1]

# uncomment this to push the X-axis end date limit further by 12 days to pad things a bit
#x_axis_end_date += datetime.timedelta(days=12)

ax.set_xlim([x_axis_start_date, x_axis_end_date])

fig.tight_layout()
plt.show()

Clearing Linux File System cache

The Linux kernel has a file system cache which can significantly speed up repeated file operations made by applications on the same file, both in terms of file / directory stat() calls (for metadata) and reading the contents. This cache makes use of unused system memory (RAM) to store the metadata and contents of previously-accessed files.

However, when testing or benchmarking file I/O performance on Linux, this file system caching system can obfuscate performance of file operations because the first time a stat() or read operation is done on a file can take some time, and then repeated subsequent tries can then be almost instant due to the file system cache.

In this situation, clearing (or ‘dropping’) the file system cache is the solution, whereby this operation is done before every test run.

The command to do this and completely flush / drop the file system cache (using sudo) is:

echo 3 | sudo tee /proc/sys/vm/drop_caches