http-test.com - An HTTP Test Website

For the past four years, as a background side-project I’ve been (more slowly and sporadically recently) working on developing sitemon3, a re-design and re-write of sitemon2 as a distributed system (rather than the monolith the previous version was), whilst also writing it in Rust as another vehicle to help learn and get better at Rust - but also to try and (re-)learn some of the workings of modern HTTPS, web development and SQL/databases due to what knowledge I previously did have in those areas having atrophied quite considerably over the past 15 years by virtue of being in the VFX / computer graphics industry working on other problems that generally didn’t require that type of knowledge (although still with occasional basic usages of web service APIs and various databases).

The web has seemingly changed quite significantly in that time, with many more websites now using client-side rendering, in addition to many websites now being behind CDNs (like Akamai and Cloudflare) that often are configured with “bot detection” via methods like TLS- and HTTP-handshake fingerprinting, and because of this, it’s now not as easy to just use vanilla libCURL to request pages and emulate web browsers. Instead using a full browser engine really needs to be used. This is not the end of the world, however I still personally wanted to develop a libCURL-based ‘engine’ (in addition to a full browser one) for requests.

So I decided recently it might be interesting to play around with making a very basic (at least initially) website that was designed explicitly with the purpose of being able to test basic HTTP user agents. There are several applications that exist that allow doing this locally (mostly designed to test HTTP API endpoints) and run as servers, however I decided to make something basic from scratch mostly for my own education and experience that is live on the internet.

So I’ve created http-test.com as an initial website which has some basic static responses currently. Presently it’s implemented using a Caddy config, but I also have it implemented using a self-written HTTP/1.1 webserver (with both C++ and Rust ports), which in the future I plan on using (likely still behind a reverse-proxy) to handle dynamic and configurable responses (i.e. like every 10 requests with a token, return an error to test things failing correctly - or random responses).