Further Average Cloud Cover Satellite Imagery

Ever since I previously generated median average images from multiple days of satellite imagery, I’ve wanted to try and generate newer higher-resolution versions, as well as to do so for other areas of the Earth’s surface, and also to try generating per-month versions, rather than just whole-year and per-quarter versions as I did previously.

So over the past few weeks I’ve been attempting this again, however due to time constraints, I didn’t manage to achieve a significant increase in resolution.

Australasia:

I originally wanted to try and increase the original downloaded resolution of the imagery for each day (I started with 2023) of the year to 128k x 64k (131072 x 65536), however as that was around 16x larger (compression and missing tiles means it doesn’t quite scale linearly) than the previous largest resolution I attempted of 32k x 16k in terms of download size and storage size, I quickly decided to compromise on 64k x 32k instead, as that was more practical.

Because of the larger size on disk for these on my Linux desktop, I also didn’t have enough SSD storage space (at least for an entire year’s worth of daily images at once), so I ended up having to use a 4 TB 3.5" hard disk I had spare, connected using an external caddy with a USB 2.0 connection.

This ended up being very slow when processing the images into median averages, as it meant that having more than three parallel threads doing different TIFF tile positions simultaneously just ground things to a halt (previously I’d scaled it up to around 12 threads almost linearly when using an SSD).

I did think about changing my image processing algorithm to cope with this a bit by using “chunks” of consecutive TIFF tiles (so say four at once per-thread), in the hope that more sequential reading and less random reads would be better, but in the end decided to just run things in the background for as long as they took.

Northern Africa / Arabian Peninsula:

So far, I’ve only created averages for the whole of the year of 2023 as well as the quarters of that year, in addition to new sub-region projections of the world: Australasia (unfortunately the wrap-around east of New Zealand needs special handling using GDAL’s warp algorithm, so there are artefacts next to New Zealand), and also of Northern Africa and the Arabian Peninsula.

Europe / Arctic:

But I do want to try and at least get one year’s of per-month averages generated in the future, as I think that will show more interesting variation than the current per-quarter sub-sets do.

North America:

Links to 4K versions:

I will be progressively uploading the per-quarter versions to my main website’s Media section in the coming weeks.