The last update was all about improvements that made Holster more efficient on the server. I'm glad to say those changes, plus a couple of recent updates, mean server processing is looking really good now. There are no more CPU spikes for the same processing load, which means Holster is happy running on small severs when given a reasonable amount of work to do.

The big change that allows this to work efficiently is that Holster now stores all radix trees in memory. This allows for much faster gets and puts less pressure on the radisk implementation by no longer requiring parsing from disk as often. This does mean you need to configure your application to provide Holster with enough memory. There is a new memoryLimit option which has a default of 500MB, which is fine for small amounts of data. If you plan to store more data you will need to increase this, but Holster also logs how much memory it's currently using so you will know when it's an issue. When it hits the limit it resets the cache and goes back to files on disk, so it does continue working but this process will probably repeat itself.

The other change required to keep servers running well was that clients needed to throttle their message sending. This also helps browsers not spike a user's CPU and memory because it's making requests that the server couldn't keep up with anyway. This led to some changes that resulted in orderly message queuing and processing, but introduced some timing issues now that have now been fixed in 1.0.17.

To check for these timing issues there's now a whole set of new system tests. I've continued using Claude to write these tests and help with debugging the timing issues which is working really well.

My project to test Holster started hitting 100% CPU the other day during peak processing times, so have been doing some investigation. It's not a big server, but Holster shouldn't need the resources it was using and this was a great opportunity to identify what needed improving.

This has led to a few performance improvement releases this week. The first was that the caching radisk was doing was only for the most recent file. This doesn't do very much when keys come in randomly for different files, and each key caches a new file. Holster is currently not designed to hold heaps of data, so it now just caches all files, which was a big improvement as reading from disk is expensive.

Next was improving the client throttling to use an actual processing queue. It previously tried to throttle with timeouts but this just caused messages to build up later. The advantage of a processing queue is that you can also return an error if the queue gets too long, because there is probably something wrong with your Holster queries if this happens (ie requesting too much data.)

Lastly the Holster website got an update! It now has a nice little visualization of connected clients.