mal

Self Hosting

Fri, 13 Nov 2026 10:17:00 GMT

Self Hosting

Hello! This is like a First Post all over again, this time publishing from i.haza.website! 🎉️

I've held off posting a few updates so that I could finish the changes required to bring this blog over, and that day has finally come. Like with the docs and holster sites, there were a lot of incremental changes to make it happen. I will talk more about them in future posts, for now I'm happy to be writing in a nice editor, that auto saves updates, allows me to create drafts, and has some nice templating features built in.

There's quite a few features you need for a static site to publish a blog, one I really wanted from the beginning is multiple posts on the home page. This was added as Front Matter at the start of a template, which makes it capable of publishing pages that don't otherwise exist in your folder structure:

holster.haza.website

Mon, 02 Nov 2026 08:28:03 GMT

holster.haza.website

The Holster website has been gone since rebuilding i.haza.website, it's now back as a replacement for the github wiki docs with updated information for all API's and better examples 🎉

I Haza Website also got some updates as I was working on this, so it now does nice preview navigation while you're editing pages. You can start from one page in preview mode, navigate to another page you're working on and continue editing from there.

There's also a lot of code examples on the site so I added a Copy Code plugin that allows you to copy code blocks if you want to run them interactively in Node. There's also a theme switcher plugin for light and dark mode:

Holster 2.3.0

Mon, 19 Oct 2026 08:43:34 GMT

Holster 2.3.0

A new Holster release is available that provides await support.

Using Holster for building i.haza.website was very different than working on RSStream as it needed to be much more responsive to Holster data changes. Looking back at the code up until this point, it was full of await new Promise(resolve => { }) wrappers so that state could update in the correct order.

The latest Holster release removes all of this from application code by providing a Thenables interface onto terminal API calls. That sounds weird but it does exactly what you need, which is when you would otherwise provide a callback to the Holster API you can just await the call instead. Internally Holster knows when a callback isn't provided and can extend the response to include a then() function, if you were chaining the request then the extra function is ignored, otherwise it provides your data.

This works for get and next in the API, put requests can also be awaited but they now internally return a promise as they only ever returned an error or null anyway. It allows for simplifying get requests like:

// From wrapped calls:
const existing = await new Promise(resolve =>
  user.get("mediaRootId", resolve),
)

// To just:
const existing = await user.get("mediaRootId")

And previously put requests where you waited for the result were:

async function bumpProjectsVersion(user) {
  const now = Date.now()
  return new Promise((resolve, reject) =>
    user
      .get("projectsVersion")
      .put(now, err => (err ? reject(new Error(err)) : resolve(now))),
  )
}

Are now:

async function bumpProjectsVersion(user) {
  const now = Date.now()
  await user.get("projectsVersion").put(now)
  return now
}

It also makes it easier to work with try and catch blocks, but it's also nice that the old callback style isn't effected by these changes when you want to use it.

I Haza Website

Sun, 18 Oct 2026 05:14:28 GMT

I Haza Website

Happy to share that i.haza.website is now available again as a new local-first app!

It was previously a PHP app with a similar goal to what the new version is doing, which is to make it easy to publish web pages. The new version is a web app backed by Holster, and besides publishing websites is great for taking notes and transferring media between devices. Holster provides automatic syncing between devices that you're logged in to, as well as encryption and offline support.

This has been another app where the complimentary goal has been Holster improvements alongside having useful tools. This has been true again because I Haza Website is much more interactive with the Holster API then other apps, so it's caught a few things that have needed fixing.

This blog hasn't been replaced yet because I Haza Website doesn't have some of the blogging features of the old app. Expect that to change soon though, I'm looking forward to moving to the new interface it's much nicer to write in. 😊

Nested listeners

Thu, 03 Sep 2026 09:15:31 GMT

Nested listeners

Another Holster release to add a new nested listener feature that I've needed for multiple projects now. In RSStream I added it outside of Holster itself, since I wasn't sure how often I would need it.

The idea is that you add a listener to a query in Holster, but that query always refers to a particular node. That node can reference other nodes, but those references don't change. So even though the API chains nodes together you need to explicitly list which ones you want to listen to changes on. The external way to do this is to create on() callbacks for each of the child nodes in the parent's on() callback. This works fine, but the code is really the same no matter what the project. So it's now included with Holster and just needs the extra parameter to the parent query to set up all listeners below that point. They're also automatically removed when the parent off() is called.

I asked Claude for a simple example and it of course gave a stand alone example that works in Node 🤖:

import Holster from "@mblaney/holster/src/holster.js"

const holster = Holster()
const user = holster.user()
  
user.create("alice", "password", () => {
    user.auth("alice", "password", () => {
      // Without {nested: true}, this listener only fires when a NEW key is
      // added to "team" - it won't fire again for a later, value-only
      // change to an EXISTING member, because that's a write to the
      // member's own soul (every object-valued child gets its own), not to
      // "team" itself.
      user.get("team").on(members => {
        console.log("team changed:", members)
      }, true, {nested: true})

      // A brand new key - the listener above always sees this.
      user.get("team").next("alice").put({role: "engineer"})

      // A value-only change to an ALREADY-KNOWN key - only reaches the
      // listener because of {nested: true}. Without it, this write would
      // be silently missed.
      setTimeout(() => {
        user.get("team").next("alice").put({role: "lead engineer"})
      }, 100)
    })
})

Concurrency

Wed, 19 Aug 2026 08:52:52 GMT

Concurrency

New Holster release focused mainly on concurrency fixes. I've been working on a new app that runs on Holster which is very front end heavy and running all sorts of async requests, and creating lots of tests for it at the same time. The tests started getting pretty complicated, matching the types of interactions involved, and these all run on top of real holster test instances. The tests started getting a bit flaky as they exhausted their memory limit and swapped out cache files.

I realized Holster has never had any specific concurrency tests, so it was interesting adding them and looking at what sort of race conditions were causing problems. I have to say that Claude is getting really good at methodically working through scenarios without getting lost in the details. I'm impressed with the way it found the issues that have been fixed in this release and the relatively simple changes that were made.

Holster browser

Sun, 12 Jul 2026 06:25:23 GMT

Holster browser

I've created another npm package that is complimentary to holster-router, this one is called holster-browser. It's another extraction of core utilities and components from RSStream so I can use them in other projects.

So now if you want to use Holster with Express and React, it's super easy to put them all together and have a working authentication system. Holster is still completely independent, but the other two side projects assume the same endpoints and account structures. I'll probably add to both as I need extra features and not worry so much about publishing the applications that use them.

Holster router

Sun, 05 Jul 2026 02:15:13 GMT

Holster router

Lots of updates this week, I gave a presentation on Holster at the June LoFi meetup and published a new npm package. The meetup was at 1am for me, so not the best time to be trying to explain how Holster works! 😅 but it was great to be invited to speak and hear from others doing similar work. I've also been going back and watching previous LoFi episodes which are really interesting and great to see how much work is going on in the local first application space.

The npm package I've released is called holster-router and is an extraction of some of the work that went into RSStream. I've done that so I can re-use the same routes and user creation process in a new Holster based project that I'm about to start. RSStream uses the concept of invite codes to control accounts, but the new project will be more open than this, so the holster-router functions that are shared between these projects will allow for more than one way to create an account. The recent work that went into user storage limits should mean that sign-ups can be open without worrying about server storage. All the other routes for managing password updates and resets will be shared and the new project will add new routes to holster-router as the sign-up model for that is worked out.

Holster 2.1.0

Thu, 18 Jun 2026 08:24:30 GMT

Holster 2.1.0

A new version of Holster is available that provides a new feature I've wanted to add for a while. This is per-user storage limits. It is off by default, but starting Holster with a new userLimit option set to true will check how much data is stored under a public key before writing updates to disk. This is a server only feature and allows controlling your disk usage if you have limited resources.

One of the things I often thought about when learning about GunDB was how you were meant to run a relay when all data is distributed. Holster is careful to only store data that browsers request, but if you host a public relay I think you should be able to decide how those resources are used. Holster already had options for memory limits, so now it has options for disk usage too. The default is only 1MB, but can also be set in the options when starting. If you set it to zero then unlisted users are effectively blocked from storing data on your server.

Listed users should be added to a simple .user_limit.json file in your application directory, it's just an object containing public keys and their allowed storage in megabytes. Your application can update this file for Holster to read from, which is what RSStream is doing now. Holster then checks it periodically while also tracking user storage in a second file called .user_storage.json. For more information and settings to update when running a relay see the Holster API docs.

Future work for this feature can be done at the application level, for instance you can provide feedback to users about how much storage they're using and provide an endpoint to increase it. Note that this is only backup storage on your server, all their data is still stored locally if you use IndexedDB.

Switched to Vite

Wed, 10 Jun 2026 09:07:17 GMT

Switched to Vite

I just updated RSStream to build using Vite instead of the old CRA tools. I didn't realize it would be such a small change otherwise I would've switched over earlier! 🎉 I've also removed the build folder from the github repo, which was suggested I do for Holster to avoid pr's being merged where the build files don't match the source change. The Holster github workflow runs the build before publishing to npm, for RSStream this means an extra step, but it's worth it to make sure the build matches your files.

RSStream has replaced my previous (also homegrown) reader and I'm not finding I want to add any other features at this stage. That means it's time to move on to another project... the next one involves adding new Holster features that I'm looking forward to being able to use, so stay tuned for updates.

RSStream walk through

Wed, 20 May 2026 09:24:19 GMT

RSStream walk through

RSStream is working well now, check out the walk through video below!

It starts with signing into the app, then demonstrates bookmarking, loading older items on scroll-back and search. The crazy thing that I really wanted to show however is that the whole video is offline! 🤯

This is thanks to Holster storing everything in indexedDB and the service worker cache storing images. I'm now also working on storing audio and video for offline playback and allowing cache management too.

Application development

Thu, 07 May 2026 08:24:41 GMT

Application development

The last few weeks have felt like Holster is capable enough to now focus on some application development. Previously trying to work on an app that used Holster meant constantly jumping back into the Holster code trying to fix edge cases or find performance improvements.

But I've recently been able to focus on improving RSStream and haven't had to context switch too much. I've just made a small Holster 2.0.1 bug fix release, but besides that it's feeling solid.

The real improvement I've found is not in any Holster change, but in the data model I'm using. A feed reader is possibly the worst starting application to test Holster with, it's too much data arriving all the time, and it's always new! What I realised is that I was treating everything as graph data, starting from feeds all the way down to properties on an item. But unless you're using the ability to query data on the graph, it's a massive performance hit to create references for every piece of graph data. So at some point it's good to stop and decide what can just be stored as strings.

When I was looking into this Claude reported stringifying items would reduce reference look ups from O(N) to O(1)... I didn't believe this, so cleared the session and started again, but same result! So I looked further and realised that as well as saving all the property references on an item, the whole item itself didn't need a reference created for it as it became a simple property on it's parent node. This is a big deal for a feed reader since the N mentioned here is the number of items per day per feed, so with this simple change the app feels usable now.

Holster 2.0

Mon, 27 Apr 2026 11:37:04 GMT

Holster 2.0

A 2.0 release sounds like it should be a big update, and that actually has turned out to be true here, but more by coincidence. Holster uses semver for version numbers, so this 2.0 update is really just for a small breaking API change: a server now needs to be started with either an explicit port number or a websocket server in their config, there's no longer a default port as a fallback.

This was required because some browser testing frameworks were breaking on the const isNode = typeof document === "undefined" check as they also don't define document. The updated check const isNode = typeof process !== "undefined" && process.versions?.node != null, is a better test for server environments but again is also true for testing libraries like jsdom. So the fix isn't to rely on isNode in the tests, but to explicitly require a port when running a server. The real benefit of this isNode change is that Web Workers should now be supported, which is enough of a reason to make the change. But the reason this has become a big update is because I made some radisk changes, and then didn't want to publish those until I was confident it was stable. This meant I just kept adding other patches while watching how the radisk change was performing.

One other API change is that there is no need for a wait parameter on requests, so there's no reason for an options object at all now. This was previously helpful if you wanted to wait longer for certain queries, but this update has dealt with timeout issues internally so there's no longer a reason to specify it. So requests are either fast if you've requested a key previously, or given much more time to check the network if it hasn't been seen before.

My last update mentioned using private browsing mode for testing, but even better than that is going offline and seeing how Holster handles fetching data. The honest answer was not very well, requests would fall through to the network and get stuck in a queue that couldn't be processed. But this also made it quite easy to find what needed improving. When a query is made via the API, the first place to check is in-memory, ie "the graph". There were a few places where it wasn't capturing everything available, so requests would fall through to disk.

Next there were the radisk improvements I mentioned. The change was quite simple, radisk shouldn't ever really time out unless there's an actual problem reading from disk. So timeouts have been mostly removed, file lookup has got faster and file splitting has improved too. It would previously split into the configured size and a smaller file for the overflow, now it aims for half each which is what GunDB tries to do too.

So now if the browser has previously synchronised all available data, no network requests are required. Lastly if something does fall through to a network request when offline then there's much better recovery there too.

I'm back

Tue, 31 Mar 2026 07:50:01 GMT

I'm back

Hello! This is one of those "I can't believe I haven't written anything in so long" posts where I try and explain why my last update was in November... well it's enough to say that life can get busy and I'm excited to be able to get back to some Holster development.

My focus at the moment is actually on application development on top of Holster, but that has just found all sorts of things that have needed fixing. They are mostly to do with timing, either timeouts from slow networks or stale cache data arriving before other updates on a callback. The easiest way to test these issues is with a private browsing window that forces a network fetch, so that has become part of my testing routine.

The latest release fixes a lot of bugs, some of which could be considered new features... for instance it was always possible to listen to a path on the graph that doesn't exist, but the listener now waits for that path to be converted to a node and attaches the original listener to that. That simplifies listeners from an application point of view as they now don't need to check that condition. There's also more retrying of underlying wire requests which mean applications don't need to check if Holster is returning something useful.

Besides that there's now some nice data consistency fixes as in-memory data was available but not being returned before a lookup was being done, and store was missing some error callbacks that mean better data checking is now happening.

These last few fixes were found by Claude, which I was using for development last year, but as a lot of people are noticing AI is getting better at code changes. It's only been a few months but it previously wouldn't find these sorts of problems so it's definitely getting more useful.

Timestamp Signatures

Sun, 09 Nov 2025 13:05:21 GMT

Timestamp Signatures

So long per property signatures, we hardly knew ye! ...I ran the per property signature update on my test server and it blew up my CPU usage with all the extra processing it was doing.

We couldn't go back to node based signatures because they're too hard to verify, so this meant coming up with a new solution. I realised we didn't need to sign content at all, we're just trying to verify ownership of the update, so signing the timestamp associated with the the update is enough.

This is great because generally you're putting a bunch of properties onto a node at once, so they all have the same timestamp, but they only need one signature. If you do update individual properties on a node then it will get it's own timestamp and an updated signature with it.

This has meant my test server is back to running normally, update is available in Holster 1.0.25.

Holster 1.0.24

Tue, 04 Nov 2025 10:25:52 GMT

Holster 1.0.24

I've just published Holster 1.0.24 which is quite a major change, as signatures are now added per property rather than per node. Before making this change I was trying to debug a signature validation issue on a node, but there was too much data being signed to know what could've gone wrong.

This led to simplifying signatures to each property, which actually fixed a whole bunch of other problems in Holster. Signing a node means you always have to fetch the full node to update it even though you might only be updating one property. There were also timing issues around node updates that aren't a problem any more.

There were a few other recent releases too, including adding Docker support. See the updated README for how to start the server with either Docker or Node.

Migrating from Gun to Holster

Mon, 20 Oct 2025 06:20:57 GMT

Migrating from Gun to Holster

The last few weeks have seen a lot of development activity, first in Holster itself then in applications using it. This has been driven by the Free Association project, which is now capable of using Holster instead of Gun. The start of the migration process required some further updates to Holster, which is now at version 1.0.21 and feeling pretty stable!

The rest of the work was done in switching API's in the application itself. Holster's API is not only slightly different to Gun, but also conceptually different. The biggest difference is that Gun streams data whereas Holster resolves all references before returning. This doesn't mean much from a performance perspective, but it does significantly change your application code. I've done a few of these migrations now so I already had the notes from the first one, which I've used again successfully since. I'm sharing them here as hopefully they're general enough for other migrations to benefit from.

On the server I've been developing with both Node and Bun, the first step in Node is:

npm uninstall gun
npm i @mblaney/holster

My previous Gun initialization code was:

import Gun from "gun"
import "gun/lib/then.js"
import "gun/sea.js"

const gun = Gun({
  web: express().listen(8765),
  multicast: false,
  axe: false,
})

This has been replaced by:

import Holster from "@mblaney/holster/src/holster.js"

const holster = Holster()

Holster is a module so the path to the file is required. If you're not using a build tool you can switch src to build in the path above. The only option I left out of both examples above was secure: true, which means only data under user accounts will be saved (no public data).

Here is the browser code I previously used for Gun (also not including secure here):

import Gun from "gun"
require("gun/lib/radix.js")
require("gun/lib/radisk.js")
require("gun/lib/store.js")
require("gun/lib/rindexed.js")
require("gun/lib/then.js")
require("gun/sea")

const gun = Gun({
  // Assume Gun is directly available on localhost, otherwise reverse proxy.
  peers: [
    window.location.hostname === "localhost"
      ? "http://localhost:8765/gun"
      : window.location.origin + "/gun",
  ],
  axe: false,
  localStorage: false,
  store: window.RindexedDB(),
})

And here is the browser code for Holster. Note that it doesn't rewrite the scheme for WebSockets like Gun, so you need to specify it or use the default:

import Holster from "@mblaney/holster/src/holster.js"

// If on localhost assume Holster is directly available and use the default
// settings, otherwise assume a secure connection is required.
let peers
if (window.location.hostname !== "localhost") {
  peers = ["wss://" + window.location.hostname]
}

const holster = Holster({peers: peers, indexedDB: true})

The rest of these notes are for the API's in general, so apply to both browser and server. First is error handling, where Gun returns an object with an err property Holster returns just an error. This means a lot of code changes from ack.err to just err.

The SEA functions work pretty much the same, except Gun uses encryption keys under user._.sea, Holster simplifies this to rely on the user.is object. They can also be used on their own with import SEA from "@mblaney/holster/src/sea.js"

I used user.get(key).once(callback) a lot when working with Gun. In Holster, that pattern is now just user.get(key, callback). I would also previously use user.get(key), and pass that around as a reference to use later. Holster removes the context that gets created once a callback is provided, so that isn't possible and you instead need to write the full query each time.

As mentioned above, the big difference is between streaming data and resolving nodes. So when using Gun you could do: get("items").map().once((item, key) => callback) to get a stream of updates. In Holster it's just get("items", items => callback). So to get the equivalent of Gun's map() you would do: Object.entries(items).forEach(([key, item]) => {callback(item, key)}) on the returned data if you wanted to use the same callback.

Also get(key).map().once(callback) subscribes to updates in Gun, you would need to use get(key).on(callback) instead in Holster for updates.

Gun has get(key).then() which is nice for awaiting data, but it's just a convenience function which can be done in Holster with: await new Promise(res => {get(key, res)})

The biggest change in terms of lines of code has generally been modifying get().get() chains from Gun, to get().next() in Holster. One more thing to watch out for is calling put() in a loop, it's better to use an actual for loop rather than forEach and awaiting each put() so they don't make updates at the same time, especially to the same node.

Holster 1.0.19

Wed, 08 Oct 2025 13:52:33 GMT

Holster 1.0.19

This is another bug fix and performance update, message queuing felt a little slow so I've bumped up queue processing from 4 messages per second to 10, we will see how that goes.

The other fixes are more data sync related than timing, the first is that listeners can subscribe to nodes, but the underlying data can change from a property to a reference. A bit of an edge case but this led to listeners not resolving full objects, which is now fixed.

Next was that updates that didn't change the in-memory graph weren't going out on the wire, so other devices on the network weren't alerted. There's also some better checks for when a websocket isn't available messages won't be dequeued until they are actually sent.

Post-performance improvement updates

Sun, 05 Oct 2025 06:08:59 GMT

Post-performance improvement updates

The last update was all about improvements that made Holster more efficient on the server. I'm glad to say those changes, plus a couple of recent updates, mean server processing is looking really good now. There are no more CPU spikes for the same processing load, which means Holster is happy running on small severs when given a reasonable amount of work to do.

The big change that allows this to work efficiently is that Holster now stores all radix trees in memory. This allows for much faster gets and puts less pressure on the radisk implementation by no longer requiring parsing from disk as often. This does mean you need to configure your application to provide Holster with enough memory. There is a new memoryLimit option which has a default of 500MB, which is fine for small amounts of data. If you plan to store more data you will need to increase this, but Holster also logs how much memory it's currently using so you will know when it's an issue. When it hits the limit it resets the cache and goes back to files on disk, so it does continue working but this process will probably repeat itself.

The other change required to keep servers running well was that clients needed to throttle their message sending. This also helps browsers not spike a user's CPU and memory because it's making requests that the server couldn't keep up with anyway. This led to some changes that resulted in orderly message queuing and processing, but introduced some timing issues now that have now been fixed in 1.0.17.

To check for these timing issues there's now a whole set of new system tests. I've continued using Claude to write these tests and help with debugging the timing issues which is working really well.

Performance Improvements

Wed, 10 Sep 2025 09:51:43 GMT

Performance Improvements

My project to test Holster started hitting 100% CPU the other day during peak processing times, so have been doing some investigation. It's not a big server, but Holster shouldn't need the resources it was using and this was a great opportunity to identify what needed improving.

This has led to a few performance improvement releases this week. The first was that the caching radisk was doing was only for the most recent file. This doesn't do very much when keys come in randomly for different files, and each key caches a new file. Holster is currently not designed to hold heaps of data, so it now just caches all files, which was a big improvement as reading from disk is expensive.

Next was improving the client throttling to use an actual processing queue. It previously tried to throttle with timeouts but this just caused messages to build up later. The advantage of a processing queue is that you can also return an error if the queue gets too long, because there is probably something wrong with your Holster queries if this happens (ie requesting too much data.)

Lastly the Holster website got an update! It now has a nice little visualization of connected clients.