Holster Performance Improvements

The great thing about writing a complicated application that relies on Holster being responsive to requests, is that you quickly find where the bottlenecks are. Claude also knows that the two projects are heavily related and is happy to switch between code bases when it find a problem in either. So when I Haza Website started slowing down with lots of pages in a project, it will go looking at the Holster code pretty quickly when looking for a reason.

The Holster 2.3.5 release is the performance improvements it found, with an impressive result that Holster now runs it's test suite in 3.5 seconds, down from nearly 6 seconds (in parallel). This is mostly thanks to removing a retry check, which made sure references to nodes existed before they're used. This was definitely a good check to have, but a concurrency issue fixed in an earlier release inadvertently removed the need for it. It did this by serializing rel creation for the same soul and property.

The retry meant many requests were waiting at least the minimum 50 milliseconds just to be sure it existed, but it was no longer necessary and there are specific tests that show that. I Haza Website has it's own tests and these went from 150 seconds down to under 60 seconds to run the full test suite.

Some other improvements went into this release, such as only responding to a client making a request rather than broadcasting the response to all clients. This reduces wire chatter while allowing actual changes to propagate as expected. Clients already ignore data they haven't asked for so the extra broadcasts weren't being listened for anyway.

Another improvement for browser requests is relaxing the message queue rates when making requests. It was initially designed to prevent overloading both browser CPU and relays that clients request data from, but it can do this while also being responsive to application requests which tend to occur in bursts. Requests can now go out in batches of up to 100, instead of waiting 50 millisecond between each request. Browsers now also check if they're connected to any peers, which means there's no reason to wait before resolving the request when offline.

Self Hosting

Hello! This is like a First Post all over again, this time publishing from i.haza.website! 🎉️

I've held off posting a few updates so that I could finish the changes required to bring this blog over, and that day has finally come. Like with the docs and holster sites, there were a lot of incremental changes to make it happen. I will talk more about them in future posts, for now I'm happy to be writing in a nice editor, that auto saves updates, allows me to create drafts, and has some nice templating features built in.

There's quite a few features you need for a static site to publish a blog, one I really wanted from the beginning is multiple posts on the home page. This was added as Front Matter at the start of a template, which makes it capable of publishing pages that don't otherwise exist in your folder structure:

holster.haza.website

The Holster website has been gone since rebuilding i.haza.website, it's now back as a replacement for the github wiki docs with updated information for all API's and better examples 🎉

I Haza Website also got some updates as I was working on this, so it now does nice preview navigation while you're editing pages. You can start from one page in preview mode, navigate to another page you're working on and continue editing from there.

There's also a lot of code examples on the site so I added a Copy Code plugin that allows you to copy code blocks if you want to run them interactively in Node. There's also a theme switcher plugin for light and dark mode:

Holster 2.3.0

A new Holster release is available that provides await support.

Using Holster for building i.haza.website was very different than working on RSStream as it needed to be much more responsive to Holster data changes. Looking back at the code up until this point, it was full of await new Promise(resolve => { }) wrappers so that state could update in the correct order.

The latest Holster release removes all of this from application code by providing a Thenables interface onto terminal API calls. That sounds weird but it does exactly what you need, which is when you would otherwise provide a callback to the Holster API you can just await the call instead. Internally Holster knows when a callback isn't provided and can extend the response to include a then() function, if you were chaining the request then the extra function is ignored, otherwise it provides your data.

This works for get and next in the API, put requests can also be awaited but they now internally return a promise as they only ever returned an error or null anyway. It allows for simplifying get requests like:

// From wrapped calls:
const existing = await new Promise(resolve =>
  user.get("mediaRootId", resolve),
)

// To just:
const existing = await user.get("mediaRootId")

And previously put requests where you waited for the result were:

async function bumpProjectsVersion(user) {
  const now = Date.now()
  return new Promise((resolve, reject) =>
    user
      .get("projectsVersion")
      .put(now, err => (err ? reject(new Error(err)) : resolve(now))),
  )
}

Are now:

async function bumpProjectsVersion(user) {
  const now = Date.now()
  await user.get("projectsVersion").put(now)
  return now
}

It also makes it easier to work with try and catch blocks, but it's also nice that the old callback style isn't effected by these changes when you want to use it.

I Haza Website

Happy to share that i.haza.website is now available again as a new local-first app!

It was previously a PHP app with a similar goal to what the new version is doing, which is to make it easy to publish web pages. The new version is a web app backed by Holster, and besides publishing websites is great for taking notes and transferring media between devices. Holster provides automatic syncing between devices that you're logged in to, as well as encryption and offline support.

This has been another app where the complimentary goal has been Holster improvements alongside having useful tools. This has been true again because I Haza Website is much more interactive with the Holster API then other apps, so it's caught a few things that have needed fixing.

This blog hasn't been replaced yet because I Haza Website doesn't have some of the blogging features of the old app. Expect that to change soon though, I'm looking forward to moving to the new interface it's much nicer to write in. 😊