Holster 1.0.2

Another small bug fix release, this one was more of a problem in application code using Holster, but investigating what went wrong there led to some improvements.

The app I'm working on requires a server account and if one doesn't exist when it starts it would create one in Holster. The problem is the user lookup timeout was too short, so it created a second account! This was not helpful since it provides a public key to the browser to look up account information... and now it's split across multiple accounts which isn't going to work.

The answer was to remove the account creation code from the app entirely. This meant I needed a new way to set up the server account in the first place, which led to a nice use of the Node REPL to configure it manually:

// Or switch to "@mblaney/holster/src/holster.js" in your own project.
const {default: Holster} = await import("./src/holster.js")
const holster = Holster()
const user = holster.user()

I've left out the actual commands, but from there you can use holster and user to query the API. Besides fixing my problem it also provides a nice way to interact with your data from the command line.

Holster 1.0.1

I had one annoying bug that I hadn't got to the bottom of before yesterday's 1.0 release. I even added a bunch of unit tests to make sure Holster was working as I expected. The previous versions of the unit tests looked like this:

test("get unknown keys in for loop callbacks null", (t, done) => {
  for (let i = 0; i < 5; i++) {
    holster.get("unknown" + i, data => {
      assert.equal(data, null)
      if (i === 4) done()
    })
  }
})

The idea is that you should be able to make as many requests as you like and Holster will manage the context of each request and map it to the right callback. Can you see the problem with the test though? The loop will finish before the callbacks are called, so the condition will match and done() will be called regardless of if the callbacks were each called correctly. In fact only one was being called but the tests still passed.

They now look like this:

test("get unknown keys in for loop callbacks null", (t, done) => {
  let count = 0
  for (let i = 0; i < 5; i++) {
    holster.get("unknown" + i, data => {
      assert.equal(data, null)
      count++
    })
  }
  setTimeout(() => {
    if (count === 5) done()
  }, 200)
})

Which makes sure each was called correctly. Holster wasn't able to pass this version of the test because the context id was being dropped. The id is passed to the API as a parameter that I thought would form part of the closure, but it seems the callback function itself is created before that can happen. The answer was to pass the id to a function that then returns a new callback, which then appears to be called correctly.

This was enough of a problem in the app I'm working on for a 1.0.1 bug fix release.

Holster 1.0

Wow I can't believe it has been over a month since I first tried switching to Holster in an existing application. It's just what it needed though, as it brought up all sorts of edge cases and timing issues that I hadn't thought of. The end result is that Holster now feels usable! This is a 1.0 release because multiple breaking API changes were required.

The biggest change is to the user API, which I mentioned needed fixing in the last update. It actually needed a complete redesign so that calling holster.user() returns a usable API and can also be used to fetch user data on other accounts. It now does this by taking an optional array of two values when making a query, so if you log in and try user.get("key") you will get your own data, and user.get(["other-account", "key"]) will get data on another account and not interfere with your current log in status.

The other API change is to holster.on(), which now takes four parameters. Previously it was only given a lex object and a callback, but I found a lot of situations where I wanted to add a listener and get the current copy of the data at the same time. So the third parameter is a boolean to get the data, and the fourth parameter is the same options object passed to get() to update the wait time if required.

Another API change is that user.recall() previously required a flag to specify if credentials are stored in local storage or session storage, now it just tries both so no flag is required.

As well as this there are a lot of updates that make it feel like a solid 1.0 release.

Verification is now done properly on user nodes, previously it would sign an update which didn't always contain all properties on a node. Now it will fetch the node first if required to make sure it signs everything. Signing and verification also need to be performed on the exact same object, so it sorts the keys to guarantee the same output.

When I started the port I noticed GunDB was doing a lot more wire sends than Holster, I now see how that helps to keep data in sync. It also means the in-memory graph is always up to date, so get requests now don't need to go to disk unless there's been a restart.

This felt like a lot of debugging to get this release out, but looking at the changes being committed it's far from a complete overhaul to how it was originally written! Hopefully this means it's close to being reliable. Next goal is to push the update to rsstream.app and see how it performs over the next few weeks.

Holster 0.3.0

I'm still in the process of converting an app from GunDB to Holster, and haven't hit too many hurdles so far. Enough changes are required to Holster for another release though, so I'm publishing version 0.3.0 as there's a small API change.

The main change required at this point is fixing user queries, I previously added holster.user(public-key) as a way to query other accounts, but it overwrites the context of a logged in user so I think I've fixed that.

I also updated the per-query options object that allows you to wait longer for data to be returned if required. It was available in the wire spec API but wasn't exposed to the Holster API, so get() and next() now both have an optional fourth parameter after the callback.

The last change is to support projects using webpack, which didn't like some Node imports in Holster. They should be ignored anyway so adding /*webpackIgnore: true*/ to the imports seems to work for that.

Peers and a first release

The Holster options config now support some more options from GunDB, including providing a peer as a string, or as an array, or as {peers: ["url", "etc"]} along with other options in an object. This also meant updating Holster's wire.js to support multiple WebSocket clients. I also added the server and port options from GunDB, all of which are required for real applications.

As I started converting existing code over to Holster I realised that set() would be nice to have. I decided to add it via put() rather than adding a new function, by using an extra parameter that converts the data you want to put to {"random-key": data}. That makes it a pretty simple change, since all updates are added to the set.

There is no map() in Holster to fetch the values in the set, but I'm finding that to be ok. More on how to convert code from GunDB to Holster later.

I've also added publishing new releases from github to npm, which required releasing the first version of Holster so v0.2.0 is now available!