Lex and npm

I've just added an initial version of Lex to Holster. It can only query properties on nodes at the moment, while I was working on it I had hopes that queries could be chained to do more interesting things, but that's going to be a lot of work! 😅

So for now lex needs to be added along with a callback for the data, not in a context chain. That allows queries like:

const nested = {
  key: "nested value",
  other: "other value"
}

holster.get("nested").put(nested, err => {
  holster.get("nested", {".": {"*": "k"}}, console.log)
  // logs: {key: "nested value"}
})

I think I'm also just ready to try using Holster from a real application and continue working on it as required from there. With that goal in mind I've also published Holster to npm! https://www.npmjs.com/package/@mblaney/holster

Time to switch gears and try building something with it.

Holster User API

Very excited about this update, Holster now has a working User API which means it's getting very close to being usable for real applications! Some obligatory code:

const user = holster.user()

user.create("alice", "password", console.log)
user.auth("alice", "password", console.log)
console.log(user.is.username) // "alice"

user.get("hello").put("world!", console.log)
user.get("hello", console.log) // "world!"

This has been a few weeks in the making, even with the GunDB code looking in pretty good shape to be used in Holster. I followed the same sign and verify model, but while looking at how to implement it I could really only see that GunDB was verifying data that started with a public key. There are some comments in the code about improving this, with the issue being that nodes often don't start with a public key but instead have their own id which is linked to a public key via a rel.

To get around this problem I decided just to add the user's public key to all nodes. Then if an update is made to existing data, the current public key needs to match what is provided in the update. If there's no existing node then the soul of the data needs a rel on the parent node, but that would need to check the current public key too, so it's not possible to associate random nodes with another user.

Other than that the GunDB code was a great starting point for adding users to Holster. I've kept the same model for storing system data under ~@username and ~publickey. I've also added secure mode as an option so that only user space can be written to. I think I got lucky that the existing context chain in Holster worked well with adding a user API, it really only required changing the root node in the graph.

The wire API changes were a bit more work, as it now fetches all nodes before updating them to check their public keys. This isn't so bad if the graph is already in memory. This might also just be the price of distributed security, if there really is such a thing!

I would really like to switch RSStream over to using Holster, but when I was working on that application I would've liked to have Lex working with user data, but had some trouble with queries in GunDB. So I think I will take a look at adding that next.

SEA!

Another big update to Holster, I've just added the SEA implementation from GunDB with relatively few modifications. A lot of the environment checks were no longer necessary as Node now supports the Web Crypto API.

Also thanks to Joey, Holster now runs natively in the browser and SEA works exactly the same. There's still a fair way to go to introduce a Holster.user API, but the SEA primitives all work and have lots of tests to show it.

The tests pretty much copy and paste into the browser console to show that they're working there too, all it requires is a JavaScript file with:

import SEA from "../src/sea.js"
globalThis.SEA = SEA

And then you should be able to check the basic functionality. Not quite sure yet how this will integrate with user data in Holster, but very happy with it as a first step.

Next

Lexi in the GunDB chat suggested renaming then() to next() to avoid confusion with the async API's. That makes sense to me so I've pushed the change and updated the API docs.

To update examples from previous posts, that means Holster queries now look like:

holster.get("hello").next("world!").on(console.log)

// Which will get called by this update.
holster.get("hello").next("world!").put("update", console.log)

Now all I need to use Holster in a real application is to add SEA from GunDB. No idea how that's going to happen... wish me luck!

IndexedDB

Another item on the TODO list ticked off, IndexedDB is now working in the browser by using Holster({indexedDB: true}). Thankfully the GunDB code was pretty easy to re-use for this! I've kept it opt-in to use browser storage, as Holster currently works fine with server only storage and I'd like to keep both working.

I also missed an update to the examples folder I could've made earlier. It previously didn't try to run both Holster API updates at the same time due to the shared context, but now that's working it's fine to make the two updates occur at random times similar to wire.js. The main difference now being that index.js doesn't set state values on properties, so the value that is fetched at the end is whichever update happened last.