Mastering Unstorage: Syncing Data Between Browser and Server

Okay, it’s time for us to get back to Unstorage. I just finished my fourth cup of coffee for the day and thought to myself, "I can’t wait to write the next part of my article on Unstorage". So, here I am, a bit jittery, but that is not going to stop me from completing the Unstorage saga.

In my previous article, I introduced you to Unstorage, an asynchronous Key-Value storage API packed with convenient features such as multi-driver mounting, built-in servers, and state snapshots, among others.

In this article, I will apply what we learned in the previous article and include a few additional features to build a practical setup for syncing data between a browser and a server. Let's get started!

What Are We Building?

When building a web application, we often need to store certain data that isn't necessary for our backend server. We might want to store the user's preferences, the user's shopping cart, or the user's browsing history, for example. We can store this data in the browser's LocalStorage. However, there are a few drawbacks: the data is not available on other devices, and occasionally, the user might clear their browser's LocalStorage, which will result in data loss.

A common but not so elegant solution to this problem is to define an API endpoint on the server for each type of data we want to store. For example, we can define an endpoint to store the user’s preferences and another endpoint to store the user’s shopping cart. This solution is not very elegant because it requires us to define many endpoints and strongly couple our frontend with our backend.

A better solution would be to create a KV store on the client that's unique to each user and sync it with the server. This way, we can store all the data we want without having to define any additional endpoints on the server. This is exactly what we will be building in this article using Unstorage.

How Are We Going to Build It?

We will use Unstorage to build our KV store on both the client and the server. This is possible because Unstorage is designed to work in all JavaScript environments, whether it's a Browser, Node.js, Bun, Deno, Workers, etc.

I’ll use one of its drivers to persist the data on the server so it isn't lost when the server restarts. I'll also use its built-in HTTP server to sync the data between the client and the server.

This is going to be a fun project, so let's get started!

Setting Up the Server

We'll begin by setting up the server that backs our KV store. The beauty of Unstorage and our setup is that we can use any JavaScript runtime to run the server. For this article, I'll use Node.js, but you can deploy the same code to a serverless environment such as Workers or any other JavaScript runtime such as Bun or Deno.

First, let's install Unstorage.

npm install unstorage

Next, I’ll import this package and create a new storage as we did in the previous article.

// src/server.ts
import { createStorage } from "unstorage";

const storage = createStorage();

Now that we have our storage ready, I'll introduce you to a new function called createStorageServer. This function takes a storage instance and returns an HTTP handler that can then be used to create an HTTP server.

// src/server.ts
import { createStorage } from "unstorage";
import { createStorageServer } from "unstorage/server";

const storage = createStorage();
const storageServer = createStorageServer(storage);

Now, we can use this handler to create an HTTP server. It conforms to the RequestHandler interface, so we can use it with any HTTP server library. For this article, I'll be using the http module that comes with Node.js.

// src/server.ts
import http from "http";

const server = http.createServer(storageServer.handle);

server.listen(3000, () => {
  console.log("🚀 Server is ready at http://localhost:3000");
});

That’s it! We now have a server that can be used to store and retrieve data. Let's see how we can use it.

Storing and Retrieving Data

Once the server is up and running, we can interact with it over HTTP. Unstorage provides a convenient HTTP API that can be used to store and retrieve data. Let's see how we can use it.

Storing Data

To store data, we can use the PUT method. The key is specified in the path, and the value is specified in the body. For example, to store the user’s preferences, we can apply the following request:

curl -X PUT -d '{"theme": "dark"}' http://localhost:3000/user:preferences

Retrieving Data

To retrieve data, we can use the GET method. The key is specified in the path. For example, to retrieve the user’s preferences, we can use the following request:

curl http://localhost:3000/user:preferences

Checking If Data Exists

To check if data exists, we can use the HEAD method. The key is specified in the path. To check if the user’s preferences exist, for example, this request can be used:

curl -I http://localhost:3000/user:preferences

Deleting Data

To delete data, we can utilize the DELETE method. The key is specified in the path. For example, to delete the user’s preferences, we can use the following request:

curl -X DELETE http://localhost:3000/user:preferences

Isn't that cool? We now have a server that can be used to store and retrieve data. But wait, there's more! Unstorage also provides a built-in HTTP client that can be used to interact with the server. Let's see how we can use it.

Setting Up the Client

Now that we have our server up and running, let's set up the client. Your client can be anything: a web app, a mobile app, a desktop app, or even a CLI app.

The process is almost identical, except we will be using a different driver on the client side to interact with the server. Let's see how we can do that.

First, let's install Unstorage.

npm install unstorage

Next, I’ll import this package and create a new storage as we did in the previous article.

// src/client.ts
import { createStorage } from "unstorage";
import httpDriver from "unstorage/drivers/http";

const storage = createStorage({
  driver: httpDriver({
    base: "http://localhost:3000", // The base URL of the server
  }),
});

That is literally all we need to set up the client. Now, we can use the storage instance on the client side just like we did in the previous article.

// src/client.ts
await storage.setItem("user:preferences", {
  theme: "dark",
});
await storage.getItem("user:preferences"); // => { theme: "dark" }
await storage.hasItem("user:preferences"); // => true
await storage.removeItem("user:preferences");

🤯 The only difference is that the data is now stored on the server instead of the client. If we run multiple clients, or restart the client, we will get the same data back.

However, there is one thing that we need to fix now! If you haven't already noticed, there is no authentication or authorization in our setup. Anyone can access the data of any user by simply changing the key in the request. Let's see how we can fix that.

Adding Authentication

Thankfully, adding authentication to our setup is very simple. All we have to do is to add middleware to our HTTP server and require the client to send an Authorization header with a valid JWT token. Let's take a look at how we can do that.

Adding Authorization to the Server

First, let's wrap our storage server inside a middleware so that we can add our authentication logic to it:

// src/server.ts
const server = http.createServer(async (req, res) => {
  // Check for the Authorization header
  return storageServer.handle(req, res);
});

Now, we can use the request headers to check for the Authorization header and verify the JWT token. If the token is valid, we’ll allow the request to proceed; otherwise, we'll throw an error.

// src/server.ts
const server = http.createServer(async (req, res) => {
  const auth = req.headers.authorization;
  if (!auth) throw new Error("Unauthorized");

  const [type, token] = auth.split(" ");
  if (type !== "Bearer") throw new Error("Unauthorized");

  const user = await verifyToken(token);
  if (!user) throw new Error("Unauthorized");

  return storageServer.handle(req, res);
});

Let's break down what's happening here. First, we check for the Authorization header. If it doesn't exist, we throw an error. Next, we split the header into two parts: the type and the token. If the type is not Bearer, we throw an error. Finally, we verify the token, and if it's not valid, we throw an error.

Now, if the token is valid, we will let the request proceed.

Adding Authorization to the Client

Now, let's add the other half of the equation on the client side. We can do that by adding our Authorization header to the httpDriver options:

// src/client.ts
const storage = createStorage({
  driver: httpDriver({
    base: "http://localhost:3000", // The base URL of the server,
    headers: {
      Authorization: `Bearer ${getAuthToken()}`, // The JWT token
    },
  }),
});

In this case, you should use your own authentication mechanism to generate the JWT token. I'm using an imaginary getAuthToken function to get the token since that's not the focus of this article. Of course, in other cases, you might want to use a different authentication mechanism, such as cookies, secret tokens, etc.

Properly Namespacing User Data

In the previous section, we added authentication to our setup, but one problem lingers: anyone can still access the data of any user by simply changing the key in the request.

This is because we are only stopping unauthorized requests, but we are not stopping requests for data that doesn't belong to the user. If the user is logged in as user:1, they can still access the data of user:2 by simply changing the key in the request.

To fix this, we need to properly namespace user data, so that each user can only access the data that belongs to them. We can do so by updating the middleware that we added in the previous section and rewriting the request URL to prepend it with user/<user_id>, which is then converted to user:<user_id> by Unstorage.

// src/server.ts
const server = http.createServer(async (req, res) => {
  const auth = req.headers.authorization;
  if (!auth) throw new Error("Unauthorized");

  const [type, token] = auth.split(" ");
  if (type !== "Bearer") throw new Error("Unauthorized");

  const user = await verifyToken(token);
  if (!user) throw new Error("Unauthorized");

  // get the user id from the token
  const prefix = `user/${user.id}`;
  req.url = prefix + req.url;

  // if the url ends with /, remove it
  if (req.url.endsWith("/")) {
    req.url = req.url.slice(0, -1);
  }

  return storageServer.handle(req, res);
});

Now, if a user logged in as user:1, they could only access the data that starts with user:1. For example, they can access user:1:preferences, but they cannot access user:2:preferences.

Persisting the Server Data

So far, we have been using in-memory storage on the server side. That's fine for development, but it's not very useful in production. If the server restarts, all the data gets lost. For instance, if we deploy this to a serverless environment, all the data will be lost once our function cold starts.

To fix this, we need to use persistent storage on the server side. Thankfully, Unstorage provides a number of built-in drivers that we can use to persist the data. For this article, I'll use the fsDriver to persist the data to the file system, but you can use any driver you want.

First, import the fsDriver from Unstorage:

import fsDriver from "unstorage/drivers/fs";

Next, update our storage instance to use the fsDriver:

const storage = createStorage({
  driver: fsDriver({
    base: "./data",
  }),
});

That's it! Now, all the data will be persisted to the file system. You can use any driver you like, but I recommend the fsDriver as it's the most reliable and easy to use.

Wrapping Up

As you can see, Unstorage is a very powerful tool that can be used to build complex applications. It provides a simple interface that's easy to learn and use, and it offers some really powerful features that can be used to build complex applications.

The tiny sync setup we built in this article is just one of the many things you can build with Unstorage. I encourage you to explore the other features of Unstorage by visiting their API documentation.

I hope you enjoyed this article and learned something new! Until next time, happy coding!