Cache and memoization

@dotpress/cache provides two tools, for two different problems:

	Memo	Cache
Avoids	loading the same value several times within one request	loading the same value on every request
Lives	as long as the request (or the event message, the job)	minutes, hours, days
Stored in	the memory of the request	a store: memory, Redis, Upstash, Workers KV
Freshness	always fresh: each request starts with an empty memo	can be outdated: expires with a `ttl`, or when you invalidate it
Typical use	the current user, loaded by a middleware and a handler; the authors of a list of articles	a catalog, a price list, a dashboard, the response of a slow service

Start with the memo: it saves queries without any risk of serving outdated data. Add the cache for values that are expensive to load and change rarely.

npm

yarn

pnpm

bun

deno

npm install @dotpress/cache

Set up

The plugin gives each request a memo, ctx.memo, and the cache, ctx.cache:

import { createCache, createMemoryStore } from '@dotpress/cache'
import { cachePlugin } from '@dotpress/cache/plugin'

const cache = createCache({ store: createMemoryStore(), prefix: 'app' })

const app = await createApp({ adapter, plugins: [cachePlugin({ cache })] })

cachePlugin() without cache provides the memo only.

Memo

Load a value once per request

The authentication middleware loads the account of the user, and the handler needs it too. Declare how to load it, then ask the memo:

// How to load an account: declared once, in a module.
export const const loadAccount: MemoDefinition<[id: string], Account | undefined>loadAccount = defineMemo<[id: string], Account | undefined>(options: LoadMemoOptions<[id: string], Account | undefined>): MemoDefinition<[id: string], Account | undefined> (+2 overloads)Defines a memoized loader: memo(loadUser, id) loads a user once per
memo (request, message, job), and calls made while it loads share the
same load (single flight). With batch, the keys requested in the same
tick are loaded in one call.
const loadUser = defineMemo({
  name: 'user',
  load: (id: string) => db.users.findById(id),
})

const loadOrder = defineMemo({
  name: 'order',
  batch: (ids: readonly string[]) => db.orders.findManyById(ids), // a Map by id
})
defineMemo({
  BaseMemoOptions.name: stringName of the memo, used in errors: letters, digits, ., _, : and
-.name: 'account',
  LoadMemoOptions<[id: string], Account | undefined>.load: (id: string) => Account | Promise<Account | undefined> | undefinedLoads the value of the arguments.load: (id: stringid: string) => const db: {
    accounts: {
        findById(id: string): Promise<Account | undefined>;
    };
}
db.accounts: {
    findById(id: string): Promise<Account | undefined>;
}
accounts.function findById(id: string): Promise<Account | undefined>findById(id: stringid),
})

const const requireActiveAccount: Middleware<unknown>requireActiveAccount = defineMiddleware<unknown>(middleware: (ctx: RequestContext) => MiddlewareResult | Promise<MiddlewareResult>, options?: MiddlewareOptions): Middleware<unknown>Declares a middleware that guarantees context properties to handlers.
@exampleconst  requireTenant = defineMiddleware<{ tenant: Tenant }>((ctx) =>
ctx.tenant ? undefined : forbiddenError()
)
defineRoute({
path: '/settings',
middlewares: [requireTenant],
handler: ({ tenant }) => tenant.settings, // Tenant, not Tenant | undefined
})
// Documented as secured by the bearerAuth scheme (@dotpress/openapi).
const requireUser = defineMiddleware((ctx) => ..., { security: ['bearerAuth'] })defineMiddleware(async (ctx: RequestContextctx) => {
  const const account: Account | undefinedaccount = await ctx: RequestContextctx.RequestContext.memo: Memo
<[string], Account | undefined>(memo: MemoDefinition<[string], Account | undefined>, args_0: string) => Promise<Account | undefined>
Loads the value of the arguments once per memo: later calls, and
calls made while it loads, share the result. Failures are shared by
the calls in progress, not kept.memo(const loadAccount: MemoDefinition<[id: string], Account | undefined>loadAccount, ctx: RequestContextctx.RequestContext.req: DotpressRequest<Record<string, string>, QueryValues, unknown, UploadedFiles>req.DotpressRequest<Record<string, string>, QueryValues, unknown, UploadedFiles>.params: Record<string, string>params.stringaccountId) // loads it
  if (!const account: Account | undefinedaccount || const account: Accountaccount.Account.suspended: booleansuspended) return function forbiddenError(message?: string, data?: ErrorData): HttpErrorforbiddenError()
})

defineRoute<"/accounts/:accountId/billing", EmptyObject, false, readonly [unknown]>(route: RouteConfig<"/accounts/:accountId/billing", EmptyObject, EmptyObject, false, readonly [unknown], unknown>): voidDeclares a route on the global router.defineRoute({
  RouteConfig<"/accounts/:accountId/billing", EmptyObject, EmptyObject, false, readonly [unknown], unknown>.path: "/accounts/:accountId/billing"path: '/accounts/:accountId/billing',
  RouteConfig<"/accounts/:accountId/billing", EmptyObject, EmptyObject, false, readonly [unknown], unknown>.middlewares?: readonly [Middleware<unknown>] | undefinedRoute middlewares, run after global and group middlewares. Context
properties they guarantee (defineMiddleware()) are typed in the
handler.middlewares: [const requireActiveAccount: Middleware<unknown>requireActiveAccount],
  RouteConfig<"/accounts/:accountId/billing", EmptyObject, EmptyObject, false, readonly [unknown], unknown>.handler: RouteHandler<EmptyObject, {
    accountId: string;
}, false, unknown>
handler: async ({ req: RouteRequest<EmptyObject, {
    accountId: string;
}, false>
req, memo: MemoMemo of the request (cachePlugin): a new one for each request.memo }) => {
    const const account: Account | undefinedaccount = await memo: Memo
<[string], Account | undefined>(memo: MemoDefinition<[string], Account | undefined>, args_0: string) => Promise<Account | undefined>
Loads the value of the arguments once per memo: later calls, and
calls made while it loads, share the result. Failures are shared by
the calls in progress, not kept.memo(const loadAccount: MemoDefinition<[id: string], Account | undefined>loadAccount, req: RouteRequest<EmptyObject, {
    accountId: string;
}, false>
req.DotpressRequest<{ accountId: string; }, QueryValues, unknown, UploadedFiles>.params: {
    accountId: string;
}
params.accountId: stringaccountId) // no query: memoized
    return { plan: "free" | "pro" | undefinedplan: const account: Account | undefinedaccount?.Account.plan: "free" | "pro" | undefinedplan }
  },
})

For GET /accounts/42/billing, the middleware and the handler both ask the memo for account 42: the database is queried once. They stay independent: neither has to pass the account to the other.

Single flight: calls made while the value loads wait for the same load.
Failures are not kept: a failed load fails the calls in progress, and the next call loads again.
Keys: the arguments of load, compared by value; a key option handles other arguments (key: (user) => user.id).
Values are not copies: a caller that mutates one changes it for the others.
After an update, ctx.memo.set(loadAccount, id, account) replaces the value, and ctx.memo.delete(loadAccount, id) loads it again next time; memo.clear() forgets everything.

Batch the loads

A list of 50 articles shows the author of each. Loading the authors one by one makes 50 queries: the N+1 problem. With batch instead of load, the memo collects the keys requested in the same tick, and loads them in one call:

export const const loadAuthor: MemoDefinition<[key: string], User | undefined>loadAuthor = defineMemo<string, User>(options: BatchMemoOptions<string, ReadonlyMap<string, User>>): MemoDefinition<[key: string], User | undefined> (+2 overloads)Defines a memoized loader: memo(loadUser, id) loads a user once per
memo (request, message, job), and calls made while it loads share the
same load (single flight). With batch, the keys requested in the same
tick are loaded in one call.
const loadUser = defineMemo({
  name: 'user',
  load: (id: string) => db.users.findById(id),
})

const loadOrder = defineMemo({
  name: 'order',
  batch: (ids: readonly string[]) => db.orders.findManyById(ids), // a Map by id
})
defineMemo({
  BaseMemoOptions.name: stringName of the memo, used in errors: letters, digits, ., _, : and
-.name: 'author',
  // Receives every id requested in the same tick, without duplicates.
  BatchMemoOptions<string, ReadonlyMap<string, User>>.batch: (keys: readonly string[]) => ReadonlyMap<string, User> | Promise<ReadonlyMap<string, User>>Loads the values of the keys requested in the same tick, without
duplicates. Returns their values in the order of the keys (an Error
fails its key only), or a Map by key (a missing key gives
undefined).batch: async (ids: readonly string[]ids: readonly string[]) => {
    const const users: User[]users = await const db: {
    users: {
        findManyById(ids: readonly string[]): Promise<User[]>;
    };
    articles: {
        list(): Promise<{
            id: string;
            title: string;
            authorId: string;
        }[]>;
    };
}
db.users: {
    findManyById(ids: readonly string[]): Promise<User[]>;
}
users.function findManyById(ids: readonly string[]): Promise<User[]>findManyById(ids: readonly string[]ids) // WHERE id IN (…)
    return new var Map: MapConstructor
new <string, User>(iterable?: Iterable<readonly [string, User]> | null | undefined) => Map<string, User> (+3 overloads)
Map(const users: User[]users.Array<User>.map<[string, User]>(callbackfn: (value: User, index: number, array: User[]) => [string, User], thisArg?: any): [string, User][]Calls a defined callback function on each element of an array, and returns an array that contains the results.
@paramcallbackfn  A function that accepts up to three arguments. The map method calls the callbackfn function one time for each element in the array.@paramthisArg  An object to which the this keyword can refer in the callbackfn function. If thisArg is omitted, undefined is used as the this value.map((user: Useruser) => [user: Useruser.User.id: stringid, user: Useruser])) // unknown ids give undefined
  },
})

defineRoute<"/articles", EmptyObject, false, unknown[]>(route: RouteConfig<"/articles", EmptyObject, EmptyObject, false, unknown[], unknown>): voidDeclares a route on the global router.defineRoute({
  RouteConfig<"/articles", EmptyObject, EmptyObject, false, unknown[], unknown>.path: "/articles"path: '/articles',
  RouteConfig<"/articles", EmptyObject, EmptyObject, false, unknown[], unknown>.handler: RouteHandler<EmptyObject, {}, false, unknown>handler: async ({ memo: MemoMemo of the request (cachePlugin): a new one for each request.memo }) => {
    const const articles: {
    id: string;
    title: string;
    authorId: string;
}[]
articles = await const db: {
    users: {
        findManyById(ids: readonly string[]): Promise<User[]>;
    };
    articles: {
        list(): Promise<{
            id: string;
            title: string;
            authorId: string;
        }[]>;
    };
}
db.articles: {
    list(): Promise<{
        id: string;
        title: string;
        authorId: string;
    }[]>;
}
articles.function list(): Promise<{
    id: string;
    title: string;
    authorId: string;
}[]>
list()
    // 50 calls to the memo, one query to the database
    return var Promise: PromiseConstructorRepresents the completion of an asynchronous operationPromise.PromiseConstructor.all<Promise<{
    author: User | undefined;
    id: string;
    title: string;
    authorId: string;
}>[]>(values: Promise<{
    author: User | undefined;
    id: string;
    title: string;
    authorId: string;
}>[]): Promise<{
    author: User | undefined;
    id: string;
    title: string;
    authorId: string;
}[]> (+1 overload)
Creates a Promise that is resolved with an array of results when all of the provided Promises
resolve, or rejected when any Promise is rejected.
@paramvalues  An array of Promises.@returnsA  new Promise.all(
      const articles: {
    id: string;
    title: string;
    authorId: string;
}[]
articles.Array<{ id: string; title: string; authorId: string; }>.map<Promise<{
    author: User | undefined;
    id: string;
    title: string;
    authorId: string;
}>>(callbackfn: (value: {
    id: string;
    title: string;
    authorId: string;
}, index: number, array: {
    id: string;
    title: string;
    authorId: string;
}[]) => Promise<{
    author: User | undefined;
    id: string;
    title: string;
    authorId: string;
}>, thisArg?: any): Promise<{
    author: User | undefined;
    id: string;
    title: string;
    authorId: string;
}>[]
Calls a defined callback function on each element of an array, and returns an array that contains the results.
@paramcallbackfn  A function that accepts up to three arguments. The map method calls the callbackfn function one time for each element in the array.@paramthisArg  An object to which the this keyword can refer in the callbackfn function. If thisArg is omitted, undefined is used as the this value.map(async (article: {
    id: string;
    title: string;
    authorId: string;
}
article) => ({
        ...article: {
    id: string;
    title: string;
    authorId: string;
}
article,
        author: User | undefinedauthor: await memo: Memo
<[string], User | undefined>(memo: MemoDefinition<[string], User | undefined>, args_0: string) => Promise<User | undefined>
Loads the value of the arguments once per memo: later calls, and
calls made while it loads, share the result. Failures are shared by
the calls in progress, not kept.memo(const loadAuthor: MemoDefinition<[key: string], User | undefined>loadAuthor, article: {
    id: string;
    title: string;
    authorId: string;
}
article.authorId: stringauthorId),
      }))
    )
  },
})

The code that renders one article does not change: it asks for one author, and the memo groups the requests. batch returns a Map by key, or an array in the order of the keys, where an Error fails its key only. maxBatchSize limits the keys per call.

One memo per unit of work

A memo lives as long as its request: each request starts empty, so it never serves outdated data. For the same reason, never keep a memo for the whole process: it would grow forever, and serve old values. In an event listener or a job, create one per message with createMemo() (see Outside a route).

Cache

Keep a value across requests

The plans of a billing service change a few times a year, and the service takes 300 ms to answer. Declare a cache entry, and read through it:

export const const planCache: CacheEntry<[id: string], {
    id: string;
    name: string;
    price: number;
}>
planCache = defineCacheEntry<[id: string], {
    id: string;
    name: string;
    price: number;
}>(options: CacheEntryOptions<[id: string], {
    id: string;
    name: string;
    price: number;
}> & {
    schema: CacheSchema<{
        id: string;
        name: string;
        price: number;
    }>;
}): CacheEntry<[id: string], {
    id: string;
    name: string;
    price: number;
}> (+2 overloads)
Defines a kind of cached value: its key, its lifetime, and optionally
its schema.
export const userCache = defineCacheEntry({
  name: 'user',
  key: (id: string) => id,
  schema: userSchema,
  ttl: '1h',
})
defineCacheEntry({
  CacheEntryOptions<A extends readonly unknown[], T>.name: stringName of the entry, in keys and logs: letters, digits, ., _ and
-.name: 'plan',
  CacheEntryOptions<[id: string], { id: string; name: string; price: number; }>.key?: ((id: string) => unknown) | undefinedKey of the arguments: a string, a number, or a value compared like the
arguments of a memo. Defaults to the argument (or the arguments).key: (id: stringid: string) => id: stringid,
  schema: CacheSchema<{
    id: string;
    name: string;
    price: number;
}>
Validates cached values when they are read (entries from an older
version of the app) and written. JSON turns dates into strings: use
z.coerce.date().schema: import zz.function object<{
    id: z.ZodString;
    name: z.ZodString;
    price: z.ZodNumber;
}>(shape?: {
    id: z.ZodString;
    name: z.ZodString;
    price: z.ZodNumber;
} | undefined, params?: string | {
    error?: string | z.core.$ZodErrorMap<NonNullable<z.core.$ZodIssueUnrecognizedKeys | z.core.$ZodIssueInvalidType<unknown>>> | undefined;
    message?: string | undefined | undefined;
} | undefined): z.ZodObject<{
    id: z.ZodString;
    name: z.ZodString;
    price: z.ZodNumber;
}, z.core.$strip>
object({ id: z.ZodStringid: import zz.function string(params?: string | z.core.$ZodStringParams): z.ZodString (+1 overload)string(), name: z.ZodStringname: import zz.function string(params?: string | z.core.$ZodStringParams): z.ZodString (+1 overload)string(), price: z.ZodNumberprice: import zz.function number(params?: string | z.core.$ZodNumberParams): z.ZodNumbernumber() }),
  CachePolicy.ttl?: Duration | undefinedLifetime of the values. Defaults to 1 hour.ttl: '1h',
})

defineRoute<"/plans/:id", EmptyObject, false, unknown[]>(route: RouteConfig<"/plans/:id", EmptyObject, EmptyObject, false, unknown[], unknown>): voidDeclares a route on the global router.defineRoute({
  RouteConfig<"/plans/:id", EmptyObject, EmptyObject, false, unknown[], unknown>.path: "/plans/:id"path: '/plans/:id',
  RouteConfig<"/plans/:id", EmptyObject, EmptyObject, false, unknown[], unknown>.handler: RouteHandler<EmptyObject, {
    id: string;
}, false, unknown>
handler: ({ req: RouteRequest<EmptyObject, {
    id: string;
}, false>
req, cache: CacheCache (cachePlugin), logging with the request logger.cache }) =>
    // From the cache, or from the service on a miss, then cached for an hour
    cache: CacheCache (cachePlugin), logging with the request logger.cache.Cache.getOrSet<[string], {
    id: string;
    name: string;
    price: number;
}, {
    id: string;
    name: string;
    price: number;
}>(entry: CacheEntry<[string], {
    id: string;
    name: string;
    price: number;
}>, argsAndLoader_0: string, argsAndLoader_1: () => {
    id: string;
    name: string;
    price: number;
} | Promise<{
    id: string;
    name: string;
    price: number;
}>): Promise<{
    id: string;
    name: string;
    price: number;
}> (+1 overload)
The value of the arguments, loaded by the last argument when missing
and cached. Calls made while it loads share the load. Applies the
grace period, stale-while-revalidate and timeouts of the entry.
A loader may return undefined (not found): it is not cached, and
getOrSet returns it.getOrSet(const planCache: CacheEntry<[id: string], {
    id: string;
    name: string;
    price: number;
}>
planCache, req: RouteRequest<EmptyObject, {
    id: string;
}, false>
req.DotpressRequest<{ id: string; }, QueryValues, unknown, UploadedFiles>.params: {
    id: string;
}
params.id: stringid, () => const billing: {
    getPlan(id: string): Promise<{
        id: string;
        name: string;
        price: number;
    }>;
}
billing.function getPlan(id: string): Promise<{
    id: string;
    name: string;
    price: number;
}>
getPlan(req: RouteRequest<EmptyObject, {
    id: string;
}, false>
req.DotpressRequest<{ id: string; }, QueryValues, unknown, UploadedFiles>.params: {
    id: string;
}
params.id: stringid)),
})

getOrSet returns the cached value, or calls the loader and caches its result. Concurrent requests for a missing value share one load. A failed load, or a load returning undefined (not found), is not cached.
get, set and delete are the usual operations.
Durations are seconds, or strings: '500ms', '30s', '10m', '1h', '7d'. The ttl of an entry overrides the ttl of the cache, 1 hour by default.
The schema types the value, and checks it when read: a value written by an older version of the app that no longer matches is a miss, not a crash. Values are stored as JSON: use z.coerce.date() for dates.
Keys are prefix:name:v1:key; longer keys than the store accepts are hashed. Increase the version of an entry when the shape of its values changes: the old values are ignored.
Values are serialized as JSON (or with a serializer, such as superjson). null is cached; undefined is not.

Expire values when the data changes

A cached value is outdated as soon as the data changes. Delete it after an update:

await db.plans.update(id, changes)
await cache.delete(planCache, id)

When one change affects many values, tag them, and invalidate the tag:

export const productCache = defineCacheEntry({
  name: 'product',
  key: (id: string) => id,
  tags: (id: string) => [`product:${id}`, 'catalog'],
})

await cache.invalidateTags(['catalog']) // after an import: every product expires

Tags are computed from the arguments of the entry, not from the value: a value expires without being read. Reading a value with tags costs one more round trip to the store, for their tokens: see Tags.

Serve outdated values when the source fails

Some data is better a little outdated than missing. An entry decides how expired values are served:

export const catalogCache = defineCacheEntry({
  name: 'catalog',
  ttl: '5m',
  grace: '1d', // when the loader fails, serve the expired value, for one more day
  staleWhileRevalidate: '1m', // serve the expired value at once, and refresh it in the background
  timeout: '300ms', // a slow loader serves the expired value, and goes on in the background
})

Option	Effect, once the `ttl` is over
`grace`	When the loader fails or times out, the expired value is served instead of the error.
`graceBackoff`	After such a failure, the loader is not called again for this delay (`10s` by default).
`staleWhileRevalidate`	The expired value is served at once, and reloaded in the background.
`timeout`	A load slower than this serves the expired value of the grace period.
`hardTimeout`	A load slower than this fails, or serves the expired value.
`lock`	One instance loads a missing value, the others wait for it (Redis).

When the store fails

The cache is an optimization: by default, it fails open. When the store is down, a read becomes a miss (the loader runs), a write is skipped, and the app keeps working. Every failure is logged, and passed to onError:

const cache = createCache({
  store,
  prefix: 'app',
  storeTimeout: '200ms', // a slower store counts as a failure
  onError: (error, { operation }) => metrics.increment('cache.store_error', { operation }),
})

Set storeTimeout: while a Redis client reconnects, its commands wait, and the requests would wait with them. failOpen: false throws a CacheStoreError instead. onEvent receives each hit, miss and stale value, for metrics.

Stores

Store	Import	Use
`createMemoryStore()`	`@dotpress/cache`	one instance, development, tests; 10,000 entries at most by default (`maxEntries`)
`createRedisStore({ client })`	`@dotpress/cache/redis`	Redis or Valkey, with a node-redis or ioredis client
`createUpstashStore()`	`@dotpress/cache/upstash`	Upstash Redis, over HTTP
`createCloudflareKvStore()`	`@dotpress/cache/cloudflare-kv`	Cloudflare Workers KV

With several instances of the API, use a shared store: with the memory store, each instance has its own cache, and an invalidation only reaches one of them. The settings and limits of each store, Upstash and Workers KV included, are in the Cache reference.

import { createClient } from 'redis'
import { createRedisStore } from '@dotpress/cache/redis'

const client = createClient({ url: env.REDIS_URL })
client.on('error', (error) => logger.error({ error }, 'Redis error'))
await client.connect()

const cache = createCache({ store: createRedisStore({ client }), prefix: 'app', storeTimeout: '200ms' })
// On shutdown: await client.close() (node-redis 5 or later; quit() with node-redis 4 and ioredis)

Using both

The two tools combine: the cache keeps the result across requests, and the memo avoids repeated queries while computing it on a miss. In the Cache example (simplified here), the list of articles is cached for a minute; when it is computed, the authors are loaded with one batched query:

handler: ({ cache, memo }) =>
  cache.getOrSet(articleListCache, async () => {
    const articles = await db.articles.list()
    // On a miss only: one query for every author, through the memo
    return Promise.all(articles.map((article) => toView(memo, loadAuthor, article)))
  })

The example counts the queries of its fake database: its tests show what each tool saves.

Outside a route

Both tools work without an app, in a worker or a script:

import { createCache, createMemo } from '@dotpress/cache'

const memo = createMemo() // one per message or job
const account = await memo(loadAccount, id)

const cache = createCache({ store, prefix: 'app' })
const plan = await cache.getOrSet(planCache, planId, () => billing.getPlan(planId))

Memo without passing the context

Services deep in the code do not always receive ctx. With memoScope: asyncLocalMemo, each request runs in the scope of its memo (AsyncLocalStorage), and a memoized loader works anywhere during the request:

import { asyncLocalMemo, memoized } from '@dotpress/cache/als'

const app = await createApp({ adapter, plugins: [cachePlugin({ cache, memoScope: asyncLocalMemo })] })

export const getAccount = memoized(loadAccount)
// in any service called during the request:
const account = await getAccount(id)

In an event listener or a job, open a scope with runWithMemo(() => …): one memo per message. AsyncLocalStorage works on Node.js, Bun, Deno, and Cloudflare Workers with nodejs_compat. Outside a scope, a memoized loader throws: there is no global memo.

#Cache and memoization

#Set up

#Memo

#Load a value once per request

#Batch the loads

#One memo per unit of work

#Cache

#Keep a value across requests

#Expire values when the data changes

#Serve outdated values when the source fails

#When the store fails

#Stores

#Using both

#Outside a route

#Memo without passing the context