Background jobs — tcm-ts

Background jobs

import { enqueue, enqueueAfter, every } from "#codebase/app/lib/server";

export async function signup(user: string): Promise<string> {
  enqueue(sendWelcome, user);              // returns the job id at once; the work runs in the background
  enqueueAfter(3600, sendReminder, user);  // an hour from now
  return user;
}
export const nightly = every("0 3 * * *", cleanup);

Jobs come with the server library (tcm lib add tcm server). tcm start and tcm dev run them.

Queued jobs

enqueue(fn, ...args) queues fn(...args) and returns the job's id. enqueueAfter(seconds, fn, ...args) queues it to run later.

  • Durable. Jobs live in .tcm/data/jobs.sqlite and survive restarts. A failed job gets up to 5 attempts, with exponential backoff (at most an hour apart), and its last error is recorded.
  • Queued by hash. A job records the exact version of the code that enqueued it, so a deploy between enqueue and run doesn't change what it does. fn must be a codebase definition (not an inline function), and its arguments plain data.
  • Safe with several servers. Several tcm start processes can share the queue: each job is claimed atomically, with a lease, and a crashed worker's job is picked up again when its lease expires.
  • Transactional. Inside a transaction, enqueue takes effect on commit and is dropped on rollback.

Scheduled jobs

every("0 3 * * *", cleanup) takes five cron fields, in UTC. every("15m", sync) takes an interval (s, m, h, d).

  • Found from the code. tcm start and tcm dev find schedules in the app's namespace with no registration, and moving a name reschedules it.
  • Each slot runs once, however many servers share the codebase.
  • New schedules don't fire on their first deploy. After downtime, an interval schedule catches up once.

Managing jobs

  • tcm jobs [n] lists the queue.
  • tcm jobs retry <id> queues a job again; tcm jobs cancel <id> cancels a queued one. An id prefix is enough.
  • Each run is a trace of its own, linked to the call that enqueued it.
  • In tests, runJobs() runs the queued jobs that are due, inline; runJobs({ all: true }) runs them all (testing).

Built from c482a14e