Definitions and hashing — tcm-ts

Definitions and hashing

What a definition is

A definition is a top-level function (with its overloads), const, class, type or interface.

  • Destructuring binds one definition per name: const { a, b: [c] } = init() is three. The statement still runs once.
  • A type and a value can share a name (const User = z.object(…) and type User = z.infer<typeof User>). Each is its own definition, updated, diffed and merged apart; #codebase exports both under the one name.
  • Type-only declarations merge with the type of their name, as TypeScript merges them: an interface Effect<A> with a declare namespace Effect { type Success<T> = … }, a declare const s: unique symbol with type s = typeof s, a class with a namespace of types. They're one definition.
  • A statement setting up a constant is part of it. A top-level statement that only writes into one constant declared above it is hashed and run with that constant: for (…) codes[c] = i, Impl.prototype = Proto, Object.freeze(types), Object.defineProperty(Proto, …). Writing means assigning into it (x[i] =, x.p =, x.n++, delete x.p) or passing it to an Object or Reflect function that changes its argument. Method calls (server.listen()) don't count. Any other statement is refused.
  • An augmentation is a definition of its own. A file's declare global { … } blocks are <namespace>.global; a declare module "./schema" { … } block is <namespace>.augments, and names the definition it adds to by hash, so it follows renames (in a scratch file, write declare module "#codebase/lib/schema"). It has no name to import: every check includes it. Code relying on what it adds doesn't name it, so its hash doesn't include it, as with @types packages.
  • A class is one unit. Methods aren't hashed apart.
  • Refused in scratch files, with a reason: top-level var/using, enums, namespaces, decorators, declare, and top-level statements other than watch expressions and statements setting up a constant. tcm import accepts most of these.
  • Dynamic import("pkg") of a package or Node builtin is pinned like a static import. Lazy imports of local files or codebase modules are refused. A computed specifier (import(url)) is allowed, with a warning: it loads code the codebase doesn't track.

How a definition is hashed

Each definition is lowered to an IR of syntax-kind names with every child labelled, and the IR is hashed. Unknown syntax is an error, never skipped, so two different programs can't collide.

What doesn't change a hash:

  • comments, formatting and export;
  • how a literal is written: 1_000 is 0x3e8, 'a' is "a", { x } is { x: x };
  • names of locals (they become positions);
  • JSX whitespace React ignores: indentation and line breaks between tags (spaces within a line count).

What does:

  • type annotations;
  • references, resolved through the type checker: another definition by its hash, an npm import by package and compatible range (npm:lodash@4; packages), a global by name;
  • property names, JSX tags and attributes, as written (a component tag like <Counter /> is a reference);
  • for a definition with JSX, the JSX runtime it compiles to (npm:react@<version>).

Mutually recursive definitions, across files too, are hashed together as a cycle.

Identity

  • Lineage salts. In JavaScript every declaration has its own identity: two identical classes fail each other's instanceof. So each function, class and value is hashed with a salt: the path where it was first added. Updates and renames keep the salt, so renames stay free and re-adding the same file gives the same hashes. Identical code added at two paths is two definitions; use tcm alias for one definition under two names.
  • Types are unsalted, so identical types are one definition. An ambient declare const x: unique symbol is salted: each is a distinct symbol.
  • Names are hashed where JavaScript can see them (.name): function and class declarations, const f = () => …, and local bindings that name an anonymous function (function helper() {}, (cb = () => {}) =>). So tcm mv changes a definition's path but not its .name. Other locals and parameters can be renamed freely.

The hash format

  • The format is versioned. tcm --version says which (hash format v2). A codebase in another format is refused, not mixed: an older one says to run tcm migrate, a newer one says to use a newer tcm.
  • tcm migrate moves a codebase to this tcm's format, history and all. Every definition is converted and hashed again, then its history (entries keep their times and commands), and everything keyed by a hash follows: names, branches, releases, patches, test results. It's deterministic, so a clone and its server each migrating get the same hashes and keep syncing (tcm serve migrates when it starts). The old database is kept beside it (.tcm/codebase.v<n>.sqlite, n the old format), and the new one is used only if tcm verify passes on it.
  • A codebase whose tables an older tcm laid out differently is refused the same way, and tcm migrate moves it too, keeping the old database (.tcm/codebase.layout-<layout>.sqlite). Hashes don't change. A codebase copied from a package an older tcm packed (the web UI's, the registry's, a library) is migrated as it's copied. A layout tcm has no step from (a newer tcm's, or a very old one) has to be made again: imported from its files, or pulled.
  • Hashes don't depend on the compiler or its version. The order the IR visits children in is tcm's own, so a TypeScript release changes no hash. TypeScript 7 and tsrs hash identically.
  • tcm verify renders every live definition from the store and hashes it again; any mismatch means information was lost. It skips what it verified already while the names it renders with and the tcm build are the same; tcm verify --fresh checks everything. TCM_KEEP_VERIFY=1 leaves the rendered files in .tcm/cache to debug a failure.

Built from c482a14e