Importing existing projects — tcm-ts

Importing existing projects

tcm import <dir> --into ns [--exclude a,b] [--update] [--tests] [--mirror] [--verbose]

Each file becomes a namespace: core/src/analyze.ts → ns.core.analyze (src path segments and index file names are dropped). Imports between files, barrels and workspace packages resolve to hashes.

--into ns
the namespace to import into
--exclude a,b
paths to skip, relative to <dir>, comma-separated (test helpers that need a dev dependency, say)
--update
re-import a project whose files changed, as tcm update would
--tests
also import test/ and tests/ directories, for tests written as test definitions (exports named test…, plus their helpers). *.test.ts files there stay skipped, so a thin runner can register the same definitions with another framework. tcm test then runs them, cached by hash
--mirror
keep the directory in step with the codebase (mirrors)
--verbose
report every definition
  • Skipped: *.test.ts, *.spec.ts, *.test-d.ts files, and test, tests, __tests__, dist, build and node_modules directories.
  • All problems at once. Import reports every problem in the project in one pass, and imports nothing until there are none.

Compiler options for imports are in compiler options.

Mirrors

With --mirror, the imported directory follows the codebase. The codebase is the truth; the files are how it reads in git and in editors that don't know tcm. Changes made in tcm are written back as edits, and the rest of each file (comments, import style, barrels) stays as written.

Changes in tcm, written to the files:

  • An edited definition (tcm edit then tcm update, a merge, a pull) is replaced where it is. References are spelled as the file already spells them (ts.Node through a namespace import, parse through a named one). Imports it needs are added, and ones it no longer uses removed.
  • A new definition goes at the end of its namespace's file. A .ts file becomes .tsx when JSX arrives, and imports of it follow. What a file comes to import from another is exported there. A namespace with no file yet gets one where the mirror reads it back as that namespace: next to a sibling namespace's file, in its parent's directory, or by its path.
  • A removed definition is deleted.
  • A rename (tcm mv) is made at every reference, as an editor's rename would: named imports, namespace imports (math.twice), shorthand properties ({ double: twice }), re-exports. It also reaches files near the mirror that use the name (the rest of its git repository: scripts, tests, an editor extension), though only the mirror's files are checked. A function's or class's name is part of its hash, so the codebase then takes the renamed code.
  • A move to another namespace (tcm mv app.math.half app.extra.half) moves the declaration to that namespace's file. Imports follow, there and in files near the mirror (one that imported others too is split). The file it left imports it if it still uses it; the file it went to stops. Moved and renamed at once, it's moved, then renamed. A namespace import (math.half) can't follow: tcm says so and leaves the files as they were.
  • Dependents re-hashed by an update, and upgraded packages, change no text.

After writing, the files are analyzed again and must hash to what the codebase holds. If they don't, they're put back as they were and tcm says why.

What's kept:

  • Everything that isn't code (Markdown, images, configuration, scripts, hidden files) is files of the codebase, named by their directory's namespace and their own name, and written back as they change. In a git repository that's what git keeps: what it ignores (node_modules/, builds) isn't the codebase's, nor is tcm-releases.json (releases). So a whole repository can be mirrored.
  • A directory with a .tcm-files file in it is all files, .ts included (examples, fixtures): kept as written, not read as code. The mark's lines are patterns to leave out (*.vsix).
  • A file whose name can't be a name (a dot before its extension) is reported, and nothing is imported until it's renamed or left out.
  • Code that imports a file (a page, an image) is written importing it from #codebase/… (files are values). Plain TypeScript can't import a page, so the project's package.json must map #codebase to the codebase's shim ("#codebase/*": "./codebase/.tcm/codebase/*.ts"), with the codebase in the project. Without that the files don't check, and the change isn't written to them.
  • The files' style is kept. A mirrored codebase stores code as written (not reformatted), and tcm edit gives it back that way. Code edited in a scratch file is stored as the mirror spells it once written back, so a scratch file's spellings (import nodePath from "node:path") don't reach the files later.
  • The files are one branch's: the branch the mirror was linked on (usually main). A session on another branch neither writes to them nor takes their changes; its work reaches them when merged there.

Changes outside tcm (by hand, or by git: a pull, a checkout, a stash) are imported before each command runs, so it sees the files as they are:

  • A definition changed in the files is updated in the codebase, as tcm import --update would.
  • One gone from the files goes from the codebase.
  • One the files didn't change stays as the codebase has it, even if the codebase changed it meanwhile; that change is then written to the files. Changed in both, the files' change wins, and tcm says so.
  • Files that don't import yet (an edit half made) wait: the command runs without them, and nothing's written to them until they import.

What importing them says goes to stderr, so a command's output (tcm run) is its own.

tcm import <dir> --into ns --update
import the files' changes by hand
tcm mirror --check
re-analyze the files: they must hash to what the codebase holds
tcm mirror --unlink
stop writing back; the files stay as they are

Verified projects

These have been imported, verified (tcm verify: every definition renders and re-hashes to itself) and run from a scratch file:

projectdefinitionsnotes
neverthrow51strict: false, kept as its check profile
ts-pattern247namespace imports (P.infer<…>), declare const
valibot (library/src, --exclude vitest)941573 files in 4s; module state, @ts-expect-error above overloads
Effect 4.1.2 (src of its npm package; the example)15,671580 files in 6–9 minutes; interfaces with namespaces of types, statements setting up constants, augmentations, cycles of up to 62 definitions; tcm verify in about 2.5 minutes. Needs seven small changes (changes.patch): four for TypeScript 7.1's extra errors, a module object used as a value, and two imports of the package by its own name
tcm-ts itselfthe repositorysee verifying a change

What import mode accepts beyond scratch files

  • Enums, namespaces and parameter properties. Node can't strip these, so they're compiled when loaded. They hash like any other syntax.
  • Namespace imports of project modules. ns.member from import * as ns from "./mod" is a reference to member, hashed as if imported by name. Using ns itself as a value is still refused.
  • Modules that re-export a package. If ./mod has export * from "pkg" or export { x } from "pkg", then ns.x (or import { x } from "./mod") is a reference to pkg's x. If it has export * as p from "pkg", p.x (via import { p } from "./mod", or ns.p.x) is too, in types as well, and p alone is pkg's namespace.
  • declare const of anything but a unique symbol (declare const process: { cwd(): string }) is a value the host provides, read from globalThis and typed as declared. Missing, it's undefined (where reading the name would have thrown).
  • declare const x: unique symbol (the brand pattern) is a type-only definition, salted, so two identical declarations are distinct symbols. So is a declare class (a class to cast a mixin to), and a namespace holding only types (declare namespace Effect { … }; a scratch file takes one too). Each merges with a type of its name into one definition (hashing).
  • Top-level let is module state (a lazy cache, a config cell), salted so each declaration is its own cell. Every definition that assigns to it shares its component (imported bindings are read-only). Readers import it and see live updates.
  • // @ts-expect-error and // @ts-ignore directly above a declaration (or an overload) stay with it. They aren't hashed, but decide whether it typechecks.
  • Statements setting up a constant (a table a loop fills, Impl.prototype = Proto, Object.freeze(types)) go with it (hashing).
  • export default name of a definition the module declares: a default import of the module uses it.
  • declare global { … } and declare module "./x" { … } (augmenting another module's definition) become definitions, <file>.global and <file>.augments, included in every check in the codebase (hashing).
  • A namespace import beside a type of the same name reads as TypeScript reads it: in a type, Iterable<T> beside import * as Iterable is the global type, Scope beside import type * as Scope is the file's own interface Scope, and Scope.Scope is the module's.

Still refused:

  • a declare module adding to several of a module's definitions at once (write one for each), or of a package;
  • other ambient declarations (a declare namespace with values, declare function);
  • import x = require(), decorators, export default of an expression;
  • other top-level statements (a call, a statement writing into several constants or into something not the module's). Skipping one could leave a definition half-initialized, so wrap it in an immediately-invoked function. A module there only for its side effect (an enable.ts calling install()) is left out with --exclude.

Built from c482a14e