Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/pg-enums-unique-constraints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@maple-dev/effect-orm": minor
---

Postgres enum types and unique constraints are first-class. `PG.enumType(name, labels, { schema? })` is a column type (rows decode to its labels; inserts and comparisons take only them; `nullable` and `array` wrap it, and enum arrays decode from the text form drivers send) and a schema object: `generate` creates the type before the tables that use it, adds labels with `ADD VALUE ... BEFORE`, renames one behind a `confirm_rename` hint, asks before dropping the type, and reports removing or reordering labels, or using a label in the migration that adds it, as unsupported. `PG.unique(name, columns, { nullsNotDistinct? })` in a table's `uniqueConstraints`, or `unique` in `PG.column` (named `<table>_<column>_unique` by default), declares an `ADD CONSTRAINT ... UNIQUE` constraint, diffed as a constraint rather than an index. `verify` reads enums from `pg_enum` and unique constraints from `pg_constraint`, and resolves types of the current schema during its scratch rebuild, so `PG.custom("public.x", ...)` enums no longer report false drift. `generate --baseline --from-drizzle` converts drizzle-kit enums, enum columns and unique constraints.
5 changes: 5 additions & 0 deletions .changeset/runnable-typescript-5.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@maple-dev/effect-orm": patch
---

A concrete insert is assignable to `Runnable` (from `/database`) and accepted by `compile` under TypeScript 5.x again. Both took `CHInsert<any, ...>`, which TypeScript 5 compares method by method, and the generic `onConflictDoUpdate` failed that comparison; they now take an insert by its tag, state and phantom (`AnyInsert`). A workaround such as `type Runnable = Orm.Runnable | { readonly _tag: "CHInsert" }` can be removed.
15 changes: 12 additions & 3 deletions design/migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -518,6 +518,15 @@ legacy: `check` accepts them, `generate` refuses to diff until `--baseline` (fro
definitions, or `--from-drizzle` from drizzle-kit's last snapshot) starts the history.
`Migrate.baseline` records already-applied migrations without running them.

**Not done.** Check and unique constraints, enums, views, sequences beyond identity defaults,
non-`public` schemas, `CREATE INDEX CONCURRENTLY` (needs a migration outside a transaction),
rename detection, and `pull` / `push`.
**Enums and unique constraints.** `PG.enumType` is a column type and a schema object; its
`enum` entity sorts before tables, `create_enum` runs first and `drop_enum` last. Labels are
added in place (`ADD VALUE ... BEFORE`), a label replaced in place is renamed (`RENAME VALUE`)
behind a `confirm_rename` hint, and removing or reordering labels is unsupported. A migration is
one transaction, so a label it adds cannot be used in it; the diff reports a default or
predicate that would. Unique constraints are `unique` entities, kept apart from unique indexes
all the way to `verify`, which reads them from `pg_constraint`. `verify` searches the current
schema after the scratch one, so types of the current schema deparse unqualified on both sides.

**Not done.** Check constraints, views, sequences beyond identity defaults, tables in non-`public`
schemas, removing enum labels (a type rebuild), `CREATE INDEX CONCURRENTLY` (needs a migration
outside a transaction), rename detection for tables and columns, and `pull` / `push`.
77 changes: 63 additions & 14 deletions docs/migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -285,17 +285,59 @@ with `where` for a partial index and `using` for the access method. A foreign ke
`name` gets drizzle-orm's, `<table>_<columns>_<foreign table>_<foreign columns>_fk`, shortened
with drizzle-kit's hash to `<table>_<hash>_fk` when it would pass 63 characters. Types are
stored as Postgres names them (`int4` is `integer`), so snapshots compare with the catalog and
with drizzle-kit. Check and unique constraints, generated columns, enums, views, sequences and other schemas
are not modeled yet; write them in a `--custom` migration, and declare a view you query as an
with drizzle-kit.

**Unique constraints.** `uniqueConstraints: [PG.unique(name, columns, { nullsNotDistinct? })]`
on a table, or `unique` in `PG.column` for one column, declares a `UNIQUE` constraint: `ALTER
TABLE ... ADD CONSTRAINT name UNIQUE (columns)`, which is what drizzle-kit writes for `unique()`
and `.unique()`. It is not a unique index: Postgres backs it with an index of the same name, but
drops it with `DROP CONSTRAINT`, and a foreign key may reference its columns. `unique: true`
names the constraint `<table>_<column>_unique`, drizzle-orm's name; `unique: "name"` or `unique:
{ name, nullsNotDistinct }` gives another (drizzle-kit v1 names a column's unnamed `.unique()`
`<table>_<column>_key`, so say so when adopting one). `nullsNotDistinct` needs Postgres 15.
`onConflictDoNothing` / `onConflictDoUpdate` take the constraint's columns as `target`, or `{
constraint: name }`.

**Enums.** `PG.enumType(name, labels)` is a column type and a schema object; see
[Enums](./postgres.md#enums). Its `CREATE TYPE` comes before every table, and is skipped when
the type already exists (Postgres has no `IF NOT EXISTS` for it), so an enum created by hand
behind `PG.custom` can be declared with `enumType` later.

Check constraints, generated columns, views, sequences and tables in other schemas are not
modeled yet; write them in a `--custom` migration, and declare a view you query as an
[external table](./tables-and-types.md#external-tables).

**Generating.** Postgres changes a column's type, nullability, default or identity in place
(`ALTER COLUMN`), and a primary key, an index or a foreign key by dropping and re-creating it,
so `generate` reports nothing as unsupported. A type change is labeled `rewrite`: Postgres
rewrites the table under an exclusive lock. Drops still need confirmation, and renames still
read as a drop and an add. Generated files carry `"dialect": "postgres"`. With `emit: "sql"` in
the config, `generate` writes the rendered statements to `migration.sql` instead, each under a
comment naming its label, so a tool that applies drizzle-kit folders applies the migration too.
(`ALTER COLUMN`), and a primary key, an index, a unique constraint or a foreign key by dropping
and re-creating it. A type change is labeled `rewrite`: Postgres rewrites the table under an
exclusive lock. Drops need confirmation, dropping an enum type included (`"kind": "enum"`), and
renames of tables and columns still read as a drop and an add. Enum types are created first and
dropped last, unique constraints are added before the foreign keys that reference them, and
turning a `uniqueIndex` into a `unique` of the same name drops the index before adding the
constraint. Generated files carry `"dialect": "postgres"`. With `emit: "sql"` in the config,
`generate` writes the rendered statements to `migration.sql` instead, each under a comment
naming its label, so a tool that applies drizzle-kit folders applies the migration too.

Enum labels change as far as Postgres can change them in place:

- **A new label** is `ALTER TYPE ... ADD VALUE`, with `BEFORE` the label it precedes, so the
order matches the definition.
- **A renamed label**, the same labels in the same order but one replaced in place, is `ALTER
TYPE ... RENAME VALUE`. Rows holding the old label then hold the new one, so it asks for a
hint first: `{"type":"confirm_rename","kind":"enum_value","entity":"channel_type.direct","to":"dm"}`.
- **Removing or reordering labels** needs a new type and every column moved to it. `generate`
reports it as unsupported and writes nothing; write that rebuild with `--custom`.

A migration is one transaction, and Postgres does not let a transaction use a label it added
(`unsafe use of new value`). So `generate` refuses a migration that adds a label and also
uses it, in a default or a partial index's predicate: add the label, generate, then use it and
generate again. Before Postgres 12, `ADD VALUE` cannot run in a transaction at all, so a
migration that adds a label needs Postgres 12 or later.

Moving an enum from `PG.custom("public.channel_type", ...)` to `PG.enumType("channel_type",
...)` generates a `create_enum`, skipped since the type exists, and an `alter_column` per column
from `public.channel_type` to `channel_type`, the same type under its unqualified name, which
leaves the data as it is.

**Applying.** Each migration runs in one transaction with its ledger row, under a
transaction-scoped advisory lock, so concurrent deploys wait for each other rather than
Expand All @@ -307,10 +349,15 @@ CONCURRENTLY`, `ALTER TYPE ... ADD VALUE` before Postgres 12) cannot be in a mig

**Drift.** `verify` builds the expected schema in a scratch schema, inside a transaction it
always rolls back, and reads both catalogs with the same queries, so Postgres deparses both
sides and `'open'` matches the stored `'open'::text`. It checks tables, columns (type, `NOT NULL`,
default, identity), primary keys, indexes (uniqueness, method, keys, predicate) and foreign keys
in the current schema. `Migrate.verify(migrations, { ignoreTables })` skips tables another tool
owns.
sides and `'open'` matches the stored `'open'::text`. The scratch schema is searched before the
current one, so a type of the current schema resolves in both, and a column of it reads
`channel_type`, its default `'public'::channel_type`, on both sides, however the definition
qualified it. It checks enum types (labels, in order, read from `pg_enum`), tables, columns
(type, `NOT NULL`, default, identity), primary keys, unique constraints (columns, `NULLS NOT
DISTINCT`), indexes (uniqueness, method, keys, predicate) and foreign keys in the current
schema. A unique constraint is never read as an index, nor an index as a constraint. An enum
the snapshot does not declare is reported only if no column uses it, so a `PG.custom` enum still
verifies. `Migrate.verify(migrations, { ignoreTables })` skips tables another tool owns.

### Adopting a drizzle-kit folder

Expand All @@ -319,8 +366,10 @@ split on `--> statement-breakpoint`. Its `snapshot.json` files are recognized as
and set aside, so the folder runs as it is. Adoption is two steps:

1. `effect-orm generate --baseline --from-drizzle` writes a migration that runs nothing, whose
snapshot is drizzle-kit's last one converted to entities. Anything the conversion cannot model
is listed and nothing is written. Without `--from-drizzle` the snapshot comes from your
snapshot is drizzle-kit's last one converted to entities: enums, tables, columns (enum columns
included), keys, indexes, unique constraints and foreign keys. Anything the conversion cannot
model (sequences, views, check constraints, policies, tables in other schemas) is listed and
nothing is written. Without `--from-drizzle` the snapshot comes from your
`PG.table` definitions instead. Migrations before the baseline are legacy: they run, but
nothing diffs against them, and a plain `generate` refuses to run until a baseline exists.
2. On a database drizzle-kit (or anything else) already migrated, `effect-orm baseline <name>`
Expand Down
71 changes: 69 additions & 2 deletions docs/postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
The same builder writes Postgres SQL. Queries, params, tenant scoping and row decoding work
as they do for ClickHouse. `@maple-dev/effect-orm/postgres` is the one import for it: the
builder, column types whose codecs read what Postgres drivers send, functions spelled the
Postgres way, `table` with its keys, indexes and foreign keys, and a `compile` that defaults to
Postgres way, `table` with its keys, indexes, unique constraints and foreign keys, `enumType`,
and a `compile` that defaults to
the Postgres dialect. Nothing from `/clickhouse` is needed.

```ts title="postgres-quickstart.ts"
Expand Down Expand Up @@ -59,7 +60,8 @@ await db.close()
`@effect/sql-pg`'s `unsafe`. The builder never runs the query. `Id` is an identity column, so an
insert may leave it out: `PG.InsertRowOf<typeof Requests>` makes it optional. See
[Tables and column types](./tables-and-types.md) for column options and external tables, and
[Schema and migrations](./migrations.md#postgres) for indexes, foreign keys, and `generate`.
[Schema and migrations](./migrations.md#postgres) for indexes, unique constraints, foreign keys,
and `generate`; [Enums](#enums) for enum types.

## What the dialect changes

Expand Down Expand Up @@ -98,6 +100,7 @@ literal that still contained it would fail the compile with `InvalidLiteral`.
| `timestamptz` | `timestamptz` | `DateTime.Utc` | `Date`, or text such as `2026-01-01 00:00:00+00` |
| `timestamptzMillis` | `timestamptz` | epoch milliseconds (`number`) | the same |
| `jsonb(schema?)` | `jsonb` | the schema's type (`unknown` by default) | a parsed value |
| `enumType(name, labels, { schema? })` | the enum type | one of its labels | string |
| `array(type)` | `type[]` | `ReadonlyArray` | array |
| `nullable(type)` | the same type | `T \| null` | the same, or `null` |
| `custom(sql, schema, literalSchema?)` | anything | the schema's type | whatever the schema reads |
Expand All @@ -115,6 +118,70 @@ Comparisons with a literal-union column take only its members: with `status` typ
`"open" | "closed"`, `$.status.eq("opne")` is a type error. A param of the primitive
(`param.string`) still compares, for a value known only at run time.

## Enums

`PG.enumType(name, labels)` declares a Postgres enum type. The value is a column type, so rows
decode to the union of its labels and inserts and comparisons take only those, and it is a
schema object, so `generate` writes `CREATE TYPE ... AS ENUM` before any table that uses it and
alters the type when its labels change. Use it bare, in `PG.column` with a `default`, or inside
`PG.nullable` and `PG.array`:

```ts title="postgres-enums.ts"
import { Effect } from "effect"
import * as PG from "@maple-dev/effect-orm/postgres"
import * as S from "@maple-dev/effect-orm/schema"

export const ChannelType = PG.enumType("channel_type", ["public", "private", "direct"])

export const Channels = PG.table("channels", {
columns: {
id: PG.text,
org_id: PG.text,
slug: PG.column(PG.text, { unique: true }),
type: PG.column(ChannelType, { default: "public" }),
allowed: PG.nullable(PG.array(ChannelType)),
},
primaryKey: ["id"],
uniqueConstraints: [PG.unique("channels_org_slug_unique", ["org_id", "slug"])],
})

export type Channel = PG.SelectRowOf<typeof Channels>
// { ...; readonly type: "public" | "private" | "direct"; readonly allowed: ReadonlyArray<...> | null }

export const shared = PG.compileUnsafe(
PG.from(Channels)
.select("id", "type")
.where(($) => [$.type.in_("public", "private")]),
{},
)

export const upsert = PG.compileUnsafe(
PG.insertInto(Channels)
.values({ id: "c1", org_id: "o", slug: "general", type: "private" })
.onConflictDoUpdate({ target: ["slug"], set: { type: "private" } }),
{},
)

export const ddl = Effect.map(S.pgEntitiesOf([ChannelType, Channels]), (entities) => S.renderPgSchema(entities))
// CREATE TYPE "channel_type" AS ENUM ('public', 'private', 'direct') (skipped if it exists)
// CREATE TABLE IF NOT EXISTS "channels" (... "type" channel_type NOT NULL DEFAULT 'public', ...)
// ALTER TABLE "channels" ADD CONSTRAINT "channels_org_slug_unique" UNIQUE ("org_id", "slug")
// ALTER TABLE "channels" ADD CONSTRAINT "channels_slug_unique" UNIQUE ("slug")
```

`ChannelType.values` lists the labels for application code. Exporting the enum from a schema
module is optional: the tables that use it bring it along. Pass `{ schema: "auth" }` for a type
in another schema; leave it out for the schema migrations run in (usually `public`), which is
how columns of the type are spelled there.

Drivers parse arrays of built-in types but not of an enum, whose array type they do not know:
node-postgres and PGlite send `{public,private}` as text. An `array(enumType)` column reads
either form, and writes its value as array text, which Postgres casts to the column's type.

`PG.brand(PG.text, Schema.Literals([...]))` and `PG.custom("public.channel_type", ...)` still
work for an enum `generate` does not manage; see [migrations](./migrations.md#postgres) for
moving one to `enumType`.

## Functions

| Function | SQL | Notes |
Expand Down
Loading
Loading