Skip to content

field-types.md

Latest commit

 

History

History
548 lines (454 loc) · 30.3 KB

field-types.md

File metadata and controls

548 lines (454 loc) · 30.3 KB
Raw

Field types

A field is a typed leaf of an entity. The metamodel ships a closed vocabulary of field subtypes; each port maps the subtype to its idiomatic native type, its DB column type, and (where appropriate) its validator. The wire format is identical across ports — a field.currency is integer minor units everywhere; a field.timestamp is ISO-8601 with timezone everywhere.

Field subtype reference

Subtype TS Java Kotlin C# Python DB (Postgres)
field.string string String String string str varchar(@maxLength ?: 255)
field.int number Integer Int int int integer
field.long number (or bigint) Long Long long int bigint
field.double number Double Double double float double precision
field.float number Float Float float float real
field.decimal string BigDecimal BigDecimal decimal Decimal numeric(@precision, @scale)
field.boolean boolean Boolean Boolean bool bool boolean
field.date Date LocalDate LocalDate DateOnly date date
field.time string LocalTime LocalTime TimeOnly datetime.time time
field.timestamp Date Instant Instant DateTimeOffset datetime timestamp with time zone
field.currency number (minor units) Long (minor units) Long (minor units) long (minor units) int (minor units) bigint
field.uuid string UUID UUID Guid UUID uuid
field.uri string java.net.URI java.net.URI Uri AnyUrl text
field.inet string InetAddress InetAddress IPAddress IPvAnyAddress inet
field.duration string (ISO-8601) java.time.Duration java.time.Duration TimeSpan timedelta interval
field.enum union + z.enum Enum enum class enum Literal inline, Enum with UPPER_CASE members when shared by extends varchar + CHECK
field.object nested type nested class nested data class nested class nested Pydantic model per @storage
field.map Record<string, V> Map<String, V> Map<String, V> Dictionary<string, V> dict[str, V] jsonb

That is the whole registered vocabulary — 18 concrete subtypes. field.base is an abstract registry anchor, never authored (ERR_ABSTRACT_SUBTYPE_AUTHORED).

Five rows need a footnote:

  • field.decimal is exact in every port's own type. TypeScript's type is a string. Postgres and MySQL keep the declared numeric(@precision, @scale) exactly; SQLite and D1 have no decimal type and keep 15 significant digits, so meta migrate warns for a decimal there with @precision above 15. On the JSON wire, each port writes its language's standard form by default: a number in C#, Java and Kotlin (12.50), and a string in Python and TypeScript ("12.50"). The project setting "wire": { "decimals": "string" } makes every port write the canonical string of ADR-0019 Amendment 1 ("12.5", zero as "0"). Input takes a string or a number in every mode, and a value past @scale or @precision is refused, never rounded. api-contract.md, "The wire setting" has the per-port table.

  • field.uri / field.inet bind a native URL/IP type on every port but TypeScript, which has none (the same reason field.uuid is a string there). @lenient: true degrades both to a plain string with no validator, and takes the inet column down to text.

  • field.duration is a fixed-length span: days, hours, minutes and seconds. A calendar span ("every 3 months") is not one, because a month has no fixed length: model it as an int and a unit. On the wire each port writes its own language's standard form by default (a C# TimeSpan is "01:30:00"), and "wire": { "durations": "iso" } in .metaobjects/config.json makes every port write one ISO-8601 form (the wire setting). That form is [-]PT[nH][nM][n[.f]S]: hours are never folded into days (PT26H), and zero is PT0S. It is one form at each port's own precision: a port writes as many fractional digits as its own type holds (6 in TypeScript and Python, 7 in C#, 9 in Java and Kotlin) and truncates a longer fraction toward zero. A Postgres interval and the integer column below both hold microseconds, so a C#, Java or Kotlin server can answer a POST with more digits than the database kept: PT0.1234567S is answered as sent and read back as PT0.123456S. Read the row back when the stored value matters. The Java types above are generated code's: Java's metadata-driven runtime (OMDB) holds a duration as its canonical ISO-8601 String, and java.time.Duration appears only inside its JDBC codec. Every port reads that form and also leading week and day counts (P1W is 7 days, P1DT2H is 26 hours). Years and months are refused, and so are lowercase designators and fractional hours. The column is a Postgres interval; SQLite, D1 and MySQL store a 64-bit integer count of microseconds, the way Django stores a DurationField outside Postgres. Both order and compare as the span does, so every filter operator a number has (eq, ne, gt, gte, lt, lte, in, isNull) and sort work on every database. There is no shared upper bound: each port is limited only by its own type (a .NET TimeSpan, a Python timedelta) and each database by its own column, and refuses what it cannot hold rather than wrapping it. The rule and its cross-port vectors are in fixtures/duration-wire/.

  • field.map is the typed dict: string keys, and a value type set by exactly one of @valueType (a scalar subtype) or @objectRef (a value object) — V above. It is one jsonb column holding the JSON object; isArray does not apply, so no port wraps the map type in a list. Codegen is now complete on all five ports. Java emits java.util.Map<String, V> (and reaches a map's @objectRef value object in the value-object emission walk, so the referenced record is actually generated); C# emits the Dictionary<string, V> property AND the EF jsonb storage mapping — a column type plus an explicit converter/comparer pair, so the property lands on the jsonb column the TS-owned migration creates instead of on whatever an unmapped dictionary would resolve to.

    Nested map values are validated on every write path, on every port. A posted map value that violates the referenced object.value's own constraints is rejected with the cross-port {"error":"validation"} envelope before the row is written. Scalar-valued maps (@valueType) carry no nested bean, so nothing extra runs for them.

    This was a real hole until 1.0.1, and it is worth knowing what it was, because the failure mode was silent acceptance rather than an error — reading your own source would never have revealed it. Java validated nested map values on its vanilla create/PATCH handlers but not on TPH (discriminator-rooted) write paths, which validate field-by-field with validateValue — that does not cascade @Valid into a nested bean. C# validated them on no write path at all: the map never reached the recursively-validating value-object arms (they admit field.object only), and ValueObjectValidator treated a Dictionary as a plain IEnumerable, so it validated KeyValuePair structs instead of the values. TypeScript and Python were never affected, and Kotlin writes no map column. (#362; fixed in 1.0.1.) If you added a boundary check of your own while this was open, it is now redundant rather than load-bearing.

    The RUNTIME tier is partial, and no persistence conformance corpus covers it. No persistence-conformance fixture exercises field.map; the api-contract jsonb/scenarios/jsonb-value-object-map-decimals.yaml gates a map of value objects through each port's generated API. Of the runtime persistence layers, Python's ObjectManager encodes a map (its jsonb write codec names FIELD_SUBTYPE_MAP alongside FIELD_SUBTYPE_OBJECT), and Java's OMDB maps a field.map @objectRef to a jsonb column whose values are the value object (GenericSQLDriver). TypeScript's runtime-ts and the Kotlin Exposed persistence lane carry no map-of-value-object handling. Closing that gap across ports is a cross-port runtime workstream, not a codegen change, and it is the prerequisite for the shared op: roundtrip persistence scenario that would gate this subtype the way every other persistable subtype is gated.

    So: for data you intend to READ BACK THROUGH A PORT RUNTIME today, a stable key set is still better declared as an object.value behind field.object, which every runtime does round-trip. See ADR-0037 for which of the two a shape belongs in.

Common field attributes

Attr Type Purpose
name string Field name (camelCase).
@required bool If true, generated type omits ? / Optional.
@maxLength int Length for string types (drives varchar(N)).
@column string Physical column name (overrides the project's naming.columns rule).
@default any Default value (literal or canonical SQL default function).
@filterable bool Field appears in the generated server-side filter allowlist.
@sortable bool Field appears in the server-side sort allowlist (defaults to @filterable).

Column naming — @column and the naming setting

A field has two names: the one your code calls it (createdAt, the metadata name, and what every port's generated property/attribute is called) and the one the database calls it. @column sets the second. It never changes the first.

With no @column, the physical name comes from the project's column rule: snake_case (the default), literal (the field name verbatim), kebab-case or upperFirst (the first letter upper-cased, the rest verbatim). That is CONFIG, not metadata, on purpose: the same model has to be able to drive a snake_case Postgres schema and a literal-column one. It is ONE setting for the whole project, naming in .metaobjects/config.json (ADR-0058):

{ "schema_version": 1, "naming": { "columns": "literal" } }

Every port reads it: meta migrate, every port's codegen (its names generator included), every runtime ObjectManager, and verify. They cannot disagree, because none of them is configured separately. Table names follow the same pattern: naming.tables (snake_case, the default, or literal) applies to a source with no @table/@view/@proc/@function, after its own name and before the object's name is made plural.

Every port ships a per-object names generator that resolves physical names through that setting: one generated constants file per object, so a hand-written consumer references a constant instead of respelling a physical name as a string literal. Enablement differs by port; see each port's own doc (docs/ports/{typescript,csharp,java,kotlin,python}.md).

The setting is the answer. Set it once, to the rule meta migrate created the schema with (that is snake_case unless you changed it), and every generated artifact in every port agrees with the database. A per-field @column is for a column whose name genuinely cannot be derived at all (a legacy column, a table whose schema is owned outside this project), not a substitute for the setting.

Through 1.0 each port had its own column-naming option and its own default, and C#, Java and Python defaulted to literal while meta migrate created snake_case. Those options are refused in 1.1.0; the migration guide gives the one line that keeps 1.0's literal names where a database needs them.

A field can still declare @column explicitly for the case the setting cannot cover — a name that no strategy would ever produce:

{ "field.timestamp": { "name": "createdAt", "@column": "created_at" } }

That declares the column created_at. The TypeScript property is still createdAt, the C# property CreatedAt, the Python model field createdAt — the language-level name follows the metadata name in every port, always.

Currency

field.currency declares "this column stores money as integer minor units" (cents for USD, yen for JPY). The server never formats currency; all formatting is client-side via locale-aware code.

{ "field.currency": {
    "name": "priceCents",
    "@currency": "USD",
    "@required": true,
    "children": [ { "view.currency": { "@locale": "en-US" } } ]
}}

Wire format invariant: integer minor units. @currency is ISO 4217. @locale is BCP 47.

Enum

field.enum is a first-class string-backed enum field. @values is required and must be a non-empty set of unique members matching ^[A-Za-z_][A-Za-z0-9_]*$.

{ "field.enum": {
    "name": "status",
    "@required": true,
    "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"]
}}

The loader enforces members own-only and emits ERR_BAD_ATTR_VALUE on a bad member or ERR_MISSING_REQUIRED_ATTR on missing @values.

Column length: a string-stored enum is an unbounded text column unless it has a length. @maxLength on the field gives it one, and "enums": { "maxLength": n } in .metaobjects/config.json gives every enum without its own @maxLength a default:

{ "field.enum": { "name": "status", "@values": ["DRAFT", "PUBLISHED", "ARCHIVED"], "@maxLength": 16 } }

The column is then VARCHAR(n): meta migrate creates it, and the generated column bindings that carry a length name the same one: TypeScript's Drizzle and Prisma, C#'s EF Core (HasMaxLength(n)) and Kotlin's Exposed. Java and Python generate no such binding; their loaders apply the same member-length refusals, and the database column holds the length. Every member must fit. A member longer than @maxLength is refused at load, and one longer than the project default is refused when the column is sized, both as ERR_ENUM_VALUE_EXCEEDS_MAX_LENGTH, with the member named. @maxLength beside @intValueMap is refused (ERR_ENUM_MAX_LENGTH_WITH_INT_VALUE_MAP): an int-backed enum's column is an integer. An array of enums stays a text[] column. Turning a length on for a column that already exists (text to VARCHAR(n)) fails on any longer row, so meta migrate gates it behind --allow type-change; turning it off widens the column and is allowed. SQLite does not enforce the length, so there is nothing to change there.

Int-backed storage: @intValueMap switches the DB column from string to integer while preserving the string wire format and generated enum type. Keys must match @values exactly; values must be unique integers. Display labels and native Postgres ENUM types remain deferred (see enum-datatype-design.md).

Two rules follow from int-backing being a persistence-layer codec:

  • It is scalar-only. @intValueMap together with isArray: true is a load error, ERR_ENUM_INT_VALUE_MAP_ARRAY, in every port — no port implements the codec element-wise over an array column, so the combination would silently persist member symbols into an integer array. An array-of-enum stays string-backed.
  • A stored integer that maps to no member throws on read, in every port. The row holds data the model says is impossible (a hand-written INSERT, or a member removed without a migration); surfacing the raw integer would hand you a "member" that is not one — and is not even representable in the ports that type the property as a closed enum — while returning null would hide the corruption behind a nullable column. The write side is left to the database: an unmapped symbol binds unchanged, so the column type and its CHECK reject it.

Sharing one enum — abstract field.enum + extends

Reuse a constraint set across entities by declaring one abstract field.enum (with the @values) and having each concrete field extends it — including across packages (declare the shared enum in a common package and reference it by FQN). Every field that reaches the same abstract enum, directly or through other fields, collapses onto one generated enum type, so the members stay in exactly one place:

// meta.common.json — package acme::common
{ "field.enum": { "name": "RecordStatus", "abstract": true,
    "@values": ["DRAFT", "ACTIVE", "CLOSED"] } }

// meta.orders.json — package acme::orders
{ "field.enum": { "name": "status", "extends": "acme::common::RecordStatus" } }

Two rules the loader/codegen enforce:

  • A field extending a shared abstract enum must NOT declare its own @values. A shared enum is one type with one member set, so an own @values on the extending field would be silently dropped by the shared-enum collapse — it is a load error, ERR_ENUM_EXTENDS_VALUES_CONFLICT. Remove the own @values to inherit the shared set. If you need a different member set, declare a separate field.enum with its own @values rather than extending the shared one — extending is for reuse of one member set, not for overriding it.

  • A field keeps the shared type through any number of extends hops. A field that extends another field which itself reaches the shared abstract enum has the SAME generated type as a field that extends it directly, whatever object hosts it (entity, projection, value object):

    { "field.enum": { "name": "status",  "extends": "acme::common::RecordStatus" } }   // on Order
{ "field.enum": { "name": "current", "extends": "acme::orders::Order.status" } }   // on OrderView: RecordStatus too

    The walk stops at the first field that declares its own @values or @intValueMap: that field starts a new vocabulary, so it and everything extending it get a per-object inline enum instead. A chain that never reaches a package-level abstract enum is inline as well. Declaring the vocabulary at package level is how you ask for one type.

Embedded value objects — field.object + @storage

field.object declares an embedded structured value backed by another object declaration. The @storage attr controls how the embedded shape persists.

@storage Persistence shape Use case
flattened One DB column per sub-field: <prefix><sub>, the prefix <parent>_ by default EF Core OwnsOne / DDD value-objects
jsonb One jsonb column Structured-but-evolvable user-defined shapes
subdocument Single jsonb column A hint for document-store targets
not set (the default) One jsonb column, as jsonb Pre-@storage projects

Example: an Author with an embedded Address flattened to per-sub-field columns:

{ "field.object": {
    "name": "address",
    "@objectRef": "Address",
    "@storage": "flattened"
}}

For flattened, the generated table gets address_street, address_city, address_postal_code columns instead of one address jsonb.

Each member's column is a prefix followed by the member's own column (its @column, else its name under the project's column rule). The prefix defaults to the embedding field's column and _ (address_; with @column: "bill", bill_; under the upperFirst rule, Address_, EF Core's own default). @flattenedColumnPrefix on the embedding field replaces it, verbatim:

{ "field.object": { "name": "homeAddress", "@objectRef": "Address", "@storage": "flattened",
    "@flattenedColumnPrefix": "" } }

gives street, city, postal_code; "@flattenedColumnPrefix": "home_" gives home_street, and so on. The prefix belongs to the embedding field, not the value object, because it is a fact about the owning table: one Address embedded twice needs two prefixes. Every port names these columns through the same core function (resolveFlattenedColumnName and its per-port equivalents), so meta migrate, the Prisma schema, the C# EF mapping and the Kotlin Exposed tables agree, and the owning object's names artifact carries each one as a constant (ShipmentNames.ShipToStreetColumn in C#; see the codegen skill's physical-names reference for every port). The field itself has no column constant, because the table has no column of that name. On a field whose @storage is not flattened the attribute is refused at load (ERR_FLATTENED_COLUMN_PREFIX_WITHOUT_FLATTENED_STORAGE): no other storage gives the members columns of their own.

Two member columns that come out the same are refused at load (ERR_COLUMN_NAME_COLLISION): two embeddings that share a prefix (two "" embeddings of one Address both give street), or a "" member beside a field of the same name. When two files both declare the same field (a merge, not an overlay) with different prefixes, "" included, the loader reports ERR_MERGE_CONFLICT.

What each port generates

TypeScript

The physical names the strategy above resolved are emitted once, into Author.names.ts (AuthorNames.fields.createdAt.column === "created_at"); every other generated file references that constant rather than respelling it. So the table binding below names no column directly — to read the resolved spelling, open the names artifact.

A @storage: flattened field binds one Drizzle column per member, keyed <field><Member> (for the address example above, addressStreet, addressCity, …) and named by the same core function meta migrate uses. The Row and Insert types, the Zod schemas, the filter and sort allowlists, the OpenAPI parameters and the form and grid generators all expose those flat keys; a client reads and writes addressStreet, not a nested address object. A flattened field declared on a TPH subtype keeps one column named for the field, as meta migrate creates it. A flattened field that carries an origin.* is derived: it owns no columns, so it exposes no flat keys in any of those surfaces, and a write cannot set them. A flattened field on a write-through entity is refused at meta gen (source-kinds.md).

// generated/acme/blog/Author.ts
import { AuthorNames } from "./Author.names";

export const author = pgTable(AuthorNames.sources.primary.table, {
  id:         bigint(AuthorNames.fields.id.column, { mode: "number" }).primaryKey().generatedByDefaultAsIdentity(),
  name:       varchar(AuthorNames.fields.name.column, { length: 200 }).notNull(),
  bio:        varchar(AuthorNames.fields.bio.column, { length: 2000 }),
  priceCents: bigint(AuthorNames.fields.priceCents.column, { mode: "number" }).notNull(),  // currency: minor units
  status:     varchar(AuthorNames.fields.status.column, { length: 32 }).notNull(),         // enum: CHECK constraint emitted by meta migrate
  createdAt:  timestamp(AuthorNames.fields.createdAt.column, { withTimezone: true }).notNull(),
});

export const AuthorSchema = z.object({
  id:         z.number(),
  name:       z.string().max(200),
  bio:        z.string().max(2000).optional(),
  priceCents: z.number().int(),
  status:     z.enum(["DRAFT", "PUBLISHED", "ARCHIVED"]),
  createdAt:  z.date(),
});

Java

// generated/acme/blog/Author.java
public class Author {
    private Long id;
    private String name;             // required, maxLength=200
    private String bio;
    private Long priceCents;          // currency: minor units (USD)
    private Status status;            // enum
    private Instant createdAt;
    // … getters / setters …
}

public enum Status { DRAFT, PUBLISHED, ARCHIVED }

Kotlin

// generated/acme/blog/Author.kt
data class Author(
    val id: Long,
    val name: String,
    val bio: String? = null,
    val priceCents: Long,         // currency: minor units (USD)
    val status: AuthorStatus,
    val createdAt: Instant,
)

@Serializable
enum class AuthorStatus { DRAFT, PUBLISHED, ARCHIVED }
// generated/acme/blog/AuthorTable.kt
object AuthorTable : Table("authors") {
    val id         = long("id").autoIncrement()
    val name       = varchar("name", 200)
    val bio        = varchar("bio", 2000).nullable()
    val priceCents = long("price_cents")
    val status     = enumerationByName("status", 64, AuthorStatus::class)
    val createdAt  = timestampWithTimeZone("created_at")
    override val primaryKey = PrimaryKey(id)
}

C#

// generated/Author.g.cs (usings and header elided)
namespace Acme.Blog;

[Table("authors")]
public class Author
{
    public enum AuthorStatus { DRAFT, PUBLISHED, ARCHIVED }
    [Key]
    [Column("id")]
    public long Id { get; set; }
    [Column("name")]
    [Required(AllowEmptyStrings = true)]
    [MaxLength(200)]
    [MinLength(1)]
    public string Name { get; set; } = default!;
    [Column("bio")]
    [MaxLength(2000)]
    public string? Bio { get; set; }
    [Column("price_cents")]
    public long PriceCents { get; set; }                  // currency: minor units
    [Column("status")]
    public AuthorStatus Status { get; set; }
    [Column("created_at")]
    public DateTimeOffset CreatedAt { get; set; }
}

EF Core enum-as-string is wired in AppDbContext:

modelBuilder.Entity<Author>()
    .Property(a => a.Status)
    .HasConversion<string>();

Python

# generated/Author.py (the Create and Patch input models are elided)
from __future__ import annotations

import datetime
from typing import Literal

from pydantic import BaseModel, Field


class Author(BaseModel):
    id: int | None = None
    name: str = Field(max_length=200)
    bio: str | None = Field(default=None, max_length=2000)
    priceCents: int                                   # currency: minor units
    status: Literal["DRAFT", "PUBLISHED", "ARCHIVED"]
    createdAt: datetime.datetime

Verified by

The following conformance fixtures gate this feature's behavior across ports:

Scalar field attributes

Currency

Enum

Embedded value objects (field.object + @storage)

Cross-port runner coverage: TS / Java / Kotlin / C# / Python all execute these via their respective conformance runners. See docs/CONFORMANCE.md for the per-port pass/skip ledger.

See also