Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
3aba65e
feat(http): add HTTP and SSE streams extension with delivery semantics
fffonion Aug 30, 2026
6cf5a79
feat(host-api): add named host-struct schema and compiler foundation
fffonion Sep 11, 2026
00179ce
fix(host): resolve named-struct bodies in VM resource walks
fffonion Sep 11, 2026
692c398
fix(compiler): reject callable arity mismatch in named expansion
fffonion Sep 11, 2026
f68e8cb
fix(host): install named-struct schemas through real extension regist…
fffonion Sep 11, 2026
6e5dcdd
feat(http): migrate HTTP and SSE host maps to named structs
fffonion Sep 11, 2026
49dcc2b
fix(http): align typed request body and SSE contracts
fffonion Sep 11, 2026
e079b99
feat(sqlite): migrate fixed-shape host maps to named structs
fffonion Sep 11, 2026
9c41909
fix(sqlite): align optional nulls, next_cursor, and open modes
fffonion Sep 11, 2026
cf28ae0
feat(jit): migrate get_config/set_config maps to named JitConfig
fffonion Sep 11, 2026
eb1ec42
fix(jit): wire typed catalog into the standard snapshot
fffonion Sep 11, 2026
e283264
fix(host): box large catalog registration errors after Named schemas
fffonion Sep 11, 2026
0ecaeff
fix(host): preserve named import schemas through codegen bind
fffonion Sep 11, 2026
3469589
fix(http): install SSE named structs on default compile paths
fffonion Sep 11, 2026
f4e11df
fix(parser): install only HTTP structs on catalog-free parse
fffonion Sep 11, 2026
1ef5d57
fix(tests): drop hardcoded agent temp path from catalog-free tests
fffonion Sep 11, 2026
091dd60
fix(sqlite): pass take-owned connections into sqlite::close
fffonion Sep 11, 2026
689bd9b
fix(vm): expand Named schemas for resource validation and merge installs
fffonion Sep 12, 2026
c407a77
fix(vm): transport guest Named structs without weakening host catalog
fffonion Sep 12, 2026
64c932a
fix(vm): isolate guest Named provenance and fail-closed VMBC v13
fffonion Sep 12, 2026
d2acd75
fix(vm): reject duplicate nostd named-struct names and update v13 docs
fffonion Sep 12, 2026
ed41b4e
fix(vm): skip Named host schemas in nostd and reject host type args
fffonion Sep 12, 2026
eb1b297
feat(http): type public request and SSE values
fffonion Sep 12, 2026
c26bba8
fix(http): stabilize typed header arrays
fffonion Sep 12, 2026
6d1a875
feat(sqlite): type values rows and transaction results
fffonion Sep 13, 2026
c023ca5
test(host): cover recursive public catalog traversal
fffonion Sep 13, 2026
dbc0564
fix(host): finish typed catalog integration address
fffonion Sep 13, 2026
a4ce265
fix(test): align SQLite catalog cfg on wasm32
fffonion Sep 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
532 changes: 522 additions & 10 deletions Cargo.lock

Large diffs are not rendered by default.

58 changes: 57 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,17 @@ name = "vm"
default = ["runtime", "cli", "cranelift-jit"]
runtime = []
async = ["runtime", "dep:tokio"]
http-client = [
"async",
"dep:futures-util",
"dep:http-body-util",
"dep:hyper",
"dep:hyper-util",
"dep:rustls",
"dep:tokio-rustls",
"dep:url",
"dep:webpki-roots",
]
sqlite = ["runtime", "dep:rusqlite"]
edge-abi = [
"dep:edge_abi",
Expand Down Expand Up @@ -79,7 +90,6 @@ cranelift-module = { version = "0.129.1", optional = true }
cranelift-native = { version = "0.129.1", optional = true }
pd-host-function = { path = "./pd-host-function", version = "0.1.0" }
rusqlite = { version = "0.32", default-features = false, features = ["bundled", "hooks", "limits"], optional = true }
tokio = { version = "1", features = ["rt-multi-thread", "net", "time", "sync", "fs", "io-util", "process"], optional = true }
edge_abi = { package = "pd-edge-abi", version = "0.1.1", default-features = false, optional = true }
futures-channel = "0.3"
paste = "1"
Expand All @@ -90,6 +100,17 @@ rt-format = "0.3.1"
self_cell = "1"
rustyline = { version = "14", optional = true }

[target.'cfg(not(target_family = "wasm"))'.dependencies]
http-body-util = { version = "0.1", optional = true }
hyper = { version = "1", default-features = false, features = ["client", "http1"], optional = true }
hyper-util = { version = "0.1", default-features = false, features = ["tokio"], optional = true }
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"], optional = true }
tokio-rustls = { version = "0.26", default-features = false, features = ["ring", "tls12"], optional = true }
url = { version = "2", optional = true }
futures-util = { version = "0.3", optional = true }
tokio = { version = "1", features = ["rt-multi-thread", "net", "time", "sync", "fs", "io-util", "process", "macros"], optional = true }
webpki-roots = { version = "1", optional = true }

[target.'cfg(windows)'.dependencies]
windows-sys = { version = "0.59", features = ["Win32_System_Diagnostics_Debug", "Win32_System_Memory", "Win32_System_ProcessStatus", "Win32_System_Threading"] }

Expand All @@ -99,6 +120,11 @@ libc = "0.2"
[dev-dependencies]
pd-host-schema = { path = "./crates/pd-host-schema", version = "0.1.0" }
syn = { version = "2", features = ["full"] }

[target.'cfg(not(target_family = "wasm"))'.dev-dependencies]
tokio = { version = "1", features = ["macros", "rt-multi-thread", "time", "sync"] }

[target.'cfg(target_family = "wasm")'.dev-dependencies]
tokio = { version = "1", features = ["macros", "rt", "time", "sync"] }

[[test]]
Expand Down Expand Up @@ -126,6 +152,36 @@ name = "host_context_arch_tests"
path = "tests/host_context_arch_tests.rs"
required-features = ["runtime"]

[[test]]
name = "host_named_struct_foundation_tests"
path = "tests/host_named_struct_foundation_tests.rs"
required-features = ["runtime"]

[[test]]
name = "http_host_tests"
path = "tests/vm/http_host_tests.rs"
required-features = ["runtime", "http-client"]

[[test]]
name = "http_sse_tests"
path = "tests/vm/http_sse_tests.rs"
required-features = ["runtime", "http-client"]

[[test]]
name = "http_named_struct_contract_tests"
path = "tests/http_named_struct_contract_tests.rs"
required-features = ["runtime", "http-client"]

[[test]]
name = "io_http_coexistence_tests"
path = "tests/vm/io_http_coexistence_tests.rs"
required-features = ["runtime", "http-client"]

[[test]]
name = "http_feature_gating_tests"
path = "tests/http_feature_gating_tests.rs"
required-features = ["runtime"]

[build-dependencies]
pd-host-schema = { path = "./crates/pd-host-schema", version = "0.1.0" }
syn = { version = "2", features = ["full"] }
43 changes: 40 additions & 3 deletions build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,17 @@ struct NamespaceDecl {
runtime_supported_on_wasm: bool,
}

/// HTTP/SSE is a native transport extension. Keep this predicate identical to
/// the `cfg` boundary used by the runtime and public exports: the Cargo
/// feature remains selectable on wasm, but it must not publish transport
/// sources or generated host/catalog entries there.
pub(crate) fn http_transport_enabled(http_client_feature: bool, target_family: &str) -> bool {
http_client_feature
&& !target_family
.split(',')
.any(|family| family.trim() == "wasm")
}

#[derive(Clone, Debug)]
struct Group<'a> {
key: String,
Expand Down Expand Up @@ -166,7 +177,8 @@ fn main() {
catalog.retain(|entry| !entry.source_name.starts_with("sqlite::"));
}

let host_sources = vec![
let target_family = env::var("CARGO_CFG_TARGET_FAMILY").expect("missing target family");
let mut host_sources = vec![
SourceSpec {
path: "src/builtins/runtime/host.rs".to_string(),
module: "host".to_string(),
Expand All @@ -178,6 +190,21 @@ fn main() {
category: SourceCategory::DefaultHost,
},
];
if http_transport_enabled(
env::var_os("CARGO_FEATURE_HTTP_CLIENT").is_some(),
&target_family,
) {
host_sources.push(SourceSpec {
path: "src/builtins/runtime/http/mod.rs".to_string(),
module: "http".to_string(),
category: SourceCategory::DefaultHost,
});
host_sources.push(SourceSpec {
path: "src/builtins/runtime/http/sse.rs".to_string(),
module: "http::sse".to_string(),
category: SourceCategory::DefaultHost,
});
}
let async_enabled = env::var_os("CARGO_FEATURE_ASYNC").is_some();
let target_arch = env::var("CARGO_CFG_TARGET_ARCH").expect("missing target architecture");
let builtin_sources = builtin_source_specs(&namespaces, async_enabled, &target_arch);
Expand Down Expand Up @@ -2216,11 +2243,21 @@ fn find_matching_paren(source: &str) -> usize {
#[cfg(test)]
mod tests {
use super::{
HostExecutionKind, NamespaceDecl, SourceCategory, builtin_source_specs, parse_source_file,
select_io_source_path,
HostExecutionKind, NamespaceDecl, SourceCategory, builtin_source_specs,
http_transport_enabled, parse_source_file, select_io_source_path,
};
use std::path::Path;

#[test]
fn http_transport_predicate_matches_source_and_catalog_boundary() {
assert!(http_transport_enabled(true, "unix"));
assert!(http_transport_enabled(true, "windows"));
assert!(!http_transport_enabled(true, "wasm"));
assert!(!http_transport_enabled(true, "wasm,unix"));
assert!(!http_transport_enabled(false, "unix"));
assert!(!http_transport_enabled(false, "wasm"));
}

fn io_namespace() -> NamespaceDecl {
NamespaceDecl {
namespace: "io".to_string(),
Expand Down
26 changes: 19 additions & 7 deletions crates/rustscript/tests/lsp_resource_types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ const ENTRY_URI: &str = "file:///tmp/rustscript-lsp-fixture/main.rss";
const CLEAN_SOURCE: &str = r#"use sqlite;
fn main() {
let db = sqlite::open({});
sqlite::query(&db, "SELECT 1", {}, {});
sqlite::query(&db, "SELECT 1", [], {});
}
"#;

Expand All @@ -258,7 +258,7 @@ fn main() {
const WRONG_TYPE_SOURCE: &str = r#"use sqlite;
fn main() {
let db = sqlite::open({});
sqlite::query("NOT_A_DB", "SELECT 1", {}, {});
sqlite::query("NOT_A_DB", "SELECT 1", [], {});
}
"#;

Expand Down Expand Up @@ -538,6 +538,18 @@ fn signature_help_shows_borrow_resource_and_value_params() {
label.contains("sql: string"),
"signature must show the value parameter: {label}"
);
assert!(
label.contains("params: array<SqliteValue>"),
"signature must show typed SQLite parameters: {label}"
);
assert!(
label.contains("limits: SqliteLimits"),
"signature must show typed SQLite limits: {label}"
);
assert!(
label.contains("-> SqliteQueryResult"),
"signature must show the typed SQLite query result: {label}"
);
}

// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -739,15 +751,15 @@ fn run() {
const MODULE_BAD_UTIL_SOURCE: &str = r#"use sqlite;
pub fn helper() {
let db = sqlite::open({});
sqlite::query("NOT_A_DB", "SELECT 1", {}, {});
sqlite::query("NOT_A_DB", "SELECT 1", [], {});
}
"#;

/// Imported module whose `helper` body is clean.
const MODULE_CLEAN_UTIL_SOURCE: &str = r#"use sqlite;
pub fn helper() {
let db = sqlite::open({});
sqlite::query(&db, "SELECT 1", {}, {});
sqlite::query(&db, "SELECT 1", [], {});
}
"#;

Expand Down Expand Up @@ -916,10 +928,10 @@ fn unicode_source_outbound_diagnostic_range_uses_utf16_columns() {
client.request(1, "initialize", serde_json::json!({}));
client.notify("initialized", serde_json::json!({}));
// Same-line multibyte prefix, then a wrong-type call on the *same line*.
// `let s = "你好😀"; sqlite::query("NOT_A_DB", "SELECT 1", {}, {});`
// `let s = "你好😀"; sqlite::query("NOT_A_DB", "SELECT 1", [], {});`
// The wrong-argument diagnostic must be reported with UTF-16 columns, so
// a client re-navigating from the range lands on the callee.
let source = "use sqlite;\nlet s = \"\u{4f60}\u{597d}\u{1f600}\"; sqlite::query(\"NOT_A_DB\", \"SELECT 1\", {}, {});\n";
let source = "use sqlite;\nlet s = \"\u{4f60}\u{597d}\u{1f600}\"; sqlite::query(\"NOT_A_DB\", \"SELECT 1\", [], {});\n";
open_doc(&mut client, ENTRY_URI, source);
let params = client.recv_notification("textDocument/publishDiagnostics");
let diagnostics = params["diagnostics"].as_array().expect("diagnostics array");
Expand Down Expand Up @@ -1474,7 +1486,7 @@ const DISK_UTIL_GOOD: &str = "pub fn helper() -> int { 41 }\n";
const BUFFER_UTIL_GOOD: &str = "pub fn helper() -> int { 999 }\n";

/// Unsaved buffer version of `util.rss` with a wrong-type call (diagnostic).
const BUFFER_UTIL_BAD: &str = "use sqlite;\npub fn helper() -> int {\n let db = sqlite::open({});\n sqlite::query(\"NOT_A_DB\", \"SELECT 1\", {}, {});\n 0\n}\n";
const BUFFER_UTIL_BAD: &str = "use sqlite;\npub fn helper() -> int {\n let db = sqlite::open({});\n sqlite::query(\"NOT_A_DB\", \"SELECT 1\", [], {});\n 0\n}\n";

/// A syntax error in `util.rss` (unterminated block) whose parser span should
/// be reported under the module URI.
Expand Down
24 changes: 20 additions & 4 deletions docs/callable-runtime.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Script call frames and callable values

RustScript bytecode format version 12 (VMBC v12) carries runtime script call frames, first-class callable values, the static builtin ID catalog, and the direct script-call opcode. Version 11 introduced frames, callable values, and the static catalog; version 12 adds `callscript` for statically resolved named calls.
RustScript bytecode format version 13 (VMBC v13) carries runtime script call frames, first-class callable values, the static builtin ID catalog, the direct script-call opcode, and an explicit guest named-struct declaration section. Version 11 introduced frames, callable values, and the static catalog; version 12 adds `callscript` for statically resolved named calls; version 13 frames guest struct declarations so a 4-byte zero trailer cannot be mistaken for an empty table.

## Bytecode contract

Expand All @@ -18,7 +18,7 @@ The three call opcodes differ in who owns the callee and what the frame must pro
- `callvalue` — the callee is a `Value::Callable` owned by the caller operand stack at the call site, and remains the caller's responsibility after the call. This path carries environments, closures, and any callable whose identity or capture state is runtime-valued.
- `callscript` — the callee is owned by program callable metadata (the prototype table). The frame contributes only `argc` arguments and no callable value, but unlike `call` the callee is a script function rather than a builtin, so the call enters a new script frame with its own local base.

VMBC v12 is the current format. It decodes the legacy v11 stream without host-schema metadata, while v12 carries full host schemas and callable metadata. Unknown versions and malformed resource schemas are rejected deterministically. PDRC v6 recordings and AOT artifacts (format 8, ABI 8) use their corresponding bumped versions and include callable metadata in cache identity.
VMBC v13 is the current format. It decodes the legacy v11 stream without host-schema metadata and the v12 stream without a named-struct section, while v13 carries full host schemas, callable metadata, and an explicit guest named-struct table. Unknown versions and malformed resource schemas are rejected deterministically. PDRC v6 recordings and AOT artifacts (format 8, ABI 8) use their corresponding bumped versions and include callable metadata in cache identity.

## Static builtin IDs

Expand All @@ -27,7 +27,7 @@ Every VM-visible builtin (ordinary, internal, and special-call) has one explicit
- **Immutable explicit IDs.** IDs never change once assigned. Adding or reordering catalog entries never renumbers existing entries; new builtins take the next free ID in their documented block (extension `0x0000..=0xFF8F` for future builtins and host imports, special-call `0xFF90..=0xFFA1`, ordinary `0xFFA2..=0xFFFF`). The reserved sentinel gap `0xFF90..=0xFF92` stays unassigned.
- **Build-time validation.** The build fails on duplicate IDs, duplicate source names, duplicate Rust variants, out-of-block IDs, class/gate inconsistencies, a discovered runtime callable without an explicit ID, or a catalog entry without a runtime callable.
- **Shared std/no-std IDs.** `pd-vm-nostd` dispatches on the same static indices through the checked-in generated mirror `pd-vm-nostd/src/generated_builtin_ids.rs`; the workspace test `static_builtin_ids_are_frozen` fails when the mirror drifts from the catalog.
- **Format breaks are permanent.** The static ID migration bumped VMBC to v11 (and the internal bytecode ABI to 11); the `callscript` opcode break bumped both to v12. Versions below the current format are rejected, never decoded.
- **Format breaks are permanent.** The static ID migration bumped VMBC to v11 (and the internal bytecode ABI to 11); the `callscript` opcode break bumped both to v12; the guest named-struct section bumped both to v13. Versions below the current encode format are not rewritten in place: v11/v12 remain readable only in their original framing.

## Runtime model

Expand Down Expand Up @@ -78,10 +78,26 @@ PDRC recordings preserve full execution-frame metadata. Callable environments us

Polling drives execution and provides backpressure: at most one event item is buffered between polls, and the VM does not produce items while the consumer is not polling. `stream::emit` validates only the configured per-item value bound (payload bytes and nesting depth); sequence assignment, receipts, persistence, and delivery policy belong to the embedding. At most one invocation is active per VM, `Invocation::cancel(reason)` cancels with a typed `OperationCancelReason`, dropping the handle retires the invocation synchronously for immediate VM reuse, and the low-level `Vm::run` pump is unchanged for custom drivers. VM reset uses the generic execution-scope close boundary; a pending close keeps the old scope installed and blocks reuse until `poll_reset_for_reuse` reports quiescence.

## Callable-driven HTTP streams

With the `http-client` feature, `http::client::request(request)` and `http::client::sse(request, on_event)` are script-facing host imports. SSE is a long-running ordinary host call. Its handler has the schema `fn(SseEvent) -> SseCallbackAction`. The host produces one typed `SseEvent`, the VM runs one child callback frame, and the returned action controls continuation before another event can arrive at the VM boundary.

The callback may yield or wait in an ordinary async host call. Existing frame machinery resumes the callback first and returns its final action to the suspended HTTP call. The network future does not own or enter the VM and is not polled while the callback is active, so at most one item remains unacknowledged and callback completion supplies backpressure.

`HttpRequest` and `SseRequest` use ordered `HttpRequestHeader` arrays and the
discriminated `HttpRequestBody` (`text` or `bytes`). Request arrays retain
repeated entries and their supplied order; HTTP does not promise
server-visible ordering across different names. Responses and SSE summaries
use `HttpResponseHeader` arrays in deterministic normalized-name order, with
stable same-name duplicate order and raw `HttpHeaderValue` (`text` or
`bytes`) variants. `SseEvent` exposes named fields for `open`, `event`, and
`end` items. See [HTTP client callable contract](http-client.md)
for field-level examples and lifecycle details.

## Optimized backends

Whole-program AOT and Trace JIT use the same builtin call path (static catalog IDs) for environment binding, native frame dispatch for `callvalue`, and prototype-direct native dispatch for `callscript`. Script-frame entry and return preserve frame-relative locals and typed continuations.

## Embedded runtime

`pd-vm-nostd` decodes the same VMBC v12 callable metadata and executes callable binding, `callvalue`, `callscript`, recursive frames, captures, and direct host targets using `core` plus `alloc`, dispatching on the identical static builtin IDs via its checked-in generated mirror.
`pd-vm-nostd` decodes the same VMBC v13 callable metadata and executes callable binding, `callvalue`, `callscript`, recursive frames, captures, and direct host targets using `core` plus `alloc`, dispatching on the identical static builtin IDs via its checked-in generated mirror.
Loading
Loading