Serio

Binary (buffer) serialization library for Roblox built on Flamework.

Heavily inspired by @rbxts/flamework-binary-serializer. It generates schemas for ser/des from just a type just like FBS.

Caution

Depends on rbxts-transformer-flamework!

Getting started

Schemas

Schemas in Serio are just types. The most basic of schemas is just a singular type, such as boolean, u8, etc. However schemas can also contain objects, lists, sets, maps, tuples, and several Roblox data types. As an example:

import type { u8 } from "@rbxts/serio";

interface MySchema {
  readonly epic: boolean;
  readonly foo: u8;
}

Serializers

Only one (global) function is exposed by Serio, createSerializer(). It's a macro you will pass your schema into to create a serializer. Serializers have two methods. serialize() and deserialize(). Serialized data contains a buffer (buf), and a list of blobs (blobs) (values which can not, or should not, be serialized. e.x. instances)

import createSerializer from "@rbxts/serio";

const mySerializer = createSerializer<MySchema>();
const data = mySerializer.serialize({ epic: true, foo: 69 });
const result = mySerializer.deserialize(data);
print(data.buf, data.blobs);
print(result.epic, result.foo);

Data Types

Serio can serialize many data types, including numeric types not natively supported by the buffer library.

Safety

Serio automatically asserts that inputs are within the bounds of their respective type. (e.x. u8 only allows numbers 0 to 255)

Extended Support

Serio supports data types that are not natively supported by the buffer library or Roblox. Currently the only examples of this are f24, f16 and f8, which are true floating point encodings (sign/exponent/mantissa) capable of representing values across their whole magnitude range, including NaN and ±Infinity.

Note: Floating point values with low bit counts like f8 and f16 can produce pretty inaccurate results. Use with caution.

import type { f8, f16, f24 } from "@rbxts/serio";

interface CoolTypes {
  readonly a: f8;
  readonly b: f16;
  readonly c: f24;
}

Bounded/Bitpacked Numbers

If you know a number's value always falls within a fixed range, IntWithin<Min, Max> and FloatWithin<Min, Max, Step> will pack it into only as many bits as that range actually needs, rather than a full byte/word. For example, a number that only ranges from 0 to 15 only needs 4 bits, not a whole u8.

IntWithin's bit width is derived automatically from Min/Max. FloatWithin linearly quantizes its value to the nearest multiple of Step across [Min, Max], then derives its bit width the same way (e.g. a range of 0-359 stepping by 1 needs 9 bits, since there are 360 representable values). It can't represent values outside [Min, Max], NaN, or ±Infinity, so reach for f8/f16/f24/f32/f64 instead when the magnitude isn't known ahead of time.

Whenever the resulting bit width isn't a multiple of 8, the type must be wrapped in Packed<> so its leftover bits can share bytes with other packed fields.

import type { IntWithin, FloatWithin, Packed } from "@rbxts/serio";

interface Player {
  // 0-100 needs only 7 bits, so this must be packed
  readonly health: IntWithin<0, 100>;
  // 0-359 degrees, quantized to whole degrees, needs 9 bits, also packed
  readonly facingAngle: FloatWithin<0, 359, 1>;
}

type PackedPlayer = Packed<Player>;

IntWithin/FloatWithin also compose with Vector, Transform, ScaleOffset, ScaleOffset2, and Bounds as axis types, since those are ultimately just per-axis number schemas. This lets a known-bounded position (e.g. a small arena) pack into far fewer bits than even a custom i16/u16 axis would.

import type { Vector, Transform, IntWithin, Packed } from "@rbxts/serio";

interface Arena {
  // a 64x64x64 arena only needs 6 bits per axis (18 bits total), instead of 3 whole i16s (48 bits)
  readonly position: Vector<IntWithin<0, 63>>;
  readonly orientation: Transform<IntWithin<0, 63>>;
}

type PackedArena = Packed<Arena>;

Customization

Serio encourages full customization over the size of serialized values.

import type { List, String, HashSet, HashMap, Tuple, Vector, ScaleOffset, ScaleOffset2, Bounds, i8, u8, i16, u16 } from "@rbxts/serio";

interface Example {
  // serialize the length of the array/string/tuple/set/map as a u8, allowing a maximum of 255 elements (which most collections are under anyways)
  string: String<u8>;
  list: List<string, u8>;
  set: HashSet<"a" | "b" | "c", u8>;
  map: HashMap<string, i8, u8>;
  tuple: Tuple<[boolean, u8, string], u8>;

  // serialize X and Z as i16s but Y as a u16, allowing a range of:
  // X: -32,768 - 32,767
  // Y: 0 - 65,535
  // Z: -32,768 - 32,767
  positiveYVector: Vector<i16, u16, i16>;
  velocity: Vector<i8>; // serialize X, Y, and Z as i8s

  // serialize positional X, Y, and Z as i16s
  cframe: Transform<i16>;

  // serialize Scale as an f32, and Offset as a u8
  udim: ScaleOffset<f32, u8>;
  // serialize X.Scale and Y.Scale as f32s, and X.Offset and Y.Offset u8s
  udim2: ScaleOffset2<f32, u8>;
  // serialize X.Scale as an f32, Y.Scale as a u8, X.Offset as a u8, and Y.Offset as an i8
  specificUDim2: ScaleOffset2<f32, u8, u8, i8>;

  // serialize Rect.Min/Max X and Y as u16s instead of the default f32s
  bounds: Bounds<u16>;
}

const serializer = createSerializer<Example>();

Packing

Serio can bitpack your data for you using the Packed<T> datatype. It bitpacks every type under T recursively.

You may know a boolean can be represented by just one bit, but to insert a boolean into a buffer you need to serialize it as an entire byte (8 bits). This is where bitpacking comes in. We can keep track of a list of bits to later combine together into a single byte (for 8 or less booleans), rather than one byte for each of your booleans.

This doesn't just affect booleans though, it also affects:

• Optional values
  • Boolean for whether the value exists
• UDim2s
  • Optimization for UDim2 special cases
  • Boolean for whether the UDim2 was optimized
• Vector3s
  • Optimization for vector special cases
  • Boolean for whether the vector was optimized
• CFrames
  • Optimization for two vector special cases (0,0,0 and 1,1,1) and axis aligned rotation special cases
  • Boolean for whether the position was optimized
  • Boolean for whether the rotation was optimized
• Enums
  • Instead of a full byte, only the minimum number of bits needed to index the enum's items are used (e.g. a 5-item enum only costs 3 bits instead of 8)
• IntWithin<Min, Max> / FloatWithin<Min, Max, Step>
  • Only required when their bit width isn't already a multiple of 8 (see Bounded/Bitpacked Numbers)

Supported Types

• All primitives
• Tuples
• Objects
• Array
• Map<K, V>
• Set
• CFrame
• Vector3
• UDim
• UDim2
• Rect
• Color3
• ColorSequence
• NumberSequence
• Literal unions
• Tagged unions
• Complex mixed unions
• Enums

Optimization

CFrames use 6 bytes less than in FBS by default. In FBS CFrames are serialized as six f32s, 24 bytes. The default for Serio is three f32s for position, two u16s for rotation X/Z, and one i16 for rotation Y. And of course you can make this less by not using the default type for position X/Y/Z (Transform<X, Y, Z>).