Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 80 additions & 56 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,97 +1,81 @@
# memmap

Turn a YAML description of an MCU memory layout into a datasheet-style memory map image. The look
follows the STM32 reference manual figures, for your own layouts: bootloaders, app slots, config
sectors, shared RAM.

- One YAML file holds the memory (`memory`) and how to draw it (`view`), kept separate.
- Validation catches overlaps, overflow and flash sector misalignment, with YAML path, line and hint.
- Output is deterministic, and `describe` lists everything drawn, so agents and CI can verify it
as text.

Below find an example with four memory maps drawn by **memmap**.
Draw datasheet-style memory maps of MCU memory. You describe the layout in a YAML file, or
have `memmap` write that file from a linked firmware image. Either way you get the same config,
which `memmap` checks and renders to SVG, PNG or PDF.

- **Write it by hand.** One YAML file holds the memory (`memory`) and how to draw it (`view`),
kept separate. Start from one of the [examples](#examples).
- **Generate it from firmware.** `from-elf` reads a linked ELF and the linker's map file (GNU ld,
Keil armlink or IAR), and places every section at its real address and size in the chip's flash
and RAM regions.
- **Check before drawing.** Validation catches overlaps, overflow and flash sector misalignment,
with the YAML path, line and a hint for each.
- **Verify as text.** Output is deterministic, and `describe` lists everything drawn, so agents and
CI can check a drawing without looking at it.

Four memory maps drawn by `memmap`:

![](docs/examples.svg)

## Install

Each [GitHub release](https://github.com/iharhl/memmap/releases) has a wheel attached. To install
it with Python 3.10 or newer and no uv or checkout, using the version you want:
one with Python 3.10 or newer, without uv or a checkout:

```bash
pipx install https://github.com/iharhl/memmap/releases/download/v0.2.0/memmap-0.2.0-py3-none-any.whl
```

To work from a checkout:
PNG and PDF output needs the `raster` extra and the system cairo library. SVG always works.

```bash
uv sync
pipx install "memmap[raster] @ https://github.com/iharhl/memmap/releases/download/v0.2.0/memmap-0.2.0-py3-none-any.whl"
```

PNG and PDF output needs the `raster` extra and the system cairo library:

```bash
uv sync --extra raster
```
To work from a checkout instead, run `uv sync` (or `uv sync --extra raster`), and put `uv run` in
front of each `memmap` command below.

On macOS with Homebrew's cairo, the library is not on the default search path, so set this before
rendering PNG or PDF:
On macOS, Homebrew's cairo is not on the default library search path. Set this before rendering
PNG or PDF:

```bash
export DYLD_FALLBACK_LIBRARY_PATH=/opt/homebrew/lib
```

## Use
## Quick start

Start from the closest config in [examples/](examples/), listed under [Examples](#examples):
Choose a starting point. Both give you a `layout.yaml`.

**A layout you are planning.** Copy the closest example:

```bash
cp examples/custom_bootloader.yaml layout.yaml
```

```bash
uv run memmap check layout.yaml
```
**A firmware image you have built.** Pass the ELF and the map file the linker wrote next to it:

```bash
uv run memmap describe layout.yaml
memmap from-elf build/firmware.elf -o layout.yaml --map build/firmware.map
```

Then check the config, see what will be drawn, and render it:

```bash
uv run memmap render layout.yaml -o layout.svg
memmap check layout.yaml
```

| Command | Purpose |
| --- | --- |
| `memmap render CONFIG [-o OUT] [--format svg\|png\|pdf] [--json] [--strict] [--set key=value]` | Validate, then draw; `--strict` fails on warnings |
| `memmap check CONFIG [--json] [--strict]` | Validate only; `--strict` fails on warnings |
| `memmap describe CONFIG [--json]` | Resolved regions, blocks, labels and option origins |
| `memmap from-elf ELF [-o OUT] [--map FILE] [--region id=start:size[:name]] [--min-size N] [--force] [--json] [--strict]` | Write a config from a linked ELF's sections, then check it |
| `memmap schema` | JSON Schema of the config |
| `memmap guide` | Print the agent guide, for agents using an installed copy |

### From an ELF file

`from-elf` writes a config from the sections of a linked firmware image, each at its real
address and size. The ELF does not record how big flash and RAM are, so pass the map file the
linker writes, from GNU ld, Keil (armlink) or IAR:

```bash
uv run memmap from-elf build/firmware.elf -o layout.yaml --map build/firmware.map
memmap describe layout.yaml
```

An lld map, as Rust and the LLVM Embedded Toolchain write, lists no regions. Give them with
`--region` instead, copying `ORIGIN` and `LENGTH` from the linker script's `MEMORY` block:

```bash
uv run memmap from-elf build/firmware.elf -o layout.yaml --region flash=0x08000000:64K --region ram=0x20000000:20K
memmap render layout.yaml -o layout.svg
```

The output is an ordinary config: edit it, then `render` it. [docs/from-elf.md](docs/from-elf.md)
covers where each toolchain puts its map, what is read from it, what happens when it is not
enough, and how sections become blocks.
## Writing a config

A minimal config:
A minimal config, with a 64K flash split into a bootloader and an application:

```yaml
version: 1
Expand All @@ -115,16 +99,56 @@ view:
- { id: flash_col, root: flash, title: Flash }
```

Columns take an optional `title`, drawn above them. For a detail column created by `view.expand`,
set it on the expansion: `expand: [{ region: flash, title: Flash }]`.
Children without a `start` are placed one after another, and `auto` takes what is left. Columns
take an optional `title`, drawn above them. For a detail column created by `view.expand`, set it
on the expansion: `expand: [{ region: flash, title: Flash }]`.

`memmap schema` prints every key with a description and an example. Point your editor's YAML
extension at it for completion. [AGENTS.md](AGENTS.md) has recipes for common tweaks.

## Generating a config from firmware

`from-elf` writes a config from the sections of a linked firmware image: `.text`, `.data`, `.bss`
and so on, each at its real address and size. The ELF does not record how big flash and RAM
are, so memmap reads the regions from the map file the linker writes:

| Toolchain | Map file | Usually in |
| --- | --- | --- |
| GNU ld: gcc, Zephyr, ESP-IDF, Pico SDK, STM32CubeIDE | `firmware.map` (`-Wl,-Map=`) | next to the ELF |
| Arm armlink: Keil MDK | `project.map` | `Listings/` |
| IAR ILINK | `project.map` | `List/` |

An lld map, as Rust and the LLVM Embedded Toolchain write, lists no regions. Give them with
`--region` instead, copying `ORIGIN` and `LENGTH` from the linker script's `MEMORY` block:

```bash
memmap from-elf build/firmware.elf -o layout.yaml --region flash=0x08000000:64K --region ram=0x20000000:20K
```

`--region` also works together with `--map`, to replace or add a region. The result is an
ordinary config: rename or merge blocks, add `view` options, then `render` it.
[docs/from-elf.md](docs/from-elf.md) covers what is read from each map, what happens when it is
not enough, and how sections become blocks.

## Commands

| Command | Purpose |
| --- | --- |
| `memmap render CONFIG [-o OUT] [--format svg\|png\|pdf] [--json] [--strict] [--set key=value]` | Validate, then draw |
| `memmap check CONFIG [--json] [--strict]` | Validate only |
| `memmap describe CONFIG [--json]` | Resolved regions, blocks, labels and option origins |
| `memmap from-elf ELF [-o OUT] [--map FILE] [--region id=start:size[:name]] [--min-size N] [--force] [--json] [--strict]` | Write a config from a linked ELF, then check it |
| `memmap schema` | JSON Schema of the config |
| `memmap guide` | Print the agent guide, for agents using an installed copy |

`--strict` makes warnings fail the command. `--json` prints one result object on stdout, for
scripts and agents.

## Learn more

- [AGENTS.md](AGENTS.md): task guide, recipes and the full diagnostic list. `memmap guide` prints it.
- [docs/from-elf.md](docs/from-elf.md): drawing a linked firmware image, its map readers and limits.
- [examples/](examples/): see the table below.
- `memmap schema`: every key with a description and an example. Point your editor's YAML extension
at it for completion.
- [Examples](#examples): one config per feature set, below.

## Examples

Expand Down
46 changes: 37 additions & 9 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,53 @@ A release is a GitHub release with the wheel and sdist attached. Publishing the
[`release.yml`](.github/workflows/release.yml), which builds both files, checks them and uploads
them. Nothing is published to PyPI.

## 1. Bump the version
## Strategy

`main` is protected, so the bump goes through a pull request like any other change.
Releases are cut from `main`. There is no long-lived release branch:

1. A short-lived prep branch bumps the version and brings the docs and tooling up to date.
2. It is merged into `main` through a pull request, like any other change.
3. The release is published on `main`, which creates the tag `vX.Y.Z` on the merged commit.

A release branch, `release/X.Y`, is only made when an older version needs a fix while `main`
has moved on.

## 1. Prepare the release

`main` is protected, so the preparation goes through a pull request.

```bash
git switch main && git pull
```

```bash
git switch -c release/v0.2.0
git switch -c release-prep/v0.2.0
```

```bash
uv version --bump minor
```

`--bump` takes `patch` (0.1.0 to 0.1.1), `minor` (0.1.0 to 0.2.0) or `major` (0.1.0 to 1.0.0). It
updates `version` in `pyproject.toml` and in `uv.lock`. Commit both, push, open a pull request and
merge it once the checks pass.
updates `version` in `pyproject.toml` and in `uv.lock`, but not `__version__` in
`memmap/__init__.py`: set that one by hand. A test fails when the two differ.

On the same branch, bring up to date what a user of the new version reads:
- The install URLs in `README.md` and in [Install a release](#4-install-a-release) below.
- `README.md` and `docs/` for features added since the last release.
- `AGENTS.md`, which `memmap guide` ships, for new diagnostic codes and recipes.

Commit, push, open a pull request and merge it once the checks pass.

## 2. Publish the release

Publish right after the merge, so the tag lands on the prep commit and not on a later change.
From the terminal:

```bash
git switch main && git pull
```

```bash
gh release create v0.2.0 --target main --generate-notes
```
Expand All @@ -38,8 +61,9 @@ Or on GitHub, under Releases, choose "Draft a new release":
- "Generate release notes" lists the pull requests merged since the last release.
- "Publish release".

The tag is `v` followed by the version in `pyproject.toml`. A draft release does not start the
workflow until it is published. A pre-release does.
The tag is `v` followed by the version in `pyproject.toml`. Tag the commit on `main`, never one
on the prep branch: after a squash merge, the branch commits are not part of `main`. A draft
release does not start the workflow until it is published. A pre-release does.

## 3. What the workflow does

Expand All @@ -57,8 +81,12 @@ It takes about a minute. Follow it in the Actions tab.
pipx install https://github.com/iharhl/memmap/releases/download/v0.2.0/memmap-0.2.0-py3-none-any.whl
```

`pip install` with the same URL works too. Add `[raster]` after the file name for PNG and PDF
output, which also needs the system cairo library.
`pip install` with the same URL works too. For PNG and PDF output, which also needs the system
cairo library, name the `raster` extra:

```bash
pipx install "memmap[raster] @ https://github.com/iharhl/memmap/releases/download/v0.2.0/memmap-0.2.0-py3-none-any.whl"
```

## When something fails

Expand Down
2 changes: 1 addition & 1 deletion memmap/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""memmap: datasheet-style memory map images from a YAML description."""

__version__ = "0.1.0"
__version__ = "0.2.0"
# Config `version` values this release reads. Keep in step with schema.Version.
SCHEMA_VERSIONS = (1,)
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "memmap"
version = "0.1.0"
version = "0.2.0"
description = "Turn a YAML description of an MCU memory layout into a datasheet-style memory map image."
readme = "README.md"
requires-python = ">=3.10"
Expand Down
7 changes: 7 additions & 0 deletions tests/test_cli.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import json
import re
import shutil

import pytest
Expand Down Expand Up @@ -30,6 +31,12 @@ def test_version():
assert __version__ in result.stdout and "schema versions: 1" in result.stdout


def test_version_matches_pyproject():
# `uv version --bump` updates pyproject.toml but not memmap/__init__.py.
pyproject = (ROOT / "pyproject.toml").read_text()
assert re.search(r'^version = "(.+)"$', pyproject, re.MULTILINE).group(1) == __version__


def test_render_defaults_to_config_name_with_svg(tmp_path):
file = tmp_path / "layout.yaml"
shutil.copy(CUSTOM, file)
Expand Down
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading