diff --git a/AGENTS.md b/AGENTS.md index 538d834..cc02a2c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -94,11 +94,21 @@ itself, such as an unknown flag, is reported as plain text on stderr. - `labels`: every string drawn, with `role` (address, name, size, note, band, title, column_title, footer, legend), `column` and an approximate pixel `box`. - `columns`: placement, plus every option with its value and its `origin`. +- `connectors`: one per zoom, with its `region`, `from` and `to` columns and the points of its + two fan `lines`. Output is deterministic: the same input gives a byte-identical SVG, so diffing is reliable. W105 and W106 catch text that does not fit, using a built-in width table, so they are close but -not exact. Connector lines are not checked against text. +not exact. Connector lines are not checked against text, but they are kept out of columns: +a zoom whose detail column is two or more lanes to the right of its source moves that column up +or down so its fan lines bend around the columns between, and W107 reports when it cannot. One +to the left is not moved, and gets W107 when its lines cross a column. Detail columns stacked in +one lane go in the order of the blocks they zoom into, highest address on top. A column that +zooms into columns that stack up taller than it is gets stretched to their height, up to 5 +times its own, so its fan lines do not start crammed together. A stack of zooms that would run +past the bottom of the drawing moves up, unless a column in it has an `offset`, and columns +stacked in one lane keep room between them for both end addresses. ## Diagnostics Every diagnostic has a stable code, a YAML path, a line and usually a `hint`. In JSON it also has @@ -131,6 +141,7 @@ Every diagnostic has a stable code, a YAML path, a line and usually a `hint`. In | W104 | Sector list does not add up to the region size | | W105 | Block name was split inside a word to fit the column | | W106 | Text overlaps other text, or is wider than its block | +| W107 | Fan lines run through a column between a zoom's source and its detail column | | W110 | `from-elf`: section outside every memory region, placed at the top level | | W111 | `from-elf`: section overlaps another one, left out | | W112 | `from-elf`: map region left out (no size, past the address space, overlap, too small) | diff --git a/docs/from-elf.md b/docs/from-elf.md index f2d2e4f..f886208 100644 --- a/docs/from-elf.md +++ b/docs/from-elf.md @@ -154,11 +154,21 @@ Other things a map does not tell memmap: ## From sections to blocks - **Which sections.** Every allocated section with a size. The thread-local `.tbss` is skipped, - as it only holds a template. + as it only holds a template. A section the linker script reserves without allocating it, such + as the RAM vector table, heap and stacks the Pico SDK declares as `(COPY)`, is drawn too when + it lies inside a memory region and overlaps no allocated section, with the note `not + allocated in the ELF, as a (COPY) section`. One at address 0 is left out, as it looks like a + debug section there. Names lose the blanks IAR leaves around them. - **Load images.** Initialised data is drawn twice: in RAM where it runs, and in flash where startup code copies it from. Each loadable segment stored away from where it runs is one flash block: `.data (load)` with a note such as `copied to RAM at 0x2000 0000`, or `RAM init data` when it holds several sections, as Zephyr's do, and `code and data` when it includes code. +- **armlink (Keil).** An execution region's RW and ZI data are two sections with one name, drawn + as `RW_IRAM1 (RW)` and `RW_IRAM1 (ZI)`. armlink stores the RW data inside the segment of the + code before it, so its flash copy, `RW_IRAM1 (load)`, is found from the segment rather than + from a segment of its own. When armlink compresses RW data, as it does by default, the section + header gives the compressed size and the ZI data starts further on than alignment explains: + the RW data is then drawn up to the ZI data, and its flash copy is noted as compressed. - **Free space.** Gaps of at least `--min-size` (default 64 bytes) become Free blocks with their size, rounded down and marked `≈` when not exact. Smaller gaps are drawn as a split edge. - **Folding.** In a region of more than 8 blocks, runs of sections smaller than `--min-size` @@ -208,7 +218,7 @@ maps through the importer. | --- | --- | --- | | Synthetic ELFs | `tests/test_from_elf.py`, built by `tests/elf_builder.py` | One rule per test: folding, free gaps, load images, several memory banks, ids, 64-bit, big-endian, converted images, bad files, the CLI | | Real builds | `tests/fixtures/firmware*.elf`, from `tests/fixtures/elf_src/` | The reader on files binutils wrote: one firmware as 32- and 64-bit, little- and big-endian, and its GNU ld map. `build.sh` rebuilds them with `arm-none-eabi-gcc` and GNU `objcopy` | -| Real firmware | `tests/test_elf_snapshots.py`, `tests/fixtures/elf_snapshots/` | Eight images from different toolchains, OSes and architectures, against golden configs and drawings | +| Real firmware | `tests/test_elf_snapshots.py`, `tests/fixtures/elf_snapshots/` | Ten images from different toolchains, OSes and architectures, against golden configs and drawings | | Linker maps | `tests/test_linker.py`, `tests/fixtures/linker_snapshots/` | The map readers on 13 real maps, against `tests/golden/linker/regions.json`, and every case of [the limits](#limits-when-the-map-is-not-enough): lld maps, maps without `MEMORY` or sizes, overlapping regions, sections across a region's edge, files that are not maps | ### Snapshots instead of ELF files @@ -228,9 +238,10 @@ clean. The real builds and the synthetic tests cover the reader. ### The images -All come from [Renode](https://github.com/renode/renode), whose board scripts -(`scripts/single-node/*.resc`) load prebuilt firmware from `dl.antmicro.com`. Locally they live -in `build/elf/`, which git ignores. +Eight come from [Renode](https://github.com/renode/renode), whose board scripts +(`scripts/single-node/*.resc`) load prebuilt firmware from `dl.antmicro.com`. The other two are +builds committed to public repositories, pinned to a commit: a Keil one and a Pico SDK one, +whose layouts GNU ld builds do not show. Locally they live in `build/elf/`, which git ignores. | Snapshot | Firmware | Regions | | --- | --- | --- | @@ -242,9 +253,12 @@ in `build/elf/`, which git ignores. | `stm32f746_dartino_lines` | Dartino (Dart VM) demo, STM32F746 | flash 1M at 0x08000000, SRAM 320K at 0x20000000 | | `stm32f4discovery` | STM32F4 Discovery demo, STM32F407 | flash 1M at 0x08000000, SRAM 128K at 0x20000000 | | `apollo4_hello_uart` | AmbiqSuite hello world, Ambiq Apollo4 (Cortex-M4F) | MRAM 1952K at 0x18000, TCM 384K at 0x10000000, shared SRAM 1M at 0x10060000 | +| `stm32l4_keil_rfe_master` | [iCub](https://github.com/robotology/icub-firmware) RFE board, STM32L4, Keil armlink | flash 512K at 0x08000000, SRAM 160K at 0x20000000 | +| `rp2040_pico_rom` | [picoROM](https://github.com/nickbild/picoROM), RP2040, Pico SDK | flash 2M at 0x10000000, RAM 256K at 0x20000000, SCRATCH_X and SCRATCH_Y 4K each | Only the ELF files are published, so the regions are `--region` specs: the chips' real -flash and RAM, from datasheets and Renode's platform files (`platforms/cpus/*.repl`). Where the +flash and RAM, from datasheets and Renode's platform files (`platforms/cpus/*.repl`), and for +the two repository builds the regions their committed linker maps give. Where the firmware starts above the bottom of its memory, the region starts there too: the FE310 shell runs from the user part of SPI flash, and the Apollo4 application starts 96 KB into its 2 MB MRAM. The Tock kernel is linked by lld, so its regions would have to be given this way anyway. @@ -343,3 +357,17 @@ A large `.data` copied from flash, and the stack and heap reserved as a section. A conventional STM32 layout. ![STM32F4 Discovery demo](../tests/golden/elf/stm32f4discovery.svg) + +### STM32L4, Keil armlink + +RW and ZI data under one name, drawn as `(RW)` and `(ZI)`, and the RW data stored compressed +after the code: 212 bytes in flash that unpack to 460 in SRAM. + +![STM32L4 Keil RFE](../tests/golden/elf/stm32l4_keil_rfe_master.svg) + +### RP2040, Pico SDK + +The RAM vector table, heap and core 0 stack are (COPY) sections, drawn with a note as they are +not allocated in the ELF. + +![RP2040 picoROM](../tests/golden/elf/rp2040_pico_rom.svg) diff --git a/memmap/from_elf/elf.py b/memmap/from_elf/elf.py index 82e8a6f..e888eb1 100644 --- a/memmap/from_elf/elf.py +++ b/memmap/from_elf/elf.py @@ -10,8 +10,11 @@ from dataclasses import dataclass # Section header types and flags used by the importer. +SHT_PROGBITS = 1 SHT_NOBITS = 8 +SHF_WRITE = 0x1 SHF_ALLOC = 0x2 +SHF_EXECINSTR = 0x4 SHF_TLS = 0x400 PT_LOAD = 1 @@ -42,6 +45,7 @@ class Section: address: int offset: int size: int + align: int = 1 @property def allocated(self) -> bool: @@ -138,7 +142,7 @@ def read_elf(data: bytes) -> Elf: names = data[table[4] : table[4] + table[5]] sections = [ - Section(_name(names, raw[0]), raw[1], raw[2], raw[3], raw[4], raw[5]) + Section(_name(names, raw[0]), raw[1], raw[2], raw[3], raw[4], raw[5], max(raw[8], 1)) for raw in raw_sections[1:] # section 0 is always empty ] # An unlinked object file has every section at address 0. One with real addresses, as diff --git a/memmap/from_elf/importer.py b/memmap/from_elf/importer.py index 1475618..561c559 100644 --- a/memmap/from_elf/importer.py +++ b/memmap/from_elf/importer.py @@ -12,14 +12,24 @@ from ..diagnostics import Diagnostic from ..units import diagnostic_address, format_address from .blocks import Bucket, Item -from .elf import PT_LOAD, SHF_TLS, Elf, Section +from .elf import ( + PT_LOAD, + SHF_EXECINSTR, + SHF_TLS, + SHF_WRITE, + SHT_NOBITS, + SHT_PROGBITS, + Elf, + Section, + Segment, +) from .regions import ImportProblem, LeftOut, MemorySpec, check_memories, unique_id from .sizes import address_literal, floor_size from .writer import write_yaml _NOTE_NAMES = 6 # section names a note lists before "+N more" FOLD_OVER = 8 # a region with more blocks than this folds its runs of small sections -_SHF_EXECINSTR = 0x4 +UNALLOCATED_NOTE = "not allocated in the ELF, as a (COPY) section" @dataclass @@ -66,6 +76,52 @@ def _name_list(names: list[str]) -> str: return f"{shown}, +{extra} more" if extra > 0 else shown +def _align_up(value: int, align: int) -> int: + """`value` rounded up to a multiple of `align`.""" + return -(-value // align) * align + + +def _rw_zi_pairs(sections: list[Section]) -> list[tuple[Section, Section]]: + """Pairs of a writable PROGBITS and a NOBITS section with the same name, the RW and ZI + parts of one armlink execution region, such as two sections called RW_IRAM1.""" + by_name: dict[str, list[Section]] = {} + for section in sections: + by_name.setdefault(section.name.strip(), []).append(section) + pairs = [] + for group in by_name.values(): + data = [s for s in group if not s.nobits] + zero = [s for s in group if s.nobits] + if len(data) == 1 and len(zero) == 1 and data[0].flags & SHF_WRITE: + pairs.append((data[0], zero[0])) + return pairs + + +def _display_names(sections: list[Section]) -> dict[Section, str]: + """The name each section is drawn with: its own without surrounding blanks, which IAR + leaves in, and with (RW) or (ZI) when an armlink region has both under one name.""" + names = {section: section.name.strip() for section in sections} + for data, zero in _rw_zi_pairs(sections): + names[data] += " (RW)" + names[zero] += " (ZI)" + return names + + +def _unpacked_sizes(sections: list[Section]) -> dict[Section, int]: + """RAM sizes of RW data that armlink stored compressed. Its section header then gives the + compressed size, and the ZI data of the same region starts further on than alignment + explains: the RW data runs up to it.""" + sizes = {} + for data, zero in _rw_zi_pairs(sections): + packed_end = data.address + data.size + if zero.address <= _align_up(packed_end, zero.align): + continue + others = [other for other in sections if other is not data] + if any(packed_end <= other.address < zero.address for other in others): + continue + sizes[data] = zero.address - data.address + return sizes + + class _Importer: """Places the sections of one ELF into buckets, collecting diagnostics as it goes.""" @@ -113,21 +169,51 @@ def place(self) -> None: """Put every wanted section in a bucket where it runs, and each load image where it is stored.""" wanted = [section for section in self.elf.sections if _wanted(section)] + names = _display_names(wanted) + unpacked = _unpacked_sizes(wanted) for section in wanted: - item = Item(self.ids.make(section.name), section.name, section.address, section.size) + name = names[section] + size = unpacked.get(section, section.size) + item = Item(self.ids.make(name), name, section.address, size) + self.bucket_for(item).items.append(item) + for item in self.unallocated(wanted): self.bucket_for(item).items.append(item) - for image in self.load_images(wanted): + for image in self.load_images(wanted, set(unpacked)): self.bucket_for(image).items.append(image) - def load_images(self, sections: list[Section]) -> list[Item]: + def unallocated(self, wanted: list[Section]) -> list[Item]: + """Sections the linker script reserved space for without allocating them, such as the + heap and stacks the Pico SDK declares as (COPY): each with an address and a size that + lie inside one memory region and overlap no allocated section. Debug and other + non-memory sections sit at address 0, so a (COPY) section there is not told apart from + them and is left out.""" + items = [] + for section in self.elf.sections: + if section.allocated or not section.size or not section.address: + continue + if section.kind not in (SHT_PROGBITS, SHT_NOBITS): + continue + end = section.address + section.size - 1 + memory = self.memory_at(section.address) + if memory is None or end > memory.end: + continue + if any(w.address <= end and section.address < w.address + w.size for w in wanted): + continue + name = section.name.strip() + items.append( + Item(self.ids.make(name), name, section.address, section.size, UNALLOCATED_NOTE) + ) + return items + + def load_images(self, sections: list[Section], compressed: set[Section]) -> list[Item]: """One block per loadable segment that is stored somewhere other than where it runs, - such as the initial values of .data. Startup code copies each in one piece.""" + such as the initial values of .data. Startup code copies each in one piece. armlink + instead puts RW data in the segment of the code before it, so a segment loaded where + it runs can still hold sections stored apart, which get a block each.""" images = [] for segment in self.elf.segments: if segment.kind != PT_LOAD or segment.file_size == 0: continue - if segment.physical == segment.virtual: - continue members = [ section for section in sections @@ -136,12 +222,13 @@ def load_images(self, sections: list[Section]) -> list[Item]: ] if not members: continue + if segment.physical == segment.virtual: + images += self.stored_apart(segment, members, compressed) + continue target = format_address(segment.virtual, self.elf.bits // 4) - names = [section.name for section in members] + names = [section.name.strip() for section in members] if len(members) == 1: # with several banks, a bare address leaves the reader to look - bank = self.memory_at(segment.virtual) - where = f"{bank.name} at {target}" if bank else target - name, note = f"{names[0]} (load)", f"copied to {where}" + name, note = f"{names[0]} (load)", f"copied to {self.bank_at(segment.virtual)}" else: name, note = self.image_name(segment.virtual, members), f"copied to {target}: " note += _name_list(names) @@ -150,6 +237,30 @@ def load_images(self, sections: list[Section]) -> list[Item]: ) return images + def stored_apart( + self, segment: Segment, members: list[Section], compressed: set[Section] + ) -> list[Item]: + """The sections of a segment loaded where it runs whose bytes it loads somewhere other + than their own address, as armlink does with RW data. `compressed` sections are stored + packed, so their copy is smaller than the RAM they unpack to.""" + images = [] + for section in members: + stored = segment.physical + section.offset - segment.offset + if stored == section.address: + continue + size = min(section.size, segment.offset + segment.file_size - section.offset) + name = f"{section.name.strip()} (load)" + where = self.bank_at(section.address) + verb = "compressed, unpacked to" if section in compressed else "copied to" + images.append(Item(self.ids.make(name), name, stored, size, note=f"{verb} {where}")) + return images + + def bank_at(self, address: int) -> str: + """Where a load image is copied to: the memory region and address, or the address.""" + target = format_address(address, self.elf.bits // 4) + bank = self.memory_at(address) + return f"{bank.name} at {target}" if bank else target + def memory_at(self, address: int) -> MemorySpec | None: """The memory region that holds `address`, if any.""" return next((m for m in self.memories if m.start <= address <= m.end), None) @@ -157,7 +268,7 @@ def memory_at(self, address: int) -> MemorySpec | None: def image_name(self, address: int, members: list[Section]) -> str: """A name for a load image of several sections, after where it is copied to and what it holds: RAM init data, or DTIM code and data when it includes code.""" - what = "code and data" if any(s.flags & _SHF_EXECINSTR for s in members) else "init data" + what = "code and data" if any(s.flags & SHF_EXECINSTR for s in members) else "init data" target = self.memory_at(address) return f"{target.name} {what}" if target else what.capitalize() diff --git a/memmap/layout/__init__.py b/memmap/layout/__init__.py index cbd0e2b..d5bbbed 100644 --- a/memmap/layout/__init__.py +++ b/memmap/layout/__init__.py @@ -23,10 +23,12 @@ place_horizontally, place_vertically, stretch_bands, + stretch_sources, ) from .connectors import connector_lines from .labels import place_labels from .primitives import INK, PAPER, Drawing, Line, Primitive, Rect, Text +from .routing import column_box, lanes_between from .text import ADDRESS_FONT_SIZE, BASELINE_SHIFT, line_height, text_width MARGIN = 10 # px, between the canvas edge and the frame @@ -274,6 +276,24 @@ def _text_collisions(drawing: Drawing) -> list[Diagnostic]: ] +def _through_columns( + region: str, path: Path, target: ColumnLayout, between: list[ColumnLayout] +) -> Diagnostic: + """W107 for fan lines that run through the columns between a zoom's source and target.""" + lanes = sorted({column.lane for column in between}) + rightward = target.lane > lanes[0] + nearest = lanes[0] if rightward else lanes[-1] # the lane between closest to the source + columns = "column" if len(between) == 1 else "columns" + return Diagnostic( + "W107", + f"the fan lines of {region} run through {columns} " + + ", ".join(repr(column.id) for column in between), + path, + f"put {target.id!r} in lane {nearest}, or move it with `offset` or `anchor`", + region, + ) + + def layout_figure( config: Config, tree: Tree, plan: ViewPlan ) -> tuple[Drawing, list[Diagnostic]]: @@ -290,6 +310,7 @@ def layout_figure( assign_lanes(plan, by_id) place_horizontally(layouts) + stretch_sources(plan, by_id) height = content_height(layouts) header = header_height(layouts) for layout in layouts: @@ -344,10 +365,20 @@ def layout_figure( for link in plan.links: source = by_id[link.source.id] + target = by_id[link.target.id] pieces = [block for block in source.blocks if block.node is link.region] - primitives += connector_lines(pieces, by_id[link.target.id]) + between = lanes_between(layouts, source.lane, target.lane) + lines, clear = connector_lines(pieces, target, [column_box(other) for other in between]) + primitives += lines + if not clear: + out.append(_through_columns(link.region.id, link.region.path, target, between)) drawing.links.append( - {"region": link.region.id, "from": link.source.id, "to": link.target.id} + { + "region": link.region.id, + "from": link.source.id, + "to": link.target.id, + "lines": [[(round(x, 2), round(y, 2)) for x, y in line.points] for line in lines], + } ) for layout in layouts: diff --git a/memmap/layout/columns.py b/memmap/layout/columns.py index 849de5c..e1e403d 100644 --- a/memmap/layout/columns.py +++ b/memmap/layout/columns.py @@ -11,6 +11,7 @@ from ..view import ColumnView, Link, ViewPlan from .heights import scaled_height, text_height from .labels import AddressLabel, build_labels +from .routing import clear_top, column_box, lanes_between, make_fan from .text import ADDRESS_FONT_SIZE, BOLD_WIDTH_FACTOR, line_height, text_width, wrap TEXT_INSET = 8 # px, taken off the column width when wrapping text inside a block @@ -18,7 +19,8 @@ NOTE_FONT_FACTOR = 0.85 # note size, relative to style.font_size NOTE_WIDTH_EM = 20 # a note wraps after this many ems of its own font, 187 px at the default NOTE_RULE_GAP = 6 # px, between the rule beside a multi-line note and its text -LANE_STACK_GAP = 24 # px, between columns stacked in one lane +LANE_STACK_GAP = 36 # px, between columns stacked in one lane, room for both end addresses +MAX_STRETCH = 5.0 # a column stretched to the height of its zooms grows at most this many times MIN_FAN_ROOM = 16 # px of fan line kept clear of the next lane's left gutter COLUMN_TITLE_GAP = 6 # px, between a column title and the column COLUMN_GAP = 160 # px, least distance between lanes, where the fan lines run @@ -55,8 +57,9 @@ def reserved(self) -> bool: @dataclass class ColumnLayout: """A column as layout works on it. `build_column` sets the blocks and measurements, then - `assign_lanes` and the place_ functions set `lane`, `x` and `y`. The gutters are the room for - labels and notes either side of the blocks.""" + `assign_lanes` and the place_ functions set `lane`, `x` and `y`, and `stretch_sources` can + make a column that zooms out taller. The gutters are the room for labels and notes either + side of the blocks.""" view: ColumnView options: Options @@ -348,29 +351,53 @@ def place_horizontally(layouts: list[ColumnLayout]) -> None: x += max(layout.width + layout.right_gutter for layout in members) +def stretch_sources(plan: ViewPlan, layouts: dict[str, ColumnLayout]) -> None: + """Make a column that zooms into others taller when the columns its zooms land in stack up + much taller than it is, so its fan lines spread out instead of starting crammed together + from a short column. It grows to the height of the tallest of those lanes, at most + MAX_STRETCH times its own height. Columns with bands are left alone, as `stretch_bands` + fills the height with them. Sources are stretched from the right, so in a chain of zooms + each one is measured against the columns it zooms into once they have grown.""" + target_lanes: dict[str, set[int]] = {} + for link in plan.links: + target_lanes.setdefault(link.source.id, set()).add(layouts[link.target.id].lane) + columns = list(layouts.values()) + for column_id in sorted(target_lanes, key=lambda column: -layouts[column].lane): + layout = layouts[column_id] + if layout.band_groups or layout.height <= 0: + continue + tallest = max(lane_height(columns, lane) for lane in target_lanes[column_id]) + factor = min(MAX_STRETCH, tallest / layout.height) + if factor <= 1: + continue + for block in layout.blocks: + block.h *= factor + layout.height *= factor + + def header_height(layouts: list[ColumnLayout]) -> float: """The row above the blocks that holds column titles, so untitled columns line up too.""" return max(layout.title_height for layout in layouts) +def lane_height(layouts: list[ColumnLayout], lane: int) -> float: + """The height the stacked columns of one lane take below the header row: their blocks, the + gaps between them and every title but the one that sits in the header row.""" + stacked = [ + layout for layout in layouts if layout.lane == lane and layout.view.spec.anchor != "free" + ] + if not stacked: + return 0 + titles = [layout.title_height for layout in stacked] + blocks = sum(layout.height for layout in stacked) + return blocks + sum(titles) - min(titles) + LANE_STACK_GAP * (len(stacked) - 1) + + def content_height(layouts: list[ColumnLayout]) -> float: """The tallest lane including the header row, counting columns in one lane as stacked. The first column's title sits in the header row, every other one takes room of its own.""" - header = header_height(layouts) - height = header - for lane in {layout.lane for layout in layouts}: - stacked = [ - layout - for layout in layouts - if layout.lane == lane and layout.view.spec.anchor != "free" - ] - if not stacked: - continue - titles = [layout.title_height for layout in stacked] - blocks = sum(layout.height for layout in stacked) - gaps = LANE_STACK_GAP * (len(stacked) - 1) - height = max(height, header + blocks + sum(titles) - min(titles) + gaps) - return height + lanes = {layout.lane for layout in layouts} + return header_height(layouts) + max(lane_height(layouts, lane) for lane in lanes) def _desired_y( @@ -392,6 +419,21 @@ def _desired_y( return y + spec.offset +def _attachment( + layout: ColumnLayout, source: tuple[float, float] | None, y: float, height: float +) -> float: + """The height a stacked column hangs from, which orders the columns of a lane: the middle + of its source block for `source`, the drawing's bottom for `bottom`, else its own top. A + tall column clamped to the top of the drawing still sorts by where it comes from, so the + fan lines of a lane do not cross.""" + anchor = layout.view.spec.anchor + if anchor == "source" and source is not None: + return (source[0] + source[1]) / 2 + if anchor == "bottom": + return height + return y + + def set_y(layout: ColumnLayout, y: float) -> None: """Move a column's top to `y` and stack its blocks from there, last block on top.""" layout.y = y @@ -407,7 +449,9 @@ def place_vertically( plan: ViewPlan, layouts: dict[str, ColumnLayout], height: float, header: float ) -> None: """Set each column's y, lane by lane from the left. The columns of a lane stack without - overlapping, apart from `free` ones.""" + overlapping, apart from `free` ones. A detail column two or more lanes from its source + moves up or down until its fan lines can go around the columns in between, without + moving above the columns already stacked in its lane.""" first_link: dict[str, Link] = {} for link in plan.links: first_link.setdefault(link.target.id, link) @@ -419,23 +463,71 @@ def place_vertically( layout = layouts[column.id] incoming = first_link.get(column.id) source = None + fan = None if incoming is not None and incoming.source.id in placed: - pieces = [ - b for b in layouts[incoming.source.id].blocks if b.node is incoming.region - ] + origin = layouts[incoming.source.id] + pieces = [b for b in origin.blocks if b.node is incoming.region] if pieces: source = ( min(block.y for block in pieces), max(block.y + block.h for block in pieces), ) - wanted.append((_desired_y(layout, source, height, header), layout)) + fan = (origin, pieces[0], source) + y = _desired_y(layout, source, height, header) + wanted.append((_attachment(layout, source, y, height), y, layout, fan)) floor = None - for y, layout in sorted(wanted, key=lambda item: item[0]): + stacked: list[ColumnLayout] = [] + detoured = False + for _, y, layout, fan in sorted(wanted, key=lambda item: item[0]): if layout.view.spec.anchor == "free": set_y(layout, y) else: - if floor is not None: - y = max(y, floor + layout.title_height) + highest = header if floor is None else floor + layout.title_height + y = max(y, highest) + if fan is not None: + around = _around_lanes_between(layout, fan, y, layouts, placed, highest) + detoured = detoured or around != y + y = around set_y(layout, y) floor = y + layout.height + LANE_STACK_GAP + stacked.append(layout) placed.add(layout.id) + if not detoured and not any(layout.view.spec.offset for layout in stacked): + _lift_overflow(stacked, height, header) + + +def _lift_overflow(stacked: list[ColumnLayout], height: float, header: float) -> None: + """Move the stacked columns of a lane up together when they run past the bottom of the + drawing, as when tall zooms hang from blocks low in their source: as far as the overflow, + but not past the header row. Their order and the gaps between them stay. Not called for a + lane with an `offset`, which places its columns by hand.""" + if not stacked: + return + overflow = max(layout.y + layout.height for layout in stacked) - height + room = min(layout.y for layout in stacked) - header # the top title sits in the header row + lift = min(overflow, room) + if lift > 0: + for layout in stacked: + set_y(layout, layout.y - lift) + + +def _around_lanes_between( + layout: ColumnLayout, + fan: tuple[ColumnLayout, Block, tuple[float, float]], + y: float, + layouts: dict[str, ColumnLayout], + placed: set[str], + highest: float, +) -> float: + """The top for `layout` nearest to `y`, and not above `highest`, from which its fan lines + clear the placed columns in the lanes between it and its source.""" + origin, piece, (top, bottom) = fan + between = [ + other + for other in lanes_between(layouts.values(), origin.lane, layout.lane) + if other.id in placed + ] + if not between: + return y + fan_lines = make_fan(piece.x, piece.x + piece.w, top, bottom, layout).moved(y) + return clear_top(fan_lines, [column_box(other) for other in between], highest) diff --git a/memmap/layout/connectors.py b/memmap/layout/connectors.py index 07d4820..8e7f7d3 100644 --- a/memmap/layout/connectors.py +++ b/memmap/layout/connectors.py @@ -4,18 +4,21 @@ from .columns import Block, ColumnLayout from .primitives import INK, Polyline +from .routing import Box, make_fan, route, straight -def connector_lines(source_blocks: list[Block], target: ColumnLayout) -> list[Polyline]: - """One straight line for the top edge and one for the bottom edge of the expanded region.""" +def connector_lines( + source_blocks: list[Block], target: ColumnLayout, between: list[Box] +) -> tuple[list[Polyline], bool]: + """One line for the top edge and one for the bottom edge of the expanded region. Each is + straight, or bent at a corner to go around `between`, the columns in the lanes the lines + cross. The flag is False when they cannot go around and run straight through them.""" top = min(block.y for block in source_blocks) bottom = max(block.y + block.h for block in source_blocks) left = source_blocks[0].x - right = left + source_blocks[0].w - rightward = target.x >= left - source_x = right if rightward else left - target_x = target.x if rightward else target.x + target.width - return [ - Polyline([(source_x, source_y), (target_x, target_y)], INK) - for source_y, target_y in ((top, target.y), (bottom, target.y + target.height)) - ] + fan = make_fan(left, left + source_blocks[0].w, top, bottom, target) + lines = route(fan, between) + clear = lines is not None + if lines is None: + lines = straight(fan) + return [Polyline(points, INK) for points in lines], clear diff --git a/memmap/layout/primitives.py b/memmap/layout/primitives.py index d824925..5fecbb8 100644 --- a/memmap/layout/primitives.py +++ b/memmap/layout/primitives.py @@ -86,7 +86,7 @@ class Drawing: height: float primitives: list[Primitive] = field(default_factory=list) columns: list[Any] = field(default_factory=list) - links: list[dict[str, str]] = field(default_factory=list) + links: list[dict[str, Any]] = field(default_factory=list) def texts(self) -> list[Text]: """Every string, in paint order.""" diff --git a/memmap/layout/routing.py b/memmap/layout/routing.py new file mode 100644 index 0000000..2ebc503 --- /dev/null +++ b/memmap/layout/routing.py @@ -0,0 +1,180 @@ +"""Fan lines that skip a lane: keeping them clear of the columns in the lanes they cross. + +A zoom whose detail column is two or more lanes from its source crosses the lanes between. A +straight fan line through a column there reads as if that column's block were the one expanded, +so the detail column is moved up or down until its fan lines can go around the columns between, +bending at a corner beside them. +""" + +from __future__ import annotations + +from collections.abc import Iterable +from dataclasses import dataclass +from itertools import pairwise +from typing import TYPE_CHECKING + +from .text import ADDRESS_FONT_SIZE + +if TYPE_CHECKING: + from .columns import ColumnLayout + +CLEARANCE = 8 # px, between a fan line that goes around a column and that column's labels +LINE_GAP = 6 # px, between the two fan lines where they bend around a column + +Point = tuple[float, float] + + +@dataclass(frozen=True) +class Box: + """The area a column covers, with its title, address labels and notes.""" + + left: float + top: float + right: float + bottom: float + + +@dataclass(frozen=True) +class Fan: + """Where the two fan lines of a zoom start and end. The source side is the expanded + block's edge facing the detail column, the target side the detail column's edge.""" + + source_x: float + source_top: float + source_bottom: float + target_x: float + target_top: float + target_bottom: float + + def moved(self, target_top: float) -> Fan: + """The same fan with the detail column's top at `target_top`.""" + height = self.target_bottom - self.target_top + return Fan( + self.source_x, + self.source_top, + self.source_bottom, + self.target_x, + target_top, + target_top + height, + ) + + +def column_box(layout: ColumnLayout) -> Box: + """The area of a placed column, from its title down to below its bottom address label. + Address labels are centred on block edges, so the top one reaches half a line above the + column; the bottom one, under the last edge, a whole line below it.""" + return Box( + layout.x - layout.left_gutter, + layout.y - layout.title_height - ADDRESS_FONT_SIZE / 2, + layout.x + layout.width + layout.right_gutter, + layout.y + layout.height + ADDRESS_FONT_SIZE, + ) + + +def lanes_between( + layouts: Iterable[ColumnLayout], source_lane: int, target_lane: int +) -> list[ColumnLayout]: + """The columns in the lanes strictly between a source and a target lane.""" + low, high = sorted((source_lane, target_lane)) + return [layout for layout in layouts if low < layout.lane < high] + + +def make_fan( + source_left: float, source_right: float, top: float, bottom: float, target: ColumnLayout +) -> Fan: + """The fan from an expanded block's y range to a placed detail column.""" + rightward = target.x >= source_left + return Fan( + source_right if rightward else source_left, + top, + bottom, + target.x if rightward else target.x + target.width, + target.y, + target.y + target.height, + ) + + +def straight(fan: Fan) -> list[list[Point]]: + """The two fan lines drawn straight, top edge first.""" + return [ + [(fan.source_x, fan.source_top), (fan.target_x, fan.target_top)], + [(fan.source_x, fan.source_bottom), (fan.target_x, fan.target_bottom)], + ] + + +def _crosses(start: Point, end: Point, box: Box) -> bool: + """Whether the segment from `start` to `end` runs through `box`.""" + (x0, y0), (x1, y1) = sorted((start, end)) + left, right = max(x0, box.left), min(x1, box.right) + if left > right: + return False + if x1 == x0: + ys = [y0, y1] + else: + ys = [y0 + (y1 - y0) * (x - x0) / (x1 - x0) for x in (left, right)] + return min(ys) < box.bottom and max(ys) > box.top + + +def _clear(lines: list[list[Point]], boxes: list[Box]) -> bool: + """Whether no segment of `lines` runs through any of `boxes`.""" + return not any( + _crosses(start, end, box) + for line in lines + for start, end in pairwise(line) + for box in boxes + ) + + +def route(fan: Fan, boxes: list[Box]) -> list[list[Point]] | None: + """The two fan lines, top edge first, as point lists: straight when that is clear of + `boxes`, else bent at a corner beside them to pass below or above them. None when the + detail column sits where neither works.""" + lines = straight(fan) + if _clear(lines, boxes): + return lines + rightward = fan.target_x >= fan.source_x + x = ( + min(box.left for box in boxes) - CLEARANCE + if rightward + else max(box.right for box in boxes) + CLEARANCE + ) + if (x - fan.source_x) * (fan.target_x - x) <= 0: # no room before the first column + return None + below = max(box.bottom for box in boxes) + CLEARANCE + above = min(box.top for box in boxes) - CLEARANCE + if fan.target_top >= below: + corners = (below, below + LINE_GAP) + elif fan.target_bottom <= above - LINE_GAP: + corners = (above - LINE_GAP, above) + else: + return None + lines = [ + [ + (fan.source_x, fan.source_top), + (x, corners[0]), + (fan.target_x, fan.target_top), + ], + [ + (fan.source_x, fan.source_bottom), + (x, corners[1]), + (fan.target_x, fan.target_bottom), + ], + ] + return lines if _clear(lines, boxes) else None + + +def clear_top(fan: Fan, boxes: list[Box], header: float) -> float: + """The detail column top nearest to `fan.target_top` from which its fan lines can go + around `boxes`: unchanged when they already can, else just below or just above them, + whichever moves it less. Above is only an option when the column still fits under the + header row.""" + if not boxes or route(fan, boxes) is not None: + return fan.target_top + height = fan.target_bottom - fan.target_top + below = max(box.bottom for box in boxes) + CLEARANCE + above = min(box.top for box in boxes) - CLEARANCE - LINE_GAP - height + options = [max(fan.target_top, below)] + if header <= above < fan.target_top: + options.append(above) + workable = [top for top in options if route(fan.moved(top), boxes) is not None] + return min(workable or options, key=lambda top: abs(top - fan.target_top)) diff --git a/tests/elf_builder.py b/tests/elf_builder.py index 1b874a9..0c7fd87 100644 --- a/tests/elf_builder.py +++ b/tests/elf_builder.py @@ -9,7 +9,7 @@ def make_elf(sections, segments=(), bits=32, little=True, file_type=2): """A minimal ELF with the given section and segment headers and no section contents. - Each section is (name, type, flags, address, size, offset); each segment is + Each section is (name, type, flags, address, size, offset[, align]); each segment is (offset, virtual, physical, file_size, memory_size).""" order = "<" if little else ">" if bits == 32: @@ -29,8 +29,8 @@ def make_elf(sections, segments=(), bits=32, little=True, file_type=2): names_offset = ph_offset + len(segments) * segment_size sh_offset = names_offset + len(names) - def section_header(name, kind, flags, address, size, offset): - return struct.pack(order + section_fmt, name, kind, flags, address, offset, size, 0, 0, 1, 0) + def section_header(name, kind, flags, address, size, offset, align=1): + return struct.pack(order + section_fmt, name, kind, flags, address, offset, size, 0, 0, align, 0) def segment_header(offset, virtual, physical, file_size, memory_size): if bits == 32: @@ -45,8 +45,8 @@ def segment_header(offset, virtual, physical, file_size, memory_size): ) data = ident + header + b"".join(segment_header(*segment) for segment in segments) + names data += section_header(0, 0, 0, 0, 0, 0) - for name_offset, (_, kind, flags, address, size, offset) in zip(name_offsets, sections): - data += section_header(name_offset, kind, flags, address, size, offset) + for name_offset, (_, *fields) in zip(name_offsets, sections): + data += section_header(name_offset, *fields) data += section_header(name_offsets[-1], 3, 0, 0, len(names), names_offset) return data @@ -60,7 +60,7 @@ def rebuild_snapshot(snapshot): """An ELF header snapshot from tests/fixtures/elf_snapshots/ as ELF bytes. make_elf adds its own section name table.""" sections = [ - (s["name"], s["type"], s["flags"], int(s["address"], 16), s["size"], s["offset"]) + (s["name"], s["type"], s["flags"], int(s["address"], 16), s["size"], s["offset"], s["align"]) for s in snapshot["sections"] if s["name"] != ".shstrtab" ] diff --git a/tests/fixtures/elf_snapshots/apollo4_hello_uart.json b/tests/fixtures/elf_snapshots/apollo4_hello_uart.json index 4de7e41..10173d1 100644 --- a/tests/fixtures/elf_snapshots/apollo4_hello_uart.json +++ b/tests/fixtures/elf_snapshots/apollo4_hello_uart.json @@ -18,7 +18,8 @@ "flags": 6, "address": "0x18000", "size": 15608, - "offset": 32768 + "offset": 32768, + "align": 4 }, { "name": ".stack", @@ -26,7 +27,8 @@ "flags": 3, "address": "0x10000000", "size": 4096, - "offset": 196608 + "offset": 196608, + "align": 4 }, { "name": ".heap", @@ -34,7 +36,8 @@ "flags": 3, "address": "0x10001000", "size": 0, - "offset": 200704 + "offset": 200704, + "align": 4 }, { "name": ".data", @@ -42,7 +45,8 @@ "flags": 3, "address": "0x10001000", "size": 32, - "offset": 69632 + "offset": 69632, + "align": 4 }, { "name": ".bss", @@ -50,7 +54,8 @@ "flags": 3, "address": "0x10001020", "size": 1828, - "offset": 69664 + "offset": 69664, + "align": 4 }, { "name": ".shared", @@ -58,7 +63,8 @@ "flags": 3, "address": "0x10060000", "size": 64, - "offset": 131072 + "offset": 131072, + "align": 4 }, { "name": ".ARM.attributes", @@ -66,7 +72,8 @@ "flags": 0, "address": "0x0", "size": 52, - "offset": 200704 + "offset": 200704, + "align": 1 }, { "name": ".debug_info", @@ -74,7 +81,8 @@ "flags": 0, "address": "0x0", "size": 51245, - "offset": 200756 + "offset": 200756, + "align": 1 }, { "name": ".debug_abbrev", @@ -82,7 +90,8 @@ "flags": 0, "address": "0x0", "size": 2560, - "offset": 252001 + "offset": 252001, + "align": 1 }, { "name": ".debug_aranges", @@ -90,7 +99,8 @@ "flags": 0, "address": "0x0", "size": 432, - "offset": 254561 + "offset": 254561, + "align": 1 }, { "name": ".debug_line", @@ -98,7 +108,8 @@ "flags": 0, "address": "0x0", "size": 5895, - "offset": 254993 + "offset": 254993, + "align": 1 }, { "name": ".debug_str", @@ -106,7 +117,8 @@ "flags": 48, "address": "0x0", "size": 32556, - "offset": 260888 + "offset": 260888, + "align": 1 }, { "name": ".comment", @@ -114,7 +126,8 @@ "flags": 48, "address": "0x0", "size": 73, - "offset": 293444 + "offset": 293444, + "align": 1 }, { "name": ".debug_ranges", @@ -122,7 +135,8 @@ "flags": 0, "address": "0x0", "size": 328, - "offset": 293517 + "offset": 293517, + "align": 1 }, { "name": ".debug_frame", @@ -130,7 +144,8 @@ "flags": 0, "address": "0x0", "size": 1116, - "offset": 293848 + "offset": 293848, + "align": 4 }, { "name": ".symtab", @@ -138,7 +153,8 @@ "flags": 0, "address": "0x0", "size": 7296, - "offset": 294964 + "offset": 294964, + "align": 4 }, { "name": ".strtab", @@ -146,7 +162,8 @@ "flags": 0, "address": "0x0", "size": 4334, - "offset": 302260 + "offset": 302260, + "align": 1 }, { "name": ".shstrtab", @@ -154,7 +171,8 @@ "flags": 0, "address": "0x0", "size": 181, - "offset": 306594 + "offset": 306594, + "align": 1 } ], "segments": [ diff --git a/tests/fixtures/elf_snapshots/fe310_zephyr_shell.json b/tests/fixtures/elf_snapshots/fe310_zephyr_shell.json index ba464bc..9a9ed5e 100644 --- a/tests/fixtures/elf_snapshots/fe310_zephyr_shell.json +++ b/tests/fixtures/elf_snapshots/fe310_zephyr_shell.json @@ -17,7 +17,8 @@ "flags": 6, "address": "0x20400000", "size": 16, - "offset": 148 + "offset": 148, + "align": 4 }, { "name": "exceptions", @@ -25,7 +26,8 @@ "flags": 6, "address": "0x20400010", "size": 628, - "offset": 164 + "offset": 164, + "align": 4 }, { "name": "text", @@ -33,7 +35,8 @@ "flags": 7, "address": "0x20400284", "size": 17292, - "offset": 792 + "offset": 792, + "align": 4 }, { "name": "devconfig", @@ -41,7 +44,8 @@ "flags": 3, "address": "0x20404610", "size": 132, - "offset": 18084 + "offset": 18084, + "align": 4 }, { "name": "rodata", @@ -49,7 +53,8 @@ "flags": 2, "address": "0x20404694", "size": 2452, - "offset": 18216 + "offset": 18216, + "align": 4 }, { "name": "datas", @@ -57,7 +62,8 @@ "flags": 3, "address": "0x80000000", "size": 168, - "offset": 20668 + "offset": 20668, + "align": 4 }, { "name": "initlevel", @@ -65,7 +71,8 @@ "flags": 3, "address": "0x800000A8", "size": 132, - "offset": 20836 + "offset": 20836, + "align": 4 }, { "name": "initshell", @@ -73,7 +80,8 @@ "flags": 3, "address": "0x8000012C", "size": 24, - "offset": 20968 + "offset": 20968, + "align": 4 }, { "name": "bss", @@ -81,7 +89,8 @@ "flags": 3, "address": "0x80000148", "size": 1336, - "offset": 21000 + "offset": 21000, + "align": 8 }, { "name": "noinit", @@ -89,7 +98,8 @@ "flags": 3, "address": "0x80000680", "size": 6608, - "offset": 21000 + "offset": 21000, + "align": 16 }, { "name": ".debug_info", @@ -97,7 +107,8 @@ "flags": 0, "address": "0x0", "size": 120305, - "offset": 20992 + "offset": 20992, + "align": 1 }, { "name": ".debug_abbrev", @@ -105,7 +116,8 @@ "flags": 0, "address": "0x0", "size": 26865, - "offset": 141297 + "offset": 141297, + "align": 1 }, { "name": ".debug_aranges", @@ -113,7 +125,8 @@ "flags": 0, "address": "0x0", "size": 3064, - "offset": 168176 + "offset": 168176, + "align": 16 }, { "name": ".debug_line", @@ -121,7 +134,8 @@ "flags": 0, "address": "0x0", "size": 60979, - "offset": 171240 + "offset": 171240, + "align": 1 }, { "name": ".debug_str", @@ -129,7 +143,8 @@ "flags": 48, "address": "0x0", "size": 19827, - "offset": 232219 + "offset": 232219, + "align": 1 }, { "name": ".comment", @@ -137,7 +152,8 @@ "flags": 48, "address": "0x0", "size": 17, - "offset": 252046 + "offset": 252046, + "align": 1 }, { "name": ".debug_frame", @@ -145,7 +161,8 @@ "flags": 0, "address": "0x0", "size": 9288, - "offset": 252064 + "offset": 252064, + "align": 4 }, { "name": ".debug_loc", @@ -153,7 +170,8 @@ "flags": 0, "address": "0x0", "size": 33596, - "offset": 261352 + "offset": 261352, + "align": 1 }, { "name": ".debug_ranges", @@ -161,7 +179,8 @@ "flags": 0, "address": "0x0", "size": 6688, - "offset": 294960 + "offset": 294960, + "align": 16 }, { "name": ".symtab", @@ -169,7 +188,8 @@ "flags": 0, "address": "0x0", "size": 9248, - "offset": 301648 + "offset": 301648, + "align": 4 }, { "name": ".strtab", @@ -177,7 +197,8 @@ "flags": 0, "address": "0x0", "size": 11035, - "offset": 310896 + "offset": 310896, + "align": 1 }, { "name": ".shstrtab", @@ -185,7 +206,8 @@ "flags": 0, "address": "0x0", "size": 215, - "offset": 321931 + "offset": 321931, + "align": 1 } ], "segments": [ diff --git a/tests/fixtures/elf_snapshots/msp430f2619_hello.json b/tests/fixtures/elf_snapshots/msp430f2619_hello.json index 28c693d..664d8e6 100644 --- a/tests/fixtures/elf_snapshots/msp430f2619_hello.json +++ b/tests/fixtures/elf_snapshots/msp430f2619_hello.json @@ -17,7 +17,8 @@ "flags": 2, "address": "0xFFFE", "size": 2, - "offset": 798 + "offset": 798, + "align": 1 }, { "name": ".lower.rodata", @@ -25,7 +26,8 @@ "flags": 1, "address": "0x2100", "size": 0, - "offset": 800 + "offset": 800, + "align": 1 }, { "name": ".rodata", @@ -33,7 +35,8 @@ "flags": 2, "address": "0x2100", "size": 80, - "offset": 212 + "offset": 212, + "align": 1 }, { "name": ".rodata2", @@ -41,7 +44,8 @@ "flags": 1, "address": "0x2150", "size": 0, - "offset": 800 + "offset": 800, + "align": 1 }, { "name": ".data", @@ -49,7 +53,8 @@ "flags": 3, "address": "0x1100", "size": 0, - "offset": 800 + "offset": 800, + "align": 1 }, { "name": ".bss", @@ -57,7 +62,8 @@ "flags": 3, "address": "0x1100", "size": 2, - "offset": 292 + "offset": 292, + "align": 2 }, { "name": ".noinit", @@ -65,7 +71,8 @@ "flags": 1, "address": "0x1102", "size": 0, - "offset": 800 + "offset": 800, + "align": 1 }, { "name": ".heap", @@ -73,7 +80,8 @@ "flags": 3, "address": "0x1102", "size": 4, - "offset": 294 + "offset": 294, + "align": 1 }, { "name": ".lowtext", @@ -81,7 +89,8 @@ "flags": 6, "address": "0x2150", "size": 24, - "offset": 292 + "offset": 292, + "align": 1 }, { "name": ".lower.text", @@ -89,7 +98,8 @@ "flags": 1, "address": "0x2168", "size": 0, - "offset": 800 + "offset": 800, + "align": 1 }, { "name": ".text", @@ -97,7 +107,8 @@ "flags": 6, "address": "0x2168", "size": 480, - "offset": 316 + "offset": 316, + "align": 2 }, { "name": ".upper.text", @@ -105,7 +116,8 @@ "flags": 1, "address": "0x10000", "size": 0, - "offset": 800 + "offset": 800, + "align": 1 }, { "name": ".MSP430.attributes", @@ -113,7 +125,8 @@ "flags": 0, "address": "0x0", "size": 23, - "offset": 800 + "offset": 800, + "align": 1 }, { "name": ".comment", @@ -121,7 +134,8 @@ "flags": 48, "address": "0x0", "size": 57, - "offset": 823 + "offset": 823, + "align": 1 }, { "name": ".symtab", @@ -129,7 +143,8 @@ "flags": 0, "address": "0x0", "size": 4848, - "offset": 880 + "offset": 880, + "align": 4 }, { "name": ".strtab", @@ -137,7 +152,8 @@ "flags": 0, "address": "0x0", "size": 1312, - "offset": 5728 + "offset": 5728, + "align": 1 }, { "name": ".shstrtab", @@ -145,7 +161,8 @@ "flags": 0, "address": "0x0", "size": 151, - "offset": 7040 + "offset": 7040, + "align": 1 } ], "segments": [ diff --git a/tests/fixtures/elf_snapshots/nrf52840_zephyr_shell.json b/tests/fixtures/elf_snapshots/nrf52840_zephyr_shell.json index e807e97..52892c5 100644 --- a/tests/fixtures/elf_snapshots/nrf52840_zephyr_shell.json +++ b/tests/fixtures/elf_snapshots/nrf52840_zephyr_shell.json @@ -17,7 +17,8 @@ "flags": 7, "address": "0x0", "size": 256, - "offset": 192 + "offset": 192, + "align": 4 }, { "name": "text", @@ -25,7 +26,8 @@ "flags": 6, "address": "0x100", "size": 49376, - "offset": 448 + "offset": 448, + "align": 8 }, { "name": ".ARM.exidx", @@ -33,7 +35,8 @@ "flags": 130, "address": "0xC1E0", "size": 8, - "offset": 49824 + "offset": 49824, + "align": 4 }, { "name": "initlevel", @@ -41,7 +44,8 @@ "flags": 2, "address": "0xC1E8", "size": 120, - "offset": 49832 + "offset": 49832, + "align": 4 }, { "name": "sw_isr_table", @@ -49,7 +53,8 @@ "flags": 3, "address": "0xC260", "size": 384, - "offset": 49952 + "offset": 49952, + "align": 4 }, { "name": "log_const_sections", @@ -57,7 +62,8 @@ "flags": 2, "address": "0xC3E0", "size": 96, - "offset": 50336 + "offset": 50336, + "align": 4 }, { "name": "log_backends_sections", @@ -65,7 +71,8 @@ "flags": 2, "address": "0xC440", "size": 32, - "offset": 50432 + "offset": 50432, + "align": 4 }, { "name": "shell_area", @@ -73,7 +80,8 @@ "flags": 2, "address": "0xC460", "size": 48, - "offset": 50464 + "offset": 50464, + "align": 4 }, { "name": "shell_root_cmds_sections", @@ -81,7 +89,8 @@ "flags": 2, "address": "0xC490", "size": 112, - "offset": 50512 + "offset": 50512, + "align": 4 }, { "name": "rodata", @@ -89,7 +98,8 @@ "flags": 2, "address": "0xC500", "size": 10936, - "offset": 50624 + "offset": 50624, + "align": 16 }, { "name": ".ramfunc", @@ -97,7 +107,8 @@ "flags": 1, "address": "0x20000000", "size": 0, - "offset": 62236 + "offset": 62236, + "align": 1 }, { "name": "datas", @@ -105,7 +116,8 @@ "flags": 3, "address": "0x20000000", "size": 372, - "offset": 61560 + "offset": 61560, + "align": 4 }, { "name": "devices", @@ -113,7 +125,8 @@ "flags": 2, "address": "0x20000174", "size": 116, - "offset": 61932 + "offset": 61932, + "align": 4 }, { "name": "log_dynamic_sections", @@ -121,7 +134,8 @@ "flags": 3, "address": "0x200001E8", "size": 48, - "offset": 62048 + "offset": 62048, + "align": 4 }, { "name": "k_timer_area", @@ -129,7 +143,8 @@ "flags": 3, "address": "0x20000218", "size": 56, - "offset": 62096 + "offset": 62096, + "align": 8 }, { "name": "k_mutex_area", @@ -137,7 +152,8 @@ "flags": 3, "address": "0x20000250", "size": 20, - "offset": 62152 + "offset": 62152, + "align": 4 }, { "name": "k_msgq_area", @@ -145,7 +161,8 @@ "flags": 3, "address": "0x20000264", "size": 40, - "offset": 62172 + "offset": 62172, + "align": 4 }, { "name": "k_sem_area", @@ -153,7 +170,8 @@ "flags": 3, "address": "0x2000028C", "size": 24, - "offset": 62212 + "offset": 62212, + "align": 4 }, { "name": "bss", @@ -161,7 +179,8 @@ "flags": 3, "address": "0x200002A8", "size": 4142, - "offset": 62248 + "offset": 62248, + "align": 8 }, { "name": "noinit", @@ -169,7 +188,8 @@ "flags": 3, "address": "0x200012E0", "size": 8144, - "offset": 62248 + "offset": 62248, + "align": 32 }, { "name": ".comment", @@ -177,7 +197,8 @@ "flags": 48, "address": "0x0", "size": 50, - "offset": 62236 + "offset": 62236, + "align": 1 }, { "name": ".debug_aranges", @@ -185,7 +206,8 @@ "flags": 0, "address": "0x0", "size": 8216, - "offset": 62288 + "offset": 62288, + "align": 8 }, { "name": ".debug_info", @@ -193,7 +215,8 @@ "flags": 0, "address": "0x0", "size": 561067, - "offset": 70504 + "offset": 70504, + "align": 1 }, { "name": ".debug_abbrev", @@ -201,7 +224,8 @@ "flags": 0, "address": "0x0", "size": 84915, - "offset": 631571 + "offset": 631571, + "align": 1 }, { "name": ".debug_line", @@ -209,7 +233,8 @@ "flags": 0, "address": "0x0", "size": 196601, - "offset": 716486 + "offset": 716486, + "align": 1 }, { "name": ".debug_frame", @@ -217,7 +242,8 @@ "flags": 0, "address": "0x0", "size": 21284, - "offset": 913088 + "offset": 913088, + "align": 4 }, { "name": ".debug_str", @@ -225,7 +251,8 @@ "flags": 48, "address": "0x0", "size": 58864, - "offset": 934372 + "offset": 934372, + "align": 1 }, { "name": ".debug_loc", @@ -233,7 +260,8 @@ "flags": 0, "address": "0x0", "size": 203162, - "offset": 993236 + "offset": 993236, + "align": 1 }, { "name": ".debug_ranges", @@ -241,7 +269,8 @@ "flags": 0, "address": "0x0", "size": 40392, - "offset": 1196400 + "offset": 1196400, + "align": 8 }, { "name": ".ARM.attributes", @@ -249,7 +278,8 @@ "flags": 0, "address": "0x0", "size": 49, - "offset": 1236792 + "offset": 1236792, + "align": 1 }, { "name": ".symtab", @@ -257,7 +287,8 @@ "flags": 0, "address": "0x0", "size": 43536, - "offset": 1236844 + "offset": 1236844, + "align": 4 }, { "name": ".strtab", @@ -265,7 +296,8 @@ "flags": 0, "address": "0x0", "size": 27941, - "offset": 1280380 + "offset": 1280380, + "align": 1 }, { "name": ".shstrtab", @@ -273,7 +305,8 @@ "flags": 0, "address": "0x0", "size": 391, - "offset": 1308321 + "offset": 1308321, + "align": 1 } ], "segments": [ diff --git a/tests/fixtures/elf_snapshots/r52_zephyr_philosophers.json b/tests/fixtures/elf_snapshots/r52_zephyr_philosophers.json index 28790b3..5aeea5a 100644 --- a/tests/fixtures/elf_snapshots/r52_zephyr_philosophers.json +++ b/tests/fixtures/elf_snapshots/r52_zephyr_philosophers.json @@ -16,7 +16,8 @@ "flags": 6, "address": "0x0", "size": 60, - "offset": 180 + "offset": 180, + "align": 4 }, { "name": "text", @@ -24,7 +25,8 @@ "flags": 6, "address": "0x3C", "size": 25348, - "offset": 240 + "offset": 240, + "align": 4 }, { "name": ".ARM.exidx", @@ -32,7 +34,8 @@ "flags": 130, "address": "0x6340", "size": 8, - "offset": 25588 + "offset": 25588, + "align": 4 }, { "name": "rodata_start", @@ -40,7 +43,8 @@ "flags": 3, "address": "0x6348", "size": 56, - "offset": 25596 + "offset": 25596, + "align": 1 }, { "name": "initlevel", @@ -48,7 +52,8 @@ "flags": 2, "address": "0x6380", "size": 32, - "offset": 25600 + "offset": 25600, + "align": 4 }, { "name": "devices", @@ -56,7 +61,8 @@ "flags": 2, "address": "0x63A0", "size": 24, - "offset": 25632 + "offset": 25632, + "align": 4 }, { "name": "sw_isr_table", @@ -64,7 +70,8 @@ "flags": 3, "address": "0x63B8", "size": 1024, - "offset": 25656 + "offset": 25656, + "align": 4 }, { "name": "device_handles", @@ -72,7 +79,8 @@ "flags": 2, "address": "0x67B8", "size": 6, - "offset": 26680 + "offset": 26680, + "align": 2 }, { "name": "log_const_sections", @@ -80,7 +88,8 @@ "flags": 2, "address": "0x67C0", "size": 24, - "offset": 26688 + "offset": 26688, + "align": 4 }, { "name": "zephyr_dbg_info", @@ -88,7 +97,8 @@ "flags": 3, "address": "0x67D8", "size": 64, - "offset": 26712 + "offset": 26712, + "align": 4 }, { "name": "rodata", @@ -96,7 +106,8 @@ "flags": 2, "address": "0x6818", "size": 2792, - "offset": 26776 + "offset": 26776, + "align": 4 }, { "name": "bss", @@ -104,7 +115,8 @@ "flags": 3, "address": "0x7300", "size": 1654, - "offset": 29568 + "offset": 29568, + "align": 64 }, { "name": "noinit", @@ -112,7 +124,8 @@ "flags": 3, "address": "0x7978", "size": 16896, - "offset": 29568 + "offset": 29568, + "align": 8 }, { "name": "datas", @@ -120,7 +133,8 @@ "flags": 3, "address": "0xBB78", "size": 44, - "offset": 29568 + "offset": 29568, + "align": 4 }, { "name": "device_states", @@ -128,7 +142,8 @@ "flags": 3, "address": "0xBBA4", "size": 2, - "offset": 29612 + "offset": 29612, + "align": 1 }, { "name": ".comment", @@ -136,7 +151,8 @@ "flags": 48, "address": "0x0", "size": 32, - "offset": 29618 + "offset": 29618, + "align": 1 }, { "name": ".debug_aranges", @@ -144,7 +160,8 @@ "flags": 0, "address": "0x0", "size": 3272, - "offset": 29656 + "offset": 29656, + "align": 8 }, { "name": ".debug_info", @@ -152,7 +169,8 @@ "flags": 0, "address": "0x0", "size": 221284, - "offset": 32928 + "offset": 32928, + "align": 1 }, { "name": ".debug_abbrev", @@ -160,7 +178,8 @@ "flags": 0, "address": "0x0", "size": 28444, - "offset": 254212 + "offset": 254212, + "align": 1 }, { "name": ".debug_line", @@ -168,7 +187,8 @@ "flags": 0, "address": "0x0", "size": 71675, - "offset": 282656 + "offset": 282656, + "align": 1 }, { "name": ".debug_frame", @@ -176,7 +196,8 @@ "flags": 0, "address": "0x0", "size": 7076, - "offset": 354332 + "offset": 354332, + "align": 4 }, { "name": ".debug_str", @@ -184,7 +205,8 @@ "flags": 48, "address": "0x0", "size": 20816, - "offset": 361408 + "offset": 361408, + "align": 1 }, { "name": ".debug_loc", @@ -192,7 +214,8 @@ "flags": 0, "address": "0x0", "size": 71039, - "offset": 382224 + "offset": 382224, + "align": 1 }, { "name": ".debug_ranges", @@ -200,7 +223,8 @@ "flags": 0, "address": "0x0", "size": 14504, - "offset": 453264 + "offset": 453264, + "align": 8 }, { "name": ".ARM.attributes", @@ -208,7 +232,8 @@ "flags": 0, "address": "0x0", "size": 51, - "offset": 467768 + "offset": 467768, + "align": 1 }, { "name": ".last_section", @@ -216,7 +241,8 @@ "flags": 2, "address": "0xBBA6", "size": 4, - "offset": 29614 + "offset": 29614, + "align": 1 }, { "name": ".symtab", @@ -224,7 +250,8 @@ "flags": 0, "address": "0x0", "size": 17104, - "offset": 467820 + "offset": 467820, + "align": 4 }, { "name": ".strtab", @@ -232,7 +259,8 @@ "flags": 0, "address": "0x0", "size": 13828, - "offset": 484924 + "offset": 484924, + "align": 1 }, { "name": ".shstrtab", @@ -240,7 +268,8 @@ "flags": 0, "address": "0x0", "size": 326, - "offset": 498752 + "offset": 498752, + "align": 1 } ], "segments": [ diff --git a/tests/fixtures/elf_snapshots/refresh.py b/tests/fixtures/elf_snapshots/refresh.py index 1826d26..941a8a4 100644 --- a/tests/fixtures/elf_snapshots/refresh.py +++ b/tests/fixtures/elf_snapshots/refresh.py @@ -80,6 +80,28 @@ "sram=0x10060000:1M:Shared SRAM", ], }, + # Not from Renode: builds committed to public repositories, pinned to a commit. Their + # regions are the ones their linker maps give. + "stm32l4_keil_rfe_master": { + "url": "https://raw.githubusercontent.com/robotology/icub-firmware/" + "4ee61bd04f3ae1280da7a49236849a6252cd6582/emBODY/eBcode/arch-arm/libs/lowlevel/" + "stm32hal/cubemx/rfe/MDK-ARM/RFE_MASTER/RFE_MASTER.axf", + "about": "iCub RFE board firmware on STM32L4, linked by Keil armlink: RW and ZI data share " + "one name, and the RW data is stored compressed after the code", + "regions": ["flash=0x08000000:512K:Flash", "sram=0x20000000:160K:SRAM"], + }, + "rp2040_pico_rom": { + "url": "https://raw.githubusercontent.com/nickbild/picoROM/" + "11d8425c38c841c5278a6e6419f4db266f2711ae/pico_rom.elf", + "about": "picoROM on RP2040, built with the Pico SDK: the RAM vector table, heap and stack " + "are (COPY) sections, which are not allocated", + "regions": [ + "flash=0x10000000:2M:Flash", + "ram=0x20000000:256K:RAM", + "scratch_x=0x20040000:4K:SCRATCH_X", + "scratch_y=0x20041000:4K:SCRATCH_Y", + ], + }, } @@ -103,6 +125,7 @@ def snapshot(name: str, entry: dict[str, object], data: bytes) -> dict[str, obje "address": f"0x{s.address:X}", "size": s.size, "offset": s.offset, + "align": s.align, } for s in elf.sections ], diff --git a/tests/fixtures/elf_snapshots/rp2040_pico_rom.json b/tests/fixtures/elf_snapshots/rp2040_pico_rom.json new file mode 100644 index 0000000..2cb4cdd --- /dev/null +++ b/tests/fixtures/elf_snapshots/rp2040_pico_rom.json @@ -0,0 +1,202 @@ +{ + "name": "rp2040_pico_rom", + "about": "picoROM on RP2040, built with the Pico SDK: the RAM vector table, heap and stack are (COPY) sections, which are not allocated", + "url": "https://raw.githubusercontent.com/nickbild/picoROM/11d8425c38c841c5278a6e6419f4db266f2711ae/pico_rom.elf", + "sha256": "0aea9ca6b2e80d4c02f64f77d707ac360d220b8b38a06348285eb1194141f787", + "regions": [ + "flash=0x10000000:2M:Flash", + "ram=0x20000000:256K:RAM", + "scratch_x=0x20040000:4K:SCRATCH_X", + "scratch_y=0x20041000:4K:SCRATCH_Y" + ], + "bits": 32, + "little_endian": true, + "file_type": 2, + "sections": [ + { + "name": ".boot2", + "type": 1, + "flags": 6, + "address": "0x10000000", + "size": 256, + "offset": 65536, + "align": 1 + }, + { + "name": ".text", + "type": 1, + "flags": 6, + "address": "0x10000100", + "size": 23248, + "offset": 65792, + "align": 8 + }, + { + "name": ".rodata", + "type": 1, + "flags": 2, + "address": "0x10005BD0", + "size": 248, + "offset": 89040, + "align": 4 + }, + { + "name": ".binary_info", + "type": 1, + "flags": 3, + "address": "0x10005CC8", + "size": 32, + "offset": 89288, + "align": 4 + }, + { + "name": ".ram_vector_table", + "type": 1, + "flags": 1, + "address": "0x20000000", + "size": 192, + "offset": 131644, + "align": 4 + }, + { + "name": ".data", + "type": 1, + "flags": 7, + "address": "0x200000C0", + "size": 380, + "offset": 131264, + "align": 16 + }, + { + "name": ".uninitialized_data", + "type": 1, + "flags": 1, + "address": "0x2000023C", + "size": 0, + "offset": 131836, + "align": 1 + }, + { + "name": ".scratch_x", + "type": 1, + "flags": 1, + "address": "0x20040000", + "size": 0, + "offset": 131836, + "align": 1 + }, + { + "name": ".scratch_y", + "type": 1, + "flags": 1, + "address": "0x20041000", + "size": 0, + "offset": 131836, + "align": 1 + }, + { + "name": ".bss", + "type": 8, + "flags": 3, + "address": "0x20000240", + "size": 33736, + "offset": 131648, + "align": 8 + }, + { + "name": ".heap", + "type": 1, + "flags": 0, + "address": "0x20008608", + "size": 2048, + "offset": 131836, + "align": 4 + }, + { + "name": ".stack_dummy", + "type": 1, + "flags": 0, + "address": "0x20041000", + "size": 2048, + "offset": 133888, + "align": 32 + }, + { + "name": ".ARM.attributes", + "type": 1879048195, + "flags": 0, + "address": "0x0", + "size": 40, + "offset": 135936, + "align": 1 + }, + { + "name": ".comment", + "type": 1, + "flags": 48, + "address": "0x0", + "size": 121, + "offset": 135976, + "align": 1 + }, + { + "name": ".debug_frame", + "type": 1, + "flags": 0, + "address": "0x0", + "size": 40, + "offset": 136100, + "align": 4 + }, + { + "name": ".symtab", + "type": 2, + "flags": 0, + "address": "0x0", + "size": 8592, + "offset": 136140, + "align": 4 + }, + { + "name": ".strtab", + "type": 3, + "flags": 0, + "address": "0x0", + "size": 5361, + "offset": 144732, + "align": 1 + }, + { + "name": ".shstrtab", + "type": 3, + "flags": 0, + "address": "0x0", + "size": 189, + "offset": 150093, + "align": 1 + } + ], + "segments": [ + { + "offset": 65536, + "virtual": "0x10000000", + "physical": "0x10000000", + "file_size": 23784, + "memory_size": 23784 + }, + { + "offset": 131264, + "virtual": "0x200000C0", + "physical": "0x10005CE8", + "file_size": 380, + "memory_size": 380 + }, + { + "offset": 131648, + "virtual": "0x20000240", + "physical": "0x20000240", + "file_size": 0, + "memory_size": 33736 + } + ] +} diff --git a/tests/fixtures/elf_snapshots/stm32f412_tock_kernel.json b/tests/fixtures/elf_snapshots/stm32f412_tock_kernel.json index 33c135d..c3132ad 100644 --- a/tests/fixtures/elf_snapshots/stm32f412_tock_kernel.json +++ b/tests/fixtures/elf_snapshots/stm32f412_tock_kernel.json @@ -17,7 +17,8 @@ "flags": 3, "address": "0x20000000", "size": 8192, - "offset": 308 + "offset": 308, + "align": 1 }, { "name": ".text", @@ -25,7 +26,8 @@ "flags": 22, "address": "0x8000000", "size": 99516, - "offset": 512 + "offset": 512, + "align": 16 }, { "name": ".ARM.exidx", @@ -33,7 +35,8 @@ "flags": 130, "address": "0x80184BC", "size": 16, - "offset": 100028 + "offset": 100028, + "align": 4 }, { "name": ".storage", @@ -41,7 +44,8 @@ "flags": 2, "address": "0x80184CC", "size": 0, - "offset": 100044 + "offset": 100044, + "align": 1 }, { "name": ".apps", @@ -49,7 +53,8 @@ "flags": 2, "address": "0x8030000", "size": 1, - "offset": 100352 + "offset": 100352, + "align": 1 }, { "name": ".relocate", @@ -57,7 +62,8 @@ "flags": 3, "address": "0x20002000", "size": 40, - "offset": 100864 + "offset": 100864, + "align": 4 }, { "name": ".sram", @@ -65,7 +71,8 @@ "flags": 3, "address": "0x20002028", "size": 65496, - "offset": 100904 + "offset": 100904, + "align": 4 }, { "name": ".debug_loc", @@ -73,7 +80,8 @@ "flags": 0, "address": "0x0", "size": 323337, - "offset": 100904 + "offset": 100904, + "align": 1 }, { "name": ".debug_abbrev", @@ -81,7 +89,8 @@ "flags": 0, "address": "0x0", "size": 5008, - "offset": 424241 + "offset": 424241, + "align": 1 }, { "name": ".debug_info", @@ -89,7 +98,8 @@ "flags": 0, "address": "0x0", "size": 1186046, - "offset": 429249 + "offset": 429249, + "align": 1 }, { "name": ".debug_aranges", @@ -97,7 +107,8 @@ "flags": 0, "address": "0x0", "size": 7552, - "offset": 1615296 + "offset": 1615296, + "align": 8 }, { "name": ".debug_ranges", @@ -105,7 +116,8 @@ "flags": 0, "address": "0x0", "size": 83344, - "offset": 1622848 + "offset": 1622848, + "align": 1 }, { "name": ".debug_str", @@ -113,7 +125,8 @@ "flags": 48, "address": "0x0", "size": 617334, - "offset": 1706192 + "offset": 1706192, + "align": 1 }, { "name": ".debug_pubnames", @@ -121,7 +134,8 @@ "flags": 0, "address": "0x0", "size": 231974, - "offset": 2323526 + "offset": 2323526, + "align": 1 }, { "name": ".debug_pubtypes", @@ -129,7 +143,8 @@ "flags": 0, "address": "0x0", "size": 569173, - "offset": 2555500 + "offset": 2555500, + "align": 1 }, { "name": ".ARM.attributes", @@ -137,7 +152,8 @@ "flags": 0, "address": "0x0", "size": 48, - "offset": 3124673 + "offset": 3124673, + "align": 1 }, { "name": ".debug_frame", @@ -145,7 +161,8 @@ "flags": 0, "address": "0x0", "size": 16148, - "offset": 3124724 + "offset": 3124724, + "align": 4 }, { "name": ".debug_line", @@ -153,7 +170,8 @@ "flags": 0, "address": "0x0", "size": 160794, - "offset": 3140872 + "offset": 3140872, + "align": 1 }, { "name": ".comment", @@ -161,7 +179,8 @@ "flags": 48, "address": "0x0", "size": 147, - "offset": 3301666 + "offset": 3301666, + "align": 1 }, { "name": ".symtab", @@ -169,7 +188,8 @@ "flags": 0, "address": "0x0", "size": 25520, - "offset": 3301816 + "offset": 3301816, + "align": 4 }, { "name": ".shstrtab", @@ -177,7 +197,8 @@ "flags": 0, "address": "0x0", "size": 241, - "offset": 3327336 + "offset": 3327336, + "align": 1 }, { "name": ".strtab", @@ -185,7 +206,8 @@ "flags": 0, "address": "0x0", "size": 63843, - "offset": 3327577 + "offset": 3327577, + "align": 1 } ], "segments": [ diff --git a/tests/fixtures/elf_snapshots/stm32f4discovery.json b/tests/fixtures/elf_snapshots/stm32f4discovery.json index 000e58f..c3c85ec 100644 --- a/tests/fixtures/elf_snapshots/stm32f4discovery.json +++ b/tests/fixtures/elf_snapshots/stm32f4discovery.json @@ -17,7 +17,8 @@ "flags": 3, "address": "0x8000000", "size": 392, - "offset": 32768 + "offset": 32768, + "align": 4 }, { "name": ".text", @@ -25,7 +26,8 @@ "flags": 6, "address": "0x8000188", "size": 77680, - "offset": 33160 + "offset": 33160, + "align": 8 }, { "name": ".ARM", @@ -33,7 +35,8 @@ "flags": 130, "address": "0x80130F8", "size": 8, - "offset": 110840 + "offset": 110840, + "align": 4 }, { "name": ".data", @@ -41,7 +44,8 @@ "flags": 3, "address": "0x20000000", "size": 2716, - "offset": 131072 + "offset": 131072, + "align": 8 }, { "name": ".bss", @@ -49,7 +53,8 @@ "flags": 3, "address": "0x20000A9C", "size": 22212, - "offset": 133788 + "offset": 133788, + "align": 4 }, { "name": "._user_heap_stack", @@ -57,7 +62,8 @@ "flags": 3, "address": "0x20006160", "size": 1024, - "offset": 133788 + "offset": 133788, + "align": 1 }, { "name": ".comment", @@ -65,7 +71,8 @@ "flags": 48, "address": "0x0", "size": 31, - "offset": 133788 + "offset": 133788, + "align": 1 }, { "name": ".ARM.attributes", @@ -73,7 +80,8 @@ "flags": 0, "address": "0x0", "size": 55, - "offset": 133819 + "offset": 133819, + "align": 1 }, { "name": ".debug_info", @@ -81,7 +89,8 @@ "flags": 0, "address": "0x0", "size": 120272, - "offset": 133874 + "offset": 133874, + "align": 1 }, { "name": ".debug_abbrev", @@ -89,7 +98,8 @@ "flags": 0, "address": "0x0", "size": 24653, - "offset": 254146 + "offset": 254146, + "align": 1 }, { "name": ".debug_aranges", @@ -97,7 +107,8 @@ "flags": 0, "address": "0x0", "size": 2256, - "offset": 278800 + "offset": 278800, + "align": 8 }, { "name": ".debug_line", @@ -105,7 +116,8 @@ "flags": 0, "address": "0x0", "size": 28197, - "offset": 281056 + "offset": 281056, + "align": 1 }, { "name": ".debug_str", @@ -113,7 +125,8 @@ "flags": 48, "address": "0x0", "size": 8026, - "offset": 309253 + "offset": 309253, + "align": 1 }, { "name": ".debug_loc", @@ -121,7 +134,8 @@ "flags": 0, "address": "0x0", "size": 76069, - "offset": 317279 + "offset": 317279, + "align": 1 }, { "name": ".debug_ranges", @@ -129,7 +143,8 @@ "flags": 0, "address": "0x0", "size": 3520, - "offset": 393348 + "offset": 393348, + "align": 1 }, { "name": ".debug_frame", @@ -137,7 +152,8 @@ "flags": 0, "address": "0x0", "size": 3764, - "offset": 396868 + "offset": 396868, + "align": 4 }, { "name": ".shstrtab", @@ -145,7 +161,8 @@ "flags": 0, "address": "0x0", "size": 206, - "offset": 400632 + "offset": 400632, + "align": 1 }, { "name": ".symtab", @@ -153,7 +170,8 @@ "flags": 0, "address": "0x0", "size": 25152, - "offset": 401640 + "offset": 401640, + "align": 4 }, { "name": ".strtab", @@ -161,7 +179,8 @@ "flags": 0, "address": "0x0", "size": 18649, - "offset": 426792 + "offset": 426792, + "align": 1 } ], "segments": [ diff --git a/tests/fixtures/elf_snapshots/stm32f746_dartino_lines.json b/tests/fixtures/elf_snapshots/stm32f746_dartino_lines.json index 129682c..5c3cf49 100644 --- a/tests/fixtures/elf_snapshots/stm32f746_dartino_lines.json +++ b/tests/fixtures/elf_snapshots/stm32f746_dartino_lines.json @@ -17,7 +17,8 @@ "flags": 2, "address": "0x8000000", "size": 456, - "offset": 65536 + "offset": 65536, + "align": 1 }, { "name": ".text", @@ -25,7 +26,8 @@ "flags": 6, "address": "0x80001D0", "size": 158288, - "offset": 66000 + "offset": 66000, + "align": 16 }, { "name": ".rodata", @@ -33,7 +35,8 @@ "flags": 2, "address": "0x8026C20", "size": 29008, - "offset": 224288 + "offset": 224288, + "align": 8 }, { "name": ".ARM", @@ -41,7 +44,8 @@ "flags": 130, "address": "0x802DD70", "size": 8, - "offset": 253296 + "offset": 253296, + "align": 4 }, { "name": ".init_array", @@ -49,7 +53,8 @@ "flags": 3, "address": "0x802DD78", "size": 56, - "offset": 253304 + "offset": 253304, + "align": 4 }, { "name": ".fini_array", @@ -57,7 +62,8 @@ "flags": 3, "address": "0x802DDB0", "size": 4, - "offset": 253360 + "offset": 253360, + "align": 4 }, { "name": ".data", @@ -65,7 +71,8 @@ "flags": 3, "address": "0x20000000", "size": 37928, - "offset": 262144 + "offset": 262144, + "align": 16 }, { "name": ".bss", @@ -73,7 +80,8 @@ "flags": 3, "address": "0x20009428", "size": 23276, - "offset": 300072 + "offset": 300072, + "align": 8 }, { "name": "._user_heap_stack", @@ -81,7 +89,8 @@ "flags": 3, "address": "0x2000EF14", "size": 1540, - "offset": 323348 + "offset": 323348, + "align": 1 }, { "name": ".ARM.attributes", @@ -89,7 +98,8 @@ "flags": 0, "address": "0x0", "size": 48, - "offset": 300072 + "offset": 300072, + "align": 1 }, { "name": ".comment", @@ -97,7 +107,8 @@ "flags": 48, "address": "0x0", "size": 110, - "offset": 300120 + "offset": 300120, + "align": 1 }, { "name": ".debug_frame", @@ -105,7 +116,8 @@ "flags": 0, "address": "0x0", "size": 46524, - "offset": 300232 + "offset": 300232, + "align": 4 }, { "name": ".shstrtab", @@ -113,7 +125,8 @@ "flags": 0, "address": "0x0", "size": 149, - "offset": 486025 + "offset": 486025, + "align": 1 }, { "name": ".symtab", @@ -121,7 +134,8 @@ "flags": 0, "address": "0x0", "size": 78624, - "offset": 346756 + "offset": 346756, + "align": 4 }, { "name": ".strtab", @@ -129,7 +143,8 @@ "flags": 0, "address": "0x0", "size": 60645, - "offset": 425380 + "offset": 425380, + "align": 1 } ], "segments": [ diff --git a/tests/fixtures/elf_snapshots/stm32l4_keil_rfe_master.json b/tests/fixtures/elf_snapshots/stm32l4_keil_rfe_master.json new file mode 100644 index 0000000..01499fa --- /dev/null +++ b/tests/fixtures/elf_snapshots/stm32l4_keil_rfe_master.json @@ -0,0 +1,159 @@ +{ + "name": "stm32l4_keil_rfe_master", + "about": "iCub RFE board firmware on STM32L4, linked by Keil armlink: RW and ZI data share one name, and the RW data is stored compressed after the code", + "url": "https://raw.githubusercontent.com/robotology/icub-firmware/4ee61bd04f3ae1280da7a49236849a6252cd6582/emBODY/eBcode/arch-arm/libs/lowlevel/stm32hal/cubemx/rfe/MDK-ARM/RFE_MASTER/RFE_MASTER.axf", + "sha256": "56e176508a1ccaf0bea1e4e550f154d380a21a68c33ca10f3c08420d0e5b3803", + "regions": [ + "flash=0x08000000:512K:Flash", + "sram=0x20000000:160K:SRAM" + ], + "bits": 32, + "little_endian": true, + "file_type": 2, + "sections": [ + { + "name": "ER_IROM1", + "type": 1, + "flags": 6, + "address": "0x8000000", + "size": 36476, + "offset": 52, + "align": 4 + }, + { + "name": "RW_IRAM1", + "type": 1, + "flags": 3, + "address": "0x20000000", + "size": 212, + "offset": 36528, + "align": 4 + }, + { + "name": "RW_IRAM1", + "type": 8, + "flags": 3, + "address": "0x200001CC", + "size": 10020, + "offset": 36740, + "align": 8 + }, + { + "name": ".debug_abbrev", + "type": 1, + "flags": 0, + "address": "0x0", + "size": 1476, + "offset": 36740, + "align": 1 + }, + { + "name": ".debug_frame", + "type": 1, + "flags": 0, + "address": "0x0", + "size": 9244, + "offset": 38216, + "align": 1 + }, + { + "name": ".debug_info", + "type": 1, + "flags": 0, + "address": "0x0", + "size": 168484, + "offset": 47460, + "align": 1 + }, + { + "name": ".debug_line", + "type": 1, + "flags": 0, + "address": "0x0", + "size": 62384, + "offset": 215944, + "align": 1 + }, + { + "name": ".debug_loc", + "type": 1, + "flags": 0, + "address": "0x0", + "size": 26136, + "offset": 278328, + "align": 1 + }, + { + "name": ".debug_macinfo", + "type": 1, + "flags": 0, + "address": "0x0", + "size": 798592, + "offset": 304464, + "align": 1 + }, + { + "name": ".debug_pubnames", + "type": 1, + "flags": 0, + "address": "0x0", + "size": 13026, + "offset": 1103056, + "align": 1 + }, + { + "name": ".symtab", + "type": 2, + "flags": 0, + "address": "0x0", + "size": 20400, + "offset": 1116084, + "align": 4 + }, + { + "name": ".strtab", + "type": 3, + "flags": 0, + "address": "0x0", + "size": 20440, + "offset": 1136484, + "align": 1 + }, + { + "name": ".note", + "type": 7, + "flags": 0, + "address": "0x0", + "size": 28, + "offset": 1156924, + "align": 4 + }, + { + "name": ".comment", + "type": 1, + "flags": 0, + "address": "0x0", + "size": 68936, + "offset": 1156952, + "align": 1 + }, + { + "name": ".shstrtab", + "type": 3, + "flags": 0, + "address": "0x0", + "size": 156, + "offset": 1225888, + "align": 1 + } + ], + "segments": [ + { + "offset": 52, + "virtual": "0x8000000", + "physical": "0x8000000", + "file_size": 36688, + "memory_size": 46956 + } + ] +} diff --git a/tests/golden/column_titles.svg b/tests/golden/column_titles.svg index f5ee255..37573f5 100644 --- a/tests/golden/column_titles.svg +++ b/tests/golden/column_titles.svg @@ -1,28 +1,28 @@ - - - + + + Figure 8. STM32F303x6/8 memory map - - - - -0x0000 0000 -0x2000 0000 -0x4000 0000 -0x6000 0000 -0x8000 0000 -0xA000 0000 -0xC000 0000 -0xE000 0000 + + + + +0x0000 0000 +0x2000 0000 +0x4000 0000 +0x6000 0000 +0x8000 0000 +0xA000 0000 +0xC000 0000 +0xE000 0000 0xFFFF FFFF -0 -1 -2 -3 -4 -5 -6 -7 +0 +1 +2 +3 +4 +5 +6 +7 0x4000 0000 @@ -44,42 +44,42 @@ 0x5000 03FF Peripherals - -0x0000 0000 - -0x0001 0000 - -0x0800 0000 - -0x0801 0000 - -0x1000 0000 - -0x1000 1000 - -0x1FFF D800 - -0x1FFF F800 - -0x1FFF FFFF -Code area - -CODE - -SRAM - - -Peripherals - - - - - - -Cortex-M4 -with FPU -Internal -Peripherals + +0x0000 0000 + +0x0001 0000 + +0x0800 0000 + +0x0801 0000 + +0x1000 0000 + +0x1000 1000 + +0x1FFF D800 + +0x1FFF F800 + +0x1FFF FFFF +Code area + +CODE + +SRAM + + +Peripherals + + + + + + +Cortex-M4 +with FPU +Internal +Peripherals APB1 @@ -98,26 +98,26 @@ Reserved AHB3 - -Flash, system -memory or SRAM, -depending on BOOT -configuration - -Reserved - -Flash memory - -Reserved - -CCM RAM - -Reserved - -System memory - -Option bytes - -Reserved -MSv33150V1 + +Flash, system +memory or SRAM, +depending on BOOT +configuration + +Reserved + +Flash memory + +Reserved + +CCM RAM + +Reserved + +System memory + +Option bytes + +Reserved +MSv33150V1 diff --git a/tests/golden/elf/r52_zephyr_philosophers.svg b/tests/golden/elf/r52_zephyr_philosophers.svg index 2337d10..6c3d280 100644 --- a/tests/golden/elf/r52_zephyr_philosophers.svg +++ b/tests/golden/elf/r52_zephyr_philosophers.svg @@ -1,13 +1,13 @@ - - - + + + r52_zephyr_philosophers.elf memory map - - - - - - + + + + + + 0x0000 0000 0x0000 003C 0x0000 6340 @@ -23,35 +23,35 @@ 0x7FFF FFFF DRAM 46.9 KB of 2 GB (<1%) - -0x0000 6340 - -0x0000 6348 - -0x0000 6380 - -0x0000 63A0 - -0x0000 63B7 -4 small sections in DRAM - -0x0000 67B8 - -0x0000 67C0 - -0x0000 67BD - -0x0000 67D7 -2 small sections in DRAM - -0x0000 BB78 - -0x0000 BBA4 - -0x0000 BBA6 - -0x0000 BBA9 -3 small sections in DRAM + +0x0000 6340 + +0x0000 6348 + +0x0000 6380 + +0x0000 63A0 + +0x0000 63B7 +4 small sections in DRAM + +0x0000 67B8 + +0x0000 67C0 + +0x0000 67BD + +0x0000 67D7 +2 small sections in DRAM + +0x0000 BB78 + +0x0000 BBA4 + +0x0000 BBA6 + +0x0000 BBA9 +3 small sections in DRAM rom_start 60 B @@ -85,31 +85,31 @@ Free ≈1.99 GB - -.ARM.exidx -8 B - -rodata_start -56 B - -initlevel -32 B - -devices -24 B - -device_handles -6 B - -log_const_sections -24 B - -datas -44 B - -device_states -2 B - -.last_section -4 B + +.ARM.exidx +8 B + +rodata_start +56 B + +initlevel +32 B + +devices +24 B + +device_handles +6 B + +log_const_sections +24 B + +datas +44 B + +device_states +2 B + +.last_section +4 B diff --git a/tests/golden/elf/rp2040_pico_rom.svg b/tests/golden/elf/rp2040_pico_rom.svg new file mode 100644 index 0000000..4b8a0d4 --- /dev/null +++ b/tests/golden/elf/rp2040_pico_rom.svg @@ -0,0 +1,84 @@ + + + +rp2040_pico_rom.elf memory map +0x1000 0000 +0x1000 0100 +0x1000 5BD0 +0x1000 5CC8 +0x1000 5CE8 +0x1000 5E64 +0x101F FFFF +Flash +23.6 KB of 2 MB (1%) +0x2000 0000 +0x2000 00C0 +0x2000 0240 +0x2000 023B +0x2000 8608 +0x2000 8E08 +0x2003 FFFF +RAM +35.5 KB of 256 KB (14%) +0x2004 0000 +0x2004 0FFF +SCRATCH_X +0 B of 4 KB (0%) +0x2004 1000 +0x2004 1800 +0x2004 1FFF +SCRATCH_Y +2 KB of 4 KB (50%) + +.boot2 +256 B + +.text +23248 B + +.rodata +248 B + +.binary_info +32 B + +.data (load) +380 B +copied to RAM at 0x2000 00C0 + +Free +≈1.97 MB + +.ram_vector_table +192 B + +not allocated in the ELF, as a (COPY) +section + +.data +380 B + +.bss +33736 B + +.heap +2 KB + +not allocated in the ELF, as a (COPY) +section + +Free +≈220 KB + +Free +4 KB + +.stack_dummy +2 KB + +not allocated in the ELF, as a (COPY) +section + +Free +2 KB + diff --git a/tests/golden/elf/rp2040_pico_rom.yaml b/tests/golden/elf/rp2040_pico_rom.yaml new file mode 100644 index 0000000..92348d3 --- /dev/null +++ b/tests/golden/elf/rp2040_pico_rom.yaml @@ -0,0 +1,56 @@ +# Generated by `memmap from-elf` from rp2040_pico_rom.elf. Section ranges come from +# the ELF and memory regions from --region. Rename, merge or annotate blocks freely. +version: 1 + +figure: + title: "rp2040_pico_rom.elf memory map" + +memory: + fill_gaps: none # gaps under --min-size show as a split edge + regions: + - id: "flash" + name: "Flash" + start: 0x10000000 + size: 2M + children: + - { id: "boot2", name: ".boot2", start: 0x10000000, size: 256 } + - { id: "text", name: ".text", start: 0x10000100, size: 23248 } + - { id: "rodata", name: ".rodata", start: 0x10005BD0, size: 248 } + - { id: "binary_info", name: ".binary_info", start: 0x10005CC8, size: 32 } + - { id: "data_load", name: ".data (load)", start: 0x10005CE8, size: 380, note: "copied to RAM at 0x2000 00C0" } + - { id: "flash_free", name: "Free\n≈1.97 MB", kind: reserved, start: 0x10005E64, size: 2072988 } + - id: "ram" + name: "RAM" + start: 0x20000000 + size: 256K + children: + - { id: "ram_vector_table", name: ".ram_vector_table", start: 0x20000000, size: 192, note: "not allocated in the ELF, as a (COPY) section" } + - { id: "data", name: ".data", start: 0x200000C0, size: 380 } + - { id: "bss", name: ".bss", start: 0x20000240, size: 33736 } + - { id: "heap", name: ".heap", start: 0x20008608, size: 2K, note: "not allocated in the ELF, as a (COPY) section" } + - { id: "ram_free", name: "Free\n≈220 KB", kind: reserved, start: 0x20008E08, size: 225784 } + - id: "scratch_x" + name: "SCRATCH_X" + start: 0x20040000 + size: 4K + children: + - { id: "scratch_x_free", name: "Free\n4 KB", kind: reserved, start: 0x20040000, size: 4K } + - id: "scratch_y" + name: "SCRATCH_Y" + start: 0x20041000 + size: 4K + children: + - { id: "stack_dummy", name: ".stack_dummy", start: 0x20041000, size: 2K, note: "not allocated in the ELF, as a (COPY) section" } + - { id: "scratch_y_free", name: "Free\n2 KB", kind: reserved, start: 0x20041800, size: 2K } + +view: + defaults: + labels: { size: true } + columns: + - { id: "flash_col", root: "flash", title: "Flash\n23.6 KB of 2 MB (1%)", width: 150 } + - { id: "ram_col", root: "ram", title: "RAM\n35.5 KB of 256 KB (14%)", width: 150 } + - { id: "scratch_x_col", root: "scratch_x", title: "SCRATCH_X\n0 B of 4 KB (0%)", width: 150 } + - { id: "scratch_y_col", root: "scratch_y", title: "SCRATCH_Y\n2 KB of 4 KB (50%)", width: 150 } + +style: + reserved: { show_label: true } # names the Free blocks diff --git a/tests/golden/elf/stm32l4_keil_rfe_master.svg b/tests/golden/elf/stm32l4_keil_rfe_master.svg new file mode 100644 index 0000000..201282c --- /dev/null +++ b/tests/golden/elf/stm32l4_keil_rfe_master.svg @@ -0,0 +1,38 @@ + + + +stm32l4_keil_rfe_master.elf memory map +0x0800 0000 +0x0800 8E7C +0x0800 8F50 +0x0807 FFFF +Flash +35.8 KB of 512 KB (7%) +0x2000 0000 +0x2000 01CC +0x2000 28F0 +0x2002 7FFF +SRAM +10.2 KB of 160 KB (6%) + +ER_IROM1 +36476 B + +RW_IRAM1 (load) +212 B + +compressed, unpacked to SRAM at 0x2000 +0000 + +Free +≈476 KB + +RW_IRAM1 (RW) +460 B + +RW_IRAM1 (ZI) +10020 B + +Free +≈149 KB + diff --git a/tests/golden/elf/stm32l4_keil_rfe_master.yaml b/tests/golden/elf/stm32l4_keil_rfe_master.yaml new file mode 100644 index 0000000..4d4a384 --- /dev/null +++ b/tests/golden/elf/stm32l4_keil_rfe_master.yaml @@ -0,0 +1,36 @@ +# Generated by `memmap from-elf` from stm32l4_keil_rfe_master.elf. Section ranges come from +# the ELF and memory regions from --region. Rename, merge or annotate blocks freely. +version: 1 + +figure: + title: "stm32l4_keil_rfe_master.elf memory map" + +memory: + fill_gaps: none # gaps under --min-size show as a split edge + regions: + - id: "flash" + name: "Flash" + start: 0x08000000 + size: 512K + children: + - { id: "er_irom1", name: "ER_IROM1", start: 0x08000000, size: 36476 } + - { id: "rw_iram1_load", name: "RW_IRAM1 (load)", start: 0x08008E7C, size: 212, note: "compressed, unpacked to SRAM at 0x2000 0000" } + - { id: "flash_free", name: "Free\n≈476 KB", kind: reserved, start: 0x08008F50, size: 487600 } + - id: "sram" + name: "SRAM" + start: 0x20000000 + size: 160K + children: + - { id: "rw_iram1_rw", name: "RW_IRAM1 (RW)", start: 0x20000000, size: 460 } + - { id: "rw_iram1_zi", name: "RW_IRAM1 (ZI)", start: 0x200001CC, size: 10020 } + - { id: "sram_free", name: "Free\n≈149 KB", kind: reserved, start: 0x200028F0, size: 153360 } + +view: + defaults: + labels: { size: true } + columns: + - { id: "flash_col", root: "flash", title: "Flash\n35.8 KB of 512 KB (7%)", width: 150 } + - { id: "sram_col", root: "sram", title: "SRAM\n10.2 KB of 160 KB (6%)", width: 150 } + +style: + reserved: { show_label: true } # names the Free blocks diff --git a/tests/golden/example_f303.svg b/tests/golden/example_f303.svg index bbca449..1602c1f 100644 --- a/tests/golden/example_f303.svg +++ b/tests/golden/example_f303.svg @@ -1,28 +1,28 @@ - - - + + + Figure 8. STM32F303x6/8 memory map - - - - -0x0000 0000 -0x2000 0000 -0x4000 0000 -0x6000 0000 -0x8000 0000 -0xA000 0000 -0xC000 0000 -0xE000 0000 + + + + +0x0000 0000 +0x2000 0000 +0x4000 0000 +0x6000 0000 +0x8000 0000 +0xA000 0000 +0xC000 0000 +0xE000 0000 0xFFFF FFFF -0 -1 -2 -3 -4 -5 -6 -7 +0 +1 +2 +3 +4 +5 +6 +7 0x4000 0000 @@ -43,41 +43,41 @@ 0x5000 0000 0x5000 03FF - -0x0000 0000 - -0x0001 0000 - -0x0800 0000 - -0x0801 0000 - -0x1000 0000 - -0x1000 1000 - -0x1FFF D800 - -0x1FFF F800 - -0x1FFF FFFF - -CODE - -SRAM - - -Peripherals - - - - - - -Cortex-M4 -with FPU -Internal -Peripherals + +0x0000 0000 + +0x0001 0000 + +0x0800 0000 + +0x0801 0000 + +0x1000 0000 + +0x1000 1000 + +0x1FFF D800 + +0x1FFF F800 + +0x1FFF FFFF + +CODE + +SRAM + + +Peripherals + + + + + + +Cortex-M4 +with FPU +Internal +Peripherals APB1 @@ -96,26 +96,26 @@ Reserved AHB3 - -Flash, system -memory or SRAM, -depending on BOOT -configuration - -Reserved - -Flash memory - -Reserved - -CCM RAM - -Reserved - -System memory - -Option bytes - -Reserved -MSv33150V1 + +Flash, system +memory or SRAM, +depending on BOOT +configuration + +Reserved + +Flash memory + +Reserved + +CCM RAM + +Reserved + +System memory + +Option bytes + +Reserved +MSv33150V1 diff --git a/tests/test_elf_snapshots.py b/tests/test_elf_snapshots.py index 9c85ba9..9563022 100644 --- a/tests/test_elf_snapshots.py +++ b/tests/test_elf_snapshots.py @@ -46,8 +46,11 @@ def test_snapshot_rebuilds_to_the_same_headers(name): snapshot = load(name) elf = read_elf(rebuild(snapshot)) wanted = [s for s in snapshot["sections"] if s["name"] != ".shstrtab"] - assert [(s.name, s.kind, s.flags, s.address, s.size) for s in elf.sections[: len(wanted)]] == [ - (s["name"], s["type"], s["flags"], int(s["address"], 16), s["size"]) for s in wanted + assert [ + (s.name, s.kind, s.flags, s.address, s.size, s.align) for s in elf.sections[: len(wanted)] + ] == [ + (s["name"], s["type"], s["flags"], int(s["address"], 16), s["size"], s["align"]) + for s in wanted ] assert elf.bits == snapshot["bits"] diff --git a/tests/test_from_elf.py b/tests/test_from_elf.py index 850980e..cb9050a 100644 --- a/tests/test_from_elf.py +++ b/tests/test_from_elf.py @@ -10,6 +10,7 @@ from memmap.cli import app from memmap.from_elf import ImportProblem, MemorySpec, import_elf, parse_region_spec from memmap.from_elf.elf import SHF_ALLOC, SHF_TLS, SHT_NOBITS, ElfError, read_elf +from memmap.from_elf.importer import UNALLOCATED_NOTE from memmap.pipeline import run FIXTURES = ROOT / "tests" / "fixtures" @@ -301,6 +302,102 @@ def sections(data_flags): assert '"Init data"' in import_elf(read_elf(make_elf(sections(SHF_ALLOC), segments)), "fw", []).text +def _armlink_elf(zi_start): + """An armlink-style image: one LOAD segment holding ER_IROM1, with the RW bytes of + RW_IRAM1 stored right after the code, and RW and ZI sections sharing one name.""" + rw = SHF_ALLOC | 0x1 # SHF_WRITE + sections = [ + ("ER_IROM1", PROGBITS, SHF_ALLOC | 0x4, 0x08000000, 0x100, 0x34), + ("RW_IRAM1", PROGBITS, rw, 0x20000000, 0x10, 0x134), + ("RW_IRAM1", SHT_NOBITS, rw, zi_start, 0x40, 0x144), + ] + return make_elf(sections, [(0x34, 0x08000000, 0x08000000, 0x110, 0x200)]) + + +ARMLINK_MEMORIES = [ + MemorySpec("er_irom1", 0x08000000, 0x1000, "ER_IROM1"), + MemorySpec("rw_iram1", 0x20000000, 0x1000, "RW_IRAM1"), +] + + +@pytest.mark.parametrize( + ("zi_start", "rw_size", "verb"), + [(0x20000010, 0x10, "copied to"), (0x20000020, 0x20, "compressed, unpacked to")], +) +def test_armlink_rw_data_is_drawn_in_ram_and_stored_after_the_code( + zi_start, rw_size, verb, tmp_path +): + """armlink stores RW data in the code's segment, and with compression its section header + gives the packed size: the RW data then runs up to the ZI data.""" + imported = import_elf(read_elf(_armlink_elf(zi_start)), "fw", ARMLINK_MEMORIES) + assert check_strict(imported.text, tmp_path) == [] + index = run(tmp_path / "layout.yaml").tree.index + ram = [(c.name, c.start, c.size) for c in index["rw_iram1"].children if c.kind != "reserved"] + assert ram == [("RW_IRAM1 (RW)", 0x20000000, rw_size), ("RW_IRAM1 (ZI)", zi_start, 0x40)] + flash = [(c.name, c.start, c.size, c.note) for c in index["er_irom1"].children] + assert flash[1] == ( + "RW_IRAM1 (load)", 0x08000100, 0x10, f"{verb} RW_IRAM1 at 0x2000 0000" + ) + assert flash[2][1] == 0x08000110 # free space starts after the stored RW data + + +@pytest.mark.parametrize( + "between", + [ + [("RW_IRAM1", SHT_NOBITS, SHF_ALLOC | 0x1, 0x20000010, 0x40, 0x140, 8)], # padding only + [ + code("STACK", 0x2000000C, 0x4, 0x140), # another section in the gap + ("RW_IRAM1", SHT_NOBITS, SHF_ALLOC | 0x1, 0x20000020, 0x40, 0x144), + ], + ], + ids=["alignment", "another_section"], +) +def test_rw_data_is_not_unpacked_over_padding_or_another_section(between): + """A gap before the ZI data that its alignment explains, or that another section fills, + is not compressed RW data.""" + sections = [ + ("ER_IROM1", PROGBITS, SHF_ALLOC | 0x4, 0x08000000, 0x100, 0x34), + ("RW_IRAM1", PROGBITS, SHF_ALLOC | 0x1, 0x20000000, 0xC, 0x134), + *between, + ] + data = make_elf(sections, [(0x34, 0x08000000, 0x08000000, 0x10C, 0x200)]) + text = import_elf(read_elf(data), "fw", ARMLINK_MEMORIES).text + assert '"RW_IRAM1 (RW)", start: 0x20000000, size: 12 }' in text + assert "copied to RW_IRAM1 at 0x2000 0000" in text + + +def test_section_names_lose_the_blanks_iar_leaves_around_them(): + data = make_elf([("P2 zi ", SHT_NOBITS, SHF_ALLOC, 0x100, 0x10, 0)]) + text = import_elf(read_elf(data), "fw", [MemorySpec("mem", 0, 0x1000, "MEM")]).text + assert 'name: "P2 zi",' in text + + +def test_unallocated_sections_inside_a_region_are_drawn_with_a_note(tmp_path): + """The Pico SDK declares its RAM vector table, heap and stacks as (COPY), which leaves + them without SHF_ALLOC: drawn when they lie in a region and overlap nothing allocated.""" + data = make_elf( + [ + code(".data", 0x20000100, 0x40), + (".ram_vector_table", PROGBITS, 0, 0x20000000, 0xC0, 0), + (".heap", PROGBITS, 0, 0x20000200, 0x800, 0), + (".shadow", PROGBITS, 0, 0x20000120, 0x10, 0), # overlaps .data + (".outside", PROGBITS, 0, 0x30000000, 0x10, 0), # in no region + (".symbols", 2, 0, 0x20000300, 0x10, 0), # SHT_SYMTAB, not memory + (".debug_info", PROGBITS, 0, 0, 0x500, 0), + ] + ) + imported = import_elf(read_elf(data), "fw", [MemorySpec("ram", 0x20000000, 0x4000, "RAM")]) + assert imported.diagnostics == [] + assert check_strict(imported.text, tmp_path) == [] + ram = run(tmp_path / "layout.yaml").tree.index["ram"] + drawn_sections = [(c.name, c.note) for c in ram.children if c.kind != "reserved"] + assert drawn_sections == [ + (".ram_vector_table", UNALLOCATED_NOTE), + (".data", None), + (".heap", UNALLOCATED_NOTE), + ] + + def test_several_banks_get_a_column_each_and_copies_name_their_bank(tmp_path): """Like an STM32H7: code in flash and ITCM, data in DTCM and AXI SRAM, SRAM4 unused.""" run_code, data = SHF_ALLOC | 0x4, SHF_ALLOC | 0x1 # executable, writable diff --git a/tests/test_layout.py b/tests/test_layout.py index 1fd075e..3238f48 100644 --- a/tests/test_layout.py +++ b/tests/test_layout.py @@ -5,7 +5,9 @@ from conftest import MODES, ROOT, codes, drawn, regions from memmap.describe import describe_data -from memmap.layout.primitives import Line +from memmap.layout.columns import LANE_STACK_GAP, MAX_STRETCH +from memmap.layout.primitives import Line, Polyline +from memmap.layout.routing import _clear, column_box from memmap.layout.text import text_width, wrap from memmap.options import OptionResolver from memmap.pipeline import run @@ -393,3 +395,173 @@ def test_a_note_line_break_is_kept(load): text = regions(' - id: m\n size: 4K\n note: "one\\ntwo"') notes = [t.text for t in drawn(load(text)).texts() if t.role == "note"] assert notes == ["one", "two"] + + +def _zoom_config(lanes=""): + """Two regions with 16 peripherals each, `low` at 0 and `high` at 64K, both zoomed.""" + + def region(name, start): + children = "\n".join(f" - {{ id: {name}{i}, size: 1K }}" for i in range(16)) + head = f" - id: {name}\n start: {start:#x}\n size: 16K\n" + return f"{head} children:\n{children}" + + body = regions(region("low", 0) + "\n" + region("high", 0x10000)) + return body + "view:\n defaults:\n scale: { mode: equal, min_height: 24 }\n" + lanes + + +def test_stacked_zooms_follow_address_order_not_definition_order(load): + """Both detail columns are taller than their sources allow, so both want the top of the + drawing; the one zoomed from higher addresses still goes on top.""" + result = load(_zoom_config(" expand: [low, high]\n")) + assert codes(result) == [] + columns = {layout.id: layout for layout in drawn(result).columns} + assert columns["high_detail"].y < columns["low_detail"].y + + +def test_a_zoom_two_lanes_away_goes_around_the_column_between(load): + lanes = ( + " columns:\n - { id: main, root: '/' }\n" + " - { id: high_detail, root: high, lane: 1, anchor: top }\n" + " - { id: low_detail, root: low, lane: 2, anchor: bottom }\n" + " expand:\n" + " - { region: high, from: main, to: high_detail }\n" + " - { region: low, from: main, to: low_detail }\n" + ) + result = load(_zoom_config(lanes)) + assert codes(result) == [] + columns = {layout.id: layout for layout in drawn(result).columns} + between = column_box(columns["high_detail"]) + assert columns["low_detail"].y >= between.bottom + fans = [p.points for p in drawn(result).primitives if isinstance(p, Polyline)] + assert [len(points) for points in fans] == [2, 2, 3, 3] + assert _clear(fans[2:], [between]) # the first two end at the column between + link = describe_data(result)["connectors"][1] + assert link["to"] == "low_detail" and len(link["lines"][0]) == 3 + + +def test_fan_lines_through_a_column_warn_when_the_detail_column_is_pinned(load): + lanes = ( + " columns:\n - { id: main, root: '/' }\n" + " - { id: high_detail, root: high, lane: 1, anchor: top }\n" + " - { id: low_detail, root: low, lane: 2, anchor: free }\n" + " expand:\n" + " - { region: high, from: main, to: high_detail }\n" + " - { region: low, from: main, to: low_detail }\n" + ) + result = load(_zoom_config(lanes)) + assert codes(result) == ["W107"] + assert "lane 1" in result.diagnostics[0].hint + + +def test_a_column_zooming_into_a_much_taller_one_is_stretched(load): + """The overview has three 24 px blocks and its zoom sixteen: the overview grows to + height of the zoom, capped at MAX_STRETCH times its own height.""" + result = load(_zoom_config(" columns:\n - { id: main, root: '/' }\n expand: [low]\n")) + assert codes(result) == [] + columns = {layout.id: layout for layout in drawn(result).columns} + main, detail = columns["main"], columns["low_detail"] + natural = 24 * len(main.blocks) + assert main.height == pytest.approx(min(MAX_STRETCH * natural, detail.height)) + assert len({round(b.h, 6) for b in main.blocks}) == 1 # equal scale stays equal + assert detail.height == 16 * 24 # the zoom itself keeps its scale + + +@pytest.mark.parametrize( + "expand", + ["[low, high]", "[{ region: low, title: Low }, { region: high, title: High }]"], + ids=["untitled", "titled"], +) +def test_a_stack_of_zooms_that_runs_past_the_bottom_moves_up(load, expand): + """Both zooms hang from small blocks in a short overview, so stacked from where they come + from they would run past the bottom of the drawing: the lane moves up to the header row, + where the top one's title sits beside the overview's.""" + result = load(_zoom_config(f" expand: {expand}\n")) + columns = {layout.id: layout for layout in drawn(result).columns} + main, high, low = columns["main"], columns["high_detail"], columns["low_detail"] + assert high.y == pytest.approx(main.y) + assert low.y == pytest.approx(high.y + high.height + LANE_STACK_GAP + low.title_height) + + +def test_a_column_is_only_stretched_for_its_own_zooms(load): + """The overview zooms into a short column; a tall column elsewhere, which it does not zoom + into, leaves it at its own height.""" + children = "\n".join(f" - {{ id: b{i}, size: 1K }}" for i in range(30)) + body = regions( + " - id: small\n start: 0\n size: 2K\n children:\n" + " - { id: s0, size: 1K }\n - { id: s1, size: 1K }\n" + f" - id: big\n start: 0x10000\n size: 30K\n children:\n{children}" + ) + text = body + ( + "view:\n defaults:\n scale: { mode: equal, min_height: 24 }\n" + " columns:\n - { id: main, root: '/' }\n - { id: big_col, root: big, lane: 2 }\n" + " expand: [small]\n" + ) + result = load(text) + assert codes(result) == [] + columns = {layout.id: layout for layout in drawn(result).columns} + assert columns["big_col"].height == 30 * 24 + assert columns["main"].height == 24 * len(columns["main"].blocks) + + +def _small_regions(*specs): + """Regions at 64K steps from 0, each with `count` 1K children: specs are (id, count).""" + parts = [] + for index, (name, count) in enumerate(specs): + children = "\n".join(f" - {{ id: {name}{i}, size: 1K }}" for i in range(count)) + head = f" - id: {name}\n start: {index * 0x10000:#x}\n size: {count}K\n" + parts.append(f"{head} children:\n{children}") + return regions("\n".join(parts)) + + +def test_a_stack_with_an_offset_is_not_moved_up(load): + """The offset pushes the zoom past the bottom of the drawing on purpose: it stays there.""" + view = ( + "view:\n defaults:\n scale: { mode: equal, min_height: 40 }\n" + " columns:\n - { id: main, root: '/' }\n" + " - { id: flash_d, root: flash, anchor: bottom, offset: 60 }\n" + " expand:\n - { region: flash, from: main, to: flash_d }\n" + ) + result = load(_small_regions(("flash", 2), ("ram", 1)) + view) + columns = {layout.id: layout for layout in drawn(result).columns} + main, detail = columns["main"], columns["flash_d"] + assert detail.y == pytest.approx(main.y + main.height - detail.height + 60) + + +def test_a_detour_stays_clear_below_the_columns_stacked_above_it(load): + """`b_d` takes the room above `m_d`, so `a_d` cannot go around above it as well and goes + around below instead of being pushed back into its way.""" + view = ( + "view:\n defaults:\n scale: { mode: equal, min_height: 24 }\n" + " columns:\n - { id: main, root: '/' }\n" + " - { id: m_d, root: m, lane: 1, anchor: top, offset: 160 }\n" + " - { id: a_d, root: a, lane: 2 }\n - { id: b_d, root: b, lane: 2 }\n" + " - { id: t, root: tall, lane: 3 }\n" + " expand:\n - { region: a, from: main, to: a_d }\n" + " - { region: m, from: main, to: m_d }\n - { region: b, from: main, to: b_d }\n" + ) + result = load(_small_regions(("a", 1), ("m", 3), ("b", 1), ("tall", 24)) + view) + assert codes(result) == [] + columns = {layout.id: layout for layout in drawn(result).columns} + assert columns["b_d"].y < columns["m_d"].y < columns["a_d"].y + + +def test_a_chain_of_zooms_stretches_the_overview_to_the_grown_detail(load): + """main zooms into flash, flash into app: flash grows first, then main grows to match it.""" + apps = "\n".join(f" - {{ id: a{i}, size: 1K }}" for i in range(12)) + body = regions( + " - id: flash\n start: 0\n size: 1M\n children:\n" + " - { id: boot, size: 64K }\n" + f" - id: app\n size: 256K\n children:\n{apps}\n" + " - { id: rest, size: auto }\n" + " - { id: cfg, size: 16K }\n" + " - { id: ram, start: 0x20000000, size: 64K }\n" + ) + view = ( + "view:\n defaults:\n scale: { mode: equal, min_height: 28 }\n" + " columns:\n - { id: main, root: '/' }\n expand: [flash, app]\n" + ) + result = load(body + view) + assert codes(result) == [] + columns = {layout.id: layout for layout in drawn(result).columns} + assert columns["flash_detail"].height == pytest.approx(columns["app_detail"].height) + assert columns["main"].height == pytest.approx(columns["flash_detail"].height) diff --git a/tests/test_routing.py b/tests/test_routing.py new file mode 100644 index 0000000..a0f00a2 --- /dev/null +++ b/tests/test_routing.py @@ -0,0 +1,81 @@ +"""Fan lines that skip a lane: the geometry helpers in memmap.layout.routing.""" + +import pytest + +from memmap.layout.routing import ( + CLEARANCE, + LINE_GAP, + Box, + Fan, + _crosses, + clear_top, + route, +) + +# A column between source and target, spanning y 100..500 at x 300..400. +BETWEEN = [Box(300, 100, 400, 500)] + + +def fan(target_top, height=100, source=(200, 300, 340), target_x=600): + """A fan from a block's right edge at source[0], y source[1]..source[2], to a detail + column whose left edge is at target_x.""" + x, top, bottom = source + return Fan(x, top, bottom, target_x, target_top, target_top + height) + + +def test_fan_lines_stay_straight_when_they_miss_the_columns_between(): + lines = route(fan(0, height=50, source=(200, 0, 40)), BETWEEN) + assert lines == [[(200, 0), (600, 0)], [(200, 40), (600, 50)]] + + +def test_fan_lines_bend_below_the_columns_between(): + lines = route(fan(520), BETWEEN) + corner = 300 - CLEARANCE + assert lines == [ + [(200, 300), (corner, 500 + CLEARANCE), (600, 520)], + [(200, 340), (corner, 500 + CLEARANCE + LINE_GAP), (600, 620)], + ] + + +def test_fan_lines_bend_above_the_columns_between(): + lines = route(fan(-30, height=100), BETWEEN) + assert lines is not None + assert [line[1] for line in lines] == [ + (300 - CLEARANCE, 100 - CLEARANCE - LINE_GAP), + (300 - CLEARANCE, 100 - CLEARANCE), + ] + + +def test_fan_lines_bend_on_the_far_side_for_a_zoom_to_the_left(): + lines = route(Fan(700, 300, 340, 100, 520, 620), BETWEEN) + assert lines is not None + assert lines[0][1] == (400 + CLEARANCE, 500 + CLEARANCE) + + +@pytest.mark.parametrize( + "lanes_fan", + [ + fan(250), # the detail column sits level with the column between + fan(520, source=(350, 300, 340)), # the source edge is inside the column between + ], + ids=["level", "no_room"], +) +def test_fan_lines_that_cannot_go_around_give_none(lanes_fan): + assert route(lanes_fan, BETWEEN) is None + + +def test_a_detail_column_moves_the_shorter_way_round(): + assert clear_top(fan(250), BETWEEN, header=-50) == 500 + CLEARANCE # 258 down vs 264 up + up = 100 - CLEARANCE - LINE_GAP - 100 + assert clear_top(fan(150), BETWEEN, header=-50) == up # 164 up vs 358 down + assert clear_top(fan(150), BETWEEN, header=up + 1) == 500 + CLEARANCE # no room above + + +def test_a_detail_column_that_is_already_clear_stays(): + assert clear_top(fan(520), BETWEEN, header=0) == 520 + assert clear_top(fan(250), [], header=0) == 250 + + +def test_a_vertical_segment_crosses_a_box_it_runs_through(): + assert _crosses((350, 0), (350, 600), BETWEEN[0]) + assert not _crosses((350, 0), (350, 90), BETWEEN[0]) diff --git a/tests/test_validate.py b/tests/test_validate.py index 9dea8e0..176998d 100644 --- a/tests/test_validate.py +++ b/tests/test_validate.py @@ -264,7 +264,8 @@ def test_expansion_finds_the_column_whose_range_shows_the_region(load): text = clipped("b").replace(" expand:", " - { id: d, root: m }\n expand:") result = load(text) assert codes(result) == [] - assert drawn(result).links == [{"region": "b", "from": "d", "to": "b_detail"}] + links = [{key: link[key] for key in ("region", "from", "to")} for link in drawn(result).links] + assert links == [{"region": "b", "from": "d", "to": "b_detail"}] def test_e023_duplicate_column(load):