An ultra-lightweight, hardware-agnostic embedded vision subsystem designed for real-time edge computing, autonomous object mapping, and telemetry visualization.
AURA is engineered specifically within the ESA OSIP framework as a direct software solution to meet the core objectives of the Hera Extended Mission Phase (Autonomous Software Experiments on Hera). Developed for execution on the spacecraft's second processor core (Core 1), the system operates within a protected sandbox environment alongside flight-critical systems, achieving Technology Readiness Level 4 (TRL 4) validation.
AURA addresses the precise limitations of deep-space operations highlighted by the European Space Agency - specifically dealing with severe communication constraints, intermittent signals, and command delays reaching up to 40 minutes during close-proximity operations around the asteroid moon Dimorphos.
- The Challenge: Deep-space missions cannot rely on continuous ground intervention during critical proximity operations.
- AURA's Solution: By tracking high-contrast features through local mathematical entropy, AURA provides deterministic, autonomous navigation support. It operates independently of ground control, allowing the spacecraft to maintain situational awareness even when Earth is silent.
- The Challenge: Deep-space telemetry suffers from extremely limited downlink volume and precious bandwidth.
- AURA's Solution: The system processes asteroid imagery locally at the edge. By computing dynamic internal heatmaps, AURA flags anomalies and isolates regions of high scientific value. This enables smart data prioritisation, ensuring the spacecraft downlinks only high-value, highly compressed visual metadata rather than massive raw image payloads.
- The Challenge: Experimental payloads must execute under non-continuous windows (2–3 hours per day) without compromising the core spacecraft control logic (Core 0).
- AURA's Solution: Built with zero dynamic memory allocation (
malloc), AURA enforces absolute execution determinism and software stability by design. It operates seamlessly within a memory-protected region, guarantees zero heap fragmentation, and natively tolerates abrupt, automated shutdowns triggered by spacecraft anomalies or Safe Mode transitions as standard operating procedures.
🧬 Project Evolution & AI Compilation TimelineThe architecture of AURA is the result of a multi-stage evolutionary porting process:
- The Python Baseline: The initial mathematical concept was modeled in the open-source prototype Entropy-Image-Prioritization, which verified the use of multivariate Shannon entropy for rapid image block prioritization using Python (OpenCV/NumPy).
- The Microcontroller Proof-of-Concept: The algorithm was then successfully ported to C/C++ and verified under tight memory and clock constraints on a cheap commercial chip inside the
EIP_ESP32S3stack.3. The Aerospace Flight Grade Port: For the ESA OSIP campaign, the pipeline was fully rewritten into low-level, high-reliability C code optimized specifically for the radiation-hardened LEON3 (SPARC V8) processor architecture, removing all high-level dependencies, floating-point variables, and dynamic allocations.
This final flight-grade iteration was 100% generated via conversational AI (Vibe Coding) utilizing the Google AI interface over a targeted 7-day hobby sprint with zero operational budget. The AI was treated as a High-Level Functional Compiler, translating explicit logical boundaries, memory rules, and aerospace mathematics into verified, bare-metal C code.
To comply with strict aerospace software engineering standards (ECSS Category D) and target space-grade hardware specifications, the software pipeline is bound by the following low-level operational limits:
- Hardware Architecture: 32-bit SPARC V8 (LEON3) processor core executing at a flight-representative 250 MIPS (Tested on GR712RC Dual-Core configuration).
-
Memory Restrictions: Strict 16 MB RAM static partition sandbox. Dynamic memory allocation (
malloc,free) is entirely omitted to enforce execution determinism. -
Sensor Interfacing & Data Source: Interfaced with a simulated Hera AFC navigation camera utilizing a monochrome FaintStar2 sensor configuration: 1020x1020 pixels, strict 8-bit Grayscale (1 byte per pixel, total raw frame size: 1,040,400 bytes).
-
Note on Visual Assets: All raw flight matrices (
image.bin) are extracted directly from the official ESA Hera dataset (AFC images.tar.gz) provided via the OSIP campaign platform.
-
Note on Visual Assets: All raw flight matrices (
-
Zero Floating-Point Unit (FPU) Overhead: Fixed-point integer mathematical models completely replace standard floating-point functions (
float,double,log2f). Logarithmic probabilities are resolved using ultra-fast bitwise arithmetic. -
Histogram Footprint Optimization: Features a dedicated tracking stack that enables precise, point-by-point clearing of modified memory indexes. This bounds clearing operations to
$O(N)$ efficiency (where$N$ is the count of active grayscale channels per block), maintaining internal CPU cache efficiency.
Data serialization circumvents human-readable ASCII or string parsing inside the real-time processing loop. Instead, the firmware packs localized statistical telemetry directly into high-density 32-bit unsigned integer registers (uint32), allocating data parameters down to the exact bit level:
| Bit Range | Size (Bits) | Description |
|---|---|---|
| [31:24] | 8 | Synchronization / Data frame identifier marker (0xA5). |
| [23:17] | 7 | Column Index (col_idx), representing block X-coordinate coordinate layout (up to 2048 px). |
| [16:10] | 7 | Row Index (row_idx), representing block Y-coordinate coordinate layout (up to 1536 px). |
| [9:0] | 10 | Scaled Shannon Entropy value ( |
- Trap & Exception Mitigation: The packed 32-bit words are transmitted over the physical interface byte-by-byte via sequential register flushing. This prevents unaligned word memory access anomalies, completely eliminating the risk of critical processor exceptions (SPARC Trap 0x07 / Data Access Alignment Trap).
- Performance Metrics: Telemetry validation reports an 85.2% absolute lossless reduction in downlink data volume compared to baseline text streaming, achieving a 6.75x bandwidth optimization factor over the telemetry link.
Hello_AURA.c— The standalone core flight software application executing the fixed-point block entropy pipeline.experiment_test.elf— The final compiled space-grade executable binary containing embedded image matrices.image.bin— The raw 8-bit monochrome binary matrix extracted for hardware memory direct mapping (0x40600000). Sourced from ESA's officialAFC images.tar.gzdataset.leon3.repl— The Renode hardware platform description file enforcing the exact 16 MB memory map layout.script.resc— The automation deployment script establishing the socket bindings and CPU clock performance.telemetry_live_visualizer.py— The Ground Segment analytics visualizer decoding binary masks into a real-time heatmap.
To set up the Software-in-the-Loop (SIL) simulation environment on Windows, you must install the official ESA/Gaisler cross-compiler and the Renode emulation framework.
1. Install Aeroflex Gaisler BCC2 ToolchainThe flight software requires the BCC2 (Bare-metal C Compiler V2) based on GCC for SPARC architectures.
- Download the Mingw64 build of BCC2 from the official Cobham Gaisler website (bcc-2.2.3-mingw64).
- Extract the package so that the path matches the compiler execution script exactly:
C:\Projects\bcc-2.2.3-mingw64\bcc-2.2.3-gcc\
- (Optional) Add
C:\Projects\bcc-2.2.3-mingw64\bcc-2.2.3-gcc\binto your system environment variables (PATH) to callsparc-gaisler-elf-gccnatively.
Renode is used to emulate the GR712RC Dual-Core LEON3 SoC on a bit-level scale.
- Download the latest Windows installer (
.exe) or portable production package from the official Renode site. - Complete the standard setup wizard.
- Ensure that the
renodesystem command is mapped to your systemPATH. Open PowerShell and verify the version payload:
renode --versionRecommended Project Workspace Path: C:\Projects\AURA-main\
All command line steps below assume that your terminal is opened and executing from the project root directory (cd C:\Projects\AURA-main\).
To recompile the flight software from source using the official Aeroflex Gaisler BCC2 cross-compiler toolchain, execute the following multi-stage compilation pipeline within a Windows PowerShell terminal opened at C:\Projects\AURA-main\:
# Compiling and linking the standalone AURA firmware directly into an ELF image with memory alignment
& "C:\Projects\bcc-2.2.3-gcc-mingw64\bcc-2.2.3-gcc\bin\sparc-gaisler-elf-gcc.exe" -O2 -g Hello_AURA.c -o experiment_test.elf "-Wl,-Ttext=0x40000000" "-Wl,-z,muldefs" -lgccIn the primary command terminal opened at C:\Projects\AURA-main\, initiate the software-in-the-loop validation inside the Renode environment to boot the LEON3 processor and start streaming data:
renode .\script.resc
Once the emulation starts running and the virtual spacecraft begins processing frames, open a separate terminal window at C:\Projects\AURA-main\ and launch the telemetry live decoder to bind to the active stream:
python .\telemetry_live_visualizer.py
Upon connection, the onboard application will continue processing the 1020x1020 image grids, routing the compiled binary stream dynamically over the loopback interface (127.0.0.1:12345) to render a real-time mathematical heatmap of the asteroid terrain in the Ground Segment visualizer window.
For immediate hardware-in-the-loop and live vision pipeline exploration without setting up the local cross-compilation toolchain, an interactive web prototype is deployed on Google AI Studio.
https://www.youtube.com/shorts/gKMWbWaEmZ0
You can access the environment directly via: 👉 AURA Vision Concept Tracker (Google AI Studio App) (Requires a valid Google AI Studio login).
- Real-time Camera Injection: Turn on the built-in webcam or pair your smartphone's camera interface to stream live physical optical matrices directly into the tracking pipeline.
- Configurable Spatial Filtering: Dynamically fine-tune block dimensions, stride lengths, and mathematical scaling factors to analyze the entropy and heatmap response instantly.
- Deterministic Logic Verification: Observe how the hardware constraints and fixed-point optimizations handle dynamic real-world environments before flight compilation.
AURA is evolving along four complementary research and engineering branches. All branches originate from the same core principle:
Information should be processed according to its informational value rather than treating every pixel equally.
The roadmap below reflects the current development priorities and long-term vision of the project.
The primary objective is reducing deep-space telemetry requirements by transmitting increasingly detailed representations only when necessary.
Pipeline concept:
Raw Image
↓
Feature Point Cloud (~1 KB)
↓
Entropy Heatmap (~20 KB)
↓
Full Image (~1 MB)
- Detect local contrast maxima.
- Generate sparse feature point clouds.
- Reduce downlink requirements through hierarchical data products.
- Allow ground operators to decide whether additional data transmission is justified.
- Prioritise scientifically valuable observations before transmitting complete image frames.
A spacecraft can transmit a compact informational summary first, significantly reducing unnecessary bandwidth consumption during deep-space operations.
Identify regions that contribute little scientific value and exclude them from further processing.
Examples include:
- Deep-space background
- Uniform sky regions
- Low-detail surfaces
- Sensor noise dominated areas
- Automatically classify low-information regions.
- Reduce computational load on embedded processors.
- Focus processing resources on scientifically relevant structures.
- Improve overall onboard decision efficiency.
More processing power becomes available for high-value observations while reducing unnecessary calculations.
Investigate whether entropy-derived feature structures can support autonomous spacecraft navigation.
Instead of analysing complete image frames, AURA may operate on compact informational representations extracted from the scene.
Pipeline concept:
Image
↓
Entropy Analysis
↓
Feature Point Extraction
↓
Constellation Generation
↓
Inter-Frame Tracking
↓
Motion Estimation
↓
Navigation Support
AURA can represent high-information regions as connected structures ("feature constellations").
These constellations:
- Remain linked to physical scene features.
- Move consistently across image sequences.
- May provide a lightweight representation for visual motion estimation.
- Could support future visual odometry and autonomous navigation experiments.
Potential applications include:
- Relative motion estimation
- Surface tracking
- Landmark recognition
- Autonomous proximity operations around asteroids and small bodies
Can entropy-derived feature constellations provide a computationally efficient alternative or complement to traditional feature-tracking approaches used in spacecraft visual navigation?
Extend AURA beyond image processing by incorporating additional spacecraft sensors into a unified informational framework.
Potential data sources:
- Optical cameras
- Infrared sensors
- Laser ranging systems
- IMU measurements
- Star trackers
- Thermal sensors
- Scientific payload instruments
- Compute multidimensional joint entropy across heterogeneous sensor inputs.
- Detect environmental changes through information-state transitions.
- Trigger autonomous decision-making processes.
- Build a generalized information-awareness layer for future spacecraft systems.
A previously empty field of view suddenly contains an object.
Simultaneously:
- Optical entropy increases.
- Infrared readings change.
- Range measurements become available.
The combined entropy signature indicates a significant environmental event and can automatically trigger higher-level mission logic.
AURA evolves from an image-processing subsystem into a general-purpose information-driven perception framework for autonomous space systems.
Future development may ultimately lead toward:
- Autonomous onboard scientific prioritisation
- Information-driven spacecraft decision support
- Vision-assisted guidance, navigation and control (GNC)
- Adaptive telemetry generation
- Self-directed observation planning
The long-term vision of AURA is to transform raw sensor streams into compact, actionable informational representations suitable for resource-constrained deep-space missions.
☝️ Disclaimer
AURA is an independent research / engineering prototype. References to ESA, Hera, LEON3, GR712RC, or related mission and hardware documentation are used for context and technical compatibility only. This repository does not imply endorsement, certification, sponsorship, or official affiliation unless explicitly stated by the respective organisation.
The TRL 4 designation describes the current maturity of the demonstrated technology and should not be interpreted as flight qualification, mission acceptance, or operational certification.
Previous prototype versions used ESA-provided interfaces and publicly available Autonomous Software Experiments on Hera campaign datasets during evaluation. The current version is a standalone implementation and no longer redistributes those files.
This project is open-source and released under the terms of the MIT License.