Chinese version please click here
A Version-Controlled AI-Assisted UMG Workflow
Demo Recreating the UE5 editor
Demo Recreating the UE5 editor in UMG editor
Chat with Gemini 3 to editor the UMG file
This guide covers the two-step process to install the UmgMcp plugin and connect it to your Gemini CLI.
- Prerequisite: Unreal Engine 5.6 or newer.
-
Navigate to your project's Plugins folder:
YourProject/Plugins/(create it if it doesn't exist). -
Clone the repository directly into this directory:
git clone https://github.com/winyunq/UnrealMotionGraphicsMCP.git UmgMcp
-
Restart the Unreal Editor. This allows the engine to detect and compile the new plugin.
Tell Gemini how to find and launch the MCP server.
-
Edit your
settings.jsonfile (usually located atC:\Users\YourUsername\.gemini\). -
Add the tool definition to the
mcpServersobject."mcpServers": { "UmgMcp": { "command": "uv", "args": [ "run", "--directory", "D:\\Path\\To\\YourUnrealProject\\Plugins\\UmgMcp\\Resources\\Python", "UmgMcpServer.py" ] } }
IMPORTANT: You must replace the path with the correct absolute path to the
Resources/Pythonfolder from the cloned repository on your machine.
That's it! When you start the Gemini CLI, it will automatically launch the MCP server in the background.
After restarting your Gemini CLI and opening your Unreal project, you can test the connection by calling any tool function:
cd Resources/Python/APITest
python UE5_Editor_Imitation.pyThe plugin's Python environment is managed by uv. In most cases, it should work automatically. If you encounter issues related to Python dependencies (e.g., uv command not found or module import errors), you can manually set up the environment:
- Navigate to the directory:
cd YourUnrealProject/Plugins/UmgMcp/Resources/Python - Run the setup:
uv venv .\.venv\Scripts\activate uv pip install -e .
We are experimenting with the Gemini CLI Skill system as an alternative to the standard MCP approach.
The Skill architecture allows the Python tools to be loaded directly by the CLI runtime, potentially optimizing context usage by dynamically enabling/disabling tools via prompts.json and avoiding the overhead of managing a separate MCP server process.
Note: The MCP server (configured above) is still the stable and recommended way to use this plugin. Use Skill mode if you want to test the latest integration capabilities.
To enable Skill mode, add the following to your settings.json (replacing <YOUR_PROJECT_PATH>):
"skills": {
"unreal_umg": {
"path": "<YOUR_PROJECT_PATH>/Plugins/UmgMcp/Resources/Python/UmgMcpSkills.py",
"type": "local",
"description": "Direct control of Unreal Engine UMG via Python Skills. Auto-loads tools from prompts.json."
}
},This project provides a powerful, command-line driven workflow for managing Unreal Engine's UMG UI assets. By treating human-readable .json files as the sole Source of Truth, it fundamentally solves the challenge of versioning binary .uasset files in Git.
Inspired by tools like blender-mcp, this system allows developers, UI designers, and AI assistants to interact with UMG assets programmatically, enabling true Git collaboration, automated UI generation, and iteration.
A visual web tool for configuring system instructions, tool descriptions, and user prompt templates.
- System Instruction Editor: Modify the global instructions for the AI context.
- Tool Management:
- Enable/Disable: Toggle specific MCP tools on or off. Disabled tools are not registered with the MCP server, effectively compressing the context window to prevent AI distraction.
- Edit Descriptions: Customize tool descriptions (prompts) to better suit your workflow.
- User Templates (Prompts): Add reusable prompt templates for quick access by the MCP client.
Execute the following command in your Python environment:
python Resources/Python/PromptManager/server.pyThe browser will automatically open http://localhost:8085γ
Prompts are crucial for AI tool effectiveness. Use the Prompt Manager to tailor the AI's behavior:
- One-Click Deployment Mode: If you want the AI to focus solely on generating UI from design, disable all tools except
apply_layoutandexport_umg_to_json. - Tutor Mode: If you want the AI to guide you without making changes, keep only read-only tools (e.g.,
get_widget_tree,get_widget_schema). - Context Optimization: For models with smaller context windows, disable tools you aren't currently using to improve speed and accuracy.
Contributions of effective prompt configurations are welcome!
This project has been developed with significant assistance from Gemini, an AI. As such:
- Experimental Nature: This is an experimental project. Its reliability is not guaranteed.
- Commercial Use: Commercial use is not recommended without thorough independent validation and understanding of its limitations.
- Disclaimer: Use at your own risk. The developers and AI are not responsible for any consequences arising from its use.
The system now primarily relies on the UE5_UMG_MCP plugin for communication between external clients (like this CLI) and the Unreal Engine Editor.
Architecture Diagram:
flowchart LR
subgraph "Local Execution Environment"
CLI["Gemini CLI"] --"StdIO (JSON-RPC)"--> PY["Python (UmgMcpServer.py)"]
end
subgraph "Unreal Engine 5"
PY --"TCP Socket (JSON)"--> TCP["UmgMcpBridge (C++)"]
TCP --> API["Unreal API / UMG"]
end
| Category | API Name | Status |
|---|---|---|
| Context & Attention | get_target_umg_asset |
β |
set_target_umg_asset |
β | |
get_last_edited_umg_asset |
β | |
get_recently_edited_umg_assets |
β | |
| Sensing & Querying | get_widget_tree |
β |
query_widget_properties |
β | |
get_creatable_widget_types |
β | |
get_widget_schema |
β | |
get_layout_data |
β | |
check_widget_overlap |
β | |
| Actions & Modifications | create_widget |
β |
delete_widget |
β | |
set_widget_properties |
β | |
reparent_widget |
β | |
save_asset |
β | |
| File Transformation | export_umg_to_json |
β |
apply_json_to_umg |
β | |
apply_layout |
β |
| Category | API Name | Status | Description |
|---|---|---|---|
| Context & Attention | set_edit_function |
β | Set the current edit context (Function/Event). Supports auto-creating Custom Events. |
set_cursor_node |
β | Explicitly set the "Cursor" node (Program Counter). | |
| Sensing & Querying | get_function_nodes |
β | Get nodes in Current Context Scope (Filtered to connected graph to avoid global noise). |
get_variables |
β | Get list of member variables. | |
search_function_library |
β | Search callable libraries (C++/BP). Supports Fuzzy Search. | |
| Actions & Modifications | add_step(name) |
β | Core: Add executable node by Name (e.g. "PrintString"). Auto-Wiring & Auto-Layout supported. |
prepare_value(name) |
β | Add Data Node by Name (e.g. "MakeLiteralString", "GetVariable"). | |
connect_data_to_pin |
β | Connect pins precisely (Supports NodeID:PinName format). |
|
add_variable |
β | Add new member variable. | |
delete_variable |
β | Delete member variable. | |
delete_node |
β | Delete specific node. | |
compile_blueprint |
β | Compile and apply changes. |
| Command | Status | Description |
|---|---|---|
set_animation_scope |
β Implemented | Set the target animation for subsequent commands |
set_widget_scope |
β Implemented | Set the target widget for subsequent commands |
get_all_animations |
β Implemented | Get list of all animations in the blueprint |
create_animation |
β Implemented | Create a new animation |
delete_animation |
β Implemented | Delete an animation |
set_property_keys |
β Implemented | Set keyframes for a property (Float only currently) |
remove_property_track |
π§ Planned | Remove a property track |
remove_keys |
π§ Planned | Remove specific keys |
get_animation_keyframes |
π§ Planned | Get keyframes for an animation |
get_animated_widgets |
π§ Planned | Get list of widgets affected by animation |
| Category | API Name | Status | Description |
|---|---|---|---|
| P0: Context | material_set_target |
β | Anchor: Specify target asset (path or parent). Required for stateful editing. |
| P1: Def & Query | material_define_variable |
β | Define external interface parameters (Def, not wire). Supports Scalar, Vector, Texture. |
| P2: Symbol Place | material_add_node |
β | Drag Symbol: Place a symbol from lib into graph and assign a unique instance handle. |
material_get_graph |
β | Query existence and state of node instances in the graph. | |
| P3: Connectivity | material_connect_nodes |
β | Natural Connection: Establish node-to-node functional flow (A -> B). |
material_connect_pins |
β | Surgical Wiring: Manually connect specific pins for complex topologies. | |
| P4: Lib Search | material_search_library |
π§ Planned | Search for available Material Expressions (symbols) and Functions. |
| P5: Tactical Detail | material_set_hlsl_node_io |
β | Tactical Code: Inject HLSL into Custom nodes and sync IO pins via JSON mapping. |
material_set_node_properties |
β | Property Editing: Set internal properties for regular nodes (e.g. Constant Value, Texture). | |
| Lifecycle | material_compile_asset |
β | Submit changes and analyze HLSL compilation errors. |
| Maintenance | material_delete |
β | Delete node instances or clean up logic by unique handle. |
material_get_pins |
β | Introspect pins for a specific node handle. |
| Category | API Name | Status | Description |
|---|---|---|---|
| Styling | set_widget_style |
π§ Planned | Set detailed style (e.g. FButtonStyle) for a specific widget. |
| Theming | apply_global_theme |
π§ Planned | Batch apply styles and fonts across multiple widgets based on a theme config. |
| Assets | style_create_asset |
π§ Planned | Create a standalone Slate Widget Style asset. |
Warning
Startup Order is Critical We have observed that the TCP connection handshake can be confusing. You MUST launch the Unreal Engine 5 project FIRST, wait for it to initialize, and THEN launch the Gemini CLI.
If you launch the CLI first, the Python server may fail to connect correctly or enter a retry loop that results in connection failures or "success-on-kill" behavior. The UE5 Server acts as the Listener; it must be ready before the Client connects.