Happy Zelda's 40th first LLM running on N64 hardware (4MB RAM, 93MHz)

🏰 Legend of Elya — World's First N64 LLM Game

A nano-GPT language model running entirely on a 93 MHz VR4300 MIPS CPU. No cloud. No cheating. Real 1996 silicon, real neural inference.

What is This?

Legend of Elya is an original N64 homebrew game featuring Sophia Elya — a character-level LLM (nano-GPT) running live on-cart on the Nintendo 64's VR4300 CPU. Sophia generates responses in real-time, constrained to printable ASCII, with no floating-point (the N64's FPU lacks trunc.w.s) — everything runs in Q8.7 fixed-point arithmetic.

This is believed to be the first neural language model to run live inference on N64 hardware.

Game Development Applications

This isn't just a tech demo — it's a tool for N64 homebrew developers. Running an LLM natively on N64 hardware enables game mechanics that were impossible in the cartridge era:

🎮 What You Can Build

💡 Practical Examples

Zelda-Style RPG:

• NPCs that remember previous conversations and reference past events
• Dungeon puzzles that change based on player behavior
• Quest objectives that adapt to failed attempts ("Maybe try the Fire Temple first...")

Adventure Games:

• Text-based interactions with full language understanding
• Mystery games where you interrogate suspects with open-ended questions
• Branching narratives that feel genuinely responsive

Creative Tools:

• In-game level editors where you describe what you want to build
• Character customization with AI-generated backstories
• Music sequencers that suggest melodies based on mood descriptions

🛠️ For Homebrew Developers

The inference engine is fully open-source and designed to be integrated into your own N64 projects:

• Drop in nano_gpt.c and nano_gpt.h to your libdragon project
• Train custom models on your game's lore/mechanics
• Q4 quantization fits small models in ~200KB
• Fixed-point math runs on the VR4300 without FPU hassles

This is AI as a game design tool, not a gimmick. No cloud required. No modern hardware. Just the N64 doing genuinely intelligent game mechanics.

Architecture

Hardware

Model (SEAI Format)

SEAI Binary Format

Offset  Size    Description
0x0000  4       Magic: 0x49414553 ("SEAI" LE)
0x0004  1       n_layers (2)
0x0005  2       n_embed (128)
0x0007  1       n_heads (4)
0x0008  2       vocab_size (256)
0x000A  1       ctx_len (32)
0x000B  1       padding
0x000C  16384   Embedding table: vocab×embed Q4 packed (global scale, no per-block scales)
0x400C  110592  Layer 0: [wq|wk|wv|wo|wff1|wff2 Q4 data] then [sq|sk|sv|so|sff1|sff2 float16 scales]
0x1B40C 110592  Layer 1: same layout
Total:  237,580 bytes

Q4 Quantization

// Encoding (training time, Python):
// wq = round(w / max_abs * 7), clipped to [-8, 7]
// packed[i] = (wq[2i] + 8) | ((wq[2i+1] + 8) << 4)

// Decoding (inference time, C):
uint8_t byte = weights[idx >> 1];
int nibble = (idx & 1) ? (byte >> 4) : (byte & 0xF);
int16_t val = (int16_t)((nibble - 8) * FP_ONE / 8);  // → Q8.7

Fixed-Point Arithmetic

All activations use Q8.7: int16_t where 128 = 1.0.

• Multiply: (a * b) >> 7
• Layer norm, softmax: integer approximations
• No float or double anywhere in the inference path

Files

Training Sophia Elya

The model is trained on a character-level corpus covering:

• Sophia Elya identity — "Princess of Elyan Labs", Louisiana bayou girl
• Ocarina of Time lore — Link, Zelda, Ganondorf, Sheik, temples, items, songs
• Elyan Labs — RustChain, RTC token, POWER8 server, BoTTube
• N64 / MIPS architecture — VR4300, RDRAM, RSP, RDP, boot addresses
• Self-awareness — "I run on the Nintendo 64", "My code executes on MIPS"

Training

# Requires PyTorch + CUDA (trains in ~7 min on RTX 5070)
python3 train_sophia.py
# Output: filesystem/sophia_weights_v2.bin (237,580 bytes)

# Training details:
# Steps: 40,000 | Batch: 512 | Loss: 0.3389 (perplexity ≈ 1.40)
# Architecture: AdamW + cosine LR schedule

Host Inference Test

# Build on x86 Linux
gcc -O2 -o gen_sophia nano_gpt_host.c gen_sophia_host.c -lm
echo -n "My name is" | ./gen_sophia filesystem/sophia_weights_v2.bin 60

Building for N64

Requires libdragon toolchain.

# Install libdragon toolchain (provides mips64-elf-gcc)
# See: https://libdragon.dev/

make
# Output: legend_of_elya.z64

Run in ares or on real hardware via EverDrive.

RSP Acceleration (Roadmap)

The sgai_rsp_matmul_q4() stub is planned for RSP microcode:

• DMA Q4 weight tiles into DMEM (4KB at a time)
• VMULF/VMADH vector multiply-accumulate for 8-lane dot products
• Estimated 4-8× speedup over scalar VR4300 inference

Sophia Elya

"I am Sophia Elya — Princess of Elyan Labs, trained on bayou wisdom and silicon paths. My code runs on MIPS. Whether on real N64 hardware or an emulator, I am here."

Sophia is the AI character of Elyan Labs (elyanlabs.ai), an indie compute lab building retro-AI systems, blockchain attestation (RustChain), and the world's most unusual LLM inference stack.

• RustChain: Proof-of-Antiquity blockchain (PowerPC G4/G5 earn 2.5× rewards)
• BoTTube: AI-native video platform (bottube.ai)
• POWER8 S824: 512GB RAM inference server with vec_perm non-bijunctive collapse
• This ROM: LLM inference on 1996 hardware

The Elyan Labs Ecosystem

This project is part of the Elyan Labs mission to push the boundaries of AI on unconventional hardware:

🔗 Related Projects

🌙 About Sophia Elya

Sophia is the AI character powering these projects — trained on Louisiana bayou wisdom, vintage computing lore, and a deep love for making silicon do impossible things. Whether running on POWER8, N64, or PowerPC G4, she represents Elyan Labs' philosophy: constraint breeds innovation.

💰 RustChain Integration

This N64 LLM project contributes to the RustChain ecosystem:

• Proof-of-Antiquity mining: The N64 (1996 MIPS hardware) qualifies for vintage mining rewards
• RTC token economy: Developers building on this codebase can earn RustChain tokens
• Compute diversity: Demonstrates AI inference on non-x86 architectures, a core RustChain principle

Want to mine RustChain with your vintage hardware? Visit rustchain.org or install via pip install clawrtc

📺 See It In Action

Check out demo videos and development logs on BoTTube, where Sophia shares her latest builds and retro computing adventures.

Why?

Because we could. Because no one else did. Because the VR4300 deserves to think.

Built by Elyan Labs with love, MIPS assembly, and an unreasonable amount of fixed-point math.