Walks an extracted tree and removes every BLP/DBC/M2/skin/WMO/ADT
that has a confirmed open-format sidecar at least as new. Dry-run
by default — requires --confirm-purge to actually delete:
asset_extract --purge-proprietary Data/expansions/wotlk
Dry-run: would purge proprietary files...
would remove: 21570 files (16380.4 MB)
(re-run with --confirm-purge to actually delete)
asset_extract --purge-proprietary Data/expansions/wotlk --confirm-purge
Purging...
removed: 21570 files (16380.4 MB)
Pairing rules:
.blp → .png
.dbc → .json
.m2 → .wom
.skin → matching .m2's .wom (foo00.skin pairs with foo.wom)
.wmo → .wob (root)
.wmo group sub-files (foo_NNN.wmo) → parent foo.wob
.adt → .whm
Servers can't run without proprietary files, so this only makes
sense for wowee-runtime-only setups. Files without a sidecar are
left untouched.
Compares the source file's mtime against the sidecar's; if the
sidecar is newer, the conversion is skipped and counted into
stats.skipped. Re-running --upgrade-extract on a fully-converted
tree is now nearly free (just an mtime check per file).
asset_extract --upgrade-extract Data/expansions/wotlk
Walking ... (first run)
JSON (DBC→JSON) : 240 ok
asset_extract --upgrade-extract Data/expansions/wotlk
Walking ... (second run, all sidecars up to date)
up-to-date (skip) : 240
JSON (DBC→JSON) : 0 ok
emitOpenFormats() takes a new optional 'incremental' flag (default
false to preserve the asset_extract main-loop's overwrite behavior
since fresh extraction always wants new sidecars).
Verified end-to-end with a hand-built DBC: first run converts,
second run reports 'up-to-date (skip): 1'.
emitOpenFormats now takes an optional threadCount parameter (0 =
auto). The asset_extract --upgrade-extract path forwards opts.threads
so users can override the auto-detect when running on a CI machine
with limited cores or wanting deterministic timing.
Also wraps the upgrade pass with a chrono timer and prints elapsed
seconds so the parallelization payoff is visible at a glance:
asset_extract --upgrade-extract Data/expansions/wotlk --threads 8
Walking Data/expansions/wotlk for open-format upgrades...
elapsed : 47.2 s
PNG (BLP→PNG) : 12340 ok
...
Verified end-to-end: --threads 2 on 5 hand-built DBCs converts all
5 in well under a second.
Standalone post-extract pass: walks an existing extracted asset
tree and writes open-format sidecars in place, without re-running
MPQ extraction.
asset_extract --upgrade-extract Data/expansions/wotlk
Lets users with old extractions opt into the open-format pipeline
without losing the extracted state. Implies --emit-open if no
individual --emit-* flag is set.
Verified end-to-end: created a hand-built DBC in a temp dir, ran
--upgrade-extract, observed test.json appear with correct
metadata. Servers continue to read .dbc from manifest; the
runtime client picks up the new .json sidecar via the existing
pickup path.
Final piece of the open-format emit pipeline:
--emit-terrain foo.adt → foo.whm + foo.wot + foo.woc
With this, --emit-open now produces a fully open-format zone
alongside every Blizzard MPQ extraction:
BLP → PNG (textures)
DBC → JSON (data tables)
M2 → WOM (models, with skin merge)
WMO → WOB (buildings, with group merge)
ADT → WHM/WOT (terrain heights + metadata)
→ WOC (collision mesh derived from heights)
Originals stay on disk and indexed by manifest.json so private
servers continue to load proprietary formats; wowee runtime/editor
read the open formats directly. One extraction now feeds both
audiences with no separate conversion pass.
Implementation:
- Inline WHM+WOT writer in open_format_emitter.cpp (mirrors the
editor's WoweeTerrain::exportOpen but without the PNG-preview /
normal-map deps so the extractor stays editor-independent).
- Tile coords (x,y) parsed from <map>_<x>_<y>.adt filename.
- Collision mesh derived via WoweeCollisionBuilder::fromTerrain
(terrain triangles only — WMO collision overlays would need
asset manager and aren't worth the extractor complexity).
Extends asset_extract with two more open-format emitters:
--emit-wom foo.m2 (+ foo00.skin) → foo.wom
--emit-wob foo.wmo (+ foo_NNN.wmo groups) → foo.wob
--emit-open now also turns these on
Originals are preserved so private servers still load .m2/.wmo
through the manifest path; the wowee runtime/editor pick up the
.wom/.wob next to them via the existing open-format search rules.
Implementation:
- New WoweeModelLoader::fromM2Bytes(m2Data, skinData) shares the
conversion body with fromM2(path, am) via a static helper
(convertM2ToWom). Lets the extractor convert without standing
up an AssetManager.
- fromM2(path, am) moved to a separate translation unit
(wowee_model_fromm2.cpp) so asset_extract doesn't have to
link the AssetManager dependency.
- WoweeBuildingLoader::fromWMO already takes a WMOModel directly,
so emitWobFromWmo just needs to read root + group files and
call save().
- Group sub-files (<base>_NNN.wmo) are skipped during the walk
since they're merged into the root WMO.
The asset_extract tool now optionally writes wowee open-format
copies next to each extracted proprietary file:
--emit-png foo.blp → foo.png
--emit-json-dbc foo.dbc → foo.json
--emit-open shortcut for both
Originals are left untouched, so private servers (AzerothCore,
TrinityCore) that load from the manifest's .blp/.dbc paths
continue to work unchanged. The wowee runtime / editor can now
consume the open formats directly without an extra conversion pass.
Implementation:
- New tools/asset_extract/open_format_emitter.{hpp,cpp} encapsulates
the post-extract walk + per-file conversion.
- BLP→PNG uses BLPLoader::load + stbi_write_png with the same
dimension/buffer-size sanity guards the editor's texture exporter
applies.
- DBC→JSON mirrors the editor's DBCExporter::exportAsJson schema
(string/float/uint heuristic) so the runtime DBC overlay loader
can consume the output drop-in.
Drop PROT_EXEC from warden module mmap when using Unicorn emulation
(not needed — module image is copied into emulator address space). Use
MAP_JIT on macOS for the native fallback path.
Add --listfile option to asset_extract and SFileAddListFileEntries
support for resolving unnamed MPQ hash table entries from external
listfiles.
Remove HDPackManager, expansion overlay manifests, and BLP size-comparison
logic. Assets now resolve through a single manifest with a simple override
directory (Data/override/) for future HD upgrades.
Extracts each expansion's assets as a CRC-compared overlay against a
base manifest, storing only files that differ. Auto-detects overlay mode
when a base manifest already exists. Adds --as-overlay, --full-base
flags and manifest merge for partial extractions.
Expansion overlays allow each expansion to supplement the base asset data
via an assetManifest field in expansion.json, loaded at priority 50 (below
HD packs). The asset extractor gains --reference-manifest for delta-only
extraction. Also fixes CharSections field indices (VariationIndex=4,
ColorIndex=5, Texture1=6) across all DBC layout references.
Replace hardcoded WotLK protocol constants with a data-driven architecture
supporting Classic 1.12.1, TBC 2.4.3, and WotLK 3.3.5a. Each expansion
has JSON profiles for opcodes, update fields, and DBC layouts, plus C++
polymorphic packet parsers for binary format differences (movement flags,
speed fields, transport data, spline format, char enum layout).
Key components:
- ExpansionRegistry: scans Data/expansions/*/expansion.json at startup
- OpcodeTable: logical enum <-> wire values loaded from JSON
- UpdateFieldTable: field indices loaded from JSON per expansion
- DBCLayout: schema-driven DBC field lookups replacing magic numbers
- PacketParsers: WotLK/TBC/Classic parsers with correct flag positions
- Multi-manifest AssetManager: layered manifests with priority ordering
- HDPackManager: overlay texture packs with expansion compatibility
- Auth screen expansion picker replacing hardcoded version dropdown
