Commit graph

12 commits

Author SHA1 Message Date
Paul
d54e262048 feat(rendering): GPU architecture + visual quality fixes
M2 GPU instancing
- M2InstanceGPU SSBO (96 B/entry, double-buffered, 16384 max)
- Group opaque instances by (modelId, LOD); single vkCmdDrawIndexed per group
- boneBase field indexes into mega bone SSBO via gl_InstanceIndex

Indirect terrain drawing
- 24 MB mega index buffer (6M uint32) + 64 MB mega vertex buffer
- CPU builds VkDrawIndexedIndirectCommand per visible chunk
- Single VB/IB bind per frame; shadow pass reuses mega buffers
- Replaced vkCmdDrawIndexedIndirect with direct vkCmdDrawIndexed to fix
  host-mapped buffer race condition that caused terrain flickering

GPU frustum culling (compute shader)
- m2_cull.comp.glsl: 64-thread workgroups, sphere-vs-6-planes + distance cull
- CullInstanceGPU SSBO input, uint visibility[] output, double-buffered
- dispatchCullCompute() runs before main pass via render graph node

Consolidated bone matrix SSBOs
- 16 MB double-buffered mega bone SSBO (2048 instances × 128 bones)
- Eliminated per-instance descriptor sets; one megaBoneSet_ per frame
- prepareRender() packs bone matrices consecutively into current frame slot

Render graph / frame graph
- RenderGraph: RGResource handles, RGPass nodes, Kahn topological sort
- Automatic VkImageMemoryBarrier/VkBufferMemoryBarrier between passes
- Passes: minimap_composite, worldmap_composite, preview_composite,
  shadow_pass, reflection_pass, compute_cull
- beginFrame() uses buildFrameGraph() + renderGraph_->execute(cmd)

Pipeline derivatives
- PipelineBuilder::setFlags/setBasePipeline for VK_PIPELINE_CREATE_DERIVATIVE_BIT
- M2 opaque = base; alphaTest/alpha/additive are derivatives
- Applied to terrain (wireframe) and WMO (alpha-test) renderers

Rendering bug fixes:
- fix(shadow): compute lightSpaceMatrix before updatePerFrameUBO to eliminate
  one-frame lag that caused shadow trails and flicker on moving objects
- fix(shadow): scale depth bias with shadowDistance_ instead of hardcoded 0.8f
  to prevent acne at close range and gaps at far range
- fix(visibility): WMO group distance threshold 500u → 1200u to match terrain
  view distance; buildings were disappearing on the horizon
- fix(precision): camera near plane 0.05 → 0.5 (ratio 600K:1 → 60K:1),
  eliminating Z-fighting and improving frustum plane extraction stability
- fix(streaming): terrain load radius 4 → 6 tiles (~2133u → ~3200u) to exceed
  M2 render distance (2800u) and eliminate pop-in when camera turns;
  unload radius 7 → 9; spawn radius 3 → 4
- fix(visibility): ground-detail M2 distance multiplier 0.75 → 0.9 to reduce
  early pop of grass and debris
2026-04-04 13:43:16 +03:00
k
b3fa8cf5f3 fix: warden mmap on macOS, add external listfile support to asset extractor
Drop PROT_EXEC from warden module mmap when using Unicorn emulation
(not needed — module image is copied into emulator address space). Use
MAP_JIT on macOS for the native fallback path.

Add --listfile option to asset_extract and SFileAddListFileEntries
support for resolving unnamed MPQ hash table entries from external
listfiles.
2026-04-04 00:22:07 -07:00
Kelsi
7264ba1706 fix(extractor): lowercase all output paths to prevent duplicate folders
WoW archives contain mixed-case variants of the same path (e.g.,
ARMLOWERTEXTURE vs ArmLowerTexture) which created duplicate directories
on case-sensitive Linux filesystems. Now mapPath() lowercases the entire
output. Also keeps TextureComponents and ObjectComponents directory
names instead of abbreviating them (item/texturecomponents/ instead of
item/texture/) so filesystem paths match the WoW virtual paths used in
manifest lookups.
2026-04-03 21:26:20 -07:00
Kelsi
b31a2a66b6 tools: fix DBC string-column detection false positives in both dbc_to_csv and asset_extract
The string-column auto-detector in both tools had two gaps that caused small
integer fields (RaceID=1, SexID=0/1, BaseSection, ColorIndex) to be falsely
classified as string columns, corrupting the generated CSVs:

1. No boundary check: a value of N was accepted as a valid string offset even
   when N landed inside a longer string (e.g. offset 3 inside "Character\...").
   Fix: precompute valid string-start boundaries (offset 0 plus every position
   immediately after a null byte); reject offsets that are not boundaries.

2. No diversity check: a column whose only non-zero value is 1 would pass the
   boundary test because offset 1 is always a valid boundary (it follows the
   mandatory null at offset 0). Fix: require at least 2 distinct non-empty
   string values before marking a column as a string column. Columns like
   SexID (all values are 0 or 1, resolving to "" and the same path fragment)
   are integer fields, not string fields.

Both dbc_to_csv and asset_extract now produce correct column metadata,
e.g. CharSections.dbc yields "strings=6,7,8" instead of "strings=0,1,...,9".
2026-03-10 03:49:06 -07:00
Kelsidavis
1fab17e639 Add Windows build scripts, fix multi-threaded MPQ extraction, and cross-platform temp paths
- Add build.ps1/bat, rebuild.ps1/bat, debug_texture.ps1/bat (Windows equivalents
  of existing bash scripts, using directory junctions for Data link)
- Fix asset extractor: StormLib is not thread-safe even with separate handles per
  thread. Serialize all MPQ reads behind a mutex while keeping CRC computation and
  disk writes parallel. Previously caused 99.8% extraction failures with >1 thread.
- Add SFileHasFile() check during enumeration to skip listfile-only entries
- Add diagnostic logging for extraction failures (first 5 per thread + summary)
- Use std::filesystem::temp_directory_path() instead of hardcoded /tmp/ in
  character_renderer.cpp debug dumps
- Update debug_texture.sh to use $TMPDIR fallback and glob for actual dump filenames
2026-02-25 08:22:45 -08:00
Kelsi
d7e2b26af7 Unify asset system: one asset set, always high-res
Remove HDPackManager, expansion overlay manifests, and BLP size-comparison
logic. Assets now resolve through a single manifest with a simple override
directory (Data/override/) for future HD upgrades.
2026-02-15 04:18:34 -08:00
Kelsi
a67dca5787 Add overlay extraction for multi-expansion asset deduplication
Extracts each expansion's assets as a CRC-compared overlay against a
base manifest, storing only files that differ. Auto-detects overlay mode
when a base manifest already exists. Adds --as-overlay, --full-base
flags and manifest merge for partial extractions.
2026-02-14 03:09:17 -08:00
Kelsi
03013e751a Cache WoW.exe during asset extraction for Warden MEM_CHECK 2026-02-14 02:17:50 -08:00
Kelsi
886f4daf2e Add per-expansion asset overlay system and fix CharSections DBC layout
Expansion overlays allow each expansion to supplement the base asset data
via an assetManifest field in expansion.json, loaded at priority 50 (below
HD packs). The asset extractor gains --reference-manifest for delta-only
extraction. Also fixes CharSections field indices (VariationIndex=4,
ColorIndex=5, Texture1=6) across all DBC layout references.
2026-02-14 00:00:26 -08:00
Kelsi
f247d53309 Add expansion DBC CSVs, Turtle support, and server-specific login 2026-02-13 00:10:01 -08:00
Kelsi
7092844b5e Add multi-expansion support with data-driven protocol layer
Replace hardcoded WotLK protocol constants with a data-driven architecture
supporting Classic 1.12.1, TBC 2.4.3, and WotLK 3.3.5a. Each expansion
has JSON profiles for opcodes, update fields, and DBC layouts, plus C++
polymorphic packet parsers for binary format differences (movement flags,
speed fields, transport data, spline format, char enum layout).

Key components:
- ExpansionRegistry: scans Data/expansions/*/expansion.json at startup
- OpcodeTable: logical enum <-> wire values loaded from JSON
- UpdateFieldTable: field indices loaded from JSON per expansion
- DBCLayout: schema-driven DBC field lookups replacing magic numbers
- PacketParsers: WotLK/TBC/Classic parsers with correct flag positions
- Multi-manifest AssetManager: layered manifests with priority ordering
- HDPackManager: overlay texture packs with expansion compatibility
- Auth screen expansion picker replacing hardcoded version dropdown
2026-02-12 22:56:36 -08:00
Kelsi
aa16a687c2 Replace MPQ runtime with loose file asset system
Extract assets from MPQ archives into organized loose files indexed by
manifest.json, enabling fully parallel reads without StormLib serialization.
Add asset_extract and blp_convert tools, PNG texture override support.
2026-02-12 20:32:14 -08:00