vidya/crates/renderer_device/AGENTS.md

# renderer_device

## Purpose
`renderer_device` is the Vulkan instance, logical-device, queue, synchronization, and device-owned-object layer extracted from `renderer`. It owns Vulkan device lifecycle and shared device services; higher-level resource and rendering abstractions belong in `renderer`.

## Module map
- `lib.rs`: crate error/result types, physical-device feature/property descriptions and queries, surface capabilities, extension helpers, and public module exports. `extern crate self as renderer_device` supports generated derive paths.
- `device.rs`: `Device`/`DeviceInner`, logical-device creation, pools, ownership traits/wrappers, extension and allocation helpers.
- `instance.rs`: `Instance`/`InstanceInner`, instance setup, adapter discovery/selection, surface capability queries, debug messenger lifecycle.
- `queue.rs`: queue-family selection, `Queue`, queue synchronization, and `DeviceQueues`.
- `sync.rs`: fence/semaphore wrappers and pools, `FenceFuture`, and background fence waiters.
- `pipeline_cache.rs`: keyed Vulkan pipeline cache with `pipeline_cache.bin` load/export.
- `debug.rs`: Vulkan debug callback forwarding messages to tracing.
- `testing.rs` (feature `testing`): `TestVulkan` creates real validation-enabled Vulkan devices and captures validation messages for smoke/integration tests.
- `renderer_macros/src/lib.rs`: separate workspace proc-macro source; derives `DeviceRef` for a struct field marked `#[device]`.

## Key types and ownership
- `Device(Arc<DeviceInner>)` is the cloneable owning handle. It dereferences to `DeviceInner`; equality is `Arc::ptr_eq`.
- `DeviceInner` keeps the raw `ash::Device`, VMA (`vk_mem`) allocator mutex, adapter and instance, queues, synchronization threadpool, extension loaders, and inline `DevicePools`.
- `Device::enabled_features()` returns the feature snapshot requested for logical-device creation; it is distinct from `Device::features()`, which reports adapter-supported features. Descriptor capability validation must use the enabled snapshot, not support data.
- `PhysicalDeviceInfo::create_logical_device` selects queues, enables requested/supported extensions and features, initializes the allocator/cache, and constructs `DeviceInner` with `Arc::new_cyclic`.
- `DeviceInner::dev`, `instance`, `queues`, `physical_device`, `main_queue`, `compute_queue`, and `transfer_queue` are the main convenience accessors for downstream code.
- `DevicePools` contains a `PipelineCache` wrapper plus pools for raw fences and binary/timeline semaphore newtypes. A `Pool<T>` stores bare handles in `Mutex<Vec<T>>` and a `Weak<DeviceInner>`; `get()` reuses a handle or creates one through `Pooled`.
- `PoolObject<T: Recycle>` strongly owns a `Device`; dropping it returns its handle to the relevant pool. The pool's weak reference prevents a `DeviceInner` ownership cycle.
- `asdf::DeviceObject<T>` strongly owns its `Device`, dereferences to its handle, and destroys that handle through `asdf::traits::ExternallyManagedObject<Device>` on drop. `DeviceOwned<T>` exposes the owning device and raw handle.
- `DeviceRef` supplies `device()` for `Device`, Arc/reference forms, and structs supported by the `DeviceRef` derive (`#[device]` identifies the field). `Pooled` creates pooled handles; `Recycle` returns them.
- The `DeviceRef` proc macro is implemented in the separate `renderer_macros` package and emits paths rooted at `::renderer_device::`.
- `sync::Fence` and `sync::Semaphore` distinguish `Dedicated` (`DeviceObject`) from `Pooled` (`PoolObject`) ownership. `SemaphoreInner`, `BinarySemaphore`, and `TimelineSemaphore` represent semaphore-kind handles.
- `PhysicalDeviceInfo`, `PhysicalDeviceFeatures`, `PhysicalDeviceProperties`, and `SurfaceCapabilities` carry adapter/capability data. `Instance::choose_adapter_default` accepts `Option<vk::SurfaceKHR>` (a raw handle), not a swapchain/surface wrapper.
- `Error` and `Result<T>` are the crate-wide error types; `Error` covers Vulkan/VMA (`vk_mem`), loading, I/O, and renderer-facing resource/capability failures.
- `InstanceDesc` configures borrowed instance extensions/layers, optional app metadata, and a raw display handle; its default requests surface and validation layers, with validation settings in debug builds. `Instance::new` creates a shared `Instance` and requests debug-utils plus optional capability-query extensions alongside requested/platform extensions.
- `Instance::expose_adapter` queries one adapter. `choose_adapter_default` ranks enumerated adapters against requested extensions/features and prefers discrete devices, then newer APIs, device-local memory, and texture size.
- `PhysicalDeviceFeatures` stores core, Vulkan 1.1/1.2/1.3, and optional mesh-shader feature structures; `push_to_device_create_info` chains enabled feature structs.
- `PhysicalDeviceProperties` stores supported extensions, core/versioned properties, optional mesh-shader properties, memory properties, and API version; `supports_extension` checks name and minimum version.
- `get_physical_device_features` and `get_physical_device_properties` query the feature/property chains. Extension query results are consumed by the instance and logical-device setup helpers.
- `DeviceExtensions` holds the debug-utils device loader and optional swapchain/mesh-shader loaders. `DeviceInner.enabled_extensions` records enabled extension names.
- `MemoryLocation` is the public allocator-independent memory preference; `AllocationDesc` combines it with a resource name and `AllocationStrategy` (`AllocatorManaged` or `Dedicated`). `GpuAllocation` wraps the private `vk_mem::Allocation` and retains memory properties needed by callers.
- `DeviceInner::allocate_for_buffer` / `allocate_for_image` obtain VMA allocations, and `bind_buffer_memory` / `bind_image_memory` bind them to Vulkan resources without exposing `vk_mem` types to upper layers. CPU-mapped locations require host-visible, host-coherent memory.
- `Allocation` represents `Owned`, `Shared`, or `Unmanaged` GPU allocations.
- `get_extensions` returns supported/enabled and unsupported requested extension sets; `get_layers` returns available requested layers or the supported/unsupported split.
- `Queue` shares a `QueueInner` through `Arc`; `with_locked` serializes operations on the Vulkan queue. `QueueFamily` records index, count, flags, granularity, and timestamp bits.
- `DeviceQueues` provides graphics, compute, and transfer queues; absent compute/transfer queues alias the graphics queue. Queue selection requires presentation support only when a display handle is provided.
- `DeviceQueueInfos` chooses queue families and produces `vk::DeviceQueueCreateInfo` values before `retrieve_queues` obtains the raw Vulkan queues.
- `QueueFlags` in `device.rs` describes requested graphics/async-compute/transfer/present families; `queue::QueueFlags` describes discovered Vulkan family capabilities. They are distinct types.
- `FenceFuture` waits and resets a fence, and `SyncThreadpool` schedules blocking fence waits for async wakers. `Fence::from_pool` and `Semaphore::from_pool` wrap recycled pool entries.
- `SemaphoreType` selects binary or timeline construction. `SemaphoreInner` wraps the common Vulkan handle while `BinarySemaphore` and `TimelineSemaphore` tag their respective pools.
- `Fence::wait_on`, `is_signaled`, and `reset` operate on either dedicated or pooled fence handles through their owning device.
- Pooled timeline semaphores are created at value zero; `Semaphore::from_pool` signals the requested initial value when it obtains one.
- `PipelineCache::new` computes its key from adapter vendor/API/device/driver properties and tries to load compatible data; cache persistence uses a magic and format version.
- `DebugName` is the wrapper for Vulkan object debug names.
- `testing::TestVulkan::acquire` returns a real Vulkan test device or skips with a reason; `expect` requires Vulkan. `take_messages`, `assert_no_validation_errors`, and `assert_no_warnings` inspect the captured debug-utils callback output. Run the device smoke tests using `direnv exec . cargo test -p renderer_device --features testing` from the workspace root.
- `Extension` is the name/version pair used by extension helpers and adapter creation. `make_extension!`, `get_extensions`, `get_layers`, and the support/intersection functions centralize Vulkan extension discovery.
- `SurfaceCapabilities` returns raw `SurfaceCapabilitiesKHR` plus the effective minimum image count, surface formats, and present modes queried for the raw surface.
- `DebugUtilsCreateInfo` is the small severity/message-type configuration record used with debug utilities; `DebugUtils` itself owns the instance loader and messenger handle.
- `InstanceExtensions` stores optional surface-capability-query extension loaders and an optional surface-maintenance marker.
- `DeviceHandle` is a separate unsafe mutable-handle destruction trait used by `vk::DescriptorPool`; the normal `DeviceObject` destruction path uses `ExternallyManagedObject` instead.
- `asdf::ExternallyManagedObject<T, O>` is the generic owner/inner wrapper: it uses `ManuallyDrop`, invokes `destroy` exactly once, and retains the owner until that call.
- `DeviceObject<T>` exposes `device()`, `handle()` for `Copy` handles, and optional `name()`. It only implements `DeviceOwned<T>` when `T` is a Vulkan handle.
- `Queue::with_locked` serializes calls on a queue; `DeviceQueues::family_indices` deduplicates family IDs for resource-sharing modes.
- `Fence::new_dedicated` and `Semaphore::new_dedicated` create per-object device-owned primitives; `from_pool` constructors create pooled sync objects.
- `FenceFuture::block` waits then resets. Polling an unsignaled future schedules a blocking waiter through `SyncThreadpool`; the pool's worker count is bounded by its configured maximum.

## Invariants and gotchas
- `asdf` is the literal legacy module name for `DeviceObject` and its traits. Do not silently rename paths; generated/client code refers to these APIs.
- Vulkan child handles left bare in pools at device teardown are implicitly released by `vkDestroyDevice`; pool entries must not independently retain device-owning wrappers.
- `_drop: DeviceDrop` is the private shutdown path: it waits for the device to idle, exports and destroys the pipeline cache to `pipeline_cache.bin`, then destroys the Vulkan device. The cache is also exposed as a copyable wrapper through `DevicePools` for pipeline creation.
- `Device` cloning is cheap and prolongs the full device lifetime; any live `DeviceObject`, `PoolObject`, sync wrapper, or other owner retaining a `Device` can delay shutdown.
- Device-owned wrapper structs live outside `DeviceInner` and its pools. Pools hold bare handles, never `DeviceObject` or `PoolObject` wrappers.
- Keep enabled-feature state distinct from adapter-supported feature queries: `enabled_features()` records what the logical device was created with, while `features()` is support information and must not be treated as enabled capability.
- `Pool::device()` upgrades its `Weak` and panics if the pool outlives its device; don't retain pools independently.
- `PoolObject` uses `ManuallyDrop` to transfer its inner handle to `Recycle` exactly once. New `Recycle` implementations must route to the matching pool and handle type.
- The derive parser requires a named-field struct with exactly one `#[device]` field and does not add generic bounds itself; generic field types must satisfy `DeviceRef` through the input declaration. Its generated paths require the crate self-alias in `lib.rs`.
- `vk::Semaphore` and `SemaphoreInner` have destruction implementations for dedicated ownership; pooled semaphore ownership instead recycles `SemaphoreInner` into a kind-specific pool.
- `vk::Buffer`, `vk::Image`, `vk::ImageView`, descriptor/pipeline objects, swapchains, semaphores, and GPU allocations have concrete destruction paths; inspect `device.rs` and `sync.rs` before introducing another handle wrapper.
- Queue `lock()`/`unlock()` and `DeviceInner::lock_queues`/`unlock_queues` are unsafe paired operations. Prefer `Queue::with_locked` or `wait_idle` unless the caller can uphold their contracts.
- `Instance` shares its raw Vulkan instance through `Arc`; `DebugUtils` owns and destroys its messenger. Device references retain their `Instance`.
- `superset_of` compares requested-true flags with `a != b`, which looks inverted for a superset test — verify before relying on adapter feature selection.
- Physical-device feature/property queries chain Vulkan 1.1/1.2/1.3 structures and optional mesh-shader data without checking each version's availability; property querying marks beyond-1.0 probing as not yet fallible.
- `choose_adapter_default` ignores its optional raw surface argument (`TODO`); it does not test surface compatibility.
- Instance/queue selection accepts `RawDisplayHandle`; surface capability lookup uses a raw `vk::SurfaceKHR`. This crate has no `renderer::swapchain::Surface` dependency.
- `queue::QueueFlags` and `device::QueueFlags` intentionally serve different purposes. Keep imports qualified when adding queue-family selection logic.
- `PipelineCache` persistence uses a single working-directory path and one cache record. Changes to the key or file header should account for compatibility with existing cache files.
- Logical-device setup always includes the swapchain extension in its required extension list, regardless of the caller's extra requested extensions.
- Queue-family selection uses the optional display handle to query presentation support and requires its chosen graphics family to be present-capable only when a display handle is provided.
- Extension selectors and feature checks drive selection; do not infer that every optional extension is enabled just because a loader type exists.
- `Instance::query_presentation_support` handles platform-specific display variants and has an unimplemented XCB branch.
- `DebugUtils` is stored inside `InstanceInner`; its destructor destroys the messenger, so retain the loader/messenger for the instance lifetime.
- `DeviceObject`'s concrete destruction impls are split between `device.rs` and `sync.rs`; search both before adding a duplicate or assuming a type has an implementation.
- `PhysicalDeviceInfo` construction can discard adapters that fail property queries; `Instance::choose_adapter` filters failed exposures before sorting.
- `DeviceQueueInfos::retrieve_queues` reuses the graphics queue handle for absent compute or transfer roles. These aliases share the same `QueueInner` lock.
- `QueueFamily::granularity` uses the source spelling `TransferGranuality`; avoid renaming it as unrelated cleanup.
- `sync::Semaphore::from_pool` signals pooled timeline semaphores to the provided value but does not reset them to zero first; ensure requested timeline value semantics remain valid on reuse.
- Fence waiters wake on successful waits, log and stop on non-timeout errors, and can time out of the worker pool after its idle duration.
- `PipelineCache` uses MD5 only to derive a local adapter key; this is a cache identity, not a security mechanism.
- Device-level extension loaders are constructed only for extensions recorded as enabled; callers should check optional loaders such as `device_extensions.swapchain` before use.
- `Device::PartialEq` compares `Arc` identity, not raw Vulkan handles or physical-device identity.
- `Instance::get_adapter_surface_capabilities` receives the surface loader separately from the raw handle and returns format/mode data alongside generic capabilities.
- In debug builds `DebugName` stores its string; in non-debug builds its dereference returns a placeholder and does not retain the supplied name. `debug_name_object` assigns names through debug utils.
- GPU allocation free is routed through `DeviceObject<GpuAllocation>` into the VMA allocator mutex with `free_memory`. Do not call allocator free on an allocation after ownership has moved or dropped.
- `ExternallyManagedObject<Device>` must destroy each owned handle once. The `vk::CommandBuffer` implementation is intentionally a no-op: its outer command-buffer wrapper frees it through its command pool. `vk::CommandPool` itself is really destroyed.
- Device handle types are hand-written around `DeviceObject` with `DeviceOwned` implementations where needed; the old `define_device_owned_handle!` macro no longer exists.

## Conventions
- Nightly crate features: `debug_closure_helpers`, `slice_partition_dedup`.
- Keep unsafe Vulkan destruction and queue locking contracts explicit; inspect each trait's safety requirements when adding implementations.
- Do not commit changes. Verify with `cargo check --workspace` and `cargo test -p renderer_device` from the workspace root.
- Keep `renderer_device` independent of higher-level render resource/swapchain wrapper types; public raw Vulkan handles and capability records form this crate's boundary.

## Change guidance
- A new device-owned Vulkan handle needs a concrete `ExternallyManagedObject<Device>` implementation before it can be wrapped in `DeviceObject<T>`.
- Keep `DeviceOwned<T>` aligned with its wrapper: `device()` returns the strong owner used for destruction, and `handle()` returns the wrapped raw handle.
- A pooled type needs both `Pooled::create_from_pool` and `Recycle::recycle`, a matching inline entry in `DevicePools`, and initialization from its `Weak<DeviceInner>`.
- Keep dedicated and pooled sync paths semantically separate. Reused fences may require reset; timeline semaphore reuse must honor counter values.
- `DeviceObject::new_debug_named` requires `DebugNameable`; use `DeviceObject::new` when no valid debug naming route exists.
- Never put `Device` or `Arc<Device>` inside a value stored inline in `DeviceInner` unless the ownership cycle is deliberately redesigned.
- Raw Vulkan calls must use the instance/device associated with their handle. Document synchronization and lifetime obligations when exposing raw handles.
- New queue operations must use `Queue::with_locked` or explicitly document their external-synchronization safety contract.
- When adding required features/extensions, update adapter support checks, physical-device queries, and logical-device enablement together.
- `required_extensions` warns when an additional requested device extension is unsupported and always includes swapchain; warnings are not a hard-failure policy.
- Keep platform surface creation out of adapter APIs; selection/capability methods use raw Vulkan surface and display handles.
- Pipeline cache data is an optimization: load failures fall back to an empty cache; export/write failures are logged during teardown.
- Preserve `extern crate self as renderer_device`: proc-macro expansion uses this absolute path inside crate tests as well as downstream code.
- The derive parser's diagnostics require named-field structs and exactly one `#[device]` field. Do not rely on the incomplete tuple-field branch.
- `get_available_extensions` ignores failed per-layer extension enumeration results; extension compatibility otherwise compares names and minimum spec versions.
- On `Instance`/`Device` teardown changes, inspect explicit `Drop` implementations and field drop order, not only local wrapper drops.