Vulkan RTX Ray Tracing Prototype

A real-time hardware ray tracing renderer built on the Vulkan 1.4 ray tracing pipeline, integrated with the Qt 6 windowing system.

Overview

This project implements hardware-accelerated ray tracing using the VK_KHR_ray_tracing_pipeline and VK_KHR_acceleration_structure extensions. It supports physically based materials, ray traced shadows and reflections, and free-fly navigation through a procedurally generated test scene.

The codebase is intended as a reference implementation for integrating Vulkan ray tracing with a Qt-based application window, rather than a full game engine. It focuses on correctness and clarity of the ray tracing pipeline setup: acceleration structure construction, shader binding tables, and descriptor management.

Features

• Hardware ray tracing via bottom-level and top-level acceleration structures (BLAS/TLAS)
• Ray tracing pipeline with a shader binding table (SBT) covering ray generation, miss, and closest-hit stages
• Physically based rendering using a metallic/roughness material model
• Ray traced shadows computed via shadow ray dispatch from the closest-hit shader
• Recursive ray traced reflections on metallic surfaces, with configurable bounce depth
• Native window management and Vulkan surface creation through Qt 6
• WASD + mouse FPS-style camera with acceleration-based smoothing
• Procedurally generated scene with multiple material types for testing PBR response
• VMA-based memory allocation for buffers and images

Requirements

Component	Minimum	Recommended
GPU	NVIDIA RTX 2060 / AMD RX 6600	NVIDIA RTX 4060 or higher
Vulkan SDK	1.3.0	1.4.350+
Qt	6.5	6.9+
CMake	3.25	4.0+
Compiler	MSVC 2022 / GCC 12	MSVC 2025 / GCC 14
Operating System	Windows 10, Linux	Windows 11

Hardware ray tracing support is required. GPUs without VK_KHR_ray_tracing_pipeline support (e.g., pre-RTX NVIDIA cards, older AMD GPUs) cannot run this project.

Getting Started

Prerequisites

1. Install the Vulkan SDK (1.3.0 or later).
2. Install Qt 6.5 or later, including the Qt6::Gui and Qt6::Widgets modules.
3. Verify your GPU driver supports ray tracing extensions:

   vulkaninfo | grep -i "ray_tracing"

Build (Windows / MSVC)

# Configure
cmake --preset msvc-release

# Build
cmake --build --preset msvc-release

# Run
./build/release/VulkanPrototype.exe

Available presets are defined in CMakePresets.json and include debug and release configurations for both MSVC and GCC toolchains.

Build (Linux)

export VULKAN_SDK=/path/to/vulkan/sdk/x.x.x.x/x86_64

cmake --preset linux-release
cmake --build --preset linux-release
./build/release/VulkanPrototype

Debug Build

cmake --preset msvc-debug
cmake --build --preset msvc-debug

Debug presets enable Vulkan validation layers and disable shader optimization for easier debugging with tools such as RenderDoc or NSight Graphics.

Controls

Key	Action
W / A / S / D	Move forward / left / backward / right
Space	Move up
Ctrl	Move down
Shift	Sprint
Mouse	Look around
Esc	Exit

Project Structure

VulkanPrototype/
├── CMakeLists.txt
├── CMakePresets.json
├── src/
│   ├── main.cpp              # Application entry point
│   ├── VulkanEngine.*        # Instance, device, and swapchain management
│   ├── RayTracingPipeline.*  # RT pipeline and acceleration structures
│   ├── Camera.*              # FPS camera system
│   ├── Scene.*               # Scene graph and geometry
│   ├── QtVulkanWindow.*      # Qt-Vulkan window integration
│   └── InputHandler.*        # Input management
├── shaders/
│   ├── raygen.rgen           # Ray generation shader
│   ├── miss.rmiss            # Miss shader (sky)
│   └── closesthit.rchit      # Closest-hit shader (PBR shading)
└── assets/                   # Textures and models (reserved for future use)

Architecture

Qt Window System
        │
        ▼
QtVulkanWindow + InputHandler
  - VkSurfaceKHR creation
  - Keyboard / mouse event dispatch
        │
        ▼
VulkanEngine + VMA
  - Instance, physical/logical device selection
  - Swapchain management
  - Buffer and image allocation
        │
        ▼
RayTracingPipeline + Scene + Camera
  - BLAS/TLAS construction
  - Shader binding table
  - View / projection matrices
        │
        ▼
GLSL Ray Tracing Shaders (compiled to SPIR-V)
  - raygen.rgen
  - miss.rmiss
  - closesthit.rchit

Rendering Pipeline

1. Acceleration structure build. Scene geometry is uploaded to GPU buffers and used to build a BLAS per mesh, then combined into a single TLAS with per-instance transforms.
2. Descriptor setup. The TLAS, output image, camera uniform buffer, and material buffers are bound via a descriptor set consumed by the ray tracing pipeline.
3. Ray dispatch. vkCmdTraceRaysKHR is issued once per frame, generating a primary ray per pixel from the raygen shader.
4. Shading. The closest-hit shader evaluates PBR shading, dispatches a shadow ray toward the light source, and recursively traces a reflection ray for metallic surfaces up to a fixed bounce depth.
5. Presentation. The output image is copied to the swapchain image and presented through the Qt-managed surface.

Configuration

Runtime parameters can be adjusted in src/Scene.cpp:

Parameter	Description	Default
kMaxRecursionDepth	Maximum reflection ray bounces	4
kShadowRayBias	Offset to avoid shadow acne	0.001
kSampleCount	Samples per pixel per frame	1
kFieldOfView	Camera vertical FOV in degrees	60.0

Camera movement speed, sprint multiplier, and mouse sensitivity are configured in src/Camera.cpp.

Performance Notes

• The current implementation traces one sample per pixel per frame without temporal accumulation or denoising. Expect visible noise on rough or reflective surfaces.
• TLAS rebuilds occur every frame; for static scenes, consider building once and updating only instance transforms.
• Shader binding table alignment must match VkPhysicalDeviceRayTracingPipelinePropertiesKHR::shaderGroupBaseAlignment. Misalignment will cause validation errors or undefined behavior on some drivers.
• Recursive reflections beyond a depth of 4-6 bounces provide diminishing visual return relative to their cost; the default is tuned for a balance between quality and frame time on RTX 4060-class hardware.

Troubleshooting

Validation layer errors on startup Ensure the Vulkan SDK's validation layers are installed and VK_LAYER_KHRONOS_validation is available. Debug presets enable validation by default.

vkCreateRayTracingPipelinesKHR fails Confirm the physical device reports support for both VK_KHR_ray_tracing_pipeline and VK_KHR_acceleration_structure. Older GPUs or outdated drivers are the most common cause.

Qt window does not display Vulkan output Verify that platform-specific private headers required for native surface creation are available for your platform, as this project relies on platform-specific surface handles.

Shader compilation errors Shaders are compiled to SPIR-V as part of the build via glslc. Confirm the Vulkan SDK's bin directory (containing glslc) is on your PATH.

Black screen with no errors Check that the TLAS build completed successfully and that the ray generation shader's output image layout transition occurs before vkCmdTraceRaysKHR is called.

Roadmap

• Texture and model loading via assets/
• Temporal accumulation and denoising
• Multiple light source support
• Screenshot capture and example scene presets
• Compute-based post-processing pass (tone mapping, bloom)
• glTF scene import

Contributing

Issues and pull requests are welcome. For significant changes, please open an issue first to discuss the proposed change. Ensure code follows the existing style and that new shader code compiles cleanly with glslc before submitting.

License

MIT License — see LICENSE for details.