Vulkan prototype - metal layer refactoring

XLE has a thin layer over the underlying graphics API called "Metal". This was originally built for DirectX11 and OpenGLES. But over time it became more DirectX-focused. Part of the goal of building in Vulkan support was to provide a basis for refactoring and improving the metal layer.

The goals for this layer are simple: - compile time polymorphism between underlying graphics APIs - not link time or run time. We know the target during compilation of client code - "leaky" abstraction layer - meaning that most client code is independent of the underlying graphics API - but the underlying objects are still accessible, so client code can write API-specific code when needed - very thin, minimal overhead - for example, many DeviceContext methods get inlined into client code, meaning that performance is similar to using the underlying API directly

To make this kind of layer work, we need to find abstractions that work well for all target APIs. Usually this means finding abstractions that are great for one API, and pretty good for other APIs. That can be tricky, particularly as APIs are changing and evolving over time.

Descriptor Set concept

Ideally we want the concept of "descriptor sets" to exist in the metal API somewhere. There are two reasons for this:

1. Clean up the DeviceContext interface so there are fewer BindXX(...) methods
2. pre-cook permanent descriptor sets using BoundUniforms (or otherwise), so that they can be reused frame to frame

DirectX11 has no concept of descriptor sets, though, so this could be a bit awkward. There are a couple of tricky problems

Sharing descriptors across shader stages

In modern APIs, we can bind a descriptor set to multiple shader stages; meaning that binding a resource one can make it accessible to multiple shaders. However, DirectX11 strictly separates the stages so that we explicitly bind only to a single stage.

So how to we create a single solution that handles both these cases?

One option is to expand the "root signature" concept. This defines a set of virtual binding points, which can be redirected to the true underlying binding points. The virtual binding points can know which shader stages they apply to -- and so can calls the appropriate underlying binding functions.

This might have some long-term advantages because it also allows us to abstract the C++ code completely from the true binding points expressed in the shader files. There's a little extra overhead, but maybe it's not a huge issue.

That would be a great solution for "frame-global" or long-term bindings. It would also be great for our few global SamplerState objects (which can easily become immutable in Vulkan, and handled automatically in DirectX).

But for short-term dynamic bindings, it may not be great. Often we're just binding one or two resources or constant buffers to the first few binding points, and using them for a single draw call. For this, we might need something different.

If the "dynamic" descriptor set in Vulkan was very large, we could more easily have separate binding points for resources that are shared between stages, and those that are used only in one. However, currently we need to keep this descriptor set small because we need to write to all binding points, regardless of whether they are used or not. If a shader accesses an unused descriptor, we will get a device-lost error.

This could be made better by calculating a mask of the used slots in the dynamic descriptor set for each shader. This would allow us to make better decisions about when to write to this dynamic descriptor set. Most draw calls don't need this dynamic set (it's mostly required by procedurally driven effects, and less important for data driven geometry like models), and this approach should reduce the overhead in the most common cases.