Initial FrameAssistant baseline from the vrdashboard template
Template stripped to the overlay platform (examples, demo UI and Frame test package removed), renamed to FrameAssistant, and working rules for the project recorded in CLAUDE.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -0,0 +1,162 @@
|
||||
# Architecture
|
||||
|
||||
A dashboard is **one process**. It renders QML scenes off-screen into textures and hands
|
||||
them to SteamVR as overlays. Input flows the other way: SteamVR reports laser, scrolling,
|
||||
keyboard and controller actions, and the app turns them into ordinary Qt events. The QML
|
||||
never knows it is in VR, which is why it also runs in a plain desktop window (`--desktop`).
|
||||
|
||||
## Structure
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
app["<b>Your app</b><br/>class Dashboard : DashboardApplication<br/>members, loading() / started()"]
|
||||
subgraph base["DashboardApplication"]
|
||||
direction TB
|
||||
engine["QQmlEngine<br/>(shared by all surfaces)"]
|
||||
runtime["VrRuntime<br/>SteamVR connection"]
|
||||
nav["ControllerNavigation"]
|
||||
end
|
||||
subgraph surfaces["Surfaces: one per view"]
|
||||
direction TB
|
||||
s1["<b>Surface 'main'</b><br/>QML root → Renderer → RenderBackend<br/>DashboardOverlay + InputBridge"]
|
||||
s2["<b>Surface 'settings'</b> (optional)<br/>same parts"]
|
||||
end
|
||||
steamvr(["SteamVR"])
|
||||
app -- "inherits" --> base
|
||||
base -- "creates, drives" --> surfaces
|
||||
surfaces -- "textures" --> steamvr
|
||||
steamvr -- "laser, scroll, keyboard" --> surfaces
|
||||
steamvr -- "controller actions" --> nav
|
||||
nav -- "keys to the active surface" --> surfaces
|
||||
```
|
||||
|
||||
**`DashboardApplication`** (`src/app/DashboardApplication.*`) is the base class of every app.
|
||||
It owns everything that isn't app-specific: command line, logging, the SteamVR connection,
|
||||
choosing the render backend, the surfaces, controller navigation, the auxiliary modes, and
|
||||
the order in which all of that is created and destroyed.
|
||||
|
||||
Your app derives from it:
|
||||
- **Constructor:** configuration (`addSurface()`, optionally `setAppKey()`,
|
||||
`setDisplayName()`, `setQuickStyle()`, `setQmlModule()`).
|
||||
- **Hooks:**
|
||||
- `loading()`: the shared QML engine, the surfaces and their render backends exist; QML
|
||||
isn't loaded yet. Set up what the QML needs here.
|
||||
- `started()`: everything runs. Connect to surfaces and input here.
|
||||
- **Accessors:** `mode()`, `appKey()`, `appName()`, `engine()`, `surfaces()`, `surface()`
|
||||
(the first), `surface(id)`, `vrRuntime()`, `navigation()`.
|
||||
- **Lifetime:** the base constructor creates the `QGuiApplication`, so your members can be
|
||||
QObjects with timers. All QML and surfaces are torn down at the end of `exec()`, while your
|
||||
object and its members still exist.
|
||||
|
||||
**`Surface`** (`src/app/Surface.*`) is one view: a QML root, its renderer, its render backend
|
||||
and its overlay. It's a component you configure (`SurfaceConfig`: id, QML root type, tab name,
|
||||
texture size, icon) and connect to (`visibilityChanged`, `activated`), never subclass.
|
||||
- All surfaces share one QML engine, so models and singletons are visible in every view.
|
||||
- Overlay key: `<app key>.<id>`, or the app key itself for an empty id.
|
||||
- A surface renders only while its overlay is visible.
|
||||
|
||||
**Render layer** (`src/render/`):
|
||||
- **`OffscreenQuickRenderer`** drives Qt Quick through `QQuickRenderControl`, without a window.
|
||||
- It renders on demand (scene change or animation), capped at `--max-fps`, on an absolute
|
||||
schedule so slow frames don't lower the frame rate.
|
||||
- **`RenderBackend`** is the *only* place with graphics-API code: device setup, texture,
|
||||
hand-off description for SteamVR, readback.
|
||||
- **`VulkanRenderBackend`** (default): Qt creates the device on the GPU SteamVR names, with
|
||||
the extensions SteamVR requires. The backend owns the image and moves it to
|
||||
`TRANSFER_SRC_OPTIMAL` after each frame, the layout OpenVR wants, and back before the next.
|
||||
- **`OpenGLRenderBackend`** (fallback): on the Frame it runs through zink on Vulkan.
|
||||
|
||||
**VR layer** (`src/vr/`):
|
||||
- **`VrRuntime`**: `VR_Init` as overlay app, polls SteamVR events and routes them by overlay
|
||||
handle. It also reports SteamVR's Vulkan requirements, and registers and launches manifests.
|
||||
- **`Overlay`**: base of all overlays. It handles input routing, visibility and frame
|
||||
submission.
|
||||
- **`DashboardOverlay`** is the dashboard tab with its icon. A world overlay would be
|
||||
another subclass (planned).
|
||||
|
||||
**Input layer** (`src/input/`):
|
||||
- **`OverlayInputBridge`** (one per overlay):
|
||||
- **Laser:** turned into mouse events, with the y-axis flipped (OpenVR counts from the
|
||||
bottom).
|
||||
- **Scrolling:** thumbstick scrolling becomes wheel events.
|
||||
- **Keyboard:** opens the SteamVR keyboard when a text field gets focus, and turns typed
|
||||
characters into key events.
|
||||
- **`ControllerNavigation`** (one per app): SteamVR Input actions (`resources/input/`, rebindable
|
||||
in SteamVR) become Tab/arrow/Space/Escape for the *active* surface, i.e. the one last opened
|
||||
or pointed at. Qt Quick's own focus handling does the rest.
|
||||
|
||||
## One frame
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant QML as QML scene
|
||||
participant R as OffscreenQuickRenderer
|
||||
participant B as RenderBackend (Vulkan)
|
||||
participant O as Overlay
|
||||
participant S as SteamVR
|
||||
QML->>R: scene changed / animation tick
|
||||
Note over R: wait for the next slot (≤ max-fps)
|
||||
R->>B: beginFrame (image back to COLOR_ATTACHMENT)
|
||||
R->>R: polish, sync, render (Qt Quick)
|
||||
R->>B: finishFrame (image → TRANSFER_SRC, GPU done)
|
||||
R->>O: frame sink: texture + bounds
|
||||
O->>S: SetOverlayTexture (SteamVR copies it)
|
||||
```
|
||||
|
||||
## Input
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant S as SteamVR
|
||||
participant V as VrRuntime
|
||||
participant I as OverlayInputBridge
|
||||
participant Q as QML (QQuickWindow)
|
||||
S->>V: overlay event (mouse, scroll, keyboard)
|
||||
V->>I: routed by overlay handle
|
||||
I->>Q: QMouseEvent / QWheelEvent / QKeyEvent
|
||||
Q-->>I: text field got focus
|
||||
I->>S: ShowKeyboardForOverlay
|
||||
```
|
||||
|
||||
## Start and shutdown (overlay mode)
|
||||
|
||||
1. **Start:**
|
||||
1. Connect to SteamVR (`VR_Init`).
|
||||
2. Create one render backend per surface. The first surface decides the API, including a
|
||||
fallback to OpenGL; the others use the same API, because Qt's graphics API is
|
||||
process-wide.
|
||||
3. Call `loading()`.
|
||||
4. Load each surface's QML.
|
||||
5. Create the overlays.
|
||||
6. Set up controller navigation.
|
||||
7. Call `started()`, then run the event loop.
|
||||
2. **Shutdown runs in reverse**, with one rule found on the Frame: **overlays first, then
|
||||
SteamVR, then the graphics devices.** OpenVR requires shutting down before the devices it
|
||||
got textures from are destroyed. Otherwise its client library frees objects on a dead
|
||||
`VkDevice` at exit, which crashed on the Frame. `runOverlay()` encodes this order through
|
||||
the declaration order of its objects.
|
||||
|
||||
## Building blocks in CMake
|
||||
|
||||
- **`vrdash_add_dashboard()`** (`cmake/Dashboard.cmake`) turns sources plus QML files into a
|
||||
dashboard executable. It covers:
|
||||
- the QML module, whose `QML_URI` becomes the app's default module through a generated
|
||||
source file
|
||||
- bytecode-only QML for system-Qt builds
|
||||
- `libopenvr_api.so` next to the executable
|
||||
- install rules and registration for Qt bundling
|
||||
- the check against SteamVR process names
|
||||
- **App identity** (root `CMakeLists.txt`): `project()` name and version, `VRDASH_APP_KEY`,
|
||||
`VRDASH_DISPLAY_NAME`, `VRDASH_EXECUTABLE_NAME`, texture size. CMake generates from them
|
||||
`AppConfig.h` and the SteamVR manifest, with `binary_path_linux_arm` for ARM builds and
|
||||
`binary_path_linux` for x86_64.
|
||||
|
||||
## Planned: world overlays
|
||||
|
||||
For e.g. a notification app: a `WorldOverlay` (another `Overlay` subclass, created with
|
||||
`CreateOverlay`) that the app shows and hides itself.
|
||||
- **Placement modes:** room, HUD, hand, "spawn in front, then stay".
|
||||
- **Optionally operable:** while visible, it takes the system laser
|
||||
(`VROverlayFlags_MakeOverlaysInteractiveIfVisible`).
|
||||
|
||||
The surface structure is ready for it: `SurfaceConfig` would gain a kind and a placement.
|
||||
Reference in New Issue
Block a user