GPU Context

Understanding GPU context management in Tiny-DL-Inference.

---

Overview

The GPUContext class is the foundation of Tiny-DL-Inference. It manages WebGPU device initialization, buffer allocation, and command submission.

---

Basic Usage

Initialization

import { GPUContext } from 'tiny-dl-inference';

const context = new GPUContext();
await context.initialize();

Configuration Options

interface GPUContextConfig {
  powerPreference?: 'low-power' | 'high-performance';
  forceFallbackAdapter?: boolean;
}

await context.initialize({
  powerPreference: 'high-performance'
});

---

Resource Management

Buffer Creation

const buffer = context.createBuffer({
  size: 1024,
  usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST
});

Deferred Destruction

Prevents use-after-free by delaying buffer destruction until GPU work completes:

// Queue buffer for destruction after GPU finishes
context.deferDestroy(tempBuffer);

// Wait for all GPU work to complete
await context.sync();
// Now deferred buffers are actually destroyed

---

Command Submission

Creating Commands

const encoder = context.createCommandEncoder();

// Encode compute passes
const pass = encoder.beginComputePass();
// ... dispatch compute operations
pass.end();

// Submit to GPU
const commandBuffer = encoder.finish();
context.submit([commandBuffer]);

Synchronization

// Wait for all submitted work to complete
await context.sync();

---

Memory Management

Buffer Lifecycle

Create → Use → Destroy
   │      │       │
   │      │       └── deferDestroy() queues for after GPU work
   │      └────────── encode commands, submit to queue
   └───────────────── GPUContext.createBuffer()

Best Practices

1. Use deferred destruction for temporary buffers
2. Minimize sync calls - they block the CPU
3. Reuse buffers when possible instead of creating new ones

---

Error Handling

Common Errors

Error	Cause	Solution
WebGPUNotSupportedError	Browser lacks WebGPU	Use modern browser
DeviceInitializationError	GPU init failed	Check drivers

Example

try {
  const context = new GPUContext();
  await context.initialize();
} catch (error) {
  if (error instanceof WebGPUNotSupportedError) {
    console.log('Please use a WebGPU-compatible browser');
  } else if (error instanceof DeviceInitializationError) {
    console.log('GPU initialization failed:', error.message);
  }
}

---

API Reference

See the GPUContext API Reference for complete documentation.