Quick Start

Get Tiny-DL-Inference up and running in minutes.

---

Prerequisites

System Requirements

• Node.js: 18.0 or higher (20.x recommended)
• npm: 8.0 or higher
• GPU: WebGPU-compatible graphics card

Browser Support

Requires a WebGPU-enabled browser:

• Chrome 113+ or Edge 113+ (recommended)
• Safari 18+ on macOS Sonoma+

To check WebGPU support:

if (navigator.gpu) {
  console.log('WebGPU is supported!');
} else {
  console.error('WebGPU is not supported in this browser');
}

---

Installation

From npm

npm install tiny-dl-inference

From Source

# Clone the repository
git clone https://github.com/LessUp/tiny-dl-inference.git
cd tiny-dl-inference

# Install dependencies
npm install

# Build the project
npm run build

# Run tests to verify installation
npm test

---

First Steps

1. Basic Setup

import { InferenceEngine, ModelLoader } from 'tiny-dl-inference';

// Create and initialize the engine
const engine = new InferenceEngine();
await engine.initialize();
console.log('Engine initialized successfully');

2. Create a Tensor

// Create a tensor from an array
const data = new Float32Array([1.0, 2.0, 3.0, 4.0]);
const tensor = engine.tensorFromArray(data, [1, 4]);

// Or create with zeros/ones
import { GPUContext, Tensor } from 'tiny-dl-inference';
const context = new GPUContext();
await context.initialize();

const zeros = Tensor.zeros(context, [1, 3, 224, 224]);
const ones = Tensor.ones(context, [1, 3, 224, 224]);

3. Load a Model

// Define your model structure
const modelDef = {
  layers: [
    { type: 'conv2d', name: 'conv1', inputs: ['input'], params: { channels: 32, kernelSize: 3 } },
    { type: 'relu', name: 'relu1', inputs: ['conv1'], params: {} },
    { type: 'maxpool', name: 'pool1', inputs: ['relu1'], params: { poolSize: 2 } },
    { type: 'flatten', name: 'flatten', inputs: ['pool1'], params: {} },
    { type: 'dense', name: 'output', inputs: ['flatten'], params: { units: 10 } },
    { type: 'softmax', name: 'softmax', inputs: ['output'], params: { axis: 1 } }
  ],
  weights: {
    'conv1.weight': { data: conv1WeightData, shape: [32, 1, 3, 3] },
    'conv1.bias': { data: conv1BiasData, shape: [32] },
    'output.weight': { data: denseWeightData, shape: [10, 1152] },
    'output.bias': { data: denseBiasData, shape: [10] }
  }
};

await engine.loadModel(modelDef);

4. Run Inference

// Prepare input data (e.g., 28x28 grayscale image)
const imageData = new Float32Array(28 * 28).fill(0.5);
const input = engine.tensorFromArray(imageData, [1, 1, 28, 28]);

// Run inference
const output = await engine.infer(input);
const predictions = await output.download();

console.log('Predictions:', predictions);

// Cleanup
engine.destroy();

---

MNIST Example

Complete MNIST digit classification example:

import { InferenceEngine, ModelLoader } from 'tiny-dl-inference';

async function classifyDigit(imageData: Float32Array): Promise<number> {
  const engine = new InferenceEngine();
  await engine.initialize();
  
  try {
    // Load pre-trained MNIST model
    const loader = new ModelLoader();
    const model = await loader.loadFromJSON('mnist-model.json');
    await engine.loadModel(model);
    
    // Prepare input: normalize to [0, 1]
    const normalized = imageData.map(v => v / 255.0);
    const input = engine.tensorFromArray(normalized, [1, 1, 28, 28]);
    
    // Run inference
    const output = await engine.infer(input);
    const logits = await output.download();
    
    // Get prediction (argmax)
    let maxIndex = 0;
    let maxValue = logits[0];
    for (let i = 1; i < logits.length; i++) {
      if (logits[i] > maxValue) {
        maxValue = logits[i];
        maxIndex = i;
      }
    }
    
    return maxIndex;
  } finally {
    engine.destroy();
  }
}

// Usage
const digit = await classifyDigit(yourImageData);
console.log(`Predicted digit: ${digit}`);

---

Memory Layout

Tiny-DL-Inference supports two memory layouts:

NCHW (PyTorch-style)

const input = engine.tensorFromArray(data, [1, 3, 224, 224], { layout: 'NCHW' });
// Shape: [Batch, Channel, Height, Width]

NHWC (TensorFlow-style)

const input = engine.tensorFromArray(data, [1, 224, 224, 3], { layout: 'NHWC' });
// Shape: [Batch, Height, Width, Channel]

Note

Current Conv2d and MaxPool execution paths support NCHW only. The engine includes layout conversion utilities for working with different formats.

---

Common Pitfalls

1. WebGPU Context Issues

// ❌ Wrong: Using wrong context
const engine = new InferenceEngine();
await engine.initialize();
const wrongContext = new GPUContext();
await wrongContext.initialize();
const tensor = Tensor.zeros(wrongContext, [1, 4]);
await engine.infer(tensor); // Error: tensor uses different context

// ✅ Correct
const input = engine.tensorFromArray(data, [1, 4]); // Uses engine's context

2. Shape Mismatch

// ❌ Wrong: Shape doesn't match data length
const data = new Float32Array(784); // 28*28
const tensor = engine.tensorFromArray(data, [1, 1, 32, 32]); // Error!

// ✅ Correct
const tensor = engine.tensorFromArray(data, [1, 1, 28, 28]);

3. Resource Cleanup

// Always cleanup to prevent memory leaks
const engine = new InferenceEngine();
await engine.initialize();

// ... use engine ...

// ✅ Cleanup
engine.destroy();

---

Next Steps

• Learn about available operators
• Explore the API reference
• Read about optimization techniques
• Check out more examples

---

Troubleshooting

WebGPU Not Supported

If you see WebGPUNotSupportedError:

1. Update your browser to the latest version
2. Enable WebGPU in browser flags (chrome://flags)
3. Check GPU compatibility

Out of Memory

For large models:

1. Process inputs in smaller batches
2. Use engine.destroy() between inferences
3. Monitor GPU memory usage

Build Errors

Make sure you have:

• TypeScript 5.0+
• Node.js 18+
• All dependencies installed (npm install)