Custom Model Loading

Learn how to define, create, and load a custom neural network model with Tiny-DL-Inference.

---

Overview

This guide shows you how to build a model from scratch, including:

• Defining the model architecture
• Creating weight tensors with proper shapes
• Loading the model into the inference engine
• Understanding layer connections and parameters

---

Model Definition Structure

Every model in Tiny-DL-Inference is defined as a JavaScript object with two main sections: layers and weights.

const modelDef = {
  name: 'MyModel',           // Optional: model identifier
  layers: [ /* layer definitions */ ],
  weights: { /* weight tensors */ }
};

---

Step 1: Define Layers

Each layer is an object with these properties:

Property	Type	Description
name	string	Unique identifier for this layer
type	string	Operator type ('conv2d', 'relu', 'dense', etc.)
inputs	string[]	Names of input tensors/layers
params	object	Operator-specific parameters

Example Layer Definition

{
  name: 'conv1',
  type: 'conv2d',
  inputs: ['input', 'conv1_weight', 'conv1_bias'],
  params: {
    kernelSize: [3, 3],
    stride: [1, 1],
    padding: [1, 1],
    useBias: true
  }
}

Special Input Names

• 'input' - Reserved name for the model's primary input tensor
• Layer names from previous layers - Reference outputs of earlier layers

---

Step 2: Create Weights

Weights are defined as a map from tensor names to data and shapes:

weights: {
  conv1_weight: {
    data: new Float32Array(/* weight values */),
    shape: [outputChannels, inputChannels, kernelHeight, kernelWidth]
  },
  conv1_bias: {
    data: new Float32Array(/* bias values */),
    shape: [outputChannels]
  }
}

Weight Shape Conventions

Layer Type	Weight Shape	Bias Shape
Conv2d	[outChannels, inChannels, kH, kW]	[outChannels]
Dense	[units, inFeatures]	[units]

Creating Random Weights

function randomWeights(shape: number[], scale = 0.01): Float32Array {
  const size = shape.reduce((a, b) => a * b, 1);
  return new Float32Array(size).map(() =>
    (Math.random() - 0.5) * 2 * scale
  );
}

// Usage
const conv1Weights = randomWeights([32, 1, 3, 3], 0.1);
const fcWeights = randomWeights([10, 3136], 0.01);

---

Step 3: Complete Example

Here is a complete custom model for image classification:

import { InferenceEngine } from 'tiny-dl-inference';

async function createAndLoadModel() {
  const engine = new InferenceEngine();
  await engine.initialize();

  // Define the model
  const modelDef = {
    name: 'CustomClassifier',
    layers: [
      // Convolutional block
      {
        name: 'conv1',
        type: 'conv2d',
        inputs: ['input', 'conv1_weight', 'conv1_bias'],
        params: {
          kernelSize: [3, 3],
          stride: [1, 1],
          padding: [1, 1],
          useBias: true
        }
      },
      {
        name: 'relu1',
        type: 'relu',
        inputs: ['conv1'],
        params: {}
      },
      {
        name: 'pool1',
        type: 'maxpool',
        inputs: ['relu1'],
        params: {
          poolSize: [2, 2],
          stride: [2, 2]
        }
      },
      // Dense classifier
      {
        name: 'flatten',
        type: 'flatten',
        inputs: ['pool1'],
        params: {}
      },
      {
        name: 'fc',
        type: 'dense',
        inputs: ['flatten', 'fc_weight', 'fc_bias'],
        params: {
          units: 10,
          useBias: true
        }
      },
      {
        name: 'output',
        type: 'softmax',
        inputs: ['fc'],
        params: { axis: -1 }
      }
    ],
    weights: {
      conv1_weight: {
        data: randomWeights([32, 3, 3, 3]),
        shape: [32, 3, 3, 3]
      },
      conv1_bias: {
        data: new Float32Array(32).fill(0),
        shape: [32]
      },
      fc_weight: {
        data: randomWeights([10, 32 * 16 * 16]),
        shape: [10, 32 * 16 * 16]
      },
      fc_bias: {
        data: new Float32Array(10).fill(0),
        shape: [10]
      }
    }
  };

  // Load the model
  await engine.loadModel(modelDef);
  console.log('Model loaded successfully');

  return engine;
}

function randomWeights(shape: number[], scale = 0.01): Float32Array {
  const size = shape.reduce((a, b) => a * b, 1);
  return new Float32Array(size).map(() => (Math.random() - 0.5) * 2 * scale);
}

---

Step 4: Run Inference

async function runInference(engine: InferenceEngine) {
  // Create input tensor [batch=1, channels=3, height=32, width=32]
  const inputData = new Float32Array(1 * 3 * 32 * 32);
  // Fill with actual data...
  const input = engine.tensorFromArray(inputData, [1, 3, 32, 32]);

  // Run inference
  const output = await engine.infer(input);
  const predictions = await output.download();

  console.log('Predictions:', predictions);

  // Cleanup
  input.destroy();
  output.destroy();
}

---

Supported Layer Types

Type	Description	Key Parameters
conv2d	2D convolution	kernelSize, stride, padding, useBias
relu	ReLU activation	None
maxpool	Max pooling	poolSize, stride
flatten	Flatten to 1D	None
dense	Fully connected layer	units, useBias
softmax	Softmax activation	axis

---

Loading Weights from Files

In production, weights are typically loaded from files rather than generated randomly:

async function loadWeightsFromFile(url: string): Promise<Record<string, { data: Float32Array; shape: number[] }>> {
  const response = await fetch(url);
  const weightData = await response.json();

  const weights: Record<string, { data: Float32Array; shape: number[] }> = {};
  for (const [name, entry] of Object.entries(weightData)) {
    weights[name] = {
      data: new Float32Array(entry.data),
      shape: entry.shape
    };
  }
  return weights;
}

// Usage
const externalWeights = await loadWeightsFromFile('model-weights.json');
const modelDef = {
  layers: [ /* ... */ ],
  weights: externalWeights
};

---

Common Mistakes

Shape Mismatch

// Wrong: Dense input features don't match flattened size
{
  name: 'fc',
  type: 'dense',
  inputs: ['flatten', 'fc_weight', 'fc_bias'],
  params: { units: 10 }
}
// Weight shape [10, 1000] but flatten produces 8192 features

// Correct: Match the actual flattened size
fc_weight: {
  data: randomWeights([10, 8192]),
  shape: [10, 8192]
}

Missing Bias

// If useBias: true, you must provide bias weights
{
  name: 'conv1',
  type: 'conv2d',
  inputs: ['input', 'conv1_weight', 'conv1_bias'],
  params: { useBias: true }
}

Incorrect Input References

// Wrong: referencing a layer that doesn't exist
{
  name: 'relu1',
  type: 'relu',
  inputs: ['nonexistent_layer'],  // Error!
  params: {}
}

// Correct: reference the actual previous layer name
{
  name: 'relu1',
  type: 'relu',
  inputs: ['conv1'],
  params: {}
}

---

Next Steps

• See MNIST Example for a complete classification pipeline
• Learn about Performance Benchmarking to optimize your model
• Read the API Reference for detailed operator parameters