Tensor Operations

Working with tensors in Tiny-DL-Inference.

---

Overview

The Tensor class represents multi-dimensional arrays stored on the GPU. It provides methods for data transfer, shape manipulation, and memory layout conversion.

---

Creating Tensors

From Array Data

import { InferenceEngine } from 'tiny-dl-inference';

const engine = new InferenceEngine();
await engine.initialize();

// Create from Float32Array
const data = new Float32Array([1, 2, 3, 4, 5, 6]);
const tensor = engine.tensorFromArray(data, [2, 3]);
// Shape: [2, 3]

Factory Methods

import { GPUContext, Tensor } from 'tiny-dl-inference';

const context = new GPUContext();
await context.initialize();

// Zeros
const zeros = Tensor.zeros(context, [1, 3, 224, 224]);

// Ones
const ones = Tensor.ones(context, [1, 3, 224, 224]);

// Filled with value
const filled = Tensor.filled(context, [2, 2], 3.14);

---

Data Transfer

Upload Data

const data = new Float32Array(224 * 224 * 3);
// ... fill data ...

const tensor = Tensor.zeros(context, [1, 3, 224, 224]);
await tensor.upload(data);

Download Data

const output = await tensor.download();
console.log(output); // Float32Array

---

Shape Operations

Zero-Copy Reshape

// Original tensor owns the buffer
const original = Tensor.zeros(context, [1, 3, 224, 224]);

// View shares the buffer (zero overhead)
const view = original.reshape([1, 3 * 224 * 224]);

// View is cheap to create and destroy
view.destroy();  // No-op: doesn't own buffer
original.destroy();  // Actually destroys GPU buffer

Shape Properties

console.log(tensor.shape);  // [1, 3, 224, 224]
console.log(tensor.size);   // 150528 (total elements)
console.log(tensor.rank);   // 4 (number of dimensions)

---

Memory Layout

Supported Layouts

Layout	Description	Shape Order
NCHW	Channel-first	[Batch, Channel, Height, Width]
NHWC	Channel-last	[Batch, Height, Width, Channel]

Layout Conversion

// Create NCHW tensor (default)
const nchw = Tensor.zeros(context, [1, 3, 224, 224], { layout: 'NCHW' });

// Convert to NHWC
const nhwc = await nchw.convertLayout('NHWC');
console.log(nhwc.shape);  // [1, 224, 224, 3]

// Roundtrip preserves data
const back = await nhwc.convertLayout('NCHW');
// back.shape: [1, 3, 224, 224]

---

Memory Management

Ownership Model

// Owner tensor
const owner = new Tensor(context, [1, 4]);
owner.ownsBuffer;  // true

// View tensor
const view = owner.reshape([2, 2]);
view.ownsBuffer;  // false

// Only owner destruction frees memory
view.destroy();   // No-op
owner.destroy();  // Frees GPU buffer

Cleanup Best Practices

// Always cleanup tensors when done
const tensor = Tensor.zeros(context, [1, 4]);
// ... use tensor ...
tensor.destroy();

// For inference engines
const engine = new InferenceEngine();
await engine.initialize();
// ... run inference ...
engine.destroy();  // Cleans up all resources

---

API Reference

See the Tensor API Reference for complete documentation.