RFC 0006: Logger, Config, and Profiler Infrastructure

Status

Status: Accepted Created: 2024 Last Updated: 2024

Overview

Design the infrastructure components for logging, configuration management, and performance profiling, providing observability and configurability for the inference engine.

Logger System

Design Goals

1. Thread-safe: Multiple threads can log simultaneously
2. Level-filtered: DEBUG, INFO, WARN, ERROR levels
3. Low overhead: Minimal impact on production performance
4. Structured output: Consistent formatting for parsing

Log Levels

Level	Value	Usage
DEBUG	0	Development diagnostics
INFO	1	Normal operation messages
WARN	2	Warning conditions
ERROR	3	Error conditions

API Design

enum class LogLevel {
    DEBUG = 0,
    INFO = 1,
    WARN = 2,
    ERROR = 3
};

class Logger {
public:
    // Singleton access
    static Logger& instance();

    // Configuration
    void set_level(LogLevel level);
    void set_output(std::ostream& stream);
    void set_file(const std::string& path);

    // Logging
    void log(LogLevel level, const std::string& message,
             const std::string& file = "", int line = 0);

    // Convenience macros (defined in logger.h)
    // LOG_DEBUG("message"), LOG_INFO("message"), etc.

    // Statistics
    size_t messages_logged() const;

private:
    Logger();
    LogLevel level_;
    std::mutex mutex_;
    size_t count_;
};

Convenience Macros

#define LOG_DEBUG(msg) Logger::instance().log(LogLevel::DEBUG, msg, __FILE__, __LINE__)
#define LOG_INFO(msg)  Logger::instance().log(LogLevel::INFO, msg, __FILE__, __LINE__)
#define LOG_WARN(msg)  Logger::instance().log(LogLevel::WARN, msg, __FILE__, __LINE__)
#define LOG_ERROR(msg) Logger::instance().log(LogLevel::ERROR, msg, __FILE__, __LINE__)

Configuration System

Design Goals

1. INI format: Simple, human-readable configuration files
2. Typed access: get<int>(), get<float>(), get<std::string>()
3. Defaults: Provide default values for missing keys
4. Hot reload: Ability to reload config at runtime

INI File Format

[engine]
device_id = 0
max_batch_size = 512
memory_pool_size = 268435456  ; 256MB

[gemm]
kernel_variant = optimized
enable_tuning = true
cache_size = 1000

[logging]
level = INFO
file = engine.log

API Design

class Config {
public:
    static Config& instance();

    // Loading
    bool load_from_file(const std::string& path);
    bool load_from_string(const std::string& content);

    // Typed access
    template<typename T>
    T get(const std::string& section, const std::string& key,
          const T& default_value = T{});

    // Existence check
    bool has_key(const std::string& section, const std::string& key) const;

    // Reloading
    bool reload();

    // Debugging
    void print() const;

private:
    Config();
    std::map<std::string, std::map<std::string, std::string>> data_;
    std::string config_path_;
};

Parser Implementation

• Single-pass parsing (read entire file into memory)
• Comments: lines starting with # or ;
• Sections: [section_name]
• Keys: key = value (whitespace stripped)
• No nesting, no variable substitution (keep it simple)

Profiler System

Design Goals

1. Scoped timing: RAII-based timing of code blocks
2. Hierarchical: Nested timing with parent-child relationships
3. Export: JSON/CSV output for analysis
4. Low overhead: <1% performance impact when enabled

API Design

struct ProfileRecord {
    std::string name;
    double total_time_ms;
    double avg_time_ms;
    double min_time_ms;
    double max_time_ms;
    int call_count;
};

class Profiler {
public:
    static Profiler& instance();

    // Start/stop profiling
    void start();
    void stop();

    // Scoped timer (RAII)
    class ScopeTimer {
    public:
        ScopeTimer(const std::string& name);
        ~ScopeTimer();
    private:
        std::string name_;
        cudaEvent_t start_, end_;
    };

    // Results
    std::vector<ProfileRecord> get_records() const;
    void export_json(const std::string& path) const;
    void export_csv(const std::string& path) const;
    void print_summary() const;

    // Reset
    void reset();

private:
    Profiler();
    bool enabled_;
    std::map<std::string, ProfileRecord> records_;
};

Usage Pattern

void inference_forward() {
    Profiler::ScopeTimer timer("forward_pass");

    // Layer 1
    {
        Profiler::ScopeTimer t("layer1_gemm");
        launch_gemm(...);
    }

    // Layer 2
    {
        Profiler::ScopeTimer t("layer2_gemm");
        launch_gemm(...);
    }
}

Error Handling

Component	Error Condition	Behavior
Logger	File cannot be opened	Fall back to stderr
Config	File not found	Throw std::runtime_error
Config	Invalid INI syntax	Throw std::runtime_error
Profiler	CUDA event creation fails	Throw CudaException

Testing Strategy

Logger Tests

1. Level filtering: Only messages >= level are output
2. Thread safety: Concurrent logging doesn’t crash
3. File output: Written content matches expected

Config Tests

1. Section/key parsing: All entries extracted correctly
2. Typed access: get<int>() returns correct values
3. Missing keys: Default values returned
4. Reload: Updated file content reflected after reload

Profiler Tests

1. Timing accuracy: Measured time matches expected
2. Nesting: Parent timers include child time
3. Export: JSON/CSV format is valid and parseable
4. Overhead: <1% performance impact

Implementation Files

• include/logger.h - Logger class (header-only for simplicity)
• include/config.h - Config class (header-only)
• include/profiler.h - Profiler class
• src/profiler.cu - Profiler implementation
• tests/test_logger.cpp - Logger tests
• tests/test_config.cpp - Config tests