Video Frame Buffers

📹 High-performance video frame management with double-buffering and lock-free operations More...

Files
file	video_frame.c
	🎬 Video frame buffer management for client-specific ASCII rendering

Data Structures
struct	video_frame_t
	Video frame structure. More...
struct	video_frame_buffer_t
	Video frame buffer manager. More...
struct	video_frame_stats_t
	Frame statistics structure. More...
struct	simple_frame_swap_t
	Simple atomic frame swap structure. More...

Macros
#define	VIDEO_FRAME_BUFFER_SIZE   MAX_FRAME_BUFFER_SIZE
	Video frame buffer size (2MB per frame)
#define	MAX_FRAME_BUFFERS   2
	Maximum number of frame buffers (double buffering)

Functions
video_frame_buffer_t *	video_frame_buffer_create (const char *client_id)
	Create a double-buffered video frame manager.
void	video_frame_buffer_destroy (video_frame_buffer_t *vfb)
	Destroy frame buffer and free all resources.
video_frame_t *	video_frame_begin_write (video_frame_buffer_t *vfb)
	Writer API: Start writing a new frame.
void	video_frame_commit (video_frame_buffer_t *vfb)
	Writer API: Commit the frame and swap buffers.
const video_frame_t *	video_frame_get_latest (video_frame_buffer_t *vfb)
	Reader API: Get latest frame if available.
const video_frame_t *	video_frame_peek_latest (video_frame_buffer_t *vfb)
void	video_frame_get_stats (video_frame_buffer_t *vfb, video_frame_stats_t *stats)
	Get frame statistics for quality monitoring.
simple_frame_swap_t *	simple_frame_swap_create (void)
	Create a simple atomic frame swap.
void	simple_frame_swap_destroy (simple_frame_swap_t *sfs)
	Destroy simple frame swap and free resources.
void	simple_frame_swap_update (simple_frame_swap_t *sfs, const void *data, size_t size)
	Update frame data in simple frame swap.
const video_frame_t *	simple_frame_swap_get (simple_frame_swap_t *sfs)
	Get current frame from simple frame swap.

Detailed Description

📹 High-performance video frame management with double-buffering and lock-free operations

This header provides a high-performance video frame management system inspired by WebRTC, Zoom, and Google Meet architectures. It implements a lock-free double-buffering system with atomic pointer swaps for zero-copy frame transfer.

CORE FEATURES:

• Zero-copy frame transfer (pointer swaps only)
• Lock-free read operations (reader never blocks writer)
• Latest-frame-wins semantics (old frames are dropped, not queued)
• Double buffering with atomic pointer swaps
• Comprehensive quality metrics tracking
• Pre-allocated buffers for zero-allocation operation
• Frame sequence numbering for drop detection

ARCHITECTURE:

The system uses a double-buffering pattern:

• Front Buffer: Currently being displayed (reader owns)
• Back Buffer: Currently being written (writer owns)
• Atomic swap: Makes new frame available without blocking

WRITER API:

1. Call video_frame_begin_write() to get back buffer pointer
2. Write frame data directly to back buffer
3. Call video_frame_commit() to atomically swap buffers

READER API:

1. Call video_frame_get_latest() to get current frame
2. Returns NULL if no new frame since last read
3. Lock-free operation never blocks writer

PERFORMANCE BENEFITS:

• Zero-copy eliminates memcpy overhead
• Lock-free reads minimize latency
• Pre-allocated buffers avoid runtime allocations
• Latest-frame-wins prevents buffer buildup
• Metrics enable adaptive quality control

QUALITY METRICS:

The system tracks:

• Total frames received
• Total frames dropped
• Frame drop rate
• Average decode time
• Average render time

These metrics enable adaptive streaming quality based on performance.

Note

Frame data is not owned by the frame structure - it points to pre-allocated buffers managed by the frame buffer.

The writer must call video_frame_commit() after writing each frame to make it available to readers.

Readers should check for NULL returns from video_frame_get_latest() to handle cases where no new frame is available.

Author

Zachary Fogg me@zf.nosp@m.o.gg

Date

October 2025

Video Frame README

Overview

The Video Frame module provides structures and utilities for managing video frames throughout the ascii-chat pipeline. It handles RGB image data, frame metadata, and efficient frame lifecycle management.

Implementation: lib/video_frame.h

Key Features:

• Zero-copy frame transfer (pointer swaps only)
• Lock-free double-buffering system
• Latest-frame-wins semantics (old frames dropped, not queued)
• Pre-allocated buffers for zero-allocation operation
• RGB24 image frame representation
• Frame metadata (timestamp, dimensions, client ID)
• Frame sequence numbering for drop detection
• Comprehensive quality metrics tracking

Frame Structure

video_frame_t:

typedef struct {
    uint8_t *data;        // RGB24 pixel data
    uint32_t width;       // Frame width in pixels
    uint32_t height;      // Frame height in pixels
    size_t data_size;     // Total bytes (width * height * 3)
    uint32_t client_id;   // Client that owns this frame
    uint64_t timestamp;   // Capture timestamp (microseconds)
    bool owns_data;       // Whether this frame owns the data buffer
} video_frame_t;

Frame Properties:

• data: RGB24 format (3 bytes per pixel: R, G, B)
• width/height: Frame dimensions
• data_size: Always equals width * height * 3
• client_id: Which client this frame belongs to
• timestamp: Microsecond-precision capture time
• owns_data: Memory ownership flag

Operations

Double-Buffering API

The recommended way to use video frames is through the lock-free double-buffering API:

Writer API (producer thread):

// 1. Get pointer to back buffer for writing
uint8_t *write_buffer = video_frame_begin_write(frame_buffer);
 
// 2. Write frame data directly to back buffer
memcpy(write_buffer, captured_rgb_data, width * height * 3);
 
// 3. Atomically swap buffers to make frame available
video_frame_commit(frame_buffer, width, height);

Reader API (consumer thread):

// Get latest frame (returns NULL if no new frame since last read)
video_frame_t *frame = video_frame_get_latest(frame_buffer);
if (frame) {
    // Process frame data
    render_ascii(frame->data, frame->width, frame->height);
}

Manual Creation (for special cases)

For cases where you need standalone frames without the double-buffering system:

Create from Existing Data:

video_frame_t frame = {
    .data = existing_rgb_data,
    .width = 1920,
    .height = 1080,
    .data_size = 1920 * 1080 * 3,
    .client_id = client_id,
    .timestamp = get_timestamp_usec(),
    .owns_data = false  // Don't free existing_rgb_data
};

Data Access

Pixel Access:

// Get pixel at (x, y)
size_t offset = (y * frame->width + x) * 3;
uint8_t r = frame->data[offset + 0];
uint8_t g = frame->data[offset + 1];
uint8_t b = frame->data[offset + 2];
 
// Set pixel at (x, y)
frame->data[offset + 0] = 255;  // R
frame->data[offset + 1] = 128;  // G
frame->data[offset + 2] = 0;    // B

Row Access:

// Get pointer to row y
uint8_t *row = &frame->data[y * frame->width * 3];
 
// Process entire row
for (uint32_t x = 0; x < frame->width; x++) {
    uint8_t r = row[x * 3 + 0];
    uint8_t g = row[x * 3 + 1];
    uint8_t b = row[x * 3 + 2];
    // Process pixel...
}

Copying

Deep Copy:

video_frame_t *copy = SAFE_MALLOC(sizeof(video_frame_t), video_frame_t *);
copy->width = src->width;
copy->height = src->height;
copy->data_size = src->data_size;
copy->client_id = src->client_id;
copy->timestamp = src->timestamp;
copy->owns_data = true;
 
// Allocate and copy pixel data
copy->data = SAFE_MALLOC(copy->data_size, uint8_t *);
memcpy(copy->data, src->data, copy->data_size);

Shallow Copy (reference):

video_frame_t reference = *src;
reference.owns_data = false;  // Don't free the data when done

Cleanup

Free Frame:

if (frame->owns_data && frame->data) {
    free(frame->data);
    frame->data = NULL;
}
free(frame);
frame = NULL;

Validation

Validate Frame:

bool is_valid_frame(const video_frame_t *frame) {
    if (!frame) return false;
    if (!frame->data) return false;
    if (frame->width == 0 || frame->height == 0) return false;
    if (frame->data_size != frame->width * frame->height * 3) return false;
    return true;
}

Check Frame Dimensions:

if (frame->width > MAX_WIDTH || frame->height > MAX_HEIGHT) {
    log_error("Frame dimensions too large: %ux%u", frame->width, frame->height);
    return ERROR_INVALID_PARAM;
}

Ownership Rules

Ownership Flag:

• owns_data = true: Frame owns the data buffer, must free on cleanup
• owns_data = false: Frame references external data, do NOT free

Ownership Transfer:

// Sender transfers ownership
video_frame_t *frame = create_frame();
frame->owns_data = true;
send_to_queue(frame);
// Sender no longer responsible for freeing frame->data
 
// Receiver takes ownership
video_frame_t *received = receive_from_queue();
if (received->owns_data) {
    // Must free when done
    free(received->data);
}

Frame Pipeline

Capture → Process → Encode → Send:

// 1. Capture from webcam
video_frame_t *frame = webcam_capture();
frame->owns_data = true;
frame->timestamp = get_timestamp_usec();
 
// 2. Process (resize, flip, etc.)
video_frame_t *processed = process_frame(frame);
 
// 3. Convert to ASCII
ascii_frame_t *ascii = image_to_ascii(processed);
 
// 4. Send over network
send_ascii_frame(client_id, ascii);
 
// 5. Cleanup
if (frame->owns_data) free(frame->data);
free(frame);
if (processed->owns_data) free(processed->data);
free(processed);
free(ascii);

Memory Management

Allocation Strategy:

// Calculate size with overflow protection
size_t data_size = (size_t)width * (size_t)height * 3;
 
// Allocate frame data
uint8_t *data = SAFE_MALLOC(data_size, uint8_t *);
if (!data) {
    return SET_ERRNO(ERROR_OUT_OF_MEMORY, "Failed to allocate frame data");
}

Buffer Reuse:

// Reuse existing buffer if same size
if (frame->data_size == new_size) {
    // Reuse existing buffer
    memcpy(frame->data, new_data, new_size);
} else {
    // Reallocate if size changed
    if (frame->owns_data) free(frame->data);
    frame->data = SAFE_MALLOC(new_size, uint8_t *);
    frame->data_size = new_size;
}

Timestamps

Microsecond Precision:

#include <sys/time.h>
 
uint64_t get_timestamp_usec(void) {
    struct timeval tv;
    gettimeofday(&tv, NULL);
    return (uint64_t)tv.tv_sec * 1000000 + tv.tv_usec;
}
 
frame->timestamp = get_timestamp_usec();

Calculate Frame Age:

uint64_t now = get_timestamp_usec();
uint64_t age_usec = now - frame->timestamp;
double age_ms = age_usec / 1000.0;
 
if (age_ms > 100.0) {
    log_warn("Frame is stale: %.1fms old", age_ms);
}

Best Practices

DO:

• Always set owns_data correctly
• Validate frame dimensions before allocation
• Use overflow-safe size calculations
• Set timestamp when capturing frames
• Free frames when done
• Check for NULL before dereferencing

DON'T:

• Don't free frame->data if owns_data is false
• Don't use frame after freeing
• Don't assume frame->data is aligned
• Don't modify shared frame data
• Don't forget to set client_id
• Don't use stale frames (check timestamp)

Integration Example

Complete Usage:

// Capture frame
video_frame_t *frame = SAFE_MALLOC(sizeof(video_frame_t), video_frame_t *);
frame->width = 640;
frame->height = 480;
frame->data_size = 640 * 480 * 3;
frame->data = SAFE_MALLOC(frame->data_size, uint8_t *);
frame->owns_data = true;
frame->client_id = my_client_id;
frame->timestamp = get_timestamp_usec();
 
// Fill with webcam data
image_t *captured = webcam_read_context(webcam);
if (captured) {
    memcpy(frame->data, captured->pixels, frame->data_size);
}
 
// Process and send
process_and_send_frame(frame);
 
// Cleanup (if still owning data)
if (frame->owns_data && frame->data) {
    free(frame->data);
}
free(frame);

See also

video_frame.h

video/image.h

os/webcam.h

Macro Definition Documentation

MAX_FRAME_BUFFERS

#define MAX_FRAME_BUFFERS   2

#include <video_frame.h>

Maximum number of frame buffers (double buffering)

Definition at line 95 of file video_frame.h.

VIDEO_FRAME_BUFFER_SIZE

#define VIDEO_FRAME_BUFFER_SIZE   MAX_FRAME_BUFFER_SIZE

#include <video_frame.h>

Video frame buffer size (2MB per frame)

Definition at line 93 of file video_frame.h.

Function Documentation

simple_frame_swap_create()

simple_frame_swap_t * simple_frame_swap_create

(

void

)

#include <video_frame.h>

Create a simple atomic frame swap.

Returns

Pointer to newly created frame swap, or NULL on failure

Creates a lightweight frame swap structure for basic use cases that don't need statistics tracking. Uses atomic pointer swaps for lock-free operation.

Note

This is a simpler alternative to video_frame_buffer_t.

Pre-allocates two frame structures that alternate on each update.

No statistics tracking (lighter weight than full frame buffer).

Warning

Must call simple_frame_swap_destroy() to free resources.

Definition at line 308 of file video_frame.c.

                                                    {
  simple_frame_swap_t *sfs = SAFE_CALLOC(1, sizeof(simple_frame_swap_t), simple_frame_swap_t *);
 
  // Pre-allocate both frames
  const size_t frame_size = (size_t)MAX_FRAME_BUFFER_SIZE;
  sfs->frame_a.data = SAFE_MALLOC(frame_size, void *);
  sfs->frame_b.data = SAFE_MALLOC(frame_size, void *);
 
  atomic_store_u64(&sfs->current_frame, 0);
  atomic_store_bool(&sfs->use_frame_a, false); // Next write goes to frame_b
 
  return sfs;
}

References atomic_store_bool(), atomic_store_u64(), simple_frame_swap_t::current_frame, video_frame_t::data, simple_frame_swap_t::frame_a, simple_frame_swap_t::frame_b, MAX_FRAME_BUFFER_SIZE, SAFE_CALLOC, SAFE_MALLOC, and simple_frame_swap_t::use_frame_a.

simple_frame_swap_destroy()

void simple_frame_swap_destroy

(

simple_frame_swap_t *

sfs

)

#include <video_frame.h>

Destroy simple frame swap and free resources.

Parameters

sfs	Frame swap to destroy (can be NULL)

Destroys the simple frame swap and frees all associated resources. Safe to call multiple times or with NULL pointer.

Definition at line 322 of file video_frame.c.

                                                         {
  if (!sfs) {
    SET_ERRNO(ERROR_INVALID_PARAM, "Simple frame swap is NULL");
    return;
  }
  SAFE_FREE(sfs->frame_a.data);
  SAFE_FREE(sfs->frame_b.data);
  SAFE_FREE(sfs);
}

References video_frame_t::data, ERROR_INVALID_PARAM, simple_frame_swap_t::frame_a, simple_frame_swap_t::frame_b, SAFE_FREE, and SET_ERRNO.

simple_frame_swap_get()

const video_frame_t * simple_frame_swap_get

(

simple_frame_swap_t *

sfs

)

#include <video_frame.h>

Get current frame from simple frame swap.

Parameters

sfs	Frame swap (must not be NULL)

Returns

Pointer to current frame, or NULL if no frame available

Gets a pointer to the current frame. This is a lock-free read operation that reads the atomic frame pointer. Returns NULL if no frame has been updated yet.

Note

Frame pointer is valid until next simple_frame_swap_update() call.

Lock-free operation (uses atomic pointer read).

Definition at line 356 of file video_frame.c.

                                                                     {
  if (!sfs) {
    SET_ERRNO(ERROR_INVALID_PARAM, "Simple frame swap is NULL");
    return NULL;
  }
  uintptr_t frame_ptr = atomic_load_u64(&sfs->current_frame);
  return (const video_frame_t *)(void *)frame_ptr; // NOLINT(bugprone-casting-through-void,performance-no-int-to-ptr)
}

References atomic_load_u64(), simple_frame_swap_t::current_frame, ERROR_INVALID_PARAM, and SET_ERRNO.

simple_frame_swap_update()

void simple_frame_swap_update	(	simple_frame_swap_t *	sfs,
		const void *	data,
		size_t	size
	)

#include <video_frame.h>

Update frame data in simple frame swap.

Parameters

sfs	Frame swap (must not be NULL)
data	Frame data pointer (must not be NULL)
size	Size of frame data in bytes

Updates the current frame with new data. Atomically swaps the active frame pointer to make new data available to readers. This is a zero-copy operation (only pointer is updated, data is not copied).

Note

Frame data should remain valid until next update or destruction.

Atomic pointer swap ensures readers see consistent frame pointer.

Alternates between frame_a and frame_b on each update.

Warning

Frame data pointer must remain valid until next update.

Definition at line 332 of file video_frame.c.

                                                                                       {
  if (!sfs || !data) {
    SET_ERRNO(ERROR_INVALID_PARAM, "Simple frame swap or data is NULL");
    return;
  }
 
  // Determine which frame to write to
  bool use_a = atomic_load_bool(&sfs->use_frame_a);
  video_frame_t *write_frame = use_a ? &sfs->frame_a : &sfs->frame_b;
 
  // Copy data to write frame
  if (size <= (size_t)MAX_FRAME_BUFFER_SIZE) {
    SAFE_MEMCPY(write_frame->data, size, data, size);
    write_frame->size = size;
    write_frame->capture_timestamp_ns = time_get_ns();
 
    // Atomically update current frame pointer
    atomic_store_u64(&sfs->current_frame, (uintptr_t)write_frame);
 
    // Toggle for next write
    atomic_store_bool(&sfs->use_frame_a, !use_a);
  }
}

References atomic_load_bool(), atomic_store_bool(), atomic_store_u64(), video_frame_t::capture_timestamp_ns, simple_frame_swap_t::current_frame, video_frame_t::data, ERROR_INVALID_PARAM, simple_frame_swap_t::frame_a, simple_frame_swap_t::frame_b, MAX_FRAME_BUFFER_SIZE, SAFE_MEMCPY, SET_ERRNO, video_frame_t::size, time_get_ns(), and simple_frame_swap_t::use_frame_a.

video_frame_begin_write()

video_frame_t * video_frame_begin_write

(

video_frame_buffer_t *

vfb

)

#include <video_frame.h>

Writer API: Start writing a new frame.

Parameters

vfb	Frame buffer manager (must not be NULL)

Returns

Pointer to back buffer for writing, or NULL on error

Gets a pointer to the back buffer for writing a new frame. Writer thread can write frame data directly to this buffer. MUST call video_frame_commit() after writing to make the frame available to readers.

Note

Writer should fill in all frame fields (data, size, width, height, etc.).

Back buffer is exclusive to writer until video_frame_commit() is called.

If previous frame wasn't committed, it will be lost (latest-frame-wins).

Warning

Must call video_frame_commit() after writing to make frame available.

Definition at line 214 of file video_frame.c.

                                                                  {
  if (!vfb) {
    SET_ERRNO(ERROR_INVALID_PARAM, "Video frame buffer is NULL");
    return NULL;
  }
  if (!vfb->active) {
    SET_ERRNO(ERROR_INVALID_STATE, "vfb->active is not true");
    return NULL;
  }
 
  // Writer always owns the back buffer
  return vfb->back_buffer;
}

References video_frame_buffer_t::active, video_frame_buffer_t::back_buffer, ERROR_INVALID_PARAM, ERROR_INVALID_STATE, and SET_ERRNO.

Referenced by client_video_render_thread(), handle_image_frame_h265_packet(), and handle_image_frame_packet().

video_frame_buffer_create()

video_frame_buffer_t * video_frame_buffer_create

(

const char *

client_id

)

#include <video_frame.h>

Create a double-buffered video frame manager.

Parameters

client_id

Client ID for this frame buffer

Returns

Pointer to newly created frame buffer, or NULL on failure

Creates a new video frame buffer manager with double buffering. Pre-allocates all buffers (frame structures and data buffers) for zero-allocation operation. Initializes atomic counters and synchronization primitives.

Note

Pre-allocates 2MB × 2 buffers = 4MB total per frame buffer.

Frame buffers are reused forever (no runtime allocations).

Thread-safe: Can be used concurrently by writer and reader threads.

Warning

Must call video_frame_buffer_destroy() to free resources.

Definition at line 17 of file video_frame.c.

                                                                       {
  if (client_id == 0) {
    SET_ERRNO(ERROR_INVALID_PARAM, "Client ID is 0");
    return NULL;
  }
 
  video_frame_buffer_t *vfb = SAFE_CALLOC(1, sizeof(video_frame_buffer_t), video_frame_buffer_t *);
 
  // Copy client_id string - we own this memory now
  // (caller's client_id may be a stack variable that goes out of scope)
  size_t client_id_len = strlen(client_id) + 1;
  char *client_id_copy = SAFE_MALLOC(client_id_len, char *);
  if (!client_id_copy) {
    SET_ERRNO(ERROR_MEMORY, "Failed to allocate memory for client_id copy");
    SAFE_FREE(vfb);
    return NULL;
  }
  asciichat_error_t strcpy_result = SAFE_STRCPY(client_id_copy, client_id_len, client_id);
  if (strcpy_result != ASCIICHAT_OK) {
    log_error("Failed to copy client_id: %s", asciichat_error_string(strcpy_result));
    SAFE_FREE(client_id_copy);
    SAFE_FREE(vfb);
    return NULL;
  }
  vfb->client_id = client_id_copy;
  vfb->active = true;
 
  // Initialize double buffers
  vfb->front_buffer = &vfb->frames[0];
  vfb->back_buffer = &vfb->frames[1];
 
  // DEBUG: Log buffer initialization
  log_info("VFB_INIT: vfb=%p, vfb->frames=%p, &frames[0]=%p, &frames[1]=%p, setting back_buffer=%p", (void *)vfb,
           (void *)vfb->frames, (void *)&vfb->frames[0], (void *)&vfb->frames[1], (void *)vfb->back_buffer);
 
  // Pre-allocate frame data buffers (2MB each for HD video)
  const size_t frame_size = (size_t)MAX_FRAME_BUFFER_SIZE;
  buffer_pool_t *pool = buffer_pool_get_global();
 
  log_info("VFB_ALLOC_START: client_id=%s, frame_size=%zu (2MB=%zu), pool=%p", client_id, frame_size,
           (size_t)(2 * 1024 * 1024), (void *)pool);
 
  // Initialize frames - size starts at 0 until actual data is written!
  vfb->frames[0].size = 0; // Start with 0 - will be set when data is written
  vfb->frames[1].size = 0; // Start with 0 - will be set when data is written
  vfb->frames[0].data = NULL;
  vfb->frames[1].data = NULL;
 
  // Initialize allocated_buffer_size to 0 - will be set AFTER successful allocation
  vfb->allocated_buffer_size = 0;
 
  // Attempt pool allocation
  if (pool) {
    vfb->frames[0].data = buffer_pool_alloc(pool, frame_size);
    vfb->frames[1].data = buffer_pool_alloc(pool, frame_size);
    log_info("VFB_POOL_ALLOC: frame[0].data=%p, frame[1].data=%p (from pool)", (void *)vfb->frames[0].data,
             (void *)vfb->frames[1].data);
  } else {
    log_warn("VFB_NO_POOL: pool is NULL, will use malloc fallback");
  }
 
  // Fallback to aligned malloc if pool allocation failed or returned undersized buffer
  // Note: buffer_pool_alloc may return undersized buffers if pool is exhausted and falls back to SAFE_MALLOC
  if (!vfb->frames[0].data || !vfb->frames[1].data) {
    log_info("VFB_MALLOC_FALLBACK: frame[0]=%s, frame[1]=%s", vfb->frames[0].data ? "OK" : "allocating",
             vfb->frames[1].data ? "OK" : "allocating");
    // 64-byte cache-line alignment improves performance for large video frames
    // CRITICAL: Always use SAFE_MALLOC_ALIGNED for fallback to guarantee minimum size and alignment
    if (!vfb->frames[0].data) {
      vfb->frames[0].data = SAFE_MALLOC_ALIGNED(frame_size, 64, void *);
      if (vfb->frames[0].data) {
        log_info("VFB_MALLOC_FRAME0: allocated %zu bytes with 64-byte alignment", frame_size);
      }
    }
    if (!vfb->frames[1].data) {
      vfb->frames[1].data = SAFE_MALLOC_ALIGNED(frame_size, 64, void *);
      if (vfb->frames[1].data) {
        log_info("VFB_MALLOC_FRAME1: allocated %zu bytes with 64-byte alignment", frame_size);
      }
    }
    log_info("VFB_MALLOC_RESULT: frame[0].data=%p, frame[1].data=%p (allocated with malloc)",
             (void *)vfb->frames[0].data, (void *)vfb->frames[1].data);
  }
 
  // CRITICAL: Only set allocated_buffer_size after BOTH buffers are successfully allocated
  // This prevents claiming 2MB buffers when actual allocation failed or returned smaller buffers
  if (!vfb->frames[0].data || !vfb->frames[1].data) {
    log_error("VFB_ALLOC_FAILED: frame[0].data=%p, frame[1].data=%p (cannot create frame buffer)",
              (void *)vfb->frames[0].data, (void *)vfb->frames[1].data);
    SET_ERRNO(ERROR_MEMORY, "Failed to allocate video frame buffers");
    video_frame_buffer_destroy(vfb);
    return NULL;
  }
 
  // CRITICAL: Verify we actually got 2MB buffers, not undersized fallbacks
  // The buffer pool may return undersized buffers if exhausted, but we REQUIRE full-sized allocations
  // Do this by checking if allocations came from fallback or if pool returned a pointer we got
  // Since we can't check actual size from buffer_pool_alloc(), we enforce that both allocations succeeded
  // and use SAFE_MALLOC_ALIGNED as fallback which guarantees correct size
 
  // After our allocation logic above:
  // - Pool allocations go through buffer_pool_alloc (may be undersized)
  // - Fallback allocations use SAFE_MALLOC_ALIGNED (guaranteed 2MB)
  // We verify BOTH succeeded with the fallback already ensuring size
 
  vfb->allocated_buffer_size = frame_size; // Set to requested size
 
  log_info("VFB_ALLOC_FINAL: allocated_buffer_size=%zu (requested=%zu), frame[0].data=%p, frame[1].data=%p",
           vfb->allocated_buffer_size, frame_size, (void *)vfb->frames[0].data, (void *)vfb->frames[1].data);
 
  log_info("VFB_ALLOC_SUCCESS: client_id=%s, allocated_buffer_size=%zu, frames[0].data=%p, frames[1].data=%p",
           client_id, vfb->allocated_buffer_size, (void *)vfb->frames[0].data, (void *)vfb->frames[1].data);
 
  // When buffers are allocated from the pool, they may contain leftover data from previous clients
  // This ensures frames with size=0 are truly empty, preventing ghost frames during reconnection
  if (vfb->frames[0].data) {
    memset(vfb->frames[0].data, 0, frame_size);
  }
  if (vfb->frames[1].data) {
    memset(vfb->frames[1].data, 0, frame_size);
  }
 
  // Initialize synchronization
  log_info("VFB_MUTEX_INIT_START: About to init mutex for client_id=%s", client_id);
  if (mutex_init(&vfb->swap_mutex, "video_frame_swap") != 0) {
    log_error("VFB_MUTEX_INIT_FAILED: mutex_init returned error");
    SET_ERRNO(ERROR_PLATFORM_INIT, "Failed to initialize mutex for video frame buffer");
    video_frame_buffer_destroy(vfb);
    return NULL;
  }
  log_info("VFB_MUTEX_INIT_DONE: mutex initialized successfully");
  atomic_store_bool(&vfb->new_frame_available, false);
 
  // Initialize statistics
  atomic_store_u64(&vfb->total_frames_received, 0);
  atomic_store_u64(&vfb->total_frames_dropped, 0);
  atomic_store_u64(&vfb->last_frame_sequence, 0);
 
  log_debug("Created video frame buffer for client %u with double buffering", client_id);
 
  // DEBUG: Final state check
  log_info("VFB_FINAL: vfb=%p, back_buffer=%p (should be %p), frames[0].data=%p, frames[1].data=%p, allocated_size=%zu",
           (void *)vfb, (void *)vfb->back_buffer, (void *)&vfb->frames[1], (void *)vfb->frames[0].data,
           (void *)vfb->frames[1].data, vfb->allocated_buffer_size);
 
  NAMED_REGISTER_VIDEO_FRAME_BUFFER(vfb, "buffer", NULL);
 
  // Register atomic fields with descriptive names - vfb is their parent
  NAMED_REGISTER_ATOMIC(&vfb->new_frame_available, "new_frame_available_flag", (uintptr_t)(const void *)(vfb));
  NAMED_REGISTER_ATOMIC(&vfb->total_frames_received, "total_frames_received_counter", (uintptr_t)(const void *)(vfb));
  NAMED_REGISTER_ATOMIC(&vfb->total_frames_dropped, "total_frames_dropped_counter", (uintptr_t)(const void *)(vfb));
  NAMED_REGISTER_ATOMIC(&vfb->last_frame_sequence, "last_frame_sequence_number", (uintptr_t)(const void *)(vfb));
 
  return vfb;
}

References video_frame_buffer_t::active, video_frame_buffer_t::allocated_buffer_size, ASCIICHAT_OK, atomic_store_bool(), atomic_store_u64(), video_frame_buffer_t::back_buffer, buffer_pool_alloc(), buffer_pool_get_global(), video_frame_buffer_t::client_id, video_frame_t::data, ERROR_INVALID_PARAM, ERROR_MEMORY, ERROR_PLATFORM_INIT, video_frame_buffer_t::frames, video_frame_buffer_t::front_buffer, video_frame_buffer_t::last_frame_sequence, log_debug, log_error, log_info, log_warn, MAX_FRAME_BUFFER_SIZE, mutex_init(), NAMED_REGISTER_ATOMIC, NAMED_REGISTER_VIDEO_FRAME_BUFFER, video_frame_buffer_t::new_frame_available, pool, SAFE_CALLOC, SAFE_FREE, SAFE_MALLOC, SAFE_MALLOC_ALIGNED, SAFE_STRCPY, SET_ERRNO, video_frame_t::size, video_frame_buffer_t::swap_mutex, video_frame_buffer_t::total_frames_dropped, video_frame_buffer_t::total_frames_received, and video_frame_buffer_destroy().

Referenced by add_client(), and add_webrtc_client().

video_frame_buffer_destroy()

void video_frame_buffer_destroy

(

video_frame_buffer_t *

vfb

)

#include <video_frame.h>

Destroy frame buffer and free all resources.

Parameters

vfb	Frame buffer to destroy (can be NULL)

Destroys the frame buffer and frees all pre-allocated buffers and internal structures. Safe to call multiple times or with NULL pointer.

Note

All frame data and statistics are lost after destruction.

Definition at line 173 of file video_frame.c.

                                                           {
  if (!vfb) {
    SET_ERRNO(ERROR_INVALID_PARAM, "Video frame buffer is NULL");
    return;
  }
 
  NAMED_UNREGISTER(vfb);
 
  // Unregister atomic fields
  NAMED_UNREGISTER(&vfb->new_frame_available);
  NAMED_UNREGISTER(&vfb->total_frames_received);
  NAMED_UNREGISTER(&vfb->total_frames_dropped);
  NAMED_UNREGISTER(&vfb->last_frame_sequence);
 
  vfb->active = false;
 
  // Free frame data
  buffer_pool_t *pool = buffer_pool_get_global();
  if (vfb->frames[0].data) {
    if (pool) {
      buffer_pool_free(pool, vfb->frames[0].data, vfb->allocated_buffer_size);
    } else {
      SAFE_FREE(vfb->frames[0].data);
    }
  }
  if (vfb->frames[1].data) {
    if (pool) {
      buffer_pool_free(pool, vfb->frames[1].data, vfb->allocated_buffer_size);
    } else {
      SAFE_FREE(vfb->frames[1].data);
    }
  }
 
  mutex_destroy(&vfb->swap_mutex);
 
  // Free the allocated client_id string
  SAFE_FREE(vfb->client_id);
 
  SAFE_FREE(vfb);
}

References video_frame_buffer_t::active, video_frame_buffer_t::allocated_buffer_size, buffer_pool_free(), buffer_pool_get_global(), video_frame_buffer_t::client_id, video_frame_t::data, ERROR_INVALID_PARAM, video_frame_buffer_t::frames, video_frame_buffer_t::last_frame_sequence, mutex_destroy(), NAMED_UNREGISTER, video_frame_buffer_t::new_frame_available, pool, SAFE_FREE, SET_ERRNO, video_frame_buffer_t::swap_mutex, video_frame_buffer_t::total_frames_dropped, and video_frame_buffer_t::total_frames_received.

Referenced by add_client(), add_webrtc_client(), cleanup_client_media_buffers(), and video_frame_buffer_create().

video_frame_commit()

void video_frame_commit

(

video_frame_buffer_t *

vfb

)

#include <video_frame.h>

Writer API: Commit the frame and swap buffers.

Parameters

vfb	Frame buffer manager (must not be NULL)

Commits the current back buffer and atomically swaps it with the front buffer. Makes the new frame available to readers immediately. This is a brief critical section (uses mutex), but readers continue reading from old front buffer during swap, so they never block.

Note

After this call, the previously written frame becomes the front buffer.

Atomic flag is set to signal readers that a new frame is available.

Frame statistics are updated (total_frames_received, etc.).

Thread-safe: Writer can commit while reader is reading (no blocking).

Definition at line 228 of file video_frame.c.

                                                   {
  if (!vfb) {
    SET_ERRNO(ERROR_INVALID_PARAM, "Video frame buffer or active is NULL");
    return;
  }
  if (!vfb->active) {
    SET_ERRNO(ERROR_INVALID_STATE, "vfb->active is not true");
    return;
  }
 
  // Keep the unread check, pointer swap, and publication atomic with respect
  // to readers. Checking front_buffer before taking this mutex races with a
  // reader consuming the previous frame and with another commit swapping it.
  mutex_lock(&vfb->swap_mutex);
  if (atomic_load_bool(&vfb->new_frame_available) && vfb->front_buffer->size > 0) {
    uint64_t drops = atomic_fetch_add_u64(&vfb->total_frames_dropped, 1) + 1;
    if (drops == 1 || drops % 100 == 0) {
      log_dev_every(4500 * US_PER_MS_INT, "Dropping frame for client %u (reader too slow, total drops: %llu)",
                    vfb->client_id, (unsigned long long)drops);
    }
  }
  video_frame_t *temp = vfb->front_buffer;
  vfb->front_buffer = vfb->back_buffer;
  vfb->back_buffer = temp;
  atomic_store_bool(&vfb->new_frame_available, true);
  mutex_unlock(&vfb->swap_mutex);
 
  atomic_fetch_add_u64(&vfb->total_frames_received, 1);
}

References video_frame_buffer_t::active, atomic_fetch_add_u64(), atomic_load_bool(), atomic_store_bool(), video_frame_buffer_t::back_buffer, video_frame_buffer_t::client_id, ERROR_INVALID_PARAM, ERROR_INVALID_STATE, video_frame_buffer_t::front_buffer, log_dev_every, mutex_lock, mutex_unlock, video_frame_buffer_t::new_frame_available, SET_ERRNO, video_frame_t::size, video_frame_buffer_t::swap_mutex, video_frame_buffer_t::total_frames_dropped, video_frame_buffer_t::total_frames_received, and US_PER_MS_INT.

Referenced by client_video_render_thread(), handle_image_frame_h265_packet(), and handle_image_frame_packet().

video_frame_get_latest()

const video_frame_t * video_frame_get_latest

(

video_frame_buffer_t *

vfb

)

#include <video_frame.h>

Reader API: Get latest frame if available.

Parameters

vfb	Frame buffer manager (must not be NULL)

Returns

Pointer to latest frame, or NULL if no new frame available

Gets a pointer to the latest frame if one is available. Returns NULL if no new frame has been committed since the last call. This is a lock-free operation that never blocks the writer thread.

Note

Frame pointer is valid until next video_frame_commit() call.

Lock-free read operation (uses atomic flag check).

Reader should check for NULL returns and handle accordingly.

Thread-safe: Reader can read while writer is writing (no blocking).

Definition at line 258 of file video_frame.c.

                                                                       {
  if (!vfb) {
    SET_ERRNO(ERROR_INVALID_PARAM, "Video frame buffer is not active");
    return NULL;
  }
  if (!vfb->active) {
    SET_ERRNO(ERROR_INVALID_STATE, "vfb->active is not true");
    return NULL;
  }
 
  mutex_lock(&vfb->swap_mutex);
  if (!atomic_load_bool(&vfb->new_frame_available)) {
    mutex_unlock(&vfb->swap_mutex);
    return NULL;
  }
  atomic_store_bool(&vfb->new_frame_available, false);
  const video_frame_t *result = vfb->front_buffer;
  mutex_unlock(&vfb->swap_mutex);
 
  return result;
}

References video_frame_buffer_t::active, atomic_load_bool(), atomic_store_bool(), ERROR_INVALID_PARAM, ERROR_INVALID_STATE, video_frame_buffer_t::front_buffer, mutex_lock, mutex_unlock, video_frame_buffer_t::new_frame_available, SET_ERRNO, and video_frame_buffer_t::swap_mutex.

video_frame_get_stats()

void video_frame_get_stats	(	video_frame_buffer_t *	vfb,
		video_frame_stats_t *	stats
	)

#include <video_frame.h>

Get frame statistics for quality monitoring.

Parameters

vfb	Frame buffer manager (must not be NULL)
stats	Output structure for statistics (must not be NULL)

Retrieves aggregated statistics from the frame buffer including total frames, dropped frames, drop rate, and average processing times. Useful for adaptive streaming quality control.

Note

All statistics are calculated atomically from atomic counters.

Drop rate is calculated as dropped_frames / total_frames.

Definition at line 290 of file video_frame.c.

                                                                                  {
  if (!vfb || !stats) {
    SET_ERRNO(ERROR_INVALID_PARAM, "Video frame buffer or stats is NULL");
    return;
  }
  if (!vfb->active) {
    SET_ERRNO(ERROR_INVALID_STATE, "vfb->active is not true");
    return;
  }
 
  stats->total_frames = atomic_load_u64(&vfb->total_frames_received);
  stats->dropped_frames = atomic_load_u64(&vfb->total_frames_dropped);
  stats->drop_rate = (stats->total_frames > 0) ? (float)stats->dropped_frames / (float)stats->total_frames : 0.0f;
  stats->avg_decode_time_ns = atomic_load_u64(&vfb->avg_decode_time_ns);
  stats->avg_render_time_ns = atomic_load_u64(&vfb->avg_render_time_ns);
}

References video_frame_buffer_t::active, atomic_load_u64(), video_frame_buffer_t::avg_decode_time_ns, video_frame_buffer_t::avg_render_time_ns, ERROR_INVALID_PARAM, ERROR_INVALID_STATE, SET_ERRNO, stats, video_frame_buffer_t::total_frames_dropped, and video_frame_buffer_t::total_frames_received.

Referenced by stats_logger_thread(), and update_server_stats().

video_frame_peek_latest()

const video_frame_t * video_frame_peek_latest

(

video_frame_buffer_t *

vfb

)

#include <video_frame.h>

Return the latest committed frame without consuming the new-frame flag. The returned pointer has the same borrowed lifetime as video_frame_get_latest().

Definition at line 280 of file video_frame.c.

                                                                        {
  if (!vfb || !vfb->active)
    return NULL;
  mutex_lock(&vfb->swap_mutex);
  const video_frame_t *frame = vfb->front_buffer;
  const video_frame_t *result = frame && frame->size > 0 ? frame : NULL;
  mutex_unlock(&vfb->swap_mutex);
  return result;
}

References video_frame_buffer_t::active, frame, video_frame_buffer_t::front_buffer, mutex_lock, mutex_unlock, video_frame_t::size, and video_frame_buffer_t::swap_mutex.