stream.h File Reference

Multi-client video mixing and ASCII frame generation. More...

Go to the source code of this file.

Functions
char *	create_mixed_ascii_frame_for_client (const char *target_client_id, unsigned short width, unsigned short height, bool wants_stretch, size_t *out_size, bool *out_grid_changed, int *out_sources_count, acip_render_settings_t *out_rendering)
	Generate personalized ASCII frame for a specific client.
bool	any_clients_sending_video (void)
	Queue ASCII frame for delivery to specific client.

Variables
atomic_t	g_should_exit
	Global application exit flag (shared across all modes).

Detailed Description

Multi-client video mixing and ASCII frame generation.

Definition in file src/server/stream.h.

Function Documentation

any_clients_sending_video()

bool any_clients_sending_video

(

void

)

Queue ASCII frame for delivery to specific client.

Packages a generated ASCII frame into a protocol packet and queues it for asynchronous delivery to the target client. This function handles all protocol details including headers, checksums, and metadata.

PACKET STRUCTURE CREATED:

• ascii_frame_packet_t header containing:
  • width, height: Client terminal dimensions
  • original_size: Uncompressed frame size
  • compressed_size: Compressed size (currently 0)
  • checksum: CRC32 of frame data for integrity
  • flags: Capability flags (color support, etc.)
• Frame data: Complete ASCII art string with ANSI codes

PROTOCOL INTEGRATION:

• Converts dimensions to network byte order
• Generates CRC32 checksum for error detection
• Sets appropriate capability flags based on client
• Uses PACKET_TYPE_ASCII_FRAME packet type

DELIVERY MECHANISM:

• Queues packet in client's video packet queue
• Send thread delivers packet asynchronously
• Queue overflow handling prevents memory exhaustion
• Error conditions are logged but don't affect other clients

MEMORY MANAGEMENT:

• Creates temporary packet buffer for header+data combination
• packet_queue_push_copy() copies data (safe to free temp buffer)
• No memory leaks on success or failure paths
• buffer domain allocation not used here (temporary buffer)

ERROR HANDLING:

• Invalid parameters: Return -1, no side effects
• Memory allocation failure: Logged, return -1
• Queue overflow: Logged as debug (expected under load)
• Client shutdown: Graceful failure without error spam

Parameters

client	Target client for frame delivery
ascii_frame	Generated ASCII art string (null-terminated)
frame_size	Size of ASCII frame in bytes

Returns

0 on successful queuing, -1 on error

Note

Frame delivery happens asynchronously via send thread

Queue overflow drops frames to maintain real-time performance

Checksums enable client-side integrity verification

See also

packet_queue_push_copy() For underlying queue implementation

asciichat_crc32() For checksum generation

Check if any connected clients are currently sending video

This function scans all active clients to determine if at least one is sending video frames. Used by render threads to avoid generating frames when no video sources are available (e.g., during webcam warmup).

LOCK OPTIMIZATION: Uses atomic reads only, no rwlock acquisition This is safe because client_id, active, and is_sending_video are all atomics

Returns

true if at least one client has is_sending_video flag set, false otherwise

Definition at line 1283 of file src/server/stream.c.

                                     {
  // LOCK OPTIMIZATION: Don't acquire rwlock - all fields we access are atomic
  // client_id, active, is_sending_video are all atomic variables
 
  // Iterate through all client slots
  for (int i = 0; i < MAX_CLIENTS; i++) {
    client_info_t *client = &g_client_manager.clients[i];
 
    // Skip uninitialized clients (atomic read)
    if (client->client_id[0] == '\0') {
      continue;
    }
 
    // Check if client is active and sending video (both atomic reads)
    bool is_active = atomic_load_bool(&client->active);
    bool is_sending = atomic_load_bool(&client->is_sending_video);
 
    if (is_active && is_sending) {
      return true;
    }
  }
 
  return false;
}

References client_info::active, client_info::client_id, g_client_manager, client_info::is_sending_video, and MAX_CLIENTS.

Referenced by client_video_render_thread().

create_mixed_ascii_frame_for_client()

char * create_mixed_ascii_frame_for_client	(	const char *	target_client_id,
		unsigned short	width,
		unsigned short	height,
		bool	wants_stretch,
		size_t *	out_size,
		bool *	out_grid_changed,
		int *	out_sources_count,
		acip_render_settings_t *	out_rendering )

Generate personalized ASCII frame for a specific client.

This is the core video mixing function that creates customized ASCII art frames for individual clients. It collects video from all active clients, creates an appropriate grid layout, and converts to ASCII using the target client's terminal capabilities.

ALGORITHM OVERVIEW:

1. FRAME COLLECTION PHASE:
   • Scan all active clients for available video frames
   • Use double-buffer system to get latest frames
   • Implement aggressive frame dropping under load (maintains real-time)
   • Always use the most current frame for professional-grade quality
2. LAYOUT CALCULATION PHASE:
   • Determine grid dimensions based on active client count
   • Calculate cell sizes with aspect ratio preservation
   • Handle special cases: single client, multiple clients
3. COMPOSITE GENERATION PHASE:
   • Create composite image with appropriate dimensions
   • Place each client's video in grid cell with proper scaling
   • Support both normal and half-block rendering modes
4. ASCII CONVERSION PHASE:
   • Convert composite to ASCII using client capabilities
   • Apply client-specific palette and color settings
   • Generate ANSI escape sequences if supported

PERFORMANCE OPTIMIZATIONS:

FRAME BUFFER MANAGEMENT:

• Aggressive frame dropping: always read latest available
• Double-buffer system: ensures smooth frame delivery
• buffer domain allocation: reduces malloc/free overhead
• Zero-copy operations where possible

CONCURRENCY OPTIMIZATIONS:

• Reader locks on client manager (allows parallel execution)
• Double-buffer atomic operations (lock-free access)
• Atomic snapshots of client state
• Thread-safe buffer operations

CLIENT CAPABILITY INTEGRATION:

TERMINAL AWARENESS:

• Respects client's color depth capabilities
• Uses client's preferred ASCII palette
• Handles UTF-8 vs ASCII-only terminals
• Supports half-block mode for 2x vertical resolution

RENDERING MODES SUPPORTED:

• RENDER_MODE_FOREGROUND: Traditional ASCII art
• RENDER_MODE_BACKGROUND: Inverted colors for better contrast
• RENDER_MODE_HALF_BLOCK: Unicode half-blocks for 2x resolution

ERROR HANDLING STRATEGY:

• Invalid dimensions: Logged and frame generation aborted
• Memory allocation failures: Graceful degradation, cleanup resources
• Client disconnections: Skip client in current frame, continue
• Buffer overruns: Frame dropping maintains real-time performance
• Capability missing: Wait for capabilities before generating frames

MEMORY MANAGEMENT:

• All image operations use buffer domain allocation
• Frame data is properly freed after use
• Cached frames use deep copies (prevent corruption)
• Automatic cleanup on function exit

Parameters

target_client_id	Client who will receive this customized frame
width	Terminal width in characters for this client
height	Terminal height in characters for this client
wants_stretch	Unused parameter (aspect ratio always preserved)
out_size	Pointer to store resulting ASCII frame size

Returns

Allocated ASCII frame string (caller must free), or NULL on error

Note

This function is called at 60fps per client by render threads

Generated frame is customized for target client's capabilities

Memory allocated from buffer domain - caller must free properly

Warning

Function performs extensive validation - invalid input returns NULL

Client capabilities must be received before frame generation

See also

ascii_convert_with_capabilities() For ASCII conversion implementation

framebuffer_read_multi_frame() For frame Owned Buffers operations

calculate_fit_dimensions_pixel() For aspect ratio calculations

Definition at line 1024 of file src/server/stream.c.

                                                                                                         {
  (void)wants_stretch; // Unused - we always handle aspect ratio ourselves
 
  // Register the stream counter once even when per-client render workers enter
  // this function concurrently.
  static _Atomic bool stream_atomics_registered = false;
  if (!atomic_exchange_explicit(&stream_atomics_registered, true, memory_order_acq_rel)) {
    NAMED_REGISTER_ATOMIC(&g_previous_active_video_count, "server_video_source_count_for_layout_detection", NULL);
  }
 
  uint64_t frame_gen_start_ns = time_get_ns();
 
  // Initialize output parameters
  if (out_grid_changed) {
    *out_grid_changed = false;
  }
  if (out_sources_count) {
    *out_sources_count = 0;
  }
 
  if (!out_size || width == 0 || height == 0) {
    SET_ERRNO(ERROR_INVALID_PARAM, "Invalid parameters: width=%u, height=%u, out_size=%p", width, height, out_size);
    return NULL;
  }
 
  // Collect all active clients and their image sources
  image_source_t sources[MAX_CLIENTS];
  uint64_t collect_start_ns = time_get_ns();
  int source_count = collect_video_sources(sources, MAX_CLIENTS);
  uint64_t collect_end_ns = time_get_ns();
 
  // Count sources that actually have video data
  int sources_with_video = 0;
  for (int i = 0; i < source_count; i++) {
    if (sources[i].has_video && sources[i].image) {
      sources_with_video++;
    }
  }
 
  log_dev_every(333 * NS_PER_MS_INT, "FRAME_GEN_START: target_client=%s sources=%d collect=%.1fms", target_client_id,
                sources_with_video, (collect_end_ns - collect_start_ns) / NS_PER_MS);
 
  // Return the source count for debugging/tracking
  if (out_sources_count) {
    *out_sources_count = sources_with_video;
  }
 
  // GRID LAYOUT CHANGE DETECTION:
  // Check if the number of active video sources has changed
  // NOTE: We only UPDATE the count and SIGNAL the change via out parameter
  // Broadcasting CLEAR_CONSOLE must happen AFTER the new frames are written to buffers
  // to prevent race condition where CLEAR arrives before new frame is ready
  uint64_t previous_count = atomic_load_u64(&g_previous_active_video_count);
  if (sources_with_video != (int)previous_count) {
    // Use compare-and-swap to ensure only ONE thread detects the change
    if (atomic_cas_u64(&g_previous_active_video_count, &previous_count, (uint64_t)sources_with_video)) {
      log_dev_every(
          LOG_RATE_DEFAULT,
          "Grid layout changing: %d -> %d active video sources - caller will broadcast clear AFTER buffering frame",
          previous_count, sources_with_video);
      if (out_grid_changed) {
        *out_grid_changed = true; // Signal to caller
      }
    }
  }
 
  // No active video sources - don't generate placeholder frames
  image_t *composite = NULL;
 
  if (sources_with_video == 0) {
    *out_size = 0;
    // No active video sources for client - this isn't an error.
    // Return NULL to indicate no frame should be sent
    return NULL;
  }
 
  if (sources_with_video == 1) {
    // Single source handling - create composite and convert to ASCII
    // Note: create_single_source_composite returns a reference to sources[i].image
    // which could be modified by other threads. Make a copy to prevent concurrent
    // modification during ascii_convert_with_capabilities.
    image_t *single_source = create_single_source_composite(sources, source_count, target_client_id, width, height);
    if (single_source) {
      composite = image_new_copy(single_source);
      if (!composite) {
        SET_ERRNO(ERROR_MEMORY, "Failed to copy single source composite");
        *out_size = 0;
        return NULL;
      }
    }
  } else {
    // Multiple sources - create grid layout
    composite =
        create_multi_source_composite(sources, source_count, sources_with_video, target_client_id, width, height);
  }
 
  char *out = NULL;
 
  if (!composite) {
    SET_ERRNO(ERROR_INVALID_STATE, "Per-client %s: Failed to create composite image", target_client_id);
    *out_size = 0;
    return NULL;
  }
 
  // Convert composite to ASCII using client capabilities
  // Pass terminal dimensions so the frame can be padded to full width
  char *ascii_frame = convert_composite_to_ascii(composite, target_client_id, width, height, out_rendering, sources,
                                                 source_count, sources_with_video);
 
  if (ascii_frame) {
    // The frame should have been null-terminated by the padding functions.
    // Use strlen() which is optimized and reliable
    size_t ascii_len = strlen(ascii_frame);
 
    // Safety check: don't accept unreasonably large frames (10MB limit)
    if (ascii_len > 10 * 1024 * 1024) {
      log_error("Frame size exceeds 10MB safety limit (possible buffer overflow)");
      SET_ERRNO(ERROR_INVALID_PARAM, "Frame size exceeds 10MB");
      SAFE_FREE(ascii_frame);
      image_destroy(composite);
      *out_size = 0;
      return NULL;
    }
 
    *out_size = ascii_len; // Renderer owns complete rows; never truncate to a color reset.
 
    log_dev_every(LOG_RATE_SLOW, "Final frame size=%zu bytes for client %s", *out_size, target_client_id);
 
    // Debug: Log the last 50 bytes of the frame to see what's really there
    if (*out_size >= 50) {
      char hex_buf[SERVER_STREAM_HEX_BUFFER_SIZE] = {0};
      size_t hex_len = 0;
      const uint8_t *last_bytes = (const uint8_t *)ascii_frame + (*out_size - 50);
      for (int i = 0; i < 50 && hex_len < sizeof(hex_buf) - 5; i++) {
        hex_len += snprintf(hex_buf + hex_len, sizeof(hex_buf) - hex_len, "%02X ", last_bytes[i]);
      }
      log_dev_every(4500 * US_PER_MS_INT, "FRAME_LAST_50_BYTES (hex): %s", hex_buf);
 
      // Also log as ASCII for readability
      char ascii_buf[SERVER_STREAM_ASCII_BUFFER_SIZE] = {0};
      for (int i = 0; i < 50 && i < (int)sizeof(ascii_buf) - 1; i++) {
        if (last_bytes[i] >= 32 && last_bytes[i] < 127) {
          ascii_buf[i] = (char)last_bytes[i];
        } else if (last_bytes[i] == '\n') {
          ascii_buf[i] = 'N';
        } else if (last_bytes[i] == '\0') {
          ascii_buf[i] = '0';
        } else {
          ascii_buf[i] = '.';
        }
      }
      log_dev_every(4500 * US_PER_MS_INT, "FRAME_LAST_50_ASCII: %s", ascii_buf);
    }
 
    out = ascii_frame;
  } else {
    SET_ERRNO(ERROR_TERMINAL, "Per-client %s: Failed to convert image to ASCII", target_client_id);
    *out_size = 0;
  }
 
  if (composite) {
    image_destroy(composite);
  }
  for (int i = 0; i < source_count; i++) {
    if (sources[i].image) {
      image_destroy(sources[i].image);
    }
  }
 
  uint64_t frame_gen_end_ns = time_get_ns();
  uint64_t frame_gen_duration_ns = frame_gen_end_ns - frame_gen_start_ns;
  if (frame_gen_duration_ns > 10 * NS_PER_MS_INT) { // Log if > 10ms
    char duration_str[TIME_DURATION_BUFFER_SIZE];
    time_pretty((uint64_t)((double)frame_gen_duration_ns), -1, duration_str, sizeof(duration_str));
    log_warn_every(LOG_RATE_DEFAULT, "SLOW_FRAME_GENERATION: Client %s full frame gen took %s", target_client_id,
                   duration_str);
  }
 
  return out;
}

References ERROR_INVALID_PARAM, ERROR_INVALID_STATE, ERROR_MEMORY, ERROR_TERMINAL, height, image_destroy(), image_new_copy(), log_dev_every, log_error, LOG_RATE_DEFAULT, LOG_RATE_SLOW, log_warn_every, MAX_CLIENTS, NAMED_REGISTER_ATOMIC, NS_PER_MS, NS_PER_MS_INT, out, SAFE_FREE, SERVER_STREAM_ASCII_BUFFER_SIZE, SERVER_STREAM_HEX_BUFFER_SIZE, SET_ERRNO, TIME_DURATION_BUFFER_SIZE, time_pretty(), US_PER_MS_INT, and width.

Referenced by client_video_render_thread().

Variable Documentation

g_should_exit

atomic_t g_should_exit

extern

Global application exit flag (shared across all modes).

Global shutdown flag referenced by lib/logging.c and lib/debug/lock.c In production builds, this is defined in main.c and set during shutdown. For tests, we stub it out as false (tests don't perform shutdown).

Definition at line 34 of file globals.c.

{0};

Referenced by ATOMIC_INIT_AUTO(), client_audio_render_thread(), client_video_render_thread(), handle_image_frame_h265_packet(), handle_image_frame_packet(), server_main(), should_exit(), signal_exit(), and stats_logger_thread().