Media Capture

📹 Webcam video capture and transmission to server More...

Files
file	capture.c
	📹 Client webcam capture: dedicated capture thread with frame rate limiting and network transmission
file	capture.h
	ascii-chat Client Media Capture Management Interface

Functions
int	capture_init ()
	Initialize capture subsystem.
int	capture_start_thread ()
	Start capture thread.
void	capture_stop_thread ()
	Stop capture thread.
bool	capture_thread_exited ()
	Check if capture thread has exited.
void	capture_cleanup ()
	Cleanup capture subsystem.

Detailed Description

📹 Webcam video capture and transmission to server

Media Capture

Overview

The media capture subsystem manages webcam video capture, frame compression, and transmission to the server. Supports platform-specific webcam backends and test pattern mode for testing without hardware.

Implementation: src/client/capture.c, src/client/capture.h

Platform Support

Webcam Backends:

• macOS: AVFoundation (lib/video/webcam/macos/webcam_avfoundation.m)
• Linux: Video4Linux2 (lib/video/webcam/linux/webcam_v4l2.c)
• Windows: Media Foundation (lib/video/webcam/windows/webcam_mediafoundation.c)

Capture Thread

static void *capture_thread_func(void *arg)
{
  (void)arg;
 
  // Calculate frame interval for target FPS
  uint64_t frame_interval_usec = 1000000 / VIDEO_CAPTURE_FPS;
  uint64_t start_time = get_monotonic_time_usec();
 
  while (!should_exit() && server_connection_is_active()) {
    // Capture frame from webcam
    video_frame_t *frame = webcam_capture();
    if (!frame) {
      platform_sleep_usec(frame_interval_usec);
      continue;
    }
 
    // Optional compression
    void *send_data = frame->data;
    size_t send_size = frame->data_size;
    void *compressed_data = NULL;
 
    if (opt_compress) {
      compressed_data = compress_frame(frame->data, frame->data_size, &send_size);
      if (compressed_data) {
        send_data = compressed_data;
      }
    }
 
    // Send frame to server
    send_packet_to_server(PACKET_TYPE_IMAGE_FRAME, send_data, send_size,
                         server_connection_get_client_id());
 
    if (compressed_data) free(compressed_data);
    if (frame->owns_data && frame->data) free(frame->data);
    free(frame);
 
    // Rate limiting
    platform_sleep_usec(frame_interval_usec);
 
    // Snapshot mode: exit after delay
    if (opt_snapshot) {
      uint64_t elapsed = (get_monotonic_time_usec() - start_time) / 1000000;
      if (elapsed >= opt_snapshot_delay) {
        signal_exit();
        break;
      }
    }
  }
 
  atomic_store(&g_capture_thread_exited, true);
  return NULL;
}

Snapshot Mode

Snapshot mode (--snapshot, --snapshot-delay <seconds>):

• Useful for testing without continuous streaming
• --snapshot: Capture single frame and exit
• --snapshot-delay N: Capture for N seconds then exit

# Single frame test
./bin/ascii-chat client --snapshot
 
# 5-second test
./bin/ascii-chat client --snapshot --snapshot-delay 5

Test Pattern Mode

Test pattern mode (--test-pattern):

• Generates synthetic video frames without webcam hardware
• Useful for testing on Windows or systems without webcam
• Uses platform-specific test pattern generation

# Use test pattern instead of webcam
./bin/ascii-chat client --test-pattern

See also

src/client/capture.c

src/client/capture.h

lib/video/webcam/webcam.h

Client Overview

Function Documentation

capture_cleanup()

void capture_cleanup

(

)

#include <capture.c>

Cleanup capture subsystem.

Cleanup capture subsystem

Stops capture thread and cleans up media source resources. Called during client shutdown.

Releases all platform resources associated with webcam access. This must be called after capture_stop_thread() and before process exit.

The cleanup process:

1. Closes webcam device (platform-specific close)
2. Frees frame buffers
3. Releases platform driver resources
4. Cleans up thread handles

Must be called in proper order during shutdown to avoid resource leaks and to restore the platform to a clean state.

See also

capture_init "Initialize capture subsystem"

Definition at line 554 of file client/capture.c.

                       {
  capture_stop_thread();
 
  // Destroy capture context
  if (g_capture_capture_ctx) {
    session_capture_destroy(g_capture_capture_ctx);
    g_capture_capture_ctx = NULL;
  }
}

References capture_stop_thread(), and session_capture_destroy().

Referenced by client_crypto_handshake().

capture_init()

int capture_init

(

)

#include <capture.c>

Initialize capture subsystem.

Initialize capture subsystem

Sets up media source (webcam, file, or stdin) and prepares capture system for operation. Must be called once during client initialization.

Returns

0 on success, negative on error

Initializes the media source and capture resources. Determines media source type from command-line options, then sets up platform-specific drivers and buffers. This must be called before any other capture functions.

The initialization process:

1. Determines media source type from options (test pattern, file, stdin, or webcam)
2. Creates appropriate media source (media_source_create())
3. For WEBCAM: Initializes platform driver (V4L2/AVFoundation/Media Foundation)
4. For FILE/STDIN: Opens FFmpeg decoder with format detection
5. For TEST: Initializes procedural test pattern generator
6. Initializes frame buffers and capture parameters

Media source selection (in order):

1. If --test-pattern → MEDIA_SOURCE_TEST
2. Else if --file - → MEDIA_SOURCE_STDIN
3. Else if --file <path> → MEDIA_SOURCE_FILE
4. Else if --url <url> → MEDIA_SOURCE_FILE (network URL)
5. Else → MEDIA_SOURCE_WEBCAM (with –webcam-index)

Options affecting initialization:

• --test-pattern: Use synthetic test pattern (any platform, no hardware needed)
• --file <path>: Media file to play (mp4, avi, mkv, gif, etc.)
• --file -: Read media from stdin (any FFmpeg format)
• --url <url>: Network stream (HTTP/HTTPS/RTSP/YouTube/etc.)
• --webcam-index <N>: Which camera to open (default: 0, first camera)
• --media-loop: Loop file playback when EOF reached
• --webcam-flip: Flip video vertically (webcam and test patterns only)
• --fps <rate>: Target frame rate in FPS (1-144, default: 60)

Error handling:

• WEBCAM: Device not found, driver error, device busy
• FILE: File not found, unsupported format, FFmpeg error
• STDIN: FFmpeg format detection failure
• TEST: Rarely fails (only if display dimensions unavailable)

Returns

ASCIICHAT_OK on success, error code on failure:

• ERROR_WEBCAM: Device not found or driver error
• ERROR_WEBCAM_IN_USE: Device busy (another app using it)
• ERROR_FILE_NOT_FOUND: Media file not found
• ERROR_FORMAT: Unsupported media format
• ERROR_NOT_SUPPORTED: Platform doesn't support this source type

Note

This is called during client initialization, before entering the connection loop.

The actual frame capture happens in the capture thread (capture_start_thread).

Frame resizing to 800x600 (max) happens during capture, not initialization.

See also

media_source_create "Media source creation"

capture_start_thread "Start capture thread"

capture_cleanup "Cleanup resources"

Definition at line 413 of file client/capture.c.

                   {
  // Build capture configuration from options
  session_capture_config_t config = {0};
  const char *media_url = GET_OPTION(media_url);
  const char *media_file = GET_OPTION(media_file);
  bool media_from_stdin = GET_OPTION(media_from_stdin);
 
  if (media_url && media_url[0] != '\0') {
    // Network URL streaming (takes priority over --file)
    // Don't open webcam when streaming from URL
    config.type = MEDIA_SOURCE_FILE;
    config.path = media_url;
    config.loop = false; // Network URLs cannot be looped
    log_debug("Using network URL: %s (webcam disabled)", media_url);
  } else if (media_file && media_file[0] != '\0') {
    // File or stdin streaming - don't open webcam
    config.type = media_from_stdin ? MEDIA_SOURCE_STDIN : MEDIA_SOURCE_FILE;
    config.path = media_file;
    config.loop = GET_OPTION(media_loop) && !media_from_stdin;
    log_debug("Using media %s: %s (webcam disabled)", media_from_stdin ? "stdin" : "file", media_file);
  } else if (GET_OPTION(test_pattern)) {
    // Test pattern mode - don't open real webcam
    config.type = MEDIA_SOURCE_TEST;
    config.path = NULL;
    log_debug("Using test pattern mode");
  } else {
    // Webcam mode (default)
    static char webcam_index_str[32];
    safe_snprintf(webcam_index_str, sizeof(webcam_index_str), "%u", GET_OPTION(webcam_index));
    config.type = MEDIA_SOURCE_WEBCAM;
    config.path = webcam_index_str;
    log_debug("Using webcam device %u", GET_OPTION(webcam_index));
  }
  config.target_fps = CAPTURE_TARGET_FPS;
  config.resize_for_network = true; // Client always resizes for network transmission
 
  // Configure audio capture with fallback to microphone
  config.enable_audio = true;
  config.audio_fallback_to_mic = true;
  config.mic_audio_ctx = audio_get_context();
 
  // Add seek timestamp if specified
  config.initial_seek_timestamp = GET_OPTION(media_seek_timestamp);
 
  // Create capture context using session library
  g_capture_capture_ctx = session_capture_create(&config);
  if (!g_capture_capture_ctx) {
    // Check if there's already an error set (e.g., ERROR_WEBCAM_IN_USE)
    asciichat_error_t existing_error = GET_ERRNO();
    log_debug("session_capture_create failed, GET_ERRNO() returned: %d", existing_error);
    if (existing_error != ASCIICHAT_OK) {
      log_debug("Returning existing error code %d", existing_error);
      return existing_error;
    }
    SET_ERRNO(ERROR_MEDIA_INIT, "Failed to initialize capture source");
    return -1;
  }
 
  return 0;
}

References ASCIICHAT_OK, session_capture_config_t::audio_fallback_to_mic, audio_get_context(), CAPTURE_TARGET_FPS, session_capture_config_t::enable_audio, ERROR_MEDIA_INIT, GET_ERRNO, GET_OPTION, session_capture_config_t::initial_seek_timestamp, log_debug, session_capture_config_t::loop, MEDIA_SOURCE_FILE, MEDIA_SOURCE_STDIN, MEDIA_SOURCE_TEST, MEDIA_SOURCE_WEBCAM, session_capture_config_t::mic_audio_ctx, session_capture_config_t::path, session_capture_config_t::resize_for_network, safe_snprintf(), session_capture_create(), SET_ERRNO, session_capture_config_t::target_fps, and session_capture_config_t::type.

capture_start_thread()

int capture_start_thread

(

)

#include <capture.c>

Start capture thread.

Start capture thread

Creates and starts the webcam capture thread. Also sends stream start notification to server.

Returns

0 on success, negative on error

Spawns the capture thread in the global client worker thread pool. The thread runs the main capture loop, continuously acquiring frames from the media source, processing them (resize, format conversion), and transmitting to the server with FPS rate limiting.

The capture loop implements this pipeline:

Frame Acquisition:

1. Read frame from media source (media_source_read_video())
2. Verify frame is valid (non-null, valid dimensions)
3. Measure acquisition time for timing/debug

Frame Processing (Resize & Format):

1. Calculate optimal resize dimensions (fit-to-bounds, aspect ratio preserved)
2. Scale frame to max 800x600 using high-quality resizing
3. Format frame as RGB (already normalized by media source)
4. Add black padding if frame smaller than target area

FPS Synchronization:

1. Calculate frame interval from target FPS (e.g., 60 FPS = 16.67ms)
2. Measure actual processing time (acquisition + resize)
3. Sleep for remaining interval to maintain target FPS
4. Account for monotonic clock drift and OS scheduling

Network Transmission:

1. Optionally compress frame if –compress is enabled
2. Serialize frame to PACKET_TYPE_IMAGE_FRAME format (width + height + pixels)
3. Send packet to server via send_packet_to_server()
4. Log transmission metrics (size, latency, dropped frames)

Loop Control:

1. Check if in snapshot mode and should exit (–snapshot mode)
2. Check for shutdown signal (capture_stop_thread() called)
3. Check for connection loss (reconnect trigger)
4. Loop back to step 1

FPS Behavior:

• Target FPS from --fps option (default: 60)
• Achieved via sleep-based rate limiting (monotonic clock)
• Maintains consistent frame intervals even if processing takes variable time
• Adapts to system load and media source frame availability

Must be called after server connection is established and protocol thread started.

Returns

ASCIICHAT_OK on success, error code on failure:

• ERROR_THREAD: Thread pool full or system thread creation failed
• ERROR_NO_MEMORY: Insufficient memory for thread resources

Note

Must be paired with capture_stop_thread() to properly clean up.

Frame resizing and FPS limiting are handled transparently by the capture thread.

Rate limiting is frame-by-frame, not based on network round-trip time.

See also

capture_stop_thread "Stop capture thread"

capture_thread_exited "Check if thread has exited"

media_source_read_video "Frame acquisition from media source"

Definition at line 483 of file client/capture.c.

                           {
  if (THREAD_IS_CREATED(g_capture_thread_created)) {
    log_warn("Capture thread already created");
    return 0;
  }
 
  // Register capture atomics with named debug registry
  static bool capture_atomics_registered = false;
  if (!capture_atomics_registered) {
    NAMED_REGISTER_ATOMIC(&g_capture_thread_exited, "capture_webcam_thread_exit_confirmation", NULL);
    capture_atomics_registered = true;
  }
 
  // Start webcam capture thread
  atomic_store_bool(&g_capture_thread_exited, false);
  if (thread_pool_spawn(g_client_worker_pool, webcam_capture_thread_func, NULL, 2, "webcam_capture") != ASCIICHAT_OK) {
    SET_ERRNO(ERROR_THREAD, "Webcam capture thread creation failed");
    LOG_ERRNO_IF_SET("Webcam capture thread creation failed");
    return -1;
  }
 
  g_capture_thread_created = true;
  log_debug("Webcam capture thread created successfully");
 
  return 0;
}

References ASCIICHAT_OK, atomic_store_bool(), ERROR_THREAD, g_client_worker_pool, log_debug, LOG_ERRNO_IF_SET, log_warn, NAMED_REGISTER_ATOMIC, SET_ERRNO, THREAD_IS_CREATED, and thread_pool_spawn().

Referenced by protocol_start_connection().

capture_stop_thread()

void capture_stop_thread

(

)

#include <capture.c>

Stop capture thread.

Stop capture thread

Gracefully stops the capture thread and cleans up resources. Safe to call multiple times.

Signals the capture thread to exit and waits for it to join cleanly. Called during connection loss handling or shutdown.

The function:

1. Sets shutdown flag
2. Polls capture_thread_exited() until thread exits
3. Releases thread-specific resources

Must be called before reconnection to prevent resource leaks.

Note

This function blocks until the capture thread exits. Typical wait time: <1 second (one frame interval).

See also

capture_start_thread "Start capture thread"

Definition at line 517 of file client/capture.c.

                           {
  if (!THREAD_IS_CREATED(g_capture_thread_created)) {
    return;
  }
 
  // Wait for thread to exit gracefully
  int wait_count = 0;
  while (wait_count < 20 && !atomic_load_bool(&g_capture_thread_exited)) {
    platform_sleep_us(100 * US_PER_MS_INT); // 100ms
    wait_count++;
  }
 
  if (!atomic_load_bool(&g_capture_thread_exited)) {
    log_warn("Capture thread not responding after 2 seconds - will be joined by thread pool");
  }
 
  // Thread will be joined by thread_pool_stop_all() in protocol_stop_connection()
  g_capture_thread_created = false;
}

References atomic_load_bool(), log_warn, platform_sleep_us(), THREAD_IS_CREATED, and US_PER_MS_INT.

Referenced by capture_cleanup(), and protocol_stop_connection().

capture_thread_exited()

bool capture_thread_exited

(

)

#include <capture.c>

Check if capture thread has exited.

Check if capture thread has exited

Returns

true if thread has exited, false otherwise

Atomically checks whether the capture thread has finished execution. The main thread uses this to wait for clean thread shutdown before reconnection or final cleanup.

Returns

true if thread exited, false otherwise

Note

The main thread typically polls this in a 10-100ms loop until true.

See also

capture_stop_thread "Stop capture thread"

Definition at line 543 of file client/capture.c.

                             {
  return atomic_load_bool(&g_capture_thread_exited);
}

References atomic_load_bool().