ascii-chat
Binary v0.16.50-198-g78f51665f
Library lib/v0.3.2-2240-g78f51665f
Video chat in your terminal
Loading...
Searching...
No Matches
capture.h File Reference

ascii-chat Client Media Capture Management Interface More...

Go to the source code of this file.

Functions

int capture_init ()
 Initialize capture subsystem.
int capture_start_thread (client_connection_t *connection)
 Start capture thread.
void capture_stop_thread (client_connection_t *connection)
 Stop capture thread.
bool capture_thread_exited (client_connection_t *connection)
 Check if capture thread has exited.
void capture_cleanup ()
 Cleanup capture subsystem.
void * capture_get_media_source (void)
struct session_capture_ctx * capture_get_context (void)

Detailed Description

ascii-chat Client Media Capture Management Interface

Defines the interface for video capture and transmission management in the ascii-chat client. Abstracts over multiple media sources (webcam, files, stdin, test patterns) and handles frame processing, resizing, and FPS sync.

The capture subsystem:

  • Manages platform-specific webcam access via abstraction layer
  • Supports multiple media sources (webcam, local files, stdin, test patterns)
  • Performs frame resizing and format conversion for network efficiency
  • Synchronizes frame transmission with configurable FPS rates
  • Handles periodic frame transmission to the server

Media Source Types

The capture subsystem supports four media source types via the media_source_t abstraction (see Media):

  • WEBCAM: Hardware camera device (webcam/camera)
    • Platform-specific access (V4L2 on Linux, AVFoundation on macOS)
    • Real-time capture with variable frame rate
    • Selected via --webcam-index <N> (default: 0, first camera)
  • FILE: Local media file or network stream
    • Supported formats: mp4, avi, mkv, webm, mov, flv, wmv, gif
    • Specified via --file <path> option
    • Playback at configurable frame rate
    • Loop support via --media-loop option
  • STDIN: Piped or redirected media data
    • Specified via --file - (dash indicates stdin)
    • Any format supported by FFmpeg (detected automatically)
    • Useful for streaming pipes: ffmpeg ... | ascii-chat client --file -
    • Does not support seeking/rewinding
  • TEST: Procedural test pattern generator
    • Enabled via --test-pattern option
    • Generates animated test pattern without hardware
    • Useful for testing without webcam access
    • Always available (no hardware requirements)

Media source selection is automatic based on command-line options:

./ascii-chat client --test-pattern # Use test pattern
./ascii-chat client --file video.mp4 # Play file
./ascii-chat client --file - # Read from stdin
./ascii-chat client --webcam-index 0 # Use default camera
./ascii-chat client # Auto-detect (webcam or test)
static bool size_t index
Definition queries.h:8

Frame Format and Negotiation

All media sources are normalized to a common frame format:

  • Color Space: RGB (24-bit) - three bytes per pixel (R, G, B)
  • Byte Order: Platform-native (little-endian on x86/ARM)
  • Frame Dimensions: Variable (see Resize Behavior below)
  • Pixel Layout: Continuous row-major array (no padding between rows)

Frame format is opaque to the caller - the media source abstraction handles all conversions. Consumers always receive RGB frames ready to use:

// Get frame from any media source (format already normalized)
// Frame is guaranteed to be:
// - RGB format (8-bit per channel)
// - Dimensions: frame->width x frame->height pixels
// - Memory: frame->data[y * frame->width * 3 + x * 3 + channel]
audio_visualization_source_t source
Definition fft.c:25
image_t * media_source_read_video(media_source_t *source)
Read next video frame from media source.
Definition source.c:568

Different input types undergo conversion:

  • WEBCAM: V4L2/AVFoundation -> RGB (platform layer conversion)
  • FILE/STDIN: FFmpeg decoder -> RGB (FFmpeg handles all codecs)
  • TEST: Generated directly as RGB (no conversion needed)

Frame Resizing and Aspect Ratio

Captured frames are resized for network efficiency while preserving aspect ratio:

  • Maximum Dimensions: 800x600 pixels (default) - reasonable bandwidth
  • Aspect Ratio: Original aspect ratio preserved via fit-to-bounds
  • Scaling Algorithm: Letterboxing - frame fits within bounds with padding
  • Padding Color: Black (0, 0, 0) for empty regions

Resize process:

  1. Calculate scale factor to fit frame within 800x600
  2. Compute new dimensions maintaining aspect ratio
  3. Scale frame using resizing algorithm (maintains quality)
  4. Add black padding to fill remaining space (if needed)

Example resizes:

  • 1920x1440 (4:3) → 800x600 (fits exactly)
  • 1280x720 (16:9) → 800x450 (with 75px black bars top/bottom)
  • 640x480 (4:3) → 640x480 (no resize, smaller than max)
  • 3840x2160 (16:9) → 800x450 (same 16:9 aspect)

Frame resizing options (future):

  • --resize-width <W> - Custom max width (default: 800)
  • --resize-height <H> - Custom max height (default: 600)
  • --no-resize - Disable resizing (use original dimensions)

Frame Rate Synchronization (FPS Control)

The capture system implements frame rate limiting for bandwidth efficiency and smooth playback:

  • Capture Rate: Up to 144 FPS (high refresh rate support)
  • Target FPS: Configurable via --fps <rate> (default: 60)
  • Range: 1 to 144 FPS (1 = slowest, 144 = fastest)
  • Timing Method: Monotonic clock for accuracy
  • Adaptive Delay: Dynamic sleep adjustment for consistent frame intervals

FPS configuration:

./ascii-chat client --fps 30 # 30 FPS (slower, less bandwidth)
./ascii-chat client --fps 60 # 60 FPS (default, balanced)
./ascii-chat client --fps 144 # 144 FPS (max, high refresh)

FPS behavior by source type:

  • WEBCAM: Captures at hardware rate, then throttles to target FPS
  • FILE: Plays at media file's native FPS, then throttles to target
  • STDIN: Decodes frames, then throttles to target FPS
  • TEST: Generates at target FPS directly (no throttling needed)

Frame rate calculation:

  1. Calculate frame interval: interval_ms = 1000 / fps
  2. Record capture start time (monotonic clock)
  3. Capture and process frame
  4. Calculate elapsed time
  5. Sleep for remaining interval to achieve consistent rate

Example timing (60 FPS target = 16.67ms per frame):

  • Frame start: 0ms
  • Capture: 5ms
  • Process: 2ms
  • Elapsed: 7ms
  • Sleep: 9.67ms (to reach 16.67ms total)
  • Frame complete: 16.67ms ✓ (on schedule)

Capture Lifecycle

  1. Initialization (capture_init): Set up platform-specific webcam drivers
  2. Thread Start (capture_start_thread): Spawn capture thread in connection loop
  3. Capture Loop: Continuously grab frames and send to server
  4. Thread Stop (capture_stop_thread): Signal thread to exit and wait
  5. Cleanup (capture_cleanup): Release platform resources
Author
Zachary Fogg me@zf.nosp@m.o.gg
Date
2025
See also
topic_client_capture "Video Capture Architecture"
media/source.h "Media Source Abstraction"

Definition in file client/capture.h.

Function Documentation

◆ capture_cleanup()

void capture_cleanup ( )

Cleanup capture subsystem.

Releases all platform resources associated with webcam access. This must be called after capture_stop_thread() and before process exit.

The cleanup process:

  1. Closes webcam device (platform-specific close)
  2. Frees frame buffers
  3. Releases platform driver resources
  4. Cleans up thread handles

Must be called in proper order during shutdown to avoid resource leaks and to restore the platform to a clean state.

See also
capture_init "Initialize capture subsystem"

Cleanup capture subsystem

Stops capture thread and cleans up media source resources. Called during client shutdown.

Definition at line 535 of file client/capture.c.

535 {
536
537 // Destroy capture context
538 if (g_capture_capture_ctx) {
539 session_capture_destroy(g_capture_capture_ctx);
540 g_capture_capture_ctx = NULL;
541 }
542}
void session_capture_destroy(session_capture_ctx_t *ctx)
Destroy session capture context and free resources.

References session_capture_destroy().

◆ capture_get_context()

struct session_capture_ctx * capture_get_context ( void )

Definition at line 548 of file client/capture.c.

548 {
549 return g_capture_capture_ctx;
550}

◆ capture_get_media_source()

void * capture_get_media_source ( void )

Definition at line 544 of file client/capture.c.

544 {
545 return session_capture_get_media_source(g_capture_capture_ctx);
546}
void * session_capture_get_media_source(session_capture_ctx_t *ctx)
Get the underlying media source from capture context.

References session_capture_get_media_source().

◆ capture_init()

int capture_init ( )

Initialize capture subsystem.

Initializes the media source and capture resources. Determines media source type from command-line options, then sets up platform-specific drivers and buffers. This must be called before any other capture functions.

The initialization process:

  1. Determines media source type from options (test pattern, file, stdin, or webcam)
  2. Creates appropriate media source (media_source_create())
  3. For WEBCAM: Initializes platform driver (V4L2/AVFoundation/Media Foundation)
  4. For FILE/STDIN: Opens FFmpeg decoder with format detection
  5. For TEST: Initializes procedural test pattern generator
  6. Initializes frame buffers and capture parameters

Media source selection (in order):

  1. If --test-pattern → MEDIA_SOURCE_TEST
  2. Else if --file - → MEDIA_SOURCE_STDIN
  3. Else if --file <path> → MEDIA_SOURCE_FILE
  4. Else if --url <url> → MEDIA_SOURCE_FILE (network URL)
  5. Else → MEDIA_SOURCE_WEBCAM (with –webcam-index)

Options affecting initialization:

  • --test-pattern: Use synthetic test pattern (any platform, no hardware needed)
  • --file <path>: Media file to play (mp4, avi, mkv, gif, etc.)
  • --file -: Read media from stdin (any FFmpeg format)
  • --url <url>: Network stream (HTTP/HTTPS/RTSP/YouTube/etc.)
  • --webcam-index <N>: Which camera to open (default: 0, first camera)
  • --media-loop: Loop file playback when EOF reached
  • --webcam-flip: Flip video vertically (webcam and test patterns only)
  • --fps <rate>: Target frame rate in FPS (1-144, default: 60)

Error handling:

  • WEBCAM: Device not found, driver error, device busy
  • FILE: File not found, unsupported format, FFmpeg error
  • STDIN: FFmpeg format detection failure
  • TEST: Rarely fails (only if display dimensions unavailable)
Returns
ASCIICHAT_OK on success, error code on failure:
  • ERROR_WEBCAM: Device not found or driver error
  • ERROR_WEBCAM_IN_USE: Device busy (another app using it)
  • ERROR_FILE_NOT_FOUND: Media file not found
  • ERROR_FORMAT: Unsupported media format
  • ERROR_NOT_SUPPORTED: Platform doesn't support this source type
Note
This is called during client initialization, before entering the connection loop.
The actual frame capture happens in the capture thread (capture_start_thread).
Frame resizing to 800x600 (max) happens during capture, not initialization.
See also
media_source_create "Media source creation"
capture_start_thread "Start capture thread"
capture_cleanup "Cleanup resources"

Initialize capture subsystem

Sets up media source (webcam, file, or stdin) and prepares capture system for operation. Must be called once during client initialization.

Returns
0 on success, negative on error

Definition at line 394 of file client/capture.c.

394 {
395 // Build capture configuration from options
396 session_capture_config_t config = {0};
397 const char *media_url = GET_OPTION(media_url);
398 const char *media_file = GET_OPTION(media_file);
399 bool media_from_stdin = GET_OPTION(media_from_stdin);
400
401 if (media_url && media_url[0] != '\0') {
402 // Network URL streaming (takes priority over --file)
403 // Don't open webcam when streaming from URL
404 config.type = MEDIA_SOURCE_FILE;
405 config.path = media_url;
406 config.loop = false; // Network URLs cannot be looped
407 log_debug("Using network URL: %s (webcam disabled)", media_url);
408 } else if (media_file && media_file[0] != '\0') {
409 // File or stdin streaming - don't open webcam
410 config.type = media_from_stdin ? MEDIA_SOURCE_STDIN : MEDIA_SOURCE_FILE;
411 config.path = media_file;
412 config.loop = GET_OPTION(media_loop) && !media_from_stdin;
413 log_debug("Using media %s: %s (webcam disabled)", media_from_stdin ? "stdin" : "file", media_file);
414 } else if (GET_OPTION(test_pattern)) {
415 // Test pattern mode - don't open real webcam
416 config.type = MEDIA_SOURCE_TEST;
417 config.path = NULL;
418 log_debug("Using test pattern mode");
419 } else {
420 // Webcam mode (default)
421 static char webcam_index_str[CLIENT_CAPTURE_WEBCAM_INDEX_BUFFER_SIZE];
422 safe_snprintf(webcam_index_str, sizeof(webcam_index_str), "%u", GET_OPTION(webcam_index));
423 config.type = MEDIA_SOURCE_WEBCAM;
424 config.path = webcam_index_str;
425 log_debug("Using webcam device %u", GET_OPTION(webcam_index));
426 }
427 int configured_fps = GET_OPTION(fps);
428 config.target_fps = configured_fps > 0 ? (uint32_t)configured_fps : CAPTURE_TARGET_FPS;
429 config.resize_for_network = true; // Client always resizes for network transmission
430
431 // Configure audio capture with fallback to microphone
432 config.enable_audio = true;
433 config.audio_fallback_to_mic = true;
435
436 // Add seek timestamp if specified
437 config.initial_seek_timestamp = GET_OPTION(media_seek_timestamp);
438
439 // Create capture context using session library
441 g_capture_capture_ctx = session_capture_create(&config);
442 if (!g_capture_capture_ctx) {
443 // Check if there's already an error set (e.g., ERROR_WEBCAM_IN_USE)
445 asciichat_error_t existing_error = errno_peek_since(create_scope, &failure) ? failure.code : ASCIICHAT_OK;
446 log_debug("session_capture_create failed %s", asciichat_error_format(NULL, existing_error));
447 if (existing_error != ASCIICHAT_OK) {
448 log_debug("Returning existing error %s", asciichat_error_format(NULL, existing_error));
449 return existing_error;
450 }
451 SET_ERRNO(ERROR_MEDIA_INIT, "Failed to initialize capture source");
452 return -1;
453 }
454
455 return 0;
456}
bool errno_peek_since(asciichat_errno_scope_t scope, asciichat_error_context_t *out)
asciichat_errno_scope_t errno_checkpoint(void)
const char * asciichat_error_format(const char *message, asciichat_error_t code)
#define SET_ERRNO(code,...)
#define CAPTURE_TARGET_FPS
#define CLIENT_CAPTURE_WEBCAM_INDEX_BUFFER_SIZE
unsigned int uint32_t
asciichat_error_t
Error and exit codes - unified status values (0-255).
Definition error_codes.h:48
@ ASCIICHAT_OK
Definition error_codes.h:50
@ ERROR_MEDIA_INIT
Definition error_codes.h:69
#define log_debug(...)
Log a DEBUG message.
Definition log/log.h:589
@ MEDIA_SOURCE_STDIN
Piped or redirected input.
Definition source.h:104
@ MEDIA_SOURCE_WEBCAM
Hardware webcam device.
Definition source.h:102
@ MEDIA_SOURCE_FILE
Media file (video/audio).
Definition source.h:103
@ MEDIA_SOURCE_TEST
Test pattern generator.
Definition source.h:105
#define GET_OPTION(field)
Definition options.h:1333
session_capture_ctx_t * session_capture_create(const session_capture_config_t *config)
Legacy function - creates either mirror or network capture based on options.
audio_context_t * audio_get_context(void)
Get the global audio context for use by other subsystems.
When code needs to work with asciichat errno snapshot, it keeps total, copied here instead of scatter...
Media-source and loader settings used to create a session capture context.
media_source_type_t type
Media source type (WEBCAM, FILE, STDIN, TEST).
const char * path
Device index (for webcam) or file path (for file sources).
uint32_t target_fps
Target frame rate in FPS (e.g., 60 for display, 144 for network).
double initial_seek_timestamp
Seek to this timestamp after opening media (0.0 = no seek).
bool audio_fallback_to_mic
Fall back to microphone if file audio is not available.
bool resize_for_network
Resize frames to network-optimal dimensions (MAX_FRAME_WIDTH x MAX_FRAME_HEIGHT).
bool loop
Enable loop playback for file sources.
bool enable_audio
Enable audio capture from media source.
void * mic_audio_ctx
Microphone audio context for fallback (borrowed, not owned).
test pattern is intentionally a small, concrete record. Fields such as image, rgba give callers the i...

References asciichat_error_format(), ASCIICHAT_OK, session_capture_config_t::audio_fallback_to_mic, audio_get_context(), CAPTURE_TARGET_FPS, CLIENT_CAPTURE_WEBCAM_INDEX_BUFFER_SIZE, asciichat_error_context_t::code, session_capture_config_t::enable_audio, errno_checkpoint(), errno_peek_since(), ERROR_MEDIA_INIT, GET_OPTION, session_capture_config_t::initial_seek_timestamp, log_debug, session_capture_config_t::loop, MEDIA_SOURCE_FILE, MEDIA_SOURCE_STDIN, MEDIA_SOURCE_TEST, MEDIA_SOURCE_WEBCAM, session_capture_config_t::mic_audio_ctx, session_capture_config_t::path, session_capture_config_t::resize_for_network, session_capture_create(), SET_ERRNO, session_capture_config_t::target_fps, and session_capture_config_t::type.

◆ capture_start_thread()

int capture_start_thread ( client_connection_t * connection)

Start capture thread.

Spawns the capture thread in the global client worker thread pool. The thread runs the main capture loop, continuously acquiring frames from the media source, processing them (resize, format conversion), and transmitting to the server with FPS rate limiting.

The capture loop implements this pipeline:

Frame Acquisition:

  1. Read frame from media source (media_source_read_video())
  2. Verify frame is valid (non-null, valid dimensions)
  3. Measure acquisition time for timing/debug

Frame Processing (Resize & Format):

  1. Calculate optimal resize dimensions (fit-to-bounds, aspect ratio preserved)
  2. Scale frame to max 800x600 using high-quality resizing
  3. Format frame as RGB (already normalized by media source)
  4. Add black padding if frame smaller than target area

FPS Synchronization:

  1. Calculate frame interval from target FPS (e.g., 60 FPS = 16.67ms)
  2. Measure actual processing time (acquisition + resize)
  3. Sleep for remaining interval to maintain target FPS
  4. Account for monotonic clock drift and OS scheduling

Network Transmission:

  1. Optionally compress frame if –compress is enabled
  2. Serialize frame to PACKET_TYPE_IMAGE_FRAME format (width + height + pixels)
  3. Send packet to server via send_packet_to_server()
  4. Log transmission metrics (size, latency, dropped frames)

Loop Control:

  1. Check if in snapshot mode and should exit (–snapshot mode)
  2. Check for shutdown signal (capture_stop_thread() called)
  3. Check for connection loss (reconnect trigger)
  4. Loop back to step 1

FPS Behavior:

  • Target FPS from --fps option (default: 60)
  • Achieved via sleep-based rate limiting (monotonic clock)
  • Maintains consistent frame intervals even if processing takes variable time
  • Adapts to system load and media source frame availability

Must be called after server connection is established and protocol thread started.

Returns
ASCIICHAT_OK on success, error code on failure:
  • ERROR_THREAD: Thread pool full or system thread creation failed
  • ERROR_NO_MEMORY: Insufficient memory for thread resources
Note
Must be paired with capture_stop_thread() to properly clean up.
Frame resizing and FPS limiting are handled transparently by the capture thread.
Rate limiting is frame-by-frame, not based on network round-trip time.
See also
capture_stop_thread "Stop capture thread"
capture_thread_exited "Check if thread has exited"
media_source_read_video "Frame acquisition from media source"

Start capture thread

Creates and starts the webcam capture thread. Also sends stream start notification to server.

Returns
0 on success, negative on error

Definition at line 467 of file client/capture.c.

467 {
468 if (!connection->capture_context)
469 return 0;
470 if (THREAD_IS_CREATED(connection->capture_thread_created)) {
471 log_warn("Capture thread already created");
472 return 0;
473 }
474
475 // Start webcam capture thread
476 atomic_store_bool(&connection->capture_thread_exited, false);
477 if (thread_pool_spawn(connection->client_worker_pool, webcam_capture_thread_func, connection, 2, "webcam_capture") != ASCIICHAT_OK) {
478 SET_ERRNO(ERROR_THREAD, "Webcam capture thread creation failed");
479 LOG_ERRNO_IF_SET("Webcam capture thread creation failed");
480 return -1;
481 }
482
483 connection->capture_thread_created = true;
484 log_debug("Webcam capture thread created successfully");
485
486 return 0;
487}
#define LOG_ERRNO_IF_SET(message)
void atomic_store_bool(atomic_t *a, bool value)
Atomically store a boolean value.
@ ERROR_THREAD
#define log_warn(...)
Log a WARN message.
Definition log/log.h:615
thread_pool_t * client_worker_pool
session_capture_ctx_t * capture_context
#define THREAD_IS_CREATED(created_flag)
Definition thread.h:66
asciichat_error_t thread_pool_spawn(thread_pool_t *pool, void *(*thread_func)(void *), void *thread_arg, int stop_id, const char *thread_name)
Spawn a worker thread in the pool.

References ASCIICHAT_OK, atomic_store_bool(), client_connection::capture_context, client_connection::capture_thread_created, client_connection::capture_thread_exited, client_connection::client_worker_pool, ERROR_THREAD, log_debug, LOG_ERRNO_IF_SET, log_warn, SET_ERRNO, THREAD_IS_CREATED, and thread_pool_spawn().

Referenced by client_protocol_start().

◆ capture_stop_thread()

void capture_stop_thread ( client_connection_t * connection)

Stop capture thread.

Signals the capture thread to exit and waits for it to join cleanly. Called during connection loss handling or shutdown.

The function:

  1. Sets shutdown flag
  2. Polls capture_thread_exited() until thread exits
  3. Releases thread-specific resources

Must be called before reconnection to prevent resource leaks.

Note
This function blocks until the capture thread exits. Typical wait time: <1 second (one frame interval).
See also
capture_start_thread "Start capture thread"

Stop capture thread

Gracefully stops the capture thread and cleans up resources. Safe to call multiple times.

Definition at line 496 of file client/capture.c.

496 {
497 if (!connection)
498 return;
499 if (!THREAD_IS_CREATED(connection->capture_thread_created)) {
500 return;
501 }
502
503 // Wait for thread to exit gracefully
504 int wait_count = 0;
505 while (wait_count < 20 && !atomic_load_bool(&connection->capture_thread_exited)) {
506 platform_sleep_us(100 * US_PER_MS_INT); // 100ms
507 wait_count++;
508 }
509
510 if (!atomic_load_bool(&connection->capture_thread_exited)) {
511 log_warn("Capture thread not responding after 2 seconds - will be joined by thread pool");
512 }
513
514 // Thread will be joined by thread_pool_stop_all() in client_connection_stop()
515 connection->capture_thread_created = false;
516}
bool atomic_load_bool(atomic_t *a)
Atomic pointer wrapper.
#define US_PER_MS_INT
Definition time.h:176

References atomic_load_bool(), client_connection::capture_thread_created, client_connection::capture_thread_exited, log_warn, THREAD_IS_CREATED, and US_PER_MS_INT.

◆ capture_thread_exited()

bool capture_thread_exited ( client_connection_t * connection)

Check if capture thread has exited.

Atomically checks whether the capture thread has finished execution. The main thread uses this to wait for clean thread shutdown before reconnection or final cleanup.

Returns
true if thread exited, false otherwise
Note
The main thread typically polls this in a 10-100ms loop until true.
See also
capture_stop_thread "Stop capture thread"

Check if capture thread has exited

Returns
true if thread has exited, false otherwise

Definition at line 524 of file client/capture.c.

524 {
525 return atomic_load_bool(&connection->capture_thread_exited);
526}

References atomic_load_bool(), and client_connection::capture_thread_exited.