ascii-chat
Binary v0.16.50-198-g78f51665f
Library lib/v0.3.2-2240-g78f51665f
Video chat in your terminal
Loading...
Searching...
No Matches
Application: Server

🖥️ ascii-chat server implementation and architecture

🖥️ ascii-chat server implementation and architecture

Multi-client real-time video and audio streaming server with per-client threading, end-to-end encryption, and terminal-aware ASCII rendering.

Server Application

The server combines a shared network runtime with per-peer media state. The network layer accepts and drives connections; server callbacks authenticate peers and connect them to the media mixer, renderer, statistics, and TUI.

Layers

  • network_server_t owns listeners, backend workers, and connection registry.
  • network_connection_t owns one peer transport, callbacks, handshake, and lifecycle state.
  • client_info_t owns peer-specific terminal/media state and render workers.
  • ACIP handlers validate and dispatch packets above the transport boundary.
  • Stream/mixer/render modules produce personalized video and audio output.

Peer flow

A listener creates a connection, the server callback installs application context, and ACIP negotiation authenticates the peer. Once ready, the server sends state, accepts media into bounded buffers/mixers, and produces output through type-specific ACIP send helpers.

Concurrency

Network callbacks are serialized per connection and run outside backend event loops. Media render workers may run per client. Shared registries and mixers use their synchronization APIs; callbacks must not hold locks while joining workers or performing blocking I/O.

Shutdown

Stop accepting work, request network shutdown, stop and join media producers, drain connection callbacks, remove mixer sources, and release application state. The transport remains owned by the network connection until close callbacks complete.

See also
topic_server_main
topic_server_client
topic_network

Server Client State

src/server/client.c manages application state for each authenticated peer. It is not the socket server: network_server_t owns listeners and connection callbacks, while each client_info_t associates a connection with media, terminal capabilities, queues, and render workers.

Open and negotiate

server_connection_open() creates per-peer state and attaches it to the network_connection_t context. The packet callback negotiates through the shared transport, marks the connection ready, sends initial state, and dispatches ACIP packets to server handlers.

Media workers

Once a peer is ready, the server may create per-client video and audio render workers. These workers personalize ASCII output and mixed audio; they do not perform network reads or own connection lifecycle. Outbound packets go through the connection transport and bounded packet queues.

Close and ownership

Closing requests the network connection to stop and stops media workers, then removes mixer sources, drains/destroys client queues, releases media buffers, and clears application context. The network runtime releases the transport after callbacks drain.

See also
topic_server
topic_network
topic_server_render

Cryptographic Operations

Overview

The cryptographic operations module manages per-client cryptographic handshakes, X25519 key exchange, session encryption, and client authentication for the ascii-chat server. This module embodies the security-first philosophy of ascii-chat, providing end-to-end encryption with multiple authentication modes to ensure secure multi-client video chat. The module integrates seamlessly with the client lifecycle, performing cryptographic handshakes during connection establishment and managing session encryption throughout the client's connection.

Implementation: src/server/crypto.c, src/server/crypto.h

Key Responsibilities:

  • Initialize server crypto system and validate encryption configuration
  • Perform cryptographic handshake with each connecting client
  • Manage per-client crypto contexts stored in client_info_t structures
  • Provide encryption/decryption functions for secure packet transmission
  • Support multiple authentication modes (password, SSH key, passwordless)
  • Integrate with client whitelist for authenticated access control

Cryptographic Handshake Architecture

The cryptographic handshake follows a multi-phase protocol:

Phase 0: Protocol Negotiation

Step 0a: Receive Client Protocol Version

// Receive client's protocol version
result = receive_packet(socket, &packet_type, &payload, &payload_len);
if (packet_type != PACKET_TYPE_PROTOCOL_VERSION) {
return -1; // Protocol mismatch
}
if (!client_version.supports_encryption) {
return -1; // Client doesn't support encryption
}
int receive_packet(socket_t sockfd, packet_type_t *type, void **data, size_t *len)
Receive a basic packet without encryption.
Definition packet.c:594
@ PACKET_TYPE_PROTOCOL_VERSION
Protocol version and capabilities negotiation.
Definition packet.h:317
Protocol version negotiation packet structure (Packet Type 1).
Definition packet.h:871
uint8_t supports_encryption
Encryption & authentication mode bitmask (ACIP_CRYPTO_* constants: NONE, ENCRYPT, AUTH,...
Definition packet.h:879

Step 0b: Send Server Protocol Version

// Send server's protocol version
protocol_version_packet_t server_version = {0};
server_version.protocol_version = htons(1);
server_version.supports_encryption = 1;
result = send_protocol_version_packet(socket, &server_version);
int send_protocol_version_packet(socket_t sockfd, const protocol_version_packet_t *version)
Send protocol version packet.
Definition packet.c:942
uint16_t protocol_version
Major protocol version (must match for compatibility).
Definition packet.h:873

Step 0c: Receive Client Crypto Capabilities

// Receive client's supported algorithms
result = receive_packet(socket, &packet_type, &payload, &payload_len);
// Parse supported algorithms
uint16_t supported_kex = ntohs(client_caps.supported_kex_algorithms);
uint16_t supported_auth = ntohs(client_caps.supported_auth_algorithms);
uint16_t supported_cipher = ntohs(client_caps.supported_cipher_algorithms);
unsigned short uint16_t
Crypto capabilities packet structure (Packet Type 14).
Definition packet.h:1027
uint16_t supported_auth_algorithms
Supported authentication algorithms bitmask (AUTH_ALGO_*).
Definition packet.h:1031
uint16_t supported_kex_algorithms
Supported key exchange algorithms bitmask (KEX_ALGO_*).
Definition packet.h:1029
uint16_t supported_cipher_algorithms
Supported cipher algorithms bitmask (CIPHER_ALGO_*).
Definition packet.h:1033

Step 0d: Select Algorithms and Send Parameters

// Select algorithms (currently only X25519 + Ed25519 + XSalsa20-Poly1305)
crypto_parameters_packet_t server_params = {0};
server_params.selected_kex = KEX_ALGO_X25519;
// Select authentication algorithm based on server configuration
}
result = send_crypto_parameters_packet(socket, &server_params);
@ KEY_TYPE_ED25519
Definition key_types.h:58
#define AUTH_ALGO_ED25519
Ed25519 authentication (Edwards-curve signatures).
Definition packet.h:1140
#define KEX_ALGO_X25519
X25519 key exchange (Curve25519).
Definition packet.h:1139
int send_crypto_parameters_packet(socket_t sockfd, const crypto_parameters_packet_t *params)
Send crypto parameters packet.
Definition packet.c:970
#define CIPHER_ALGO_XSALSA20_POLY1305
XSalsa20-Poly1305 authenticated encryption.
Definition packet.h:1142
bool g_server_encryption_enabled
Global flag indicating if server encryption is enabled.
private_key_t g_server_private_key
Global server private key (first identity key, for backward compatibility).
Crypto parameters packet structure (Packet Type 15).
Definition packet.h:1056
uint8_t selected_kex
Selected key exchange algorithm (KEX_ALGO_*).
Definition packet.h:1058
uint8_t selected_cipher
Selected cipher algorithm (CIPHER_ALGO_*).
Definition packet.h:1062
uint8_t selected_auth
Selected authentication algorithm (AUTH_ALGO_*).
Definition packet.h:1060

Phase 1: Key Exchange

Step 1: Send Server's Ephemeral Public Key

// Generate ephemeral key pair for this client session
x25519_keypair_t ephemeral_keys;
x25519_generate_keypair(&ephemeral_keys);
// Store ephemeral private key in client's crypto context
client->crypto_ctx.ephemeral_private_key = ephemeral_keys.private_key;
// Send ephemeral public key to client
key_exchange_init_packet_t keyex_init = {0};
memcpy(keyex_init.public_key, ephemeral_keys.public_key, X25519_PUBLIC_KEY_SIZE);
// If server has identity key, sign the key exchange
ed25519_sign(&keyex_init.signature, keyex_init.public_key,
X25519_PUBLIC_KEY_SIZE, &g_server_private_key);
memcpy(keyex_init.identity_key, g_server_public_key, ED25519_PUBLIC_KEY_SIZE);
}
result = send_key_exchange_init_packet(socket, &keyex_init);
#define ED25519_PUBLIC_KEY_SIZE
Ed25519 public key size in bytes.
Definition sizes.h:37

Step 2: Receive Client's Public Key and Derive Shared Secret

// Receive client's ephemeral public key
key_exchange_response_packet_t client_keyex;
result = receive_packet(socket, &packet_type, &payload, &payload_len);
// Derive shared secret using X25519 key exchange
x25519_shared_secret_t shared_secret;
x25519_derive_shared_secret(&shared_secret, client_keyex.public_key,
&client->crypto_ctx.ephemeral_private_key);
// Store shared secret in client's crypto context
memcpy(client->crypto_ctx.shared_secret, shared_secret, X25519_SHARED_SECRET_SIZE);

Phase 2: Authentication

Step 2: Send Authentication Challenge

// Receive client's public key and send auth challenge
auth_request_packet_t auth_req;
result = receive_packet(socket, &packet_type, &payload, &payload_len);
// If whitelist enabled, verify client's public key
bool client_in_whitelist = false;
for (size_t i = 0; i < g_num_whitelisted_clients; i++) {
if (memcmp(auth_req.public_key, g_client_whitelist[i],
client_in_whitelist = true;
break;
}
}
if (!client_in_whitelist) {
return -1; // Client not in whitelist
}
}
// Generate authentication challenge
auth_challenge_packet_t challenge;
random_bytes(challenge.challenge, AUTH_CHALLENGE_SIZE);
// Sign challenge with server's identity key (if available)
ed25519_sign(&challenge.signature, challenge.challenge,
}
result = send_auth_challenge_packet(socket, &challenge);
size_t g_num_whitelisted_clients
Number of whitelisted clients.
public_key_t g_client_whitelist[MAX_CLIENTS]
Global client public key whitelist.
#define AUTH_CHALLENGE_SIZE
Challenge nonce size (32 bytes).
Definition sizes.h:103

Step 3: Receive Authentication Response and Complete Handshake

// Receive client's authentication response
auth_response_packet_t auth_resp;
result = receive_packet(socket, &packet_type, &payload, &payload_len);
// Verify client's signature on challenge
if (!ed25519_verify(auth_resp.signature, challenge.challenge,
AUTH_CHALLENGE_SIZE, auth_req.public_key)) {
return -1; // Authentication failed
}
// Derive session encryption keys from shared secret
derive_session_keys(&client->crypto_ctx.session_keys, &client->crypto_ctx.shared_secret);
// Mark handshake as complete
atomic_store(&client->crypto_initialized, true);

Authentication Modes

Password Authentication

How It Works:

  • Uses Argon2id key derivation from shared password
  • Both server and client derive same key from password
  • No identity keys required (password-only mode)
  • Suitable for trusted networks or simple deployments

Configuration:

// Set password via --password command-line option
// Password is used for key derivation, not transmitted
// Both server and client must use same password

SSH Key Authentication

How It Works:

  • Server uses Ed25519 private key for identity verification
  • Client provides Ed25519 public key for authentication
  • Identity verification via known_hosts and whitelist
  • Strong authentication with cryptographic signatures

Configuration:

// Server loads SSH key via --key option
// Supports SSH Ed25519 key files
// Supports gpg:keyid format for GPG keys (if GPG support enabled)
// Client provides public key during handshake

Key Loading:

  • Validates SSH key file format
  • Native encrypted key support (bcrypt_pbkdf + BearSSL AES-256-CTR)
  • Validates key type (Ed25519 required)
  • Loads private key for signing operations

Passwordless Mode

How It Works:

  • Ephemeral keys only (no long-term identity)
  • Key exchange provides confidentiality but not authentication
  • Suitable for trusted networks or testing
  • No identity verification performed

Configuration:

// No --key or --password provided
// Server runs in passwordless mode
// Key exchange provides encryption but no authentication

Per-Client Crypto Contexts

Each client has an independent crypto context stored in client_info_t:

typedef struct {
// Key exchange state
x25519_private_key_t ephemeral_private_key;
x25519_shared_secret_t shared_secret;
// Session encryption keys
session_keys_t session_keys;
// Authentication state
ed25519_public_key_t client_public_key;
bool authenticated;
// Handshake state
handshake_state_t state;
} crypto_handshake_ctx_t;
bool initialized
Definition mmap.c:41

Context Lifecycle:

  • Created during connection establishment
  • Initialized during cryptographic handshake
  • Used throughout client connection for encryption/decryption
  • Cleaned up on client disconnect

Thread Safety:

  • Each client has independent crypto context (no shared state)
  • Socket access protected by client_state_mutex
  • Per-client encryption/decryption operations are isolated
  • Global server crypto state (g_server_private_key) read-only after init

Encryption and Decryption Operations

Packet Encryption

After handshake completion, all packets are encrypted before transmission:

int crypto_server_encrypt_packet(uint32_t client_id,
const void *plaintext, size_t plaintext_len,
void *ciphertext, size_t *ciphertext_len)
{
// Get client's crypto context
client_info_t *client = get_client_by_id(client_id);
// Check if encryption is enabled
if (!client->crypto_initialized) {
// Encryption not initialized - passthrough
memcpy(ciphertext, plaintext, plaintext_len);
*ciphertext_len = plaintext_len;
return 0;
}
// Encrypt packet using XSalsa20-Poly1305
int result = xsalsa20poly1305_encrypt(ciphertext, ciphertext_len,
plaintext, plaintext_len,
NULL, 0, // No additional data
client->crypto_ctx.session_keys.encryption_key);
return result;
}
unsigned int uint32_t
struct client_info client_info_t

Packet Decryption

All received packets are decrypted before processing:

int crypto_server_decrypt_packet(uint32_t client_id,
const void *ciphertext, size_t ciphertext_len,
void *plaintext, size_t *plaintext_len)
{
// Get client's crypto context
client_info_t *client = get_client_by_id(client_id);
// Check if encryption is enabled
if (!client->crypto_initialized) {
// Encryption not initialized - passthrough
memcpy(plaintext, ciphertext, ciphertext_len);
*plaintext_len = ciphertext_len;
return 0;
}
// Decrypt packet using XSalsa20-Poly1305
int result = xsalsa20poly1305_decrypt(plaintext, plaintext_len,
ciphertext, ciphertext_len,
NULL, 0, // No additional data
client->crypto_ctx.session_keys.decryption_key);
return result;
}

Automatic Passthrough:

  • When encryption is disabled (–no-encrypt), packets pass through unchanged
  • No performance overhead when encryption disabled
  • Seamless integration with non-encrypted mode

Client Whitelist Integration

Whitelist Loading

Client whitelist is loaded during server initialization:

// Load client whitelist if provided
if (strlen(opt_client_keys) > 0) {
if (parse_client_keys(opt_client_keys, g_client_whitelist,
return -1; // Whitelist loading failed
}
log_info("Server will only accept %zu whitelisted clients",
}
#define MAX_CLIENTS
Maximum possible clients (static array size) - actual runtime limit set by –max-clients (1-128).
#define log_info(...)
Log an INFO message.
Definition log/log.h:602

Whitelist Format:

  • Supports multiple key formats (SSH public keys, raw hex, etc.)
  • Multiple keys can be provided (comma-separated)
  • Keys are validated during loading
  • Only Ed25519 keys are supported

Whitelist Verification

During handshake, client's public key is verified against whitelist:

// Verify client's public key against whitelist
bool client_in_whitelist = false;
for (size_t i = 0; i < g_num_whitelisted_clients; i++) {
if (memcmp(client_public_key, g_client_whitelist[i],
client_in_whitelist = true;
break;
}
}
if (!client_in_whitelist) {
log_warn("Client public key not in whitelist - rejecting connection");
return -1; // Authentication failed
}
}
#define log_warn(...)
Log a WARN message.
Definition log/log.h:615

Verification Behavior:

  • If whitelist enabled, only whitelisted clients can connect
  • Clients not in whitelist are rejected during handshake
  • Detailed error logging for troubleshooting
  • Clean disconnect on whitelist rejection

Error Handling

Handshake Errors:

  • Client disconnection during handshake: Logged and return error
  • Protocol mismatch: Detailed error logging and disconnect
  • Authentication failure: Logged and disconnect (whitelist rejection)
  • Network errors: Detected and handled gracefully
  • Invalid packets: Validated before processing

Encryption/Decryption Errors:

  • Decryption failures: Logged and packet dropped
  • Encryption failures: Logged and connection marked lost
  • Key derivation failures: Handled gracefully
  • Session key errors: Clean disconnect

Algorithm Support

Current Algorithms:

  • Key Exchange: X25519 (Elliptic Curve Diffie-Hellman)
  • Cipher: XSalsa20-Poly1305 (Authenticated Encryption)
  • Authentication: Ed25519 (when server has identity key)
  • Key Derivation: Argon2id (for password-based authentication)
  • HMAC: HMAC-SHA256 (for additional integrity protection)

Future Algorithm Support:

  • Additional key exchange algorithms (ECDH-P256, etc.)
  • Additional cipher algorithms (ChaCha20-Poly1305, etc.)
  • Additional authentication algorithms (ECDSA, etc.)

Integration with Other Modules

Integration with client.c

Called By:

  • add_client(): Performs cryptographic handshake during connection
  • Handshake integrated into client connection flow

Provides To:

  • Per-client crypto contexts
  • Encryption/decryption functions
  • Authentication verification

Integration with protocol.c

Used By:

  • Protocol handlers decrypt received packets
  • Packet encryption before transmission
  • Seamless integration with packet processing

Provides To:

  • Packet encryption/decryption functions
  • Crypto context access

Integration with main.c

Called By:

  • init_server_crypto(): Initializes server crypto system
  • Server key loading and validation
  • Whitelist loading and validation

Provides To:

  • Global server crypto state
  • Server identity key
  • Client whitelist

Best Practices

DO:

  • Always validate packets before processing
  • Use per-client crypto contexts (no shared state)
  • Check crypto_initialized before encryption/decryption
  • Handle errors gracefully without disconnecting clients unnecessarily
  • Log detailed errors for troubleshooting

DON'T:

  • Don't share crypto contexts between clients
  • Don't skip packet validation
  • Don't ignore encryption/decryption errors
  • Don't expose private keys in logs
  • Don't skip whitelist verification when enabled
See also
src/server/crypto.c
src/server/crypto.h
Server Overview
Client Management
topic_crypto
topic_handshake
topic_keys

Server Lifecycle

Server mode creates shared listeners and one network_server_t. TCP, WebSocket, native QUIC, and WebTransport listeners feed the same connection callback contract where enabled. The server application does not create a separate receive/send thread pair for each protocol.

Startup

Startup parses options, initializes crypto/media services, configures optional QUIC/WebTransport endpoints, creates the network server, and starts listeners. The server callback installs client_info_t state for each peer and begins ACIP negotiation.

Run loop

network_server_run() drives connection callbacks and backend I/O. Server callbacks use network_connection_transport() and ACIP helpers; render and audio workers publish outbound media without bypassing the transport layer.

Shutdown

Signal handlers only call network_server_request_stop(). After the run loop returns, network_server_stop() drains connections and backend workers while preserving dependencies if a join fails. The owner then destroys the server, QUIC endpoints, media services, and UI in dependency order.

See also
topic_server_client
topic_network
topic_quic

Server ACIP Handling

The server receives packets through network_connection_t. Its connection callback handles negotiation, then passes READY traffic to the shared ACIP handlers with the connection's acip_transport_t and client_info_t context.

Receive path

The transport supplies complete packet boundaries. The ACIP layer checks packet type, payload length, CRC, security state, and negotiated capabilities. Handlers then update terminal capabilities, stream state, media buffers, audio mixer sources, or discovery state. Payload pointers are borrowed for the callback; queues copy or take ownership explicitly.

Media path

Incoming video enters the client's media state and is available to the server stream/render pipeline. Incoming PCM, batched audio, and Opus batches enter the client's audio buffer and mixer. Outbound ASCII, encoded video, and mixed audio use the shared ACIP send helpers and bounded queues.

Control and shutdown

Ping/pong, stream controls, terminal-size updates, errors, and server state stay on the reliable control path. A protocol or authentication failure stops the connection; it is not silently downgraded to an unauthenticated session. Closing stops producers before the connection and its transport are released.

See also
topic_server_client
topic_packet_types
topic_network

Render Threads

Overview

The render thread module manages per-client rendering threads that generate personalized ASCII frames at 60fps and mix audio streams at 172fps. This module embodies the real-time performance guarantees of ascii-chat, ensuring every client receives smooth video and low-latency audio regardless of the number of connected clients. The module implements sophisticated rate limiting, precise timing control, and thread lifecycle management to achieve professional-grade performance.

Implementation: src/server/render.c, src/server/render.h

Key Responsibilities:

  • Manage per-client video rendering threads (60fps per client)
  • Manage per-client audio rendering threads (172fps per client)
  • Coordinate frame generation timing and rate limiting
  • Ensure thread-safe access to client state and media buffers
  • Provide graceful thread lifecycle management
  • Handle platform-specific timing and synchronization

Per-Client Threading Model

Each connected client spawns exactly 2 dedicated threads:

Video Render Thread

Purpose: Generate personalized ASCII frames at 60fps Performance: 16.67ms intervals (60 frames per second) Operations:

  • Collects video frames from all active clients (via stream.c)
  • Creates composite frame with appropriate grid layout
  • Converts composite to ASCII using client's terminal capabilities
  • Queues ASCII frame in client's video packet queue
  • Rate limits to exactly 60fps with precise timing

Thread Implementation:

static void *video_render_thread_func(void *arg) {
uint32_t client_id = (uint32_t)(uintptr_t)arg;
// Initialize timing
struct timespec last_frame_time;
clock_gettime(CLOCK_MONOTONIC, &last_frame_time);
while (true) {
// Check shutdown flag
if (atomic_load(&g_server_should_exit) ||
!atomic_load(&client->video_render_thread_running) ||
!atomic_load(&client->active)) {
break;
}
// Generate personalized ASCII frame for client
ascii_frame_t *frame = generate_personalized_ascii_frame(client_id);
if (frame) {
// Queue frame for delivery via send thread
packet_queue_push_copy(client->video_packet_queue, PACKET_TYPE_ASCII_FRAME, frame, frame_size, 0);
}
// Rate limit to exactly 60fps
struct timespec current_time;
clock_gettime(CLOCK_MONOTONIC, &current_time);
double elapsed = timespec_diff_ms(&current_time, &last_frame_time);
if (elapsed < 16.67) {
// Ahead of schedule - sleep until next frame time
platform_sleep_usec((int)((16.67 - elapsed) * 1000));
}
last_frame_time = current_time;
}
return NULL;
}
@ PACKET_TYPE_ASCII_FRAME
Complete ASCII frame with all metadata.
Definition packet.h:397
asciichat_error_t packet_queue_push_copy(packet_queue_t *queue, packet_type_t type, const void *data, size_t length, uint32_t client_id)
Definition queue.c:154
When code needs to work with ascii frame, it keeps output, output_target, width, source_height here i...

Rate Limiting Strategy:

  • Uses CLOCK_MONOTONIC for precise timing
  • Calculates elapsed time since last frame
  • Sleeps only if ahead of schedule
  • Prevents CPU spinning under light load
  • Maintains exactly 60fps timing

Audio Render Thread

Purpose: Mix audio streams at 172fps (excluding client's own audio) Performance: 5.8ms intervals (172 frames per second) Operations:

  • Reads audio samples from all clients via audio mixer
  • Excludes client's own audio to prevent echo
  • Mixes remaining audio streams with ducking and compression
  • Queues mixed audio in client's audio packet queue
  • Rate limits to exactly 172fps (5.8ms intervals)

Thread Implementation:

static void *audio_render_thread_func(void *arg) {
uint32_t client_id = (uint32_t)(uintptr_t)arg;
// Audio mixer provides per-client mixed audio
while (true) {
// Check shutdown flag
if (atomic_load(&g_server_should_exit) ||
!atomic_load(&client->audio_render_thread_running) ||
!atomic_load(&client->active)) {
break;
}
// Get mixed audio (excluding client's own audio)
audio_sample_t *mixed_audio = mixer_get_mixed_audio(mixer, client_id);
if (mixed_audio) {
// Queue audio for delivery via send thread
packet_queue_push_copy(client->audio_packet_queue, PACKET_TYPE_AUDIO_BATCH, mixed_audio,
mixed_audio_size, 0);
}
// Rate limit to exactly 172fps (5.8ms intervals)
platform_sleep_usec(5800); // 5.8ms
}
return NULL;
}
@ PACKET_TYPE_AUDIO_BATCH
Batched audio packets for efficiency.
Definition packet.h:410
mixer_t *volatile g_audio_mixer
Global audio mixer instance for multi-client audio processing.
For ducking, this is the live record that ties threshold_dB, leader_margin_dB, atten_dB,...
Definition mixer.h:342

Audio Processing Features:

  • Echo cancellation (excludes client's own audio)
  • Audio ducking (quiets other clients when client speaks)
  • Compression (normalizes audio levels)
  • Noise gate (suppresses background noise)
  • Low-latency delivery (5.8ms intervals)

Rate Limiting and Timing Control

Timing Precision

Platform-Specific Timing:

  • Windows: Uses Sleep() with millisecond precision
  • POSIX: Uses condition variables for responsive interruption
  • High-resolution timing with clock_gettime()
  • Interruptible sleep for responsive shutdown

Rate Limiting Algorithm:

// Calculate elapsed time since last frame
double elapsed = timespec_diff_ms(&current_time, &last_frame_time);
if (elapsed < target_interval) {
// Ahead of schedule - sleep until next frame time
platform_sleep_usec((int)((target_interval - elapsed) * 1000));
} else {
// Behind schedule - skip sleep to catch up
// This prevents frame accumulation under load
}

Timing Guarantees:

  • Video: Exactly 60fps (16.67ms intervals)
  • Audio: Exactly 172fps (5.8ms intervals)
  • Platform-specific high-resolution timers
  • Interruptible sleep for responsive shutdown

CPU Usage Management

Prevent CPU Spinning:

  • Sleep when ahead of schedule
  • Skip sleep only when behind schedule
  • Minimal CPU usage under light load
  • Responsive to load changes

Catch-Up Strategy:

  • Behind schedule: Skip sleep to catch up
  • Prevents frame accumulation
  • Maintains real-time performance
  • Adaptive to load changes

Thread Safety and Synchronization

Client State Access

All client state access uses the snapshot pattern:

// Snapshot client state under mutex
mutex_lock(&client->client_state_mutex);
bool should_continue = client->video_render_thread_running &&
client->active;
uint32_t client_id_snapshot = client->client_id;
unsigned short width_snapshot = client->width;
unsigned short height_snapshot = client->height;
terminal_capabilities_t caps_snapshot = client->capabilities;
mutex_unlock(&client->client_state_mutex);
// Process using local copies (no locks held)
if (should_continue) {
generate_frame(client_id_snapshot, width_snapshot, height_snapshot,
&caps_snapshot);
}

Snapshot Benefits:

  • Minimal lock holding time
  • No blocking operations while holding locks
  • Prevents deadlocks from complex call chains
  • Enables parallel processing across clients

Thread Control Flags

Atomic Thread Control:

  • video_render_thread_running: Atomic boolean for thread control
  • audio_render_thread_running: Atomic boolean for thread control
  • active: Atomic boolean for client connection status
  • g_server_should_exit: Atomic boolean for server shutdown

Shutdown Coordination:

  • Threads check flags frequently in loops
  • Clean thread exit on flag change
  • No blocking operations that can't be interrupted
  • Responsive shutdown (threads exit within frame interval)

Media Buffer Access

Video Buffer Access:

  • Double-buffer system (atomic operations)
  • Thread-safe frame reading
  • Always get latest available frame
  • Frame dropping under load handled gracefully

Audio Buffer Access:

  • Lock-free ring buffer
  • Thread-safe audio sample reading
  • Automatic overflow handling
  • Low-latency access

Thread Lifecycle Management

Thread Creation

Creation Order:

  1. Video render thread created first (60fps requirement)
  2. Audio render thread created second (172fps requirement)

Creation Process:

// Create video render thread
atomic_store(&client->video_render_thread_running, true);
if (platform_thread_create(&client->video_render_thread,
video_render_thread_func,
(void *)(uintptr_t)client->client_id) != 0) {
return -1;
}
// Create audio render thread
atomic_store(&client->audio_render_thread_running, true);
if (platform_thread_create(&client->audio_render_thread,
audio_render_thread_func,
(void *)(uintptr_t)client->client_id) != 0) {
// Cleanup video thread on failure
atomic_store(&client->video_render_thread_running, false);
thread_join(&client->video_render_thread, NULL);
return -1;
}
return 0;
}
int create_client_render_threads(server_context_t *server_ctx, client_info_t *client)
Create and initialize per-client rendering threads.
platform_thread_t audio_render_thread
platform_thread_t video_render_thread
char client_id[MAX_CLIENT_ID_LEN]

Thread Termination

Termination Sequence:

  1. Set thread running flags to false
  2. Threads detect flag change and exit loops
  3. Join threads to ensure complete cleanup
  4. Clear thread handles

Termination Process:

void destroy_client_render_threads(client_info_t *client) {
// Signal threads to exit
atomic_store(&client->video_render_thread_running, false);
atomic_store(&client->audio_render_thread_running, false);
// Join threads with timeout
thread_join_timeout(&client->video_render_thread, NULL, 1000);
thread_join_timeout(&client->audio_render_thread, NULL, 1000);
// Clear thread handles
thread_init(&client->video_render_thread);
thread_init(&client->audio_render_thread);
}

Timeout Handling:

  • Thread join timeouts prevent blocking
  • Logged warnings for timeout cases
  • Graceful degradation on thread hang
  • Clean shutdown even if threads don't exit cleanly

Platform Abstraction Integration

Cross-Platform Timing

Windows Timing:

  • Uses Sleep() with millisecond precision
  • Requires timer resolution adjustment (timeBeginPeriod)
  • Minimal overhead for sleep operations

POSIX Timing:

  • Uses condition variables for responsive interruption
  • High-resolution timing with clock_gettime()
  • Interruptible sleep for shutdown responsiveness

Cross-Platform Threading

Windows Threading:

  • Uses platform abstraction layer
  • Handles Windows-specific thread initialization
  • Platform-safe thread join operations

POSIX Threading:

  • Uses pthreads via platform abstraction
  • Standard POSIX thread operations
  • Cross-platform thread lifecycle management

Integration with Other Modules

Integration with client.c

Called By:

Provides To:

  • Render thread management functions
  • Thread lifecycle coordination

Integration with stream.c

Called By:

  • Video render threads call generate_personalized_ascii_frame()
  • Frame generation at 60fps per client

Provides To:

  • Per-client frame generation requests
  • Terminal capability awareness

Integration with mixer.c

Called By:

  • Audio render threads call mixer_get_mixed_audio()
  • Audio mixing at 172fps per client

Provides To:

  • Per-client audio mixing requests
  • Echo cancellation (excludes client's own audio)

Performance Characteristics

Linear Scaling:

  • Each client gets dedicated render threads
  • No shared bottlenecks between clients
  • Performance scales linearly up to 9+ clients
  • Real-time guarantees maintained per client

CPU Usage:

  • Minimal CPU usage when ahead of schedule
  • Adaptive to load changes
  • No CPU spinning under light load
  • Efficient sleep operations

Timing Precision:

  • Exactly 60fps for video (16.67ms intervals)
  • Exactly 172fps for audio (5.8ms intervals)
  • Platform-specific high-resolution timers
  • Consistent timing across platforms

Best Practices

DO:

  • Always check shutdown flags in loops
  • Use snapshot pattern for client state access
  • Sleep when ahead of schedule to prevent CPU spinning
  • Use precise timing for rate limiting
  • Join threads with timeout to prevent blocking

DON'T:

  • Don't hold locks during frame generation
  • Don't skip shutdown flag checks
  • Don't use blocking operations without timeout
  • Don't ignore thread join failures
  • Don't modify thread control flags without synchronization
See also
src/server/render.c
src/server/render.h
Server Overview
Client Management
Stream Generation
topic_mixer
Concurrency Documentation

Statistics and Monitoring

Overview

The statistics and monitoring module provides comprehensive performance tracking and operational visibility for the ascii-chat server. It operates as a background thread that collects metrics from all system components every 30 seconds and generates detailed reports for troubleshooting, performance optimization, and operational monitoring. This module embodies the operational excellence philosophy of ascii-chat, providing the visibility needed to maintain high-performance multi-client video chat.

Implementation: src/server/stats.c, src/server/stats.h

Key Responsibilities:

  • Continuous monitoring of server performance metrics
  • Per-client statistics collection and reporting
  • buffer domain utilization tracking
  • Packet queue performance analysis
  • Hash table efficiency monitoring
  • Periodic statistics logging (every 30 seconds)
  • Operational visibility for administrators

Monitoring Architecture

Statistics Collection Thread

The statistics module runs as a dedicated background thread:

static void *stats_logger_thread_func(void *arg) {
(void)arg;
// Main monitoring loop
while (!atomic_load(&g_server_should_exit)) {
// Sleep for reporting interval (30 seconds)
for (int i = 0; i < 300 && !atomic_load(&g_server_should_exit); i++) {
platform_sleep_usec(100000); // 0.1 second intervals
}
if (atomic_load(&g_server_should_exit)) {
break;
}
// Collect statistics from all system components
collect_and_log_statistics();
}
return NULL;
}

Thread Characteristics:

  • Non-intrusive background monitoring
  • 30-second reporting intervals (configurable)
  • Interruptible sleep for responsive shutdown
  • Thread-safe data collection from all system components
  • Minimal impact on operational performance

Statistics Collection Methodology

Non-Intrusive Monitoring:

  • Uses reader locks to avoid blocking operational threads
  • Takes atomic snapshots of volatile data
  • Minimal impact on render thread performance
  • Safe concurrent access to client data

Statistics Atomicity:

  • Global statistics protected by dedicated mutex
  • Consistent reporting even during concurrent updates
  • Thread-safe access to shared counters
  • Snapshot pattern for volatile data

Performance Metrics Collected

Client Management Metrics

Client Statistics:

  • Total active clients
  • Clients with audio capabilities
  • Clients with video capabilities
  • Connection duration and activity patterns
  • Per-client connection metadata

Collection Process:

// Collect client statistics
rwlock_rdlock(&g_client_manager_rwlock);
int active_clients = 0;
int clients_with_audio = 0;
int clients_with_video = 0;
for (int i = 0; i < MAX_CLIENTS; i++) {
client_info_t *client = &g_client_manager.clients[i];
if (atomic_load(&client->active)) {
active_clients++;
if (client->audio_enabled) clients_with_audio++;
if (client->video_enabled) clients_with_video++;
}
}
rwlock_rdunlock(&g_client_manager_rwlock);
rwlock_t g_client_manager_rwlock
Reader-writer lock protecting the global client manager.
client_manager_t g_client_manager
Hash table entry for client packet dispatch.

buffer domain Performance

buffer domain Metrics:

  • Global buffer domain utilization
  • Allocation/deallocation rates
  • Peak usage patterns
  • Memory efficiency metrics
  • buffer domain fragmentation indicators

Collection Process:

log_info("Buffers: reserved=%zu live=%zu cached=%zu outstanding=%zu",
stats.reserved_bytes, stats.live_bytes, stats.cached_bytes, stats.outstanding);
void buffer_domain_stats(buffer_domain_t *domain, buffer_stats_t *out)
Definition buffer.c:378
buffer_domain_t * buffer_default_domain(void)
Definition buffer.c:419
size_t live_bytes
Definition buffer.h:46
size_t outstanding
Definition buffer.h:50
size_t reserved_bytes
Definition buffer.h:45
size_t cached_bytes
Definition buffer.h:48

Packet Queue Performance

Packet Queue Statistics:

  • Per-client queue depths
  • Enqueue/dequeue rates
  • Packet drop rates under load
  • Queue overflow incidents
  • Queue utilization patterns

Collection Process:

// Collect packet queue statistics
rwlock_rdlock(&g_client_manager_rwlock);
for (int i = 0; i < MAX_CLIENTS; i++) {
client_info_t *client = &g_client_manager.clients[i];
if (atomic_load(&client->active)) {
packet_queue_stats_t video_stats =
packet_queue_get_statistics(client->video_packet_queue);
packet_queue_stats_t audio_stats =
packet_queue_get_statistics(client->audio_packet_queue);
log_info("Client %u queues: video=%zu/%zu, audio=%zu/%zu",
client->client_id, video_stats.size, video_stats.capacity,
audio_stats.size, audio_stats.capacity);
}
}
rwlock_rdunlock(&g_client_manager_rwlock);

Hash Table Efficiency

Hash Table Metrics:

  • Client lookup performance
  • Hash collision rates
  • Load factor monitoring
  • Access pattern analysis
  • Table efficiency indicators

Collection Process:

// Collect hash table statistics
size_t client_count = HASH_COUNT(g_client_manager.clients_by_id);
log_info("Hash Table: %zu entries, %zu capacity, %.2f%% load factor, %zu collisions",
stats.entries, stats.capacity, stats.load_factor * 100.0, stats.collisions);

Frame Processing Metrics

Frame Processing Statistics:

  • Total frames captured from clients
  • Total frames sent to clients
  • Frame drop rate under load
  • Blank frame count (no video sources)
  • Frame generation performance

Collection Process:

// Collect frame processing statistics
frame_stats_t stats = get_frame_processing_statistics();
log_info("Frames: %zu captured, %zu sent, %zu dropped, %.2f%% drop rate",
stats.captured, stats.sent, stats.dropped,
(stats.dropped * 100.0) / stats.captured);

Statistics Reporting

Report Format

Statistics are logged in a structured format:

[STATS] Client Count: 3 active (2 audio, 3 video)
[STATS] buffer domain: 45/100 allocated (45.00% utilization), 52 peak
[STATS] Packet Queues: avg_depth=12.3, max_depth=25, drops=3
[STATS] Hash Table: 3 entries, 16 capacity (18.75% load), 0 collisions
[STATS] Frames: 5420 captured, 5340 sent, 80 dropped (1.48% drop rate)

Report Sections:

  1. Client Statistics: Active client count and capabilities
  2. buffer domain: Memory utilization and efficiency
  3. Packet Queues: Queue performance and overflow incidents
  4. Hash Table: Lookup performance and efficiency
  5. Frame Processing: Frame generation and delivery metrics

Reporting Interval

Default Interval: 30 seconds

  • Configurable via compile-time constants
  • Interruptible sleep for responsive shutdown
  • Adaptive to system load (may skip reports under heavy load)

Interval Tuning:

  • Longer intervals: Lower overhead, less granular visibility
  • Shorter intervals: Higher overhead, more granular visibility
  • Default 30 seconds: Balanced visibility and overhead

Integration with Other Modules

Client Management Metrics

Monitored From:

  • Client lifecycle statistics
  • Per-client connection metadata
  • Client activity patterns
  • Thread status and health

Provides To:

  • Client count and activity reports
  • Connection duration metrics
  • Client capability statistics

buffer domain Performance

Monitored From:

  • Global buffer domain utilization
  • Allocation/deallocation rates
  • Memory efficiency metrics

Provides To:

  • Memory usage reports
  • buffer domain performance analysis
  • Memory leak detection indicators

Packet Queue Performance

Monitored From:

  • Per-client packet queue depths
  • Enqueue/dequeue rates
  • Overflow incidents

Provides To:

  • Queue performance reports
  • Overflow detection
  • Network congestion indicators

Hash Table Efficiency

Monitored From:

  • Hash table efficiency
  • Lookup performance
  • Collision statistics

Provides To:

  • Hash table performance reports
  • Lookup efficiency metrics
  • Table optimization indicators

Operational Visibility

Troubleshooting Support

Performance Bottleneck Identification:

  • buffer domain utilization identifies memory pressure
  • Packet queue depths identify network congestion
  • Frame drop rates identify processing bottlenecks
  • Hash table collisions identify lookup inefficiency

System Health Monitoring:

  • Client count trends indicate connection stability
  • buffer domain peak usage identifies memory leaks
  • Packet queue overflow identifies network issues
  • Frame processing rates identify performance degradation

Debugging Support

Extensive Debug Logging:

  • Thread startup/shutdown tracking
  • Statistics collection reliability
  • Shutdown detection timing
  • Sleep/wake cycle behavior

Performance Profiling:

  • Statistics collection overhead tracking
  • Thread execution time monitoring
  • Resource utilization trends

Error Handling

Statistics Collection Errors:

  • Graceful degradation: Missing statistics logged but don't crash
  • Thread-safe error handling: Errors in one collection don't affect others
  • Detailed error logging: Troubleshooting information preserved

Thread Errors:

  • Thread join timeouts: Logged but don't block shutdown
  • Statistics collection failures: Logged but server continues
  • Resource cleanup errors: Handled gracefully

Performance Impact

Minimal Overhead:

  • Non-intrusive background monitoring
  • Reader locks only (no blocking writes)
  • Atomic snapshots (no long-held locks)
  • Minimal CPU usage (<1% typical)

Responsive Shutdown:

  • Interruptible sleep operations
  • Frequent shutdown flag checks
  • Clean thread exit on shutdown
  • No blocking operations

Best Practices

DO:

  • Use reader locks for statistics collection
  • Take atomic snapshots of volatile data
  • Check shutdown flags frequently
  • Log detailed statistics for troubleshooting
  • Handle errors gracefully

DON'T:

  • Don't use write locks for statistics collection
  • Don't hold locks during statistics collection
  • Don't block on statistics collection
  • Don't skip error handling
  • Don't ignore shutdown flags
See also
src/server/stats.c
src/server/stats.h
Server Overview
Server Main Entry Point
topic_buffer
topic_packet_queue
uthash library for hash table implementation

Stream Generation

Overview

The stream generation module creates personalized ASCII art frames for each connected client by collecting video from all active clients, creating composite layouts, and converting to ASCII using client-specific terminal capabilities. This module embodies the core innovation of ascii-chat: real-time multi-client video chat rendered entirely in ASCII art, optimized for terminal viewing with professional-grade performance.

Implementation: src/server/stream.c, src/server/stream.h

Key Responsibilities:

  • Collect video frames from all active clients
  • Create composite video layouts (single client, 2x2, 3x3 grids)
  • Generate client-specific ASCII art with terminal capability awareness
  • Process latest frames from double-buffer system for real-time performance
  • Manage memory efficiently with owned buffers and zero-copy operations
  • Support advanced rendering modes (half-block, color, custom palettes)

Video Mixing Architecture

Processing Pipeline

The mixing system operates in five distinct stages:

Stage 1: Frame Collection

// Scan all active clients for available video frames
for (int i = 0; i < MAX_CLIENTS; i++) {
client_info_t *source_client = &g_client_manager.clients[i];
// Skip inactive clients
if (!source_client->active) continue;
// Get latest frame from double-buffer system
video_frame_t *frame = get_latest_frame(source_client);
// Store frame for layout calculation
sources[source_count++] = (image_source_t){
.frame = frame,
.aspect_ratio = (float)frame->width / (float)frame->height,
.client_id = source_client->client_id
};
}
One client's video input as seen by the compositor.
When code needs to work with simple frame swap, it keeps current_frame, frame_a, frame_b,...
uint32_t height
Frame height in pixels.
uint32_t width
Frame width in pixels.

Aggressive Frame Dropping:

  • Always reads latest available frame (discards older frames)
  • Prevents buffer overflow under load
  • Maintains real-time performance like Zoom/Google Meet
  • Logarithmic drop rate based on buffer occupancy

Stage 2: Layout Calculation

// Determine optimal grid layout based on active client count
int rows, cols;
calculate_optimal_grid_layout(sources, source_count, terminal_width,
terminal_height, &cols, &rows);
// Calculate cell dimensions with aspect ratio preservation
int cell_width = terminal_width / cols;
int cell_height = terminal_height / rows;

Layout Algorithm:

  • Tries all grid configurations (1x1, 2x1, 2x2, 3x2, 3x3)
  • Calculates total area utilization for each configuration
  • Chooses configuration with highest area utilization
  • Preserves aspect ratios in grid cells

Stage 3: Composite Generation

// Create composite image with proper dimensions
image_t *composite = image_create(terminal_width, terminal_height);
// Place each client's video in appropriate grid cell
for (int i = 0; i < source_count; i++) {
int row = i / cols;
int col = i % cols;
int x = col * cell_width;
int y = row * cell_height;
// Scale and place frame in cell
image_scale_and_place(source->frame, composite, x, y,
cell_width, cell_height);
}
audio_visualization_source_t source
Definition fft.c:25

Composite Features:

  • Aspect ratio preservation in grid cells
  • High-quality scaling (bilinear interpolation)
  • Support for different grid sizes (1x1, 2x2, 3x3)
  • Efficient memory usage via owned buffers

Stage 4: ASCII Conversion

// Convert composite to ASCII using client's terminal capabilities
ascii_frame_t *ascii_frame = image_to_ascii(composite, &client->capabilities,
client->palette, client->color_mode);
// Apply client-specific rendering options
if (client->half_block_mode) {
ascii_frame = apply_half_block_rendering(ascii_frame);
}
if (client->color_mode) {
ascii_frame = apply_color_escape_sequences(ascii_frame,
client->color_depth);
}

ASCII Conversion Features:

  • Terminal capability awareness (color depth, UTF-8 support)
  • Custom ASCII palettes (brightness-to-character mapping)
  • Half-block rendering (2x vertical resolution)
  • ANSI escape sequence generation for color output
  • SIMD-optimized conversion for performance

Stage 5: Packet Generation

// Wrap ASCII frame in protocol packet
ascii_frame_packet_t *packet = create_ascii_frame_packet(ascii_frame);
// Queue packet for delivery via send thread
packet_queue_push_copy(client->video_packet_queue, PACKET_TYPE_ASCII_FRAME, packet, packet_size, 0);
The wire metadata for one rendered ASCII frame. width and height describe the terminal grid,...
Definition packet.h:916

Packet Features:

  • Protocol-compliant packet headers
  • Frame metadata (dimensions, timestamp, checksum)
  • Compression support for large frames
  • Thread-safe queue operations

Per-Client Customization

Unlike traditional video mixing that generates one output, ascii-chat creates personalized frames for each client:

Terminal Capability Awareness

Color Depth:

  • 1-bit (mono): Black and white ASCII only
  • 8-color: Basic ANSI colors
  • 16-color: Standard terminal colors
  • 256-color: Extended palette
  • 24-bit RGB: Full color support (when available)

Character Support:

  • ASCII-only: Basic character set (portable)
  • UTF-8: Box drawing characters for smoother frames
  • Unicode: Special characters for enhanced rendering

Render Modes:

  • Foreground color: Standard text coloring
  • Background color: Block-based rendering
  • Half-block: 2x vertical resolution using Unicode half-blocks

Custom Palettes:

  • Brightness-to-character mapping
  • Multiple predefined palettes (simple, extended, full)
  • Custom palette support for client preferences

Frame Customization

Each client receives frames optimized for their terminal:

// Generate personalized frame for client
ascii_frame_t *frame = generate_personalized_ascii_frame(client_id);
// Frame is customized based on:
// - Client's terminal dimensions (width, height)
// - Client's color depth (1-bit, 8-color, 256-color, RGB)
// - Client's palette preference (simple, extended, full)
// - Client's UTF-8 support (ASCII-only vs box drawing)
// - Client's render mode (foreground, background, half-block)

Customization Benefits:

  • Optimal quality for each client's terminal
  • Efficient use of available terminal capabilities
  • No wasted bandwidth on unsupported features
  • Consistent rendering across diverse terminals

Performance Optimizations

Frame Buffer Management

Double-Buffer System:

  • Each client uses double-buffer for smooth frame delivery
  • Always provides latest available frame
  • No frame caching complexity or stale data concerns
  • Professional-grade real-time performance
  • Memory managed via owned buffers

Buffer Overflow Handling: When clients send frames faster than processing:

  • Aggressive frame dropping (keep only latest)
  • Logarithmic drop rate based on buffer occupancy
  • Maintains real-time performance under load
  • Prevents buffer exhaustion

Memory Management

buffer domain Integration:

// Images use owned, aligned storage
image_t *composite = image_new(terminal_width, terminal_height);
// Explicit release after use
image_destroy(composite);
void image_destroy(image_t *image)
Destroy an image allocated with image_new().
image_t * image_new(size_t width, size_t height)
Create a new image with standard allocation.

Zero-Copy Operations:

  • Frame data is reference-counted (avoid copies)
  • Double-buffer system eliminates need for frame copies
  • Efficient memory usage via owned buffers
  • Automatic cleanup on client disconnect

Memory Efficiency:

  • Frame data is shared across clients where possible
  • buffer domain reduces malloc/free overhead
  • Automatic cleanup prevents memory leaks
  • Graceful degradation on allocation failures

Concurrency Optimizations

Reader Locks on Client Manager:

  • Allows parallel frame generation across clients
  • No blocking between client frame generation
  • Minimal lock contention

Lock-Free Media Buffers:

  • Audio buffers use lock-free ring buffers
  • Video buffers use atomic operations for double-buffer
  • Minimal synchronization overhead

Per-Client Isolation:

  • Each client's frame generation is independent
  • No shared state between frame generations
  • Linear scaling performance

Integration with Other Modules

Integration with render.c

Called By:

  • Video render threads call stream generation functions at 60fps
  • generate_personalized_ascii_frame(): Main entry point for frame generation

Provides To:

  • Personalized ASCII frames for each client
  • Optimized frame generation per terminal
  • Terminal capability awareness

Integration with client.c

Provides To:

  • Client state access for frame generation
  • Terminal capabilities from client state
  • Frame queue for delivery

Uses From:

  • Client video buffers (reads frames from all clients)
  • Client terminal capabilities (for ASCII conversion)
  • Client rendering preferences (palette, color mode)

Integration with protocol.c

Consumed By:

  • Video frames stored by protocol handlers
  • Frame data accessed by stream generation

Provides To:

  • Frame generation uses stored video data
  • Client capabilities set by protocol handlers

Integration with video/

Used By:

  • ASCII conversion implementation (video/)
  • SIMD-optimized conversion routines
  • Palette management (palette.c)

Provides To:

  • Composite images for ASCII conversion
  • Terminal capabilities for conversion optimization
  • Rendering preferences for palette selection

Grid Layout System

Supported Layout Types

Single Client (1x1):

  • Full-screen video when only one client connected
  • Maximum quality for single user
  • No scaling overhead

Side-by-Side (2x1):

  • Two clients side-by-side
  • Equal width split
  • Aspect ratio preserved in each half

Grid Layouts (2x2, 3x2, 3x3):

  • Multiple clients in grid arrangement
  • Optimal space utilization
  • Aspect ratio preserved in each cell

Layout Calculation Algorithm

Algorithm:

  1. Try all grid configurations from 1x1 to NxN
  2. For each configuration, calculate total area utilization
  3. Choose configuration with highest area utilization
  4. Calculate cell dimensions with aspect ratio preservation

Area Utilization:

  • Accounts for aspect ratio preservation
  • Prefers larger frames over smaller frames
  • Optimizes for terminal space usage
  • Handles different source aspect ratios

ASCII Conversion

Conversion Process

RGB to Grayscale:

  • Weighted conversion (red, green, blue weights)
  • Configurable color weights for optimization
  • SIMD-optimized for performance

Grayscale to ASCII:

  • Brightness-to-character mapping
  • Multiple palette options (simple, extended, full)
  • Terminal capability awareness

Color Application:

  • ANSI escape sequence generation
  • Terminal color depth support
  • Palette-based color selection

Rendering Modes

Foreground Color:

  • Standard text coloring
  • Efficient escape sequences
  • Compatible with most terminals

Background Color:

  • Block-based rendering
  • Higher quality appearance
  • Better color blending

Half-Block Rendering:

  • 2x vertical resolution
  • Uses Unicode half-block characters
  • Requires UTF-8 support
  • Professional-grade quality

Performance Optimizations

Linear Scaling:

  • Each client gets dedicated render thread
  • No shared bottlenecks between clients
  • Performance scales linearly up to 9+ clients
  • Real-time guarantees maintained per client

Frame Generation Speed:

  • 60fps per client with precise timing
  • SIMD-optimized ASCII conversion
  • Efficient memory usage via owned buffers
  • Minimal CPU overhead

Memory Usage:

  • Per-client composite buffers (temporary)
  • Frame data shared across clients
  • buffer domain reduces malloc/free overhead
  • Automatic cleanup on client disconnect

Best Practices

DO:

  • Always use latest available frame (aggressive frame dropping)
  • Use owned buffers for all allocations
  • Check client active status before operations
  • Use snapshot pattern for client state access
  • Optimize for terminal capabilities

DON'T:

  • Don't cache frames unnecessarily
  • Don't hold locks during frame generation
  • Don't skip frame dropping under load
  • Don't ignore terminal capabilities
  • Don't allocate without owned buffers
See also
src/server/stream.c
src/server/stream.h
Server Overview
Render Threads
topic_video
topic_palette