Application: Server

🖥️ ascii-chat server implementation and architecture More...

Modules
	Server Main Entry Point
	🚀 Server initialization, signal handling, and connection management
	Client Management
	👥 Per-client lifecycle management and threading coordination
	Protocol Handler
	📡 Network packet processing and protocol implementation
	Stream Generation
	🎬 Multi-client video mixing and ASCII frame generation
	Render Threads
	🎨 Per-client rendering threads with rate limiting
	Statistics and Monitoring
	📊 Performance monitoring and resource utilization tracking
	Cryptographic Operations
	🔐 Per-client cryptographic handshake and session encryption management

Detailed Description

🖥️ ascii-chat server implementation and architecture

Multi-client real-time video and audio streaming server with per-client threading, end-to-end encryption, and terminal-aware ASCII rendering.

Overview of Server (src/server/*)

Overview

Welcome to the heart of ascii-chat—the server! This is where all the magic happens.

Picture a video conference with multiple people. Someone needs to coordinate everything: collecting video from each person, mixing the audio streams together, generating those cool ASCII frames, and sending everything back out to everyone. That's exactly what the server does, and it does it all in real-time at 60 frames per second!

The server is designed like a well-oiled machine. Each client gets their own dedicated threads (think of them as personal assistants)—one for receiving data, one for sending, one for video rendering, and one for audio mixing. This means when Client A's connection gets slow, it doesn't affect Client B at all. They're independent! This architecture scales beautifully—we've tested with 9+ clients without breaking a sweat.

And security? Every single packet is encrypted end-to-end using modern cryptography (X25519, XSalsa20-Poly1305—the good stuff). Your video chat stays private.

What makes this server special?

• 4 threads per client: Receive, send, video render, audio render (complete independence)
• Real-time performance: 60fps video per client, 172fps audio mixing (smooth as butter)
• Linear scalability: Add more clients, performance scales proportionally (no bottlenecks!)
• End-to-end encryption: Full cryptographic handshake—your data stays yours
• Terminal-aware: Generates ASCII optimized for each client's terminal capabilities
• Graceful shutdown: Clean resource cleanup even when things go sideways

Implementation: src/server/*.c, src/server/*.h

Architecture

Threading Model

Let's talk about how the server juggles multiple clients simultaneously—it's actually pretty clever!

Think of the server like a restaurant. You've got a host (main thread) greeting customers and showing them to tables. Then each table gets its own team of waiters (4 threads per client) who handle everything for that specific table. One waiter takes orders (receive thread), another delivers food (send thread), one prepares the visual presentation (video render), and another handles drinks (audio render). This way, if one table's order is taking forever, it doesn't slow down the other tables—everyone gets independent service!

The threading setup:

Global Threads (these run regardless of client count):

• Main Thread: The host—greets new clients, handles connections, coordinates everything
• Stats Logger Thread: The accountant—checks performance every 30 seconds, logs metrics

Per-Client Threads (4 dedicated threads for each connected client):

1. Receive Thread: Listens for incoming packets from this client (what are they sending?)
2. Send Thread: Delivers outgoing packets to this client (here's your ASCII frame!)
3. Video Render Thread: Generates personalized ASCII frames at 60fps (smooth video)
4. Audio Render Thread: Mixes audio from everyone at 172fps (clear sound)

Why this design rocks:

• Linear scalability: Add clients, performance scales proportionally (no shared chokepoints)
• Fault isolation: Client A's slow connection doesn't affect Client B at all
• Simple synchronization: Each client owns their stuff, minimal lock fighting
• Real-time guarantees: Everyone gets their own CPU time slice—fair and predictable

For the nitty-gritty details on locks, synchronization, and threading internals, check out Concurrency Architecture Documentation.

Server Modules

The server is organized into modular components:

Main Entry Point

File: src/server/main.c, src/server/main.h

Server initialization, signal handling, connection management, and overall orchestration. Handles:

• Socket binding and listening (IPv4 and IPv6)
• Client connection acceptance
• Signal handler setup (SIGINT/SIGTERM)
• Global shutdown coordination
• Resource cleanup

Client Management

File: src/server/client.c, src/server/client.h

Per-client lifecycle management, threading coordination, and state management. Handles:

• Client connection establishment (add_client())
• Client disconnection handling (remove_client())
• Thread creation and management per client
• Client state synchronization
• Hash table management for O(1) client lookups

Protocol Handler

File: src/server/protocol.c, src/server/protocol.h

Network packet processing and protocol implementation. Handles:

• Packet parsing and validation
• Client state updates based on received packets
• Media data storage (video frames, audio samples)
• Protocol compliance checking
• Packet type dispatch

Stream Generation

File: src/server/stream.c, src/server/stream.h

Multi-client video mixing, ASCII frame generation, and personalized rendering. Handles:

• Video frame collection from all active clients
• Composite frame generation (single, 2x2, 3x3 layouts)
• ASCII conversion with terminal capability awareness
• Per-client personalized frame generation
• Buffer pool integration for efficient memory management

Render Threads

File: src/server/render.c, src/server/render.h

Per-client rendering threads with rate limiting and timing control. Handles:

• Video render thread management (60fps per client)
• Audio render thread management (172fps per client)
• Frame generation timing and rate limiting
• Thread lifecycle management
• Platform-specific timing integration

Statistics and Monitoring

File: src/server/stats.c, src/server/stats.h

Performance monitoring, resource utilization tracking, and health reporting. Handles:

• Continuous performance metric collection
• Per-client statistics reporting
• Buffer pool utilization tracking
• Packet queue performance analysis
• Hash table efficiency monitoring
• Periodic statistics logging (every 30 seconds)

Cryptographic Operations

File: src/server/crypto.c, src/server/crypto.h

Per-client cryptographic handshake, key exchange, and session encryption management. Handles:

• Cryptographic handshake with each client
• X25519 key exchange per session
• Session encryption key derivation
• Client authentication (password, SSH key, passwordless)
• Client whitelist integration
• Per-client crypto context management

For complete details on cryptography, see Cryptography Module. For handshake protocol details, see Handshake Protocol.

Concurrency Architecture

Synchronization Primitives

The server uses a carefully designed synchronization architecture to ensure thread safety while maintaining high performance. See Concurrency Documentation for complete details on:

• Global RWLock (g_client_manager_rwlock): Protects global client array
• Per-Client Mutexes: Fine-grained locking for client state
• Atomic Variables: Lock-free thread control flags
• Condition Variables: Thread wakeup during shutdown

Lock Ordering Protocol

Okay, this is super important—like "your program will freeze if you mess this up" important.

Deadlocks are the nightmare of multi-threaded programming. Imagine two threads: Thread A holds Lock 1 and wants Lock 2, while Thread B holds Lock 2 and wants Lock 1. Both threads wait forever. Boom, deadlock. The program freezes. Not fun.

The solution? Always acquire locks in the same order. It's like a traffic rule—if everyone follows it, there are no collisions.

CRITICAL RULE (memorize this!):

Always acquire locks in this exact order:

1. Global RWLock (g_client_manager_rwlock) — The big one, protects the client list
2. Per-Client Mutex (client_state_mutex) — Protects individual client state
3. Specialized Mutexes (video_buffer_mutex, g_stats_mutex, etc.) — Specific subsystems

Never acquire them in a different order. Ever. Seriously. Violating this causes deadlocks, and debugging deadlocks is like finding a needle in a haystack while wearing mittens.

For detailed examples, war stories, and the rationale behind this ordering, check out Concurrency Documentation.

Snapshot Pattern

Here's a clever technique we use throughout the server to keep things fast: the snapshot pattern.

The idea is simple: hold locks for as little time as possible. Instead of keeping a lock while doing expensive work (like rendering a frame), we quickly grab the lock, copy the data we need into local variables (a "snapshot"), release the lock, and then do the work using the snapshot.

It's like taking a photo instead of staring at the original—you get what you need, then other people can look at the original while you work with your copy.

Here's what it looks like in practice:

// 1. Acquire mutex
mutex_lock(&client->client_state_mutex);
 
// 2. Quickly copy what we need to local variables
bool should_continue = client->video_render_thread_running && client->active;
uint32_t client_id_snapshot = client->client_id;
unsigned short width_snapshot = client->width;
unsigned short height_snapshot = client->height;
 
// 3. Release the lock immediately (this is the key!)
mutex_unlock(&client->client_state_mutex);
 
// 4. Now do the expensive work with our snapshot (no locks held)
if (should_continue) {
  generate_frame(client_id_snapshot, width_snapshot, height_snapshot);
  // This might take milliseconds, but we're not blocking anyone!
}

This pattern is why the server can maintain 60fps video rendering—we minimize lock contention, so threads don't spend time waiting for each other.

Cryptographic Operations

The server supports end-to-end encryption with multiple authentication modes. See Cryptography Module for algorithm details and Handshake Protocol for connection establishment.

Cryptographic Handshake

Each client connection performs a cryptographic handshake before media streaming begins:

Phase 0: Algorithm Negotiation

• Client advertises supported algorithms
• Server selects compatible algorithms
• Both agree on key exchange, cipher, and authentication methods

Phase 1: Key Exchange

• Server generates ephemeral X25519 keypair
• Client generates ephemeral X25519 keypair
• Both exchange public keys and compute shared secret via ECDH

Phase 2: Authentication

• Server verifies client identity (if whitelist enabled)
• Server signs challenge with Ed25519 identity key (if available)
• Client proves identity with password or SSH key

Phase 3: Session Establishment

• Both derive session encryption keys from shared secret
• All subsequent packets are encrypted with XSalsa20-Poly1305
• Perfect forward secrecy (ephemeral keys per session)

Keys Module

The server supports multiple key management approaches:

SSH Key Authentication:

• Server loads Ed25519 private key (via --server-key)
• Server verifies client public keys against whitelist (via --client-keys)
• Keys must be Ed25519 format (modern, secure, fast)
• Native Decryption: Encrypted keys are decrypted using bcrypt_pbkdf (libsodium-bcrypt-pbkdf)
  • BearSSL AES-256-CTR. No external tools required.

Password Authentication:

• Both server and client derive same key from shared password
• Uses Argon2id key derivation for password hashing
• No identity keys required (password-only mode)

Passwordless Mode:

• Ephemeral keys only (no long-term identity)
• Key exchange provides confidentiality but not authentication
• Suitable for trusted networks or testing

For complete key management details, see Keys Module.

Statistics Logger Thread

The stats logger thread (stats_logger_thread() in stats.c) provides continuous performance monitoring and resource utilization tracking.

Statistics Functionality

Monitoring Frequency:

• Statistics collected every 30 seconds
• Thread checks shutdown flag every 10ms for responsive shutdown
• Background processing doesn't affect real-time performance

Metrics Collected:

• Active client count
• Clients with audio capabilities
• Clients with video capabilities
• Per-client packet queue statistics (enqueued, dequeued, dropped)
• Per-client video buffer statistics (total frames, dropped frames, drop rate)
• Buffer pool utilization (global allocation/deallocation rates)
• Lock debugging statistics (mutex/RWLock acquisitions, releases, currently held)
• Hash table efficiency metrics

Statistics Output: The stats logger generates comprehensive performance reports with per-client details:

Stats: Clients: 3, Audio: 2, Video: 3
  Client 1 audio queue: 1024 enqueued, 1024 dequeued, 0 dropped
  Client 1 video buffer: 1800 frames, 12 dropped (0.7% drop rate)
  Client 2 audio queue: 512 enqueued, 512 dequeued, 0 dropped
  ...

Thread Safety:

• Uses reader locks on shared data structures
• Takes atomic snapshots of volatile counters
• Minimal impact on operational performance
• Safe concurrent access with render threads

Debug Output

The stats logger thread includes extensive debug instrumentation for troubleshooting:

• Thread startup/shutdown logging
• Loop iteration counters
• Shutdown flag state changes
• Sleep cycle progression
• Statistics collection timing

Debug output is enabled via logging system configuration. See Logging System for details.

Error Handling

The server implements comprehensive error handling throughout all modules. See Error Number System for complete error handling details.

Error Handling Patterns

Library Code (server modules):

• Use SET_ERRNO() macros for error reporting
• Provide meaningful context messages
• Capture system errors with SET_ERRNO_SYS()
• Return appropriate error codes

Application Code (main.c):

• Check for library errors with HAS_ERRNO()
• Use FATAL() macros for fatal errors
• Use FATAL_AUTO_CONTEXT() for automatic context detection
• Log errors with full context before shutdown

Fatal Errors

The server uses FATAL() macros to terminate the server on critical errors:

• Network errors: Socket creation, binding, listening failures
• Cryptographic errors: Key loading failures, handshake failures
• Memory errors: Critical allocation failures
• Configuration errors: Invalid command-line options, missing dependencies

All fatal errors include full context including file, line, function, and error message. See Exit Codes for complete exit code reference.

Non-Fatal Errors

Many errors are handled gracefully without server termination:

• Client connection errors: Individual client failures don't crash server
• Packet parsing errors: Invalid packets logged but client not disconnected
• Buffer allocation errors: Frame dropping instead of crash
• Network timeouts: Connection retry logic instead of termination

Shutdown and Exit

The server implements graceful shutdown with proper resource cleanup.

Shutdown Sequence

1. Shutdown Signal (SIGINT/SIGTERM handler):

• Sets atomic g_server_should_exit flag (signal-safe)
• Closes listening sockets to interrupt accept() calls
• Returns immediately (no complex cleanup in signal handler)

2. Main Thread Detection:

• Main loop detects g_server_should_exit flag
• Stops accepting new connections
• Initiates client cleanup sequence

3. Client Cleanup (per client, in remove_client()):

• Sets thread shutdown flags (atomic operations)
• Shuts down packet queues (wakes up blocked threads)
• Joins threads in order: send → receive → video render → audio render
• Cleans up resources: queues, buffers, mutexes
• Closes client sockets

4. Statistics Thread Cleanup:

• Statistics thread detects shutdown flag
• Exits monitoring loop gracefully
• Logs final statistics report
• Thread joined by main thread

5. Resource Cleanup:

• Closes remaining sockets
• Destroys synchronization primitives
• Frees global buffers
• Logs final shutdown message

Exit Codes

The server exits with specific codes to indicate status:

• 0: Normal shutdown (graceful termination)
• 1: Fatal error or forced termination (double Ctrl+C)
• Error codes: See Exit Codes for complete reference

Exit codes are set via FATAL() macros or explicit exit() calls.

Known Quirks and Limitations

Every system has its quirks, and ascii-chat is no exception. Here are some things you should know about—not bugs, just... characteristics!

SSH Keygen Dependency

Native Encrypted Key Support: The server supports native decryption of encrypted Ed25519 private keys without requiring any external tools.

How it works: When you load an encrypted Ed25519 private key (via --server-key), ascii-chat decrypts it using the same algorithms as OpenSSH:

• bcrypt_pbkdf: Key derivation function (via libsodium-bcrypt-pbkdf library)
• AES-256-CTR: Symmetric encryption (via BearSSL)
• OpenSSH format: Full support for openssh-key-v1 format

Supported encryption:

• Cipher: aes256-ctr, aes256-cbc (OpenSSH defaults for Ed25519 keys)
• KDF: bcrypt (bcrypt_pbkdf with configurable rounds)
• Key types: Ed25519 only (RSA/ECDSA not supported)

Password input methods:

• Interactive prompt (default, uses platform_get_password())
• Environment variable: $ASCII_CHAT_KEY_PASSWORD (for automation)
• SSH/GPG agent: $SSH_AUTH_SOCK for password-free operation

Implementation: See lib/crypto/keys/ssh_keys.c:59-134 for the native decryption code and cmake/LibsodiumBcryptPbkdf.cmake for the bcrypt_pbkdf library integration.

Stats Logger Thread Behavior

The stats logger thread (stats_logger_thread() in stats.c) has specific behavior characteristics:

30-Second Intervals:

• Statistics are collected and logged every 30 seconds
• Thread sleeps in 10ms increments to maintain shutdown responsiveness
• Shutdown flag is checked frequently (every 10ms) during sleep periods

Per-Client Details:

• Only logs per-client details if clients have active queues or buffers
• Filters out empty/zero statistics to reduce log spam
• Format: Client N audio queue: X enqueued, Y dequeued, Z dropped

Lock Debug Integration:

• Integrates with lock debugging system for mutex/RWLock statistics
• Reports total locks acquired/released and currently held count
• Helps identify lock contention issues

Final Statistics Report:

• Logs final server statistics on thread exit
• Prints error statistics summary
• Helps diagnose issues during shutdown

Double Ctrl+C Behavior

Historical Issue: Previously required double Ctrl+C to shutdown (fixed).

Root Cause: Signal handler was accessing shared client data structures without proper synchronization, causing race conditions and incomplete shutdown.

Current Behavior (after fix):

• Single Ctrl+C properly shuts down server
• Signal handler is signal-safe (only sets flags, no data structure access)
• Main thread handles all cleanup with proper synchronization

See Concurrency Documentation Bug #1 for complete fix details.

Client Limits

Current Limitation: Server supports up to MAX_CLIENTS concurrent connections (defined in client.h). Default is typically 64 clients.

Why: Client array is statically allocated for performance (no dynamic allocation).

• Faster client lookups (O(1) array access)
• No allocation overhead per connection
• Predictable memory usage

Scaling: Server scales linearly up to 9+ clients in testing. Actual capacity depends on:

• CPU cores available
• Memory bandwidth
• Network bandwidth
• Client frame rates and resolutions

Frame Dropping Under Load

Here's the thing about real-time video: When the system gets overloaded, you have two choices—buffer everything (causing latency to skyrocket) or drop frames (keeping latency low but reducing smoothness). We chose the latter.

Why drop frames instead of buffering? Think about a live conversation. Would you rather:

• See smooth video but with a 5-second delay (buffering approach), or
• See slightly choppy video but respond in real-time (frame dropping approach)?

For a chat application, real-time responsiveness beats smoothness. So when the server gets overwhelmed (lots of clients, slow CPU, whatever), it:

• Always uses the latest available frame (freshest data)
• Drops older frames logarithmically based on buffer occupancy (smart dropping)
• Maintains target frame rate as much as possible

What this means for you: Under heavy load, clients might see 30fps instead of 60fps, but the video will always feel responsive. No weird lag where someone asks a question and you respond 5 seconds later!

ACDS Integration (Discovery Service)

The server can register with the ascii-chat Discovery Service (ACDS) to enable easy connection via human-friendly session strings instead of IP addresses.

Session Registration

When started with --acds, the server:

1. Connects to ACDS server (default: 127.0.0.1:27225)
2. Sends ACIP_SESSION_CREATE with server identity, capabilities, and participant limits
3. Receives session_string (e.g., "happy-sunset-ocean") and STUN/TURN server list
4. Displays session_string to user for sharing
5. Maintains keepalive with ACDS for session validity

Usage:

# Register with default ACDS server
ascii-chat server --acds
 
# Register with custom ACDS server
ascii-chat server --acds --acds-server acds.example.com --acds-port 27225
 
# Register with explicit public IP exposure (for NAT traversal)
ascii-chat server --acds --acds-expose-ip

NAT Traversal

The server supports multiple NAT traversal techniques for remote connectivity:

UPnP/NAT-PMP (--upnp):

• Automatically configures port forwarding on home routers
• Works on ~70% of home networks
• Enables direct TCP connections (lowest latency)

WebRTC Fallback:

• When direct TCP fails, clients can connect via WebRTC DataChannels
• ACDS provides STUN/TURN servers for ICE negotiation
• TURN relay ensures connectivity even behind restrictive NATs

Privacy Considerations

By default, the server's public IP is NOT revealed to clients until:

• They provide the correct session password (if password-protected), OR
• The server explicitly opts in with --acds-expose-ip

This prevents IP harvesting attacks while allowing legitimate users to connect.

See also

ACDS Overview for complete discovery service documentation

ACDS Server for server-side ACDS implementation

Integration with Library Modules

The server integrates with many library modules:

• Network I/O (Network Module): Socket operations, packet protocol
• Cryptography (Crypto Module): Handshake, encryption, key management
• Video Processing (Video to ASCII): RGB to ASCII conversion
• Audio Processing (Audio Module): Audio mixing and playback
• Buffer Management (Buffer Pool): Efficient memory allocation
• Packet Queues (Packet Queue): Thread-safe packet delivery
• Logging (Logging System): Structured logging with levels
• Error Handling (Error System): Typed error codes and context
• Platform Abstraction (Platform Layer): Cross-platform threading/sockets
• Discovery Service (ACDS): Session discovery and NAT traversal

Performance Characteristics

Linear Scaling:

• Performance scales linearly with number of clients
• No shared bottlenecks between clients
• Each client gets dedicated CPU resources
• Real-time guarantees maintained per client

Frame Rates:

• Video: 60fps per client (16.67ms intervals)
• Audio: 172fps per client (5.8ms intervals)
• Frame rate maintained under normal load
• Frame dropping prevents latency accumulation under heavy load

Memory Usage:

• Per-client memory: ~1-2 MB per client (buffers, queues, state)
• Buffer pool: Shared pool reduces allocation overhead
• Scales linearly with number of clients

CPU Usage:

• Video rendering: ~10-20% CPU per client (depends on resolution)
• Audio mixing: ~1-2% CPU per client
• Network I/O: Minimal CPU (kernel handles most work)
• Statistics: Negligible (<1% CPU)

Best Practices

Here are some hard-won lessons from building and debugging this server. Follow these, and you'll save yourself a lot of headaches!

Thread Safety (avoiding the deadlock nightmare):

• Always follow lock ordering: Global → per-client → specialized (no exceptions!)
• Use snapshot pattern: Copy data while holding locks, process without locks
• Minimize lock time: Hold locks for as little time as possible
• Atomic flags: Use atomics for simple booleans (no lock needed)

Why? Deadlocks are terrible to debug. Following these rules prevents them entirely.

Error Handling (making bugs easy to find):

• Check all returns: Network operations can fail—always check the return value
• Use SET_ERRNO(): In library code, use this for automatic context capture
• Use FATAL(): In main code, fatal errors should be obvious and informative
• Meaningful messages: "Failed to bind" is better than "Error -1"

Why? Good error messages save hours of debugging. You'll thank yourself later.

Performance (keeping it fast):

• Use buffer pools: Hot paths should never call malloc() directly
• Avoid lock fighting: If threads are waiting for locks, rethink your design
• Lock-free when possible: Ring buffers, atomics—they're your friends
• Profile contention: Use the lock debugging system to find bottlenecks

Why? Real-time 60fps doesn't happen by accident. Every microsecond counts.

Shutdown (cleaning up gracefully):

• Flags first: Set shutdown flags before joining threads (they need to know to stop!)
• Join in order: send → receive → render (order matters for cleanup)
• Check frequently: Loops should check shutdown flags often (responsive exit)
• Interruptible sleep: Use sleep that can be interrupted by shutdown signals

Why? Nothing's worse than a program that won't exit. Clean shutdown is a feature.

See also

src/server/main.c For server entry point and initialization

src/server/client.c For client lifecycle management

src/server/protocol.c For Packet Types processing

src/server/stream.c For Video to ASCII Conversion mixing and ASCII generation

src/server/render.c For rendering threads

src/server/stats.c For performance monitoring

src/server/crypto.c For cryptographic operations

CONCURRENCY.md For complete concurrency architecture

topic_crypto For cryptography details

topic_handshake For Handshake Module protocol details

topic_keys For key management details

topic_network For network I/O details

topic_acds For ACDS discovery service integration

Client Management

Overview

The client management module orchestrates the complete lifecycle of each connected client, from initial connection through thread spawning, state synchronization, and graceful disconnection cleanup. It serves as the integration hub between the main server loop and all per-client subsystems, managing the sophisticated per-client threading architecture that enables real-time video chat at 60fps per client.

Implementation: src/server/client.c, src/server/client.h

Key Responsibilities:

• Client connection establishment and initialization
• Per-client thread creation and management (4 threads per client)
• Client state management with thread-safe access patterns
• Hash table management for O(1) client ID lookups
• Client disconnection handling and resource cleanup
• Integration point between main.c and other server modules

Client Lifecycle

Connection Establishment

When a new client connects:

int add_client(socket_t socket, const char *client_ip, int client_port)
{
  // 1. Find available slot in client array
  rwlock_wrlock(&g_client_manager_rwlock);
  int slot = find_available_client_slot();
 
  // 2. Generate unique client ID (based on local port)
  uint32_t client_id = generate_client_id(client_port);
 
  // 3. Initialize client data structures
  client_info_t *client = &g_client_manager.clients[slot];
  mutex_init(&client->client_state_mutex);
  atomic_store(&client->client_id, client_id);
  atomic_store(&client->active, true);
  client->socket = socket;
 
  // 4. Add to uthash table for O(1) lookups
  HASH_ADD_INT(g_client_manager.clients_by_id, client_id, client);
 
  // 5. Perform cryptographic handshake
  crypto_handshake_result_t handshake_result =
    perform_crypto_handshake_server(socket, &client->crypto_ctx);
 
  if (handshake_result != HANDSHAKE_SUCCESS) {
    // Handshake failed - cleanup and return error
    remove_client(client_id);
    return -1;
  }
 
  // 6. Create per-client threads (receive, send, render)
  create_client_receive_thread(client);
  create_client_send_thread(client);
  create_client_render_threads(client);  // Video + Audio render threads
 
  rwlock_wrunlock(&g_client_manager_rwlock);
 
  return client_id;
}

Thread Creation Order:

1. Send Thread: Created first (cleanest exit, no blocking I/O typically)
2. Receive Thread: Created second (handles incoming packets, may block on network)
3. Video Render Thread: Created third (60fps frame generation)
4. Audio Render Thread: Created fourth (172fps audio mixing)

Disconnection Handling

Client disconnection is detected and handled gracefully:

int remove_client(uint32_t client_id)
{
  // 1. Find client in uthash table
  rwlock_wrlock(&g_client_manager_rwlock);
  client_info_t *client = NULL;
  HASH_FIND_INT(g_client_manager.clients_by_id, &client_id, client);
 
  // 2. Mark client as inactive (causes threads to exit)
  atomic_store(&client->active, false);
  atomic_store(&client->video_render_thread_running, false);
  atomic_store(&client->audio_render_thread_running, false);
 
  // 3. Join threads in reverse creation order
  asciichat_thread_join(&client->send_thread, NULL);
  asciichat_thread_join(&client->receive_thread, NULL);
  asciichat_thread_join(&client->video_render_thread, NULL);
  asciichat_thread_join(&client->audio_render_thread, NULL);
 
  // 4. Cleanup resources
  crypto_handshake_cleanup(&client->crypto_ctx);
  packet_queue_destroy(client->video_packet_queue);
  packet_queue_destroy(client->audio_packet_queue);
  image_destroy(client->incoming_video_buffer);
 
  // 5. Remove from uthash table
  HASH_DELETE(hh, g_client_manager.clients_by_id, client);
 
  // 6. Clear client slot
  memset(client, 0, sizeof(client_info_t));
  g_client_manager.client_count--;
 
  rwlock_wrunlock(&g_client_manager_rwlock);
 
  return 0;
}

Cleanup Order:

1. Mark inactive (threads exit loops)
2. Join send thread (quickest, usually not blocked)
3. Join receive thread (may be blocked on recv(), will timeout)
4. Join render threads (computational work, clean exit)
5. Cleanup resources (crypto, queues, buffers)
6. Remove from hash table
7. Clear slot for reuse

Per-Client Threading Architecture

Each connected client spawns exactly 4 dedicated threads:

Receive Thread

Purpose: Handle incoming packets from client Module: protocol.c functions called by receive thread Operations:

• Receives encrypted packets from socket
• Validates packet headers and CRC32 checksums
• Decrypts packets using per-client crypto context
• Dispatches to protocol handler functions based on packet type
• Stores media data (video frames, audio samples) in client buffers
• Updates client state (capabilities, dimensions, stream status)

Thread Safety:

• Uses per-client mutex for state updates
• Snapshot pattern for minimal lock holding time
• Thread-safe media buffers (double-buffer system)

Send Thread

Purpose: Manage outgoing packet delivery to client Module: client.c (send thread implementation) Operations:

• Reads packets from client's video and audio packet queues
• Encrypts packets using per-client crypto context
• Sends encrypted packets to client socket
• Handles chunked transmission for large packets
• Detects send failures and marks connection lost

Thread Safety:

• Packet queues are internally thread-safe
• Socket access protected by connection state checks
• Connection loss detected atomically

Video Render Thread

Purpose: Generate personalized ASCII frames at 60fps Module: render.c (video render thread implementation) Operations:

• Collects video frames from all active clients (via stream.c)
• Creates composite frame with appropriate grid layout
• Converts composite to ASCII using client's terminal capabilities
• Queues ASCII frame in client's video packet queue
• Rate limits to exactly 60fps with precise timing

Performance:

• 16.67ms intervals (60fps)
• Per-client frame generation (each client gets personalized view)
• Terminal capability awareness (color depth, palette, UTF-8 support)
• Linear scaling (no shared bottlenecks)

Audio Render Thread

Purpose: Mix audio streams at 172fps (excluding client's own audio) Module: render.c (audio render thread implementation) Operations:

• Reads audio samples from all clients via audio mixer
• Excludes client's own audio to prevent echo
• Mixes remaining audio streams with ducking and compression
• Queues mixed audio in client's audio packet queue
• Rate limits to exactly 172fps (5.8ms intervals)

Performance:

• 5.8ms intervals (172fps)
• Low-latency audio delivery
• Professional audio processing (ducking, compression, noise gate)
• Linear scaling (per-client dedicated threads)

Data Structures

Client Manager

Global singleton managing all clients:

typedef struct {
  client_info_t clients[MAX_CLIENTS];      // Backing storage array
  client_info_t *clients_by_id;            // uthash head pointer for O(1) client_id lookup
  int client_count;                        // Current number of active clients
  mutex_t mutex;                           // Protects client_count updates
} client_manager_t;
 
extern client_manager_t g_client_manager;
extern rwlock_t g_client_manager_rwlock;   // Protects entire manager

Synchronization:

• Reader-writer lock allows concurrent reads
• Exclusive write access for client add/remove
• Hash table provides O(1) client lookups
• Array provides O(1) slot access by index

Client Information

Per-client state structure:

typedef struct {
  // Connection state
  socket_t socket;
  _Atomic uint32_t client_id;
  _Atomic bool active;
 
  // Threading
  asciichat_thread_t receive_thread;
  asciichat_thread_t send_thread;
  asciichat_thread_t video_render_thread;
  asciichat_thread_t audio_render_thread;
  _Atomic bool video_render_thread_running;
  _Atomic bool audio_render_thread_running;
 
  // Cryptography
  crypto_handshake_ctx_t crypto_ctx;
  _Atomic bool crypto_initialized;
 
  // Media buffers
  video_frame_t *incoming_video_buffer;
  ringbuffer_t *incoming_audio_buffer;
 
  // Packet queues
  packet_queue_t *video_packet_queue;
  packet_queue_t *audio_packet_queue;
 
  // Terminal capabilities
  unsigned short width;
  unsigned short height;
  terminal_capabilities_t capabilities;
 
  // Synchronization
  mutex_t client_state_mutex;
  mutex_t video_buffer_mutex;
} client_info_t;

State Protection:

• client_state_mutex: Protects most client state fields
• video_buffer_mutex: Protects video frame buffer
• Atomic variables: Thread control flags (no mutex needed)
• Atomic snapshot pattern: Copy state under mutex, process without locks

Synchronization Patterns

Lock Ordering Protocol

CRITICAL RULE: Always acquire locks in this order:

1. g_client_manager_rwlock (global reader-writer lock)
2. client->client_state_mutex (per-client mutex)
3. Specialized mutexes (video_buffer_mutex, etc.)

This ordering prevents deadlocks across all server code.

Example:

// CORRECT: Global lock first, then per-client mutex
rwlock_rdlock(&g_client_manager_rwlock);
client_info_t *client = NULL;
HASH_FIND_INT(g_client_manager.clients_by_id, &client_id, client);
mutex_lock(&client->client_state_mutex);
// ... access client state ...
mutex_unlock(&client->client_state_mutex);
rwlock_rdunlock(&g_client_manager_rwlock);

Snapshot Pattern

All client state access uses the snapshot pattern to minimize lock contention:

// 1. Acquire mutex
mutex_lock(&client->client_state_mutex);
 
// 2. Copy needed state to local variables
bool should_continue = client->video_render_thread_running && client->active;
uint32_t client_id_snapshot = client->client_id;
unsigned short width_snapshot = client->width;
unsigned short height_snapshot = client->height;
 
// 3. Release mutex immediately
mutex_unlock(&client->client_state_mutex);
 
// 4. Process using local copies (no locks held)
if (should_continue) {
  generate_frame(client_id_snapshot, width_snapshot, height_snapshot);
}

Benefits:

• Minimal lock holding time (reduces contention)
• No blocking operations while holding locks
• Prevents deadlocks from complex call chains
• Enables parallel processing across clients

Hash Table Integration

The client manager uses a hash table for O(1) client ID lookups:

// Add client to uthash table
HASH_ADD_INT(g_client_manager.clients_by_id, client_id, client);
 
// Lookup client by ID
client_info_t *client = NULL;
HASH_FIND_INT(g_client_manager.clients_by_id, &client_id, client);
 
// Remove client from uthash table
HASH_DELETE(hh, g_client_manager.clients_by_id, client);

Performance:

• O(1) average-case lookup time
• Handles hash collisions internally
• Thread-safe operations (protected by RWLock)
• Scales to 9+ clients efficiently

Client ID Generation:

• Based on client's local port number
• Ensures uniqueness across connections
• Provides stable identifier for session

Integration with Other Modules

Integration with main.c

Called By:

• add_client(): Called from main loop when new connection accepted
• remove_client(): Called from main loop during cleanup phase

Provides To:

• Global client manager (g_client_manager) for thread access
• Client lookup functions for other modules
• Thread lifecycle coordination

Integration with protocol.c

Called By:

• Receive threads call protocol handler functions
• handle_client_join_packet(): Processes CLIENT_JOIN packets
• handle_client_capabilities_packet(): Updates terminal capabilities
• handle_image_frame_packet(): Stores video frames
• handle_audio_batch_packet(): Stores audio samples

Provides To:

• Client state access for protocol handlers
• Media buffer storage (video frames, audio samples)
• State update functions

Integration with render.c

Called By:

• create_client_render_threads(): Creates video and audio render threads
• Render threads access client state and media buffers

Provides To:

• Client state snapshots for render threads
• Media buffers for frame generation
• Terminal capabilities for ASCII conversion

Integration with stream.c

Called By:

• Render threads call stream generation functions
• generate_personalized_ascii_frame(): Creates client-specific frames

Provides To:

• Client capabilities for frame generation
• Video frame access for mixing
• Frame queue for delivery

Error Handling

Connection Errors:

• Handshake failures: Client rejected, socket closed
• Thread creation failures: Client rejected, resources cleaned up
• Resource allocation failures: Graceful degradation, client rejected

Runtime Errors:

• Receive thread errors: Connection marked inactive, cleanup triggered
• Send thread errors: Connection marked inactive, cleanup triggered
• Render thread errors: Thread exits, connection continues (graceful degradation)

Cleanup Errors:

• Thread join timeouts: Logged but don't block shutdown
• Resource cleanup failures: Logged but don't crash server
• Hash table errors: Handled gracefully with fallback

Performance Characteristics

Linear Scaling:

• Each client gets dedicated CPU resources
• No shared bottlenecks between clients
• Performance scales linearly up to 9+ clients
• Real-time guarantees maintained per client

Memory Usage:

• Per-client allocations: ~50KB per client (buffers, queues)
• Hash table overhead: ~1KB per client
• Frame buffers: Variable based on terminal size
• Audio buffers: Fixed size ring buffers

Thread Overhead:

• 4 threads per client (receive, send, video render, audio render)
• Minimal context switching overhead
• CPU affinity possible for performance tuning

Best Practices

DO:

• Always use proper lock ordering (RWLock → per-client mutex)
• Use snapshot pattern for client state access
• Check client->active before operations
• Join threads before resource cleanup
• Use hash table for client lookups (not array iteration)

DON'T:

• Don't acquire per-client mutex before global RWLock
• Don't hold locks during blocking operations
• Don't access client state without mutex protection
• Don't skip thread joins during cleanup
• Don't modify client array without write lock

See also

src/server/client.c

src/server/client.h

Server Overview

Server Main Entry Point

Protocol Handler

Render Threads

Concurrency Documentation

Cryptographic Operations

Overview

The cryptographic operations module manages per-client cryptographic handshakes, X25519 key exchange, session encryption, and client authentication for the ascii-chat server. This module embodies the security-first philosophy of ascii-chat, providing end-to-end encryption with multiple authentication modes to ensure secure multi-client video chat. The module integrates seamlessly with the client lifecycle, performing cryptographic handshakes during connection establishment and managing session encryption throughout the client's connection.

Implementation: src/server/crypto.c, src/server/crypto.h

Key Responsibilities:

• Initialize server crypto system and validate encryption configuration
• Perform cryptographic handshake with each connecting client
• Manage per-client crypto contexts stored in client_info_t structures
• Provide encryption/decryption functions for secure packet transmission
• Support multiple authentication modes (password, SSH key, passwordless)
• Integrate with client whitelist for authenticated access control

Cryptographic Handshake

The cryptographic handshake follows a multi-phase protocol:

Phase 0: Protocol Negotiation

Step 0a: Receive Client Protocol Version

// Receive client's protocol version
protocol_version_packet_t client_version;
result = receive_packet(socket, &packet_type, &payload, &payload_len);
 
if (packet_type != PACKET_TYPE_PROTOCOL_VERSION) {
  return -1;  // Protocol mismatch
}
 
if (!client_version.supports_encryption) {
  return -1;  // Client doesn't support encryption
}

Step 0b: Send Server Protocol Version

// Send server's protocol version
protocol_version_packet_t server_version = {0};
server_version.protocol_version = htons(1);
server_version.supports_encryption = 1;
 
result = send_protocol_version_packet(socket, &server_version);

Step 0c: Receive Client Crypto Capabilities

// Receive client's supported algorithms
crypto_capabilities_packet_t client_caps;
result = receive_packet(socket, &packet_type, &payload, &payload_len);
 
// Parse supported algorithms
uint16_t supported_kex = ntohs(client_caps.supported_kex_algorithms);
uint16_t supported_auth = ntohs(client_caps.supported_auth_algorithms);
uint16_t supported_cipher = ntohs(client_caps.supported_cipher_algorithms);

Step 0d: Select Algorithms and Send Parameters

// Select algorithms (currently only X25519 + Ed25519 + XSalsa20-Poly1305)
crypto_parameters_packet_t server_params = {0};
server_params.selected_kex = KEX_ALGO_X25519;
server_params.selected_cipher = CIPHER_ALGO_XSALSA20_POLY1305;
 
// Select authentication algorithm based on server configuration
if (g_server_encryption_enabled && g_server_private_key.type == KEY_TYPE_ED25519) {
  server_params.selected_auth = AUTH_ALGO_ED25519;
}
 
result = send_crypto_parameters_packet(socket, &server_params);

Phase 1: Key Exchange

Step 1: Send Server's Ephemeral Public Key

// Generate ephemeral key pair for this client session
x25519_keypair_t ephemeral_keys;
x25519_generate_keypair(&ephemeral_keys);
 
// Store ephemeral private key in client's crypto context
client->crypto_ctx.ephemeral_private_key = ephemeral_keys.private_key;
 
// Send ephemeral public key to client
key_exchange_init_packet_t keyex_init = {0};
memcpy(keyex_init.public_key, ephemeral_keys.public_key, X25519_PUBLIC_KEY_SIZE);
 
// If server has identity key, sign the key exchange
if (g_server_encryption_enabled && g_server_private_key.type == KEY_TYPE_ED25519) {
  ed25519_sign(&keyex_init.signature, keyex_init.public_key,
               X25519_PUBLIC_KEY_SIZE, &g_server_private_key);
  memcpy(keyex_init.identity_key, g_server_public_key, ED25519_PUBLIC_KEY_SIZE);
}
 
result = send_key_exchange_init_packet(socket, &keyex_init);

Step 2: Receive Client's Public Key and Derive Shared Secret

// Receive client's ephemeral public key
key_exchange_response_packet_t client_keyex;
result = receive_packet(socket, &packet_type, &payload, &payload_len);
 
// Derive shared secret using X25519 key exchange
x25519_shared_secret_t shared_secret;
x25519_derive_shared_secret(&shared_secret, client_keyex.public_key,
                            &client->crypto_ctx.ephemeral_private_key);
 
// Store shared secret in client's crypto context
memcpy(client->crypto_ctx.shared_secret, shared_secret, X25519_SHARED_SECRET_SIZE);

Phase 2: Authentication

Step 2: Send Authentication Challenge

// Receive client's public key and send auth challenge
auth_request_packet_t auth_req;
result = receive_packet(socket, &packet_type, &payload, &payload_len);
 
// If whitelist enabled, verify client's public key
if (g_num_whitelisted_clients > 0) {
  bool client_in_whitelist = false;
  for (size_t i = 0; i < g_num_whitelisted_clients; i++) {
    if (memcmp(auth_req.public_key, g_client_whitelist[i],
               ED25519_PUBLIC_KEY_SIZE) == 0) {
      client_in_whitelist = true;
      break;
    }
  }
 
  if (!client_in_whitelist) {
    return -1;  // Client not in whitelist
  }
}
 
// Generate authentication challenge
auth_challenge_packet_t challenge;
random_bytes(challenge.challenge, AUTH_CHALLENGE_SIZE);
 
// Sign challenge with server's identity key (if available)
if (g_server_encryption_enabled && g_server_private_key.type == KEY_TYPE_ED25519) {
  ed25519_sign(&challenge.signature, challenge.challenge,
               AUTH_CHALLENGE_SIZE, &g_server_private_key);
}
 
result = send_auth_challenge_packet(socket, &challenge);

Step 3: Receive Authentication Response and Complete Handshake

// Receive client's authentication response
auth_response_packet_t auth_resp;
result = receive_packet(socket, &packet_type, &payload, &payload_len);
 
// Verify client's signature on challenge
if (!ed25519_verify(auth_resp.signature, challenge.challenge,
                    AUTH_CHALLENGE_SIZE, auth_req.public_key)) {
  return -1;  // Authentication failed
}
 
// Derive session encryption keys from shared secret
derive_session_keys(&client->crypto_ctx.session_keys, &client->crypto_ctx.shared_secret);
 
// Mark handshake as complete
atomic_store(&client->crypto_initialized, true);

Authentication Modes

Password Authentication

How It Works:

• Uses Argon2id key derivation from shared password
• Both server and client derive same key from password
• No identity keys required (password-only mode)
• Suitable for trusted networks or simple deployments

Configuration:

// Set password via --password command-line option
// Password is used for key derivation, not transmitted
// Both server and client must use same password

SSH Key Authentication

How It Works:

• Server uses Ed25519 private key for identity verification
• Client provides Ed25519 public key for authentication
• Identity verification via known_hosts and whitelist
• Strong authentication with cryptographic signatures

Configuration:

// Server loads SSH key via --key option
// Supports SSH Ed25519 key files
// Supports gpg:keyid format for GPG keys (if GPG support enabled)
// Client provides public key during handshake

Key Loading:

• Validates SSH key file format
• Native encrypted key support (bcrypt_pbkdf + BearSSL AES-256-CTR)
• Validates key type (Ed25519 required)
• Loads private key for signing operations

Passwordless Mode

How It Works:

• Ephemeral keys only (no long-term identity)
• Key exchange provides confidentiality but not authentication
• Suitable for trusted networks or testing
• No identity verification performed

Configuration:

// No --key or --password provided
// Server runs in passwordless mode
// Key exchange provides encryption but no authentication

Per-Client Crypto Contexts

Each client has an independent crypto context stored in client_info_t:

typedef struct {
  // Key exchange state
  x25519_private_key_t ephemeral_private_key;
  x25519_shared_secret_t shared_secret;
 
  // Session encryption keys
  session_keys_t session_keys;
 
  // Authentication state
  ed25519_public_key_t client_public_key;
  bool authenticated;
 
  // Handshake state
  handshake_state_t state;
  bool initialized;
} crypto_handshake_ctx_t;

Context Lifecycle:

• Created during connection establishment
• Initialized during cryptographic handshake
• Used throughout client connection for encryption/decryption
• Cleaned up on client disconnect

Thread Safety:

• Each client has independent crypto context (no shared state)
• Socket access protected by client_state_mutex
• Per-client encryption/decryption operations are isolated
• Global server crypto state (g_server_private_key) read-only after init

Encryption and Decryption Operations

Packet Encryption

After handshake completion, all packets are encrypted before transmission:

int crypto_server_encrypt_packet(uint32_t client_id,
                                  const void *plaintext, size_t plaintext_len,
                                  void *ciphertext, size_t *ciphertext_len)
{
  // Get client's crypto context
  client_info_t *client = get_client_by_id(client_id);
 
  // Check if encryption is enabled
  if (!client->crypto_initialized) {
    // Encryption not initialized - passthrough
    memcpy(ciphertext, plaintext, plaintext_len);
    *ciphertext_len = plaintext_len;
    return 0;
  }
 
  // Encrypt packet using XSalsa20-Poly1305
  int result = xsalsa20poly1305_encrypt(ciphertext, ciphertext_len,
                                        plaintext, plaintext_len,
                                        NULL, 0,  // No additional data
                                        client->crypto_ctx.session_keys.encryption_key);
 
  return result;
}

Packet Decryption

All received packets are decrypted before processing:

int crypto_server_decrypt_packet(uint32_t client_id,
                                  const void *ciphertext, size_t ciphertext_len,
                                  void *plaintext, size_t *plaintext_len)
{
  // Get client's crypto context
  client_info_t *client = get_client_by_id(client_id);
 
  // Check if encryption is enabled
  if (!client->crypto_initialized) {
    // Encryption not initialized - passthrough
    memcpy(plaintext, ciphertext, ciphertext_len);
    *plaintext_len = ciphertext_len;
    return 0;
  }
 
  // Decrypt packet using XSalsa20-Poly1305
  int result = xsalsa20poly1305_decrypt(plaintext, plaintext_len,
                                        ciphertext, ciphertext_len,
                                        NULL, 0,  // No additional data
                                        client->crypto_ctx.session_keys.decryption_key);
 
  return result;
}

Automatic Passthrough:

• When encryption is disabled (–no-encrypt), packets pass through unchanged
• No performance overhead when encryption disabled
• Seamless integration with non-encrypted mode

Client Whitelist Integration

Whitelist Loading

Client whitelist is loaded during server initialization:

// Load client whitelist if provided
if (strlen(opt_client_keys) > 0) {
  if (parse_client_keys(opt_client_keys, g_client_whitelist,
                        &g_num_whitelisted_clients, MAX_CLIENTS) != 0) {
    return -1;  // Whitelist loading failed
  }
  log_info("Server will only accept %zu whitelisted clients",
           g_num_whitelisted_clients);
}

Whitelist Format:

• Supports multiple key formats (SSH public keys, raw hex, etc.)
• Multiple keys can be provided (comma-separated)
• Keys are validated during loading
• Only Ed25519 keys are supported

Whitelist Verification

During handshake, client's public key is verified against whitelist:

// Verify client's public key against whitelist
if (g_num_whitelisted_clients > 0) {
  bool client_in_whitelist = false;
  for (size_t i = 0; i < g_num_whitelisted_clients; i++) {
    if (memcmp(client_public_key, g_client_whitelist[i],
               ED25519_PUBLIC_KEY_SIZE) == 0) {
      client_in_whitelist = true;
      break;
    }
  }
 
  if (!client_in_whitelist) {
    log_warn("Client public key not in whitelist - rejecting connection");
    return -1;  // Authentication failed
  }
}

Verification Behavior:

• If whitelist enabled, only whitelisted clients can connect
• Clients not in whitelist are rejected during handshake
• Detailed error logging for troubleshooting
• Clean disconnect on whitelist rejection

Error Handling

Handshake Errors:

• Client disconnection during handshake: Logged and return error
• Protocol mismatch: Detailed error logging and disconnect
• Authentication failure: Logged and disconnect (whitelist rejection)
• Network errors: Detected and handled gracefully
• Invalid packets: Validated before processing

Encryption/Decryption Errors:

• Decryption failures: Logged and packet dropped
• Encryption failures: Logged and connection marked lost
• Key derivation failures: Handled gracefully
• Session key errors: Clean disconnect

Algorithm Support

Current Algorithms:

• Key Exchange: X25519 (Elliptic Curve Diffie-Hellman)
• Cipher: XSalsa20-Poly1305 (Authenticated Encryption)
• Authentication: Ed25519 (when server has identity key)
• Key Derivation: Argon2id (for password-based authentication)
• HMAC: HMAC-SHA256 (for additional integrity protection)

Future Algorithm Support:

• Additional key exchange algorithms (ECDH-P256, etc.)
• Additional cipher algorithms (ChaCha20-Poly1305, etc.)
• Additional authentication algorithms (ECDSA, etc.)

Integration with Other Modules

Integration with client.c

Called By:

• add_client(): Performs cryptographic handshake during connection
• Handshake integrated into client connection flow

Provides To:

• Per-client crypto contexts
• Encryption/decryption functions
• Authentication verification

Integration with protocol.c

Used By:

• Protocol handlers decrypt received packets
• Packet encryption before transmission
• Seamless integration with packet processing

Provides To:

• Packet encryption/decryption functions
• Crypto context access

Integration with main.c

Called By:

• init_server_crypto(): Initializes server crypto system
• Server key loading and validation
• Whitelist loading and validation

Provides To:

• Global server crypto state
• Server identity key
• Client whitelist

Best Practices

DO:

• Always validate packets before processing
• Use per-client crypto contexts (no shared state)
• Check crypto_initialized before encryption/decryption
• Handle errors gracefully without disconnecting clients unnecessarily
• Log detailed errors for troubleshooting

DON'T:

• Don't share crypto contexts between clients
• Don't skip packet validation
• Don't ignore encryption/decryption errors
• Don't expose private keys in logs
• Don't skip whitelist verification when enabled

See also

src/server/crypto.c

src/server/crypto.h

Server Overview

Client Management

topic_crypto

topic_handshake

topic_keys

Server Main Entry Point

Overview

The server main entry point orchestrates the complete server lifecycle, from initialization and socket binding through the multi-client connection accept loop to graceful shutdown. It serves as the conductor of the ascii-chat server's modular architecture, coordinating all subsystems while maintaining real-time performance and thread safety guarantees.

Implementation: src/server/main.c, src/server/main.h

Key Responsibilities:

• Platform initialization (Windows/POSIX compatibility)
• Logging and configuration setup
• Network socket creation and binding (IPv4 and IPv6)
• Global resource initialization (audio mixer, buffer pools, crypto)
• Background thread management (statistics logging)
• Main connection accept loop with client lifecycle management
• Signal handling for graceful shutdown (SIGINT, SIGTERM)
• Resource cleanup and thread coordination

Architecture

Initialization Sequence

The server initialization follows a strict dependency order:

// 1. Parse command-line options
options_init(argc, argv, MODE_SERVER);
 
// 2. Initialize platform (Windows: Winsock, timer resolution)
platform_init();
 
// 3. Initialize logging system
asciichat_shared_init("server.log");
 
// 4. Initialize crypto system (keys, whitelist)
init_server_crypto();
 
// 5. Initialize lock debugging system
lock_debug_init();
 
// 6. Initialize statistics system
stats_init();
 
// 7. Setup signal handlers (SIGINT, SIGTERM, SIGUSR1)
platform_signal(SIGINT, sigint_handler);
platform_signal(SIGTERM, sigterm_handler);
 
// 8. Initialize synchronization primitives
rwlock_init(&g_client_manager_rwlock);
mutex_init(&g_client_manager.mutex);
 
// 9. Initialize client hash table for O(1) lookups
g_client_manager.clients_by_id = NULL;  // uthash head pointer
 
// 10. Initialize audio mixer for multi-client audio
g_audio_mixer = mixer_create(MAX_CLIENTS, AUDIO_SAMPLE_RATE);
 
// 11. Bind listening sockets (IPv4 and/or IPv6)
listenfd = bind_and_listen(ipv4_address, AF_INET, port);
listenfd6 = bind_and_listen(ipv6_address, AF_INET6, port);
 
// 12. Start statistics logger thread
asciichat_thread_create(&g_stats_logger_thread, stats_logger_thread, NULL);

Main Connection Loop

The main connection loop orchestrates the complete multi-client lifecycle:

while (!atomic_load(&g_server_should_exit)) {
  // PHASE 1: Cleanup disconnected clients (free slots)
  // Collect cleanup tasks under read lock
  rwlock_rdlock(&g_client_manager_rwlock);
  for (int i = 0; i < MAX_CLIENTS; i++) {
    if (!client->active && client->receive_thread_initialized) {
      cleanup_tasks[cleanup_count++] = client->client_id;
    }
  }
  rwlock_rdunlock(&g_client_manager_rwlock);
 
  // Process cleanup without holding lock
  for (int i = 0; i < cleanup_count; i++) {
    asciichat_thread_join(&cleanup_tasks[i].receive_thread, NULL);
    remove_client(cleanup_tasks[i].client_id);
  }
 
  // PHASE 2: Accept new connection (with timeout)
  fd_set read_fds;
  socket_select(max_fd + 1, &read_fds, NULL, NULL, &timeout);
 
  socket_t client_sock = accept_with_timeout(accept_listenfd,
                                             &client_addr, &client_len, 0);
 
  // PHASE 3: Add client (spawns all per-client threads)
  int client_id = add_client(client_sock, client_ip, client_port);
  if (client_id < 0) {
    socket_close(client_sock);  // Reject connection
    continue;
  }
 
  // Connection accepted - client threads now running
  log_info("Client %d connected, total clients: %d", client_id,
           g_client_manager.client_count);
}

Critical Loop Design:

• Cleanup MUST happen before accept() to free connection slots
• Timeout on accept() allows checking shutdown flag
• Dual-stack support (IPv4 and IPv6 sockets checked simultaneously)
• Graceful degradation if add_client() fails

Signal Handling

SIGINT Handler (Ctrl+C)

The SIGINT handler implements signal-safe shutdown:

static void sigint_handler(int sigint) {
  (void)sigint;
  static int sigint_count = 0;
  sigint_count++;
 
  // Second Ctrl+C forces immediate exit
  if (sigint_count > 1) {
    exit(1);
  }
 
  // First Ctrl+C: Graceful shutdown
  atomic_store(&g_server_should_exit, true);
  printf("SIGINT received - shutting down server...\n");
 
  // Close listening sockets to interrupt accept()
  socket_close(atomic_load(&listenfd));
  socket_close(atomic_load(&listenfd6));
 
  // NOTE: Do NOT access client data structures in signal handler
  // Signal handlers cannot safely use mutexes or complex data structures
  // Main thread will handle client cleanup when it detects g_server_should_exit
}

Signal Safety Strategy:

• Only async-signal-safe functions (atomic operations, write())
• No mutex operations (can deadlock if main thread holds mutex)
• No malloc/free (heap corruption risk if interrupted)
• Minimal work - set flag and close sockets to interrupt blocking I/O

Why Socket Closure: Without closing sockets, threads remain blocked in:

• accept() in main loop (waiting for new connections)
• recv() in client receive threads (waiting for packets)
• send() in client send threads (if network is slow)

Closing sockets causes these functions to return with errors, allowing threads to check g_server_should_exit and exit gracefully.

SIGTERM Handler

SIGTERM is sent by process managers (systemd, Docker) for graceful termination:

static void sigterm_handler(int sigterm) {
  (void)sigterm;
  atomic_store(&g_server_should_exit, true);
  printf("SIGTERM received - shutting down server...\n");
  // Return immediately - main thread handles cleanup
}

Conservative Approach:

• More conservative than SIGINT handler
• Relies on main thread for complete cleanup
• Focuses on minimal flag setting
• Prevents partial states during automated shutdown

SIGUSR1 Handler (Lock Debugging)

SIGUSR1 triggers lock debugging output for troubleshooting deadlocks:

static void sigusr1_handler(int sigusr1) {
  (void)sigusr1;
  lock_debug_trigger_print();  // Signal-safe lock state dump
}

This allows external triggering of lock debugging without modifying running server.

Shutdown Sequence

The shutdown sequence ensures complete resource cleanup:

   // 1. Signal all threads to exit
   atomic_store(&g_server_should_exit, true);
  
   // 2. Wake up blocked threads
   static_cond_broadcast(&g_shutdown_cond);
  
   // 3. Close all client sockets (unblock receive threads)
   rwlock_rdlock(&g_client_manager_rwlock);
   for (int i = 0; i < MAX_CLIENTS; i++) {
     if (client->socket != INVALID_SOCKET_VALUE) {
       socket_close(client->socket);  // Interrupts blocking recv()
     }
   }
   rwlock_rdunlock(&g_client_manager_rwlock);
  
   // 4. Wait for stats logger thread
   asciichat_thread_join(&g_stats_logger_thread, NULL);
  
   // 5. Remove all clients (joins all worker threads)
   for (int i = 0; i < client_count; i++) {
     remove_client(clients_to_remove[i]);
   }
  
   // 6. Cleanup resources in reverse dependency order
   // Clean up uthash table (iterate and delete all entries)
HASH_ITER(hh, g_client_manager.clients_by_id, ...);
   mixer_destroy(g_audio_mixer);
   rwlock_destroy(&g_client_manager_rwlock);
   mutex_destroy(&g_client_manager.mutex);
   stats_cleanup();
   lock_debug_cleanup();
   socket_close(listenfd);
   socket_close(listenfd6);
   simd_caches_destroy_all();
   data_buffer_pool_cleanup_global();
   log_destroy();

Cleanup Guarantees:

• All threads joined before resource cleanup
• Resources cleaned in reverse dependency order
• No memory leaks or hanging processes
• Deterministic shutdown regardless of thread state

IPv4 and IPv6 Dual-Stack Support

The server supports simultaneous binding to both IPv4 and IPv6:

Binding Logic:

• Default (0 arguments): Bind to both 127.0.0.1 (IPv4) and ::1 (IPv6) for localhost dual-stack
• If one IPv4 address provided: Bind only to that IPv4 address
• If one IPv6 address provided: Bind only to that IPv6 address
• If two addresses provided: Bind to both addresses (full dual-stack, order-independent)

Connection Acceptance:

• Uses select() to check both sockets simultaneously
• Accepts connections from either socket as they arrive
• Handles IPv4-mapped IPv6 addresses transparently
• Logs connection source (IPv4, IPv6, or IPv4-mapped)

Integration Points

With Client Management (client.c):

• Calls add_client() for each new connection
• Calls remove_client() for disconnections
• Accesses g_client_manager with proper locking

With Protocol Handler (protocol.c):

• Not directly called (used by client receive threads)
• Provides global state (g_server_should_exit)

With Stream Generation (stream.c):

• Not directly called (used by render threads)
• Render threads call stream functions independently

With Render Threads (render.c):

• Threads created via create_client_render_threads() in client.c
• Not directly managed by main.c

With Statistics (stats.c):

• Starts stats_logger_thread_func() in background
• Waits for stats thread during shutdown

With Cryptography (crypto.c):

• Calls init_server_crypto() during initialization
• Manages global server crypto state (g_server_private_key, g_client_whitelist)

Modular Design Philosophy

The original server.c became unmaintainable at 2408+ lines, making it:

• Too large for LLM context windows (limited AI-assisted development)
• Difficult for humans to navigate and understand
• Slow to compile and modify
• Hard to isolate bugs and add new features

The modular design enables:

• Faster development cycles: Smaller compilation units compile faster
• Better IDE support: Jump-to-definition, IntelliSense work reliably
• Easier testing: Isolated components can be unit tested
• Future extensibility: New protocols, renderers, optimizations can be added cleanly

Module Boundaries:

• main.c: Server lifecycle, signal handling, connection acceptance
• client.c: Per-client lifecycle, threading, state management
• protocol.c: Packet processing, protocol state, media storage
• stream.c: Video mixing, ASCII conversion, frame generation
• render.c: Render thread management, rate limiting, timing
• stats.c: Performance monitoring, resource tracking, reporting
• crypto.c: Cryptographic handshake, key exchange, session encryption

Error Handling

Initialization Errors:

• FATAL macros used for unrecoverable errors (crypto, network, platform)
• Detailed error context via asciichat_errno system
• Clean resource cleanup before exit

Connection Errors:

• Graceful degradation: Failed client addition doesn't crash server
• Socket closure on failure: Prevents resource leaks
• Error logging: Detailed diagnostics for troubleshooting

Runtime Errors:

• Connection loss handled by client management (not fatal)
• Thread failures logged but don't crash server
• Resource allocation failures handled gracefully

Concurrency Model

Main Thread Responsibilities:

• Accept new connections (blocking with timeout)
• Manage client lifecycle (add/remove)
• Handle disconnection cleanup
• Coordinate graceful shutdown

Background Thread Responsibilities:

• Per-client receive: Handle incoming packets (client.c)
• Per-client send: Manage outgoing packets (client.c)
• Per-client video render: Generate ASCII frames (render.c)
• Per-client audio render: Mix audio streams (render.c)
• Stats logger: Monitor server performance (stats.c)

Thread Safety:

• Global RWLock (g_client_manager_rwlock) protects client array
• Atomic variables (g_server_should_exit) for shutdown coordination
• Snapshot pattern for client state access
• Lock ordering protocol prevents deadlocks

See Concurrency Documentation for complete details.

Best Practices

DO:

• Always check g_server_should_exit in loops
• Use proper lock ordering (RWLock → per-client mutex)
• Use snapshot pattern for client state access
• Close sockets in signal handlers to interrupt blocking I/O
• Wait for all threads during shutdown

DON'T:

• Don't access client data structures in signal handlers
• Don't acquire locks in signal handlers (can deadlock)
• Don't call malloc/free in signal handlers (heap corruption risk)
• Don't skip client cleanup (memory leaks)
• Don't ignore lock ordering protocol (deadlock risk)

See also

src/server/main.c

src/server/main.h

Server Overview

Client Management

Concurrency Documentation

Server Protocol README

Overview

The protocol handler module processes all incoming packets from clients, validates packet structure, updates client state, and stores media data for render threads. It serves as the communication bridge between clients and the server, translating network packets into actionable state changes and media data storage.

Implementation: src/server/protocol.c, src/server/protocol.h

Key Responsibilities:

• Parse and validate incoming packets from clients
• Update client state based on received packet data
• Store media data (video frames, audio samples) in client buffers
• Coordinate with other modules for media processing
• Generate appropriate server responses to client requests
• Maintain protocol compliance and packet format standards

Packet Processing Architecture

Processing Flow

Packet processing follows a three-stage pipeline:

Stage 1: Packet Reception (in client.c receive thread):

// Receive raw packet data from socket
int packet_len = recv_with_timeout(client->socket, buffer, sizeof(buffer), timeout);
 
// Validate packet header (magic number, CRC32, size)
if (validate_packet_header(buffer, packet_len) < 0) {
  log_warn("Invalid packet header from client %u", client_id);
  return -1;
}
 
// Decrypt packet if encryption enabled
if (client->crypto_initialized) {
  decrypt_packet(buffer, packet_len, &client->crypto_ctx);
}
 
// Dispatch to appropriate handler based on packet type
dispatch_packet_to_handler(client_id, buffer, packet_len);

Stage 2: Handler Function (this module):

// Validate packet payload structure
if (packet_len < sizeof(packet_type_t)) {
  log_warn("Packet too small from client %u", client_id);
  return -1;
}
 
// Parse packet type
packet_type_t type = get_packet_type(buffer);
 
// Route to handler function based on type
switch (type) {
  case PACKET_TYPE_CLIENT_JOIN:
    handle_client_join_packet(client_id, buffer, packet_len);
    break;
  case PACKET_TYPE_IMAGE_FRAME:
    handle_image_frame_packet(client_id, buffer, packet_len);
    break;
  // ... other packet types ...
}

Stage 3: Response Generation (via packet queues):

// Queue response packet for delivery
packet_queue_enqueue(client->video_packet_queue, response_packet, response_len);
 
// Send thread will deliver packet asynchronously
// (no blocking I/O in handler functions)

Supported Packet Types

Client Lifecycle Packets

PACKET_TYPE_CLIENT_JOIN:

• Initial client capabilities and identity
• Contains display name, terminal capabilities, client version
• Triggers client state initialization
• Generates server welcome response

PACKET_TYPE_CLIENT_LEAVE:

• Clean disconnect notification
• Triggers graceful client cleanup
• No response required

PACKET_TYPE_CLIENT_CAPABILITIES:

• Terminal capabilities and preferences update
• Contains color depth, palette preferences, UTF-8 support
• Updates client state for frame generation
• Triggers frame regeneration with new capabilities

Media Streaming Packets

PACKET_TYPE_STREAM_START:

• Signals client is ready to send audio/video
• Contains stream type flags (audio, video, both)
• Enables render threads for this client
• Triggers frame generation activation

PACKET_TYPE_STREAM_STOP:

• Signals client is stopping audio/video transmission
• Disables render threads for this client
• Clears media buffers

PACKET_TYPE_IMAGE_FRAME:

• Raw RGB video frame data
• Contains frame dimensions, timestamp, compressed data
• Stored in client's incoming video buffer
• Processed by video render thread

PACKET_TYPE_AUDIO_BATCH:

• Batched audio samples (efficient format)
• Contains sample count, sample rate, compressed audio data
• Stored in client's incoming audio ring buffer
• Processed by audio render thread

PACKET_TYPE_AUDIO (legacy):

• Single audio sample packet (deprecated)
• Replaced by PACKET_TYPE_AUDIO_BATCH
• Still supported for backwards compatibility

Control Protocol Packets

PACKET_TYPE_PING:

• Client keepalive request
• Generates PACKET_TYPE_PONG response
• Used for connection health monitoring

PACKET_TYPE_PONG:

• Server keepalive response
• Acknowledges client ping
• Confirms bidirectional connectivity

State Management

Client State Updates

All client state modifications use the snapshot pattern:

// Handler function updates client state
void handle_client_capabilities_packet(uint32_t client_id,
                                       const void *data, size_t len)
{
  // 1. Get client (with read lock)
  rwlock_rdlock(&g_client_manager_rwlock);
  client_info_t *client = NULL;
  HASH_FIND_INT(g_client_manager.clients_by_id,
                                            client_id);
 
  // 2. Acquire per-client mutex
  mutex_lock(&client->client_state_mutex);
 
  // 3. Update client state fields
  client->width = packet->width;
  client->height = packet->height;
  client->capabilities = packet->capabilities;
  client->palette = packet->palette;
 
  // 4. Release mutex immediately
  mutex_unlock(&client->client_state_mutex);
 
  // 5. Release read lock
  rwlock_rdunlock(&g_client_manager_rwlock);
 
  // 6. Process using local copies (no locks held)
  log_info("Client %u capabilities updated: %dx%d, palette=%s", client_id,
           packet->width, packet->height, palette_name(packet->palette));
}

State Fields:

• Terminal dimensions (width, height)
• Terminal capabilities (color depth, UTF-8 support, palette)
• Stream status (audio enabled, video enabled)
• Client identity (display name, client version)
• Connection metadata (connect time, last packet time)

Media Buffer Coordination

Video Frames:

• Stored in client->incoming_video_buffer (double-buffer system)
• Thread-safe buffer access via mutex
• Processed by video render thread at 60fps
• Frame dropping under load (keep only latest frame)

Audio Samples:

• Stored in client->incoming_audio_buffer (lock-free ring buffer)
• Thread-safe lock-free operations
• Processed by audio render thread at 172fps
• Automatic overflow handling (dropped samples logged)

Packet Validation Strategy

All handlers validate packets before processing:

void handle_image_frame_packet(uint32_t client_id,
                                const void *data, size_t len)
{
  // 1. Validate packet size
  if (len < sizeof(image_frame_packet_t)) {
    log_warn("Image frame packet too small from client %u: %zu < %zu",
             client_id, len, sizeof(image_frame_packet_t));
    return;
  }
 
  // 2. Validate packet structure
  const image_frame_packet_t *packet = (const image_frame_packet_t *)data;
  if (packet->width == 0 || packet->height == 0) {
    log_warn("Invalid frame dimensions from client %u: %ux%u",
             client_id, packet->width, packet->height);
    return;
  }
 
  // 3. Validate frame data size
  size_t expected_size = packet->width * packet->height * 3; // RGB
  if (packet->data_size > expected_size) {
    log_warn("Frame data size mismatch from client %u: %zu > %zu",
             client_id, packet->data_size, expected_size);
    return;
  }
 
  // 4. Validate client capabilities
  if (!client->video_enabled) {
    log_warn("Client %u sent video but video not enabled", client_id);
    return;
  }
 
  // 5. Process packet (all validations passed)
  store_video_frame(client_id, packet);
}

Validation Checks:

• Packet size matches expected structure size
• Packet type matches handler function
• Payload fields are within valid ranges
• Client capabilities permit the operation
• Buffer pointers are valid before access
• Network byte order conversion where needed

Error Handling Philosophy

Invalid Packets:

• Logged but don't disconnect clients
• Malformed packets are silently dropped
• Detailed error logging for troubleshooting
• Protocol compliance checking prevents crashes

Buffer Allocation Failures:

• Handled gracefully with error return
• Client continues receiving other packets
• Render threads handle missing frames gracefully
• No server crash from allocation failures

Network Errors During Responses:

• Don't affect client state
• Logged for debugging
• Send thread handles connection loss detection
• No blocking I/O in handler functions

Shutdown Conditions:

• Handlers check g_server_should_exit flag
• Early return from handlers during shutdown
• Avoids error spam during cleanup
• Clean thread exit without errors

Integration with Other Modules

Integration with client.c

Called By:

• Receive threads call protocol handler functions
• Receive thread receives packets and dispatches to handlers

Provides To:

• Client state update functions
• Media buffer storage functions
• Packet validation utilities

Integration with render.c

Consumed By:

• Render threads consume media data stored by handlers
• Video render thread reads from incoming_video_buffer
• Audio render thread reads from incoming_audio_buffer

Provides To:

• Video frame data for frame generation
• Audio sample data for audio mixing
• Client state for frame customization

Integration with stream.c

Used By:

• Stream generation uses client capabilities set by handlers
• Frame generation adapts to terminal dimensions
• Palette selection based on client preferences

Provides To:

• Client terminal capabilities
• Client rendering preferences
• Frame generation parameters

Performance Characteristics

Processing Speed:

• Packet validation is O(1) per packet
• State updates use minimal locking
• Media storage uses efficient buffer systems
• No blocking I/O in handler functions

Memory Usage:

• Packet buffers: Temporary allocations (freed after processing)
• Media buffers: Per-client fixed allocations
• State structures: Minimal overhead per client

Concurrency:

• Per-client handler isolation (no shared state)
• Thread-safe state updates (mutex protection)
• Lock-free media buffers where possible
• Minimal lock contention (snapshot pattern)

Best Practices

DO:

• Always validate packet size before accessing fields
• Use snapshot pattern for client state access
• Check client capabilities before operations
• Handle errors gracefully without disconnecting clients
• Use atomic operations for thread control flags

DON'T:

• Don't access packet fields without size validation
• Don't hold locks during media processing
• Don't perform blocking I/O in handler functions
• Don't skip error checking on state updates
• Don't ignore shutdown flags during processing

See also

src/server/protocol.c

src/server/protocol.h

Server Overview

Client Management

Render Threads

topic_packet_types

Render Threads

Overview

The render thread module manages per-client rendering threads that generate personalized ASCII frames at 60fps and mix audio streams at 172fps. This module embodies the real-time performance guarantees of ascii-chat, ensuring every client receives smooth video and low-latency audio regardless of the number of connected clients. The module implements sophisticated rate limiting, precise timing control, and thread lifecycle management to achieve professional-grade performance.

Implementation: src/server/render.c, src/server/render.h

Key Responsibilities:

• Manage per-client video rendering threads (60fps per client)
• Manage per-client audio rendering threads (172fps per client)
• Coordinate frame generation timing and rate limiting
• Ensure thread-safe access to client state and media buffers
• Provide graceful thread lifecycle management
• Handle platform-specific timing and synchronization

Per-Client Threading Model

Each connected client spawns exactly 2 dedicated threads:

Video Render Thread

Purpose: Generate personalized ASCII frames at 60fps Performance: 16.67ms intervals (60 frames per second) Operations:

• Collects video frames from all active clients (via stream.c)
• Creates composite frame with appropriate grid layout
• Converts composite to ASCII using client's terminal capabilities
• Queues ASCII frame in client's video packet queue
• Rate limits to exactly 60fps with precise timing

Thread Implementation:

static void *video_render_thread_func(void *arg) {
  uint32_t client_id = (uint32_t)(uintptr_t)arg;
 
  // Initialize timing
  struct timespec last_frame_time;
  clock_gettime(CLOCK_MONOTONIC, &last_frame_time);
 
  while (true) {
    // Check shutdown flag
    if (atomic_load(&g_server_should_exit) ||
        !atomic_load(&client->video_render_thread_running) ||
        !atomic_load(&client->active)) {
      break;
    }
 
    // Generate personalized ASCII frame for client
    ascii_frame_t *frame = generate_personalized_ascii_frame(client_id);
    if (frame) {
      // Queue frame for delivery via send thread
      packet_queue_enqueue(client->video_packet_queue, frame, frame_size);
    }
 
    // Rate limit to exactly 60fps
    struct timespec current_time;
    clock_gettime(CLOCK_MONOTONIC, &current_time);
    double elapsed = timespec_diff_ms(&current_time, &last_frame_time);
 
    if (elapsed < 16.67) {
      // Ahead of schedule - sleep until next frame time
      platform_sleep_usec((int)((16.67 - elapsed) * 1000));
    }
 
    last_frame_time = current_time;
  }
 
  return NULL;
}

Rate Limiting Strategy:

• Uses CLOCK_MONOTONIC for precise timing
• Calculates elapsed time since last frame
• Sleeps only if ahead of schedule
• Prevents CPU spinning under light load
• Maintains exactly 60fps timing

Audio Render Thread

Purpose: Mix audio streams at 172fps (excluding client's own audio) Performance: 5.8ms intervals (172 frames per second) Operations:

• Reads audio samples from all clients via audio mixer
• Excludes client's own audio to prevent echo
• Mixes remaining audio streams with ducking and compression
• Queues mixed audio in client's audio packet queue
• Rate limits to exactly 172fps (5.8ms intervals)

Thread Implementation:

static void *audio_render_thread_func(void *arg) {
  uint32_t client_id = (uint32_t)(uintptr_t)arg;
 
  // Audio mixer provides per-client mixed audio
  mixer_t *mixer = g_audio_mixer;
 
  while (true) {
    // Check shutdown flag
    if (atomic_load(&g_server_should_exit) ||
        !atomic_load(&client->audio_render_thread_running) ||
        !atomic_load(&client->active)) {
      break;
    }
 
    // Get mixed audio (excluding client's own audio)
    audio_sample_t *mixed_audio = mixer_get_mixed_audio(mixer, client_id);
 
    if (mixed_audio) {
      // Queue audio for delivery via send thread
      packet_queue_enqueue(client->audio_packet_queue, mixed_audio,
                           mixed_audio_size);
    }
 
    // Rate limit to exactly 172fps (5.8ms intervals)
    platform_sleep_usec(5800);  // 5.8ms
  }
 
  return NULL;
}

Audio Processing Features:

• Echo cancellation (excludes client's own audio)
• Audio ducking (quiets other clients when client speaks)
• Compression (normalizes audio levels)
• Noise gate (suppresses background noise)
• Low-latency delivery (5.8ms intervals)

Rate Limiting and Timing Control

Timing Precision

Platform-Specific Timing:

• Windows: Uses Sleep() with millisecond precision
• POSIX: Uses condition variables for responsive interruption
• High-resolution timing with clock_gettime()
• Interruptible sleep for responsive shutdown

Rate Limiting Algorithm:

// Calculate elapsed time since last frame
double elapsed = timespec_diff_ms(&current_time, &last_frame_time);
 
if (elapsed < target_interval) {
  // Ahead of schedule - sleep until next frame time
  platform_sleep_usec((int)((target_interval - elapsed) * 1000));
} else {
  // Behind schedule - skip sleep to catch up
  // This prevents frame accumulation under load
}

Timing Guarantees:

• Video: Exactly 60fps (16.67ms intervals)
• Audio: Exactly 172fps (5.8ms intervals)
• Platform-specific high-resolution timers
• Interruptible sleep for responsive shutdown

CPU Usage Management

Prevent CPU Spinning:

• Sleep when ahead of schedule
• Skip sleep only when behind schedule
• Minimal CPU usage under light load
• Responsive to load changes

Catch-Up Strategy:

• Behind schedule: Skip sleep to catch up
• Prevents frame accumulation
• Maintains real-time performance
• Adaptive to load changes

Thread Safety and Synchronization

Client State Access

All client state access uses the snapshot pattern:

// Snapshot client state under mutex
mutex_lock(&client->client_state_mutex);
 
bool should_continue = client->video_render_thread_running &&
                       client->active;
uint32_t client_id_snapshot = client->client_id;
unsigned short width_snapshot = client->width;
unsigned short height_snapshot = client->height;
terminal_capabilities_t caps_snapshot = client->capabilities;
 
mutex_unlock(&client->client_state_mutex);
 
// Process using local copies (no locks held)
if (should_continue) {
  generate_frame(client_id_snapshot, width_snapshot, height_snapshot,
                 &caps_snapshot);
}

Snapshot Benefits:

• Minimal lock holding time
• No blocking operations while holding locks
• Prevents deadlocks from complex call chains
• Enables parallel processing across clients

Thread Control Flags

Atomic Thread Control:

• video_render_thread_running: Atomic boolean for thread control
• audio_render_thread_running: Atomic boolean for thread control
• active: Atomic boolean for client connection status
• g_server_should_exit: Atomic boolean for server shutdown

Shutdown Coordination:

• Threads check flags frequently in loops
• Clean thread exit on flag change
• No blocking operations that can't be interrupted
• Responsive shutdown (threads exit within frame interval)

Media Buffer Access

Video Buffer Access:

• Double-buffer system (atomic operations)
• Thread-safe frame reading
• Always get latest available frame
• Frame dropping under load handled gracefully

Audio Buffer Access:

• Lock-free ring buffer
• Thread-safe audio sample reading
• Automatic overflow handling
• Low-latency access

Thread Lifecycle Management

Thread Creation

Creation Order:

1. Video render thread created first (60fps requirement)
2. Audio render thread created second (172fps requirement)

Creation Process:

int create_client_render_threads(client_info_t *client) {
  // Create video render thread
  atomic_store(&client->video_render_thread_running, true);
  if (asciichat_thread_create(&client->video_render_thread,
                          video_render_thread_func,
                          (void *)(uintptr_t)client->client_id) != 0) {
    return -1;
  }
 
  // Create audio render thread
  atomic_store(&client->audio_render_thread_running, true);
  if (asciichat_thread_create(&client->audio_render_thread,
                          audio_render_thread_func,
                          (void *)(uintptr_t)client->client_id) != 0) {
    // Cleanup video thread on failure
    atomic_store(&client->video_render_thread_running, false);
    asciichat_thread_join(&client->video_render_thread, NULL);
    return -1;
  }
 
  return 0;
}

Thread Termination

Termination Sequence:

1. Set thread running flags to false
2. Threads detect flag change and exit loops
3. Join threads to ensure complete cleanup
4. Clear thread handles

Termination Process:

void destroy_client_render_threads(client_info_t *client) {
  // Signal threads to exit
  atomic_store(&client->video_render_thread_running, false);
  atomic_store(&client->audio_render_thread_running, false);
 
  // Join threads with timeout
  asciichat_thread_join_timeout(&client->video_render_thread, NULL, 1000);
  asciichat_thread_join_timeout(&client->audio_render_thread, NULL, 1000);
 
  // Clear thread handles
  asciichat_thread_init(&client->video_render_thread);
  asciichat_thread_init(&client->audio_render_thread);
}

Timeout Handling:

• Thread join timeouts prevent blocking
• Logged warnings for timeout cases
• Graceful degradation on thread hang
• Clean shutdown even if threads don't exit cleanly

Platform Abstraction Integration

Cross-Platform Timing

Windows Timing:

• Uses Sleep() with millisecond precision
• Requires timer resolution adjustment (timeBeginPeriod)
• Minimal overhead for sleep operations

POSIX Timing:

• Uses condition variables for responsive interruption
• High-resolution timing with clock_gettime()
• Interruptible sleep for shutdown responsiveness

Cross-Platform Threading

Windows Threading:

• Uses platform abstraction layer
• Handles Windows-specific thread initialization
• Platform-safe thread join operations

POSIX Threading:

• Uses pthreads via platform abstraction
• Standard POSIX thread operations
• Cross-platform thread lifecycle management

Integration with Other Modules

Integration with client.c

Called By:

• create_client_render_threads(): Creates render threads per client
• Thread creation coordinated with client lifecycle

Provides To:

• Render thread management functions
• Thread lifecycle coordination

Integration with stream.c

Called By:

• Video render threads call generate_personalized_ascii_frame()
• Frame generation at 60fps per client

Provides To:

• Per-client frame generation requests
• Terminal capability awareness

Integration with mixer.c

Called By:

• Audio render threads call mixer_get_mixed_audio()
• Audio mixing at 172fps per client

Provides To:

• Per-client audio mixing requests
• Echo cancellation (excludes client's own audio)

Performance Characteristics

Linear Scaling:

• Each client gets dedicated render threads
• No shared bottlenecks between clients
• Performance scales linearly up to 9+ clients
• Real-time guarantees maintained per client

CPU Usage:

• Minimal CPU usage when ahead of schedule
• Adaptive to load changes
• No CPU spinning under light load
• Efficient sleep operations

Timing Precision:

• Exactly 60fps for video (16.67ms intervals)
• Exactly 172fps for audio (5.8ms intervals)
• Platform-specific high-resolution timers
• Consistent timing across platforms

Best Practices

DO:

• Always check shutdown flags in loops
• Use snapshot pattern for client state access
• Sleep when ahead of schedule to prevent CPU spinning
• Use precise timing for rate limiting
• Join threads with timeout to prevent blocking

DON'T:

• Don't hold locks during frame generation
• Don't skip shutdown flag checks
• Don't use blocking operations without timeout
• Don't ignore thread join failures
• Don't modify thread control flags without synchronization

See also

src/server/render.c

src/server/render.h

Server Overview

Client Management

Stream Generation

topic_mixer

Concurrency Documentation

Statistics and Monitoring

Overview

The statistics and monitoring module provides comprehensive performance tracking and operational visibility for the ascii-chat server. It operates as a background thread that collects metrics from all system components every 30 seconds and generates detailed reports for troubleshooting, performance optimization, and operational monitoring. This module embodies the operational excellence philosophy of ascii-chat, providing the visibility needed to maintain high-performance multi-client video chat.

Implementation: src/server/stats.c, src/server/stats.h

Key Responsibilities:

• Continuous monitoring of server performance metrics
• Per-client statistics collection and reporting
• Buffer pool utilization tracking
• Packet queue performance analysis
• Hash table efficiency monitoring
• Periodic statistics logging (every 30 seconds)
• Operational visibility for administrators

Monitoring Architecture

Statistics Collection Thread

The statistics module runs as a dedicated background thread:

static void *stats_logger_thread_func(void *arg) {
  (void)arg;
 
  // Main monitoring loop
  while (!atomic_load(&g_server_should_exit)) {
    // Sleep for reporting interval (30 seconds)
    for (int i = 0; i < 300 && !atomic_load(&g_server_should_exit); i++) {
      platform_sleep_usec(100000);  // 0.1 second intervals
    }
 
    if (atomic_load(&g_server_should_exit)) {
      break;
    }
 
    // Collect statistics from all system components
    collect_and_log_statistics();
  }
 
  return NULL;
}

Thread Characteristics:

• Non-intrusive background monitoring
• 30-second reporting intervals (configurable)
• Interruptible sleep for responsive shutdown
• Thread-safe data collection from all system components
• Minimal impact on operational performance

Statistics Collection Methodology

Non-Intrusive Monitoring:

• Uses reader locks to avoid blocking operational threads
• Takes atomic snapshots of volatile data
• Minimal impact on render thread performance
• Safe concurrent access to client data

Statistics Atomicity:

• Global statistics protected by dedicated mutex
• Consistent reporting even during concurrent updates
• Thread-safe access to shared counters
• Snapshot pattern for volatile data

Performance Metrics Collected

Client Management Metrics

Client Statistics:

• Total active clients
• Clients with audio capabilities
• Clients with video capabilities
• Connection duration and activity patterns
• Per-client connection metadata

Collection Process:

// Collect client statistics
rwlock_rdlock(&g_client_manager_rwlock);
 
int active_clients = 0;
int clients_with_audio = 0;
int clients_with_video = 0;
 
for (int i = 0; i < MAX_CLIENTS; i++) {
  client_info_t *client = &g_client_manager.clients[i];
  if (atomic_load(&client->active)) {
    active_clients++;
    if (client->audio_enabled) clients_with_audio++;
    if (client->video_enabled) clients_with_video++;
  }
}
 
rwlock_rdunlock(&g_client_manager_rwlock);

Buffer Pool Performance

Buffer Pool Metrics:

• Global buffer pool utilization
• Allocation/deallocation rates
• Peak usage patterns
• Memory efficiency metrics
• Buffer pool fragmentation indicators

Collection Process:

// Collect buffer pool statistics
buffer_pool_stats_t stats = buffer_pool_get_statistics();
 
log_info("Buffer Pool: %zu allocated, %zu peak, %zu total, %.2f%% utilization",
         stats.allocated, stats.peak, stats.total,
         (stats.allocated * 100.0) / stats.total);

Packet Queue Performance

Packet Queue Statistics:

• Per-client queue depths
• Enqueue/dequeue rates
• Packet drop rates under load
• Queue overflow incidents
• Queue utilization patterns

Collection Process:

// Collect packet queue statistics
rwlock_rdlock(&g_client_manager_rwlock);
 
for (int i = 0; i < MAX_CLIENTS; i++) {
  client_info_t *client = &g_client_manager.clients[i];
  if (atomic_load(&client->active)) {
    packet_queue_stats_t video_stats = 
      packet_queue_get_statistics(client->video_packet_queue);
    packet_queue_stats_t audio_stats = 
      packet_queue_get_statistics(client->audio_packet_queue);
 
    log_info("Client %u queues: video=%zu/%zu, audio=%zu/%zu",
             client->client_id, video_stats.size, video_stats.capacity,
             audio_stats.size, audio_stats.capacity);
  }
}
 
rwlock_rdunlock(&g_client_manager_rwlock);

Hash Table Efficiency

Hash Table Metrics:

• Client lookup performance
• Hash collision rates
• Load factor monitoring
• Access pattern analysis
• Table efficiency indicators

Collection Process:

// Collect hash table statistics
size_t client_count = HASH_COUNT(g_client_manager.clients_by_id);
 
log_info("Hash Table: %zu entries, %zu capacity, %.2f%% load factor, %zu collisions",
         stats.entries, stats.capacity, stats.load_factor * 100.0, stats.collisions);

Frame Processing Metrics

Frame Processing Statistics:

• Total frames captured from clients
• Total frames sent to clients
• Frame drop rate under load
• Blank frame count (no video sources)
• Frame generation performance

Collection Process:

// Collect frame processing statistics
frame_stats_t stats = get_frame_processing_statistics();
 
log_info("Frames: %zu captured, %zu sent, %zu dropped, %.2f%% drop rate",
         stats.captured, stats.sent, stats.dropped,
         (stats.dropped * 100.0) / stats.captured);

Statistics Reporting

Report Format

Statistics are logged in a structured format:

[STATS] Client Count: 3 active (2 audio, 3 video)
[STATS] Buffer Pool: 45/100 allocated (45.00% utilization), 52 peak
[STATS] Packet Queues: avg_depth=12.3, max_depth=25, drops=3
[STATS] Hash Table: 3 entries, 16 capacity (18.75% load), 0 collisions
[STATS] Frames: 5420 captured, 5340 sent, 80 dropped (1.48% drop rate)

Report Sections:

1. Client Statistics: Active client count and capabilities
2. Buffer Pool: Memory utilization and efficiency
3. Packet Queues: Queue performance and overflow incidents
4. Hash Table: Lookup performance and efficiency
5. Frame Processing: Frame generation and delivery metrics

Reporting Interval

Default Interval: 30 seconds

• Configurable via compile-time constants
• Interruptible sleep for responsive shutdown
• Adaptive to system load (may skip reports under heavy load)

Interval Tuning:

• Longer intervals: Lower overhead, less granular visibility
• Shorter intervals: Higher overhead, more granular visibility
• Default 30 seconds: Balanced visibility and overhead

Integration with Other Modules

Client Management Metrics

Monitored From:

• Client lifecycle statistics
• Per-client connection metadata
• Client activity patterns
• Thread status and health

Provides To:

• Client count and activity reports
• Connection duration metrics
• Client capability statistics

Buffer Pool Performance

Monitored From:

• Global buffer pool utilization
• Allocation/deallocation rates
• Memory efficiency metrics

Provides To:

• Memory usage reports
• Buffer pool performance analysis
• Memory leak detection indicators

Packet Queue Performance

Monitored From:

• Per-client packet queue depths
• Enqueue/dequeue rates
• Overflow incidents

Provides To:

• Queue performance reports
• Overflow detection
• Network congestion indicators

Hash Table Efficiency

Monitored From:

• Hash table efficiency
• Lookup performance
• Collision statistics

Provides To:

• Hash table performance reports
• Lookup efficiency metrics
• Table optimization indicators

Operational Visibility

Troubleshooting Support

Performance Bottleneck Identification:

• Buffer pool utilization identifies memory pressure
• Packet queue depths identify network congestion
• Frame drop rates identify processing bottlenecks
• Hash table collisions identify lookup inefficiency

System Health Monitoring:

• Client count trends indicate connection stability
• Buffer pool peak usage identifies memory leaks
• Packet queue overflow identifies network issues
• Frame processing rates identify performance degradation

Debug Output

Extensive Debug Logging:

• Thread startup/shutdown tracking
• Statistics collection reliability
• Shutdown detection timing
• Sleep/wake cycle behavior

Performance Profiling:

• Statistics collection overhead tracking
• Thread execution time monitoring
• Resource utilization trends

Error Handling

Statistics Collection Errors:

• Graceful degradation: Missing statistics logged but don't crash
• Thread-safe error handling: Errors in one collection don't affect others
• Detailed error logging: Troubleshooting information preserved

Thread Errors:

• Thread join timeouts: Logged but don't block shutdown
• Statistics collection failures: Logged but server continues
• Resource cleanup errors: Handled gracefully

Performance Impact

Minimal Overhead:

• Non-intrusive background monitoring
• Reader locks only (no blocking writes)
• Atomic snapshots (no long-held locks)
• Minimal CPU usage (<1% typical)

Responsive Shutdown:

• Interruptible sleep operations
• Frequent shutdown flag checks
• Clean thread exit on shutdown
• No blocking operations

Best Practices

DO:

• Use reader locks for statistics collection
• Take atomic snapshots of volatile data
• Check shutdown flags frequently
• Log detailed statistics for troubleshooting
• Handle errors gracefully

DON'T:

• Don't use write locks for statistics collection
• Don't hold locks during statistics collection
• Don't block on statistics collection
• Don't skip error handling
• Don't ignore shutdown flags

See also

src/server/stats.c

src/server/stats.h

Server Overview

Server Main Entry Point

topic_buffer_pool

topic_packet_queue

uthash library for hash table implementation

Stream Generation

Overview

The stream generation module creates personalized ASCII art frames for each connected client by collecting video from all active clients, creating composite layouts, and converting to ASCII using client-specific terminal capabilities. This module embodies the core innovation of ascii-chat: real-time multi-client video chat rendered entirely in ASCII art, optimized for terminal viewing with professional-grade performance.

Implementation: src/server/stream.c, src/server/stream.h

Key Responsibilities:

• Collect video frames from all active clients
• Create composite video layouts (single client, 2x2, 3x3 grids)
• Generate client-specific ASCII art with terminal capability awareness
• Process latest frames from double-buffer system for real-time performance
• Manage memory efficiently with buffer pools and zero-copy operations
• Support advanced rendering modes (half-block, color, custom palettes)

Video Mixing Architecture

Processing Pipeline

The mixing system operates in five distinct stages:

Stage 1: Frame Collection

// Scan all active clients for available video frames
for (int i = 0; i < MAX_CLIENTS; i++) {
  client_info_t *source_client = &g_client_manager.clients[i];
 
  // Skip inactive clients
  if (!source_client->active) continue;
 
  // Get latest frame from double-buffer system
  video_frame_t *frame = get_latest_frame(source_client);
 
  // Store frame for layout calculation
  sources[source_count++] = (image_source_t){
    .frame = frame,
    .aspect_ratio = (float)frame->width / (float)frame->height,
    .client_id = source_client->client_id
  };
}

Aggressive Frame Dropping:

• Always reads latest available frame (discards older frames)
• Prevents buffer overflow under load
• Maintains real-time performance like Zoom/Google Meet
• Logarithmic drop rate based on buffer occupancy

Stage 2: Layout Calculation

// Determine optimal grid layout based on active client count
int rows, cols;
calculate_optimal_grid_layout(sources, source_count, terminal_width,
                               terminal_height, &cols, &rows);
 
// Calculate cell dimensions with aspect ratio preservation
int cell_width = terminal_width / cols;
int cell_height = terminal_height / rows;

Layout Algorithm:

• Tries all grid configurations (1x1, 2x1, 2x2, 3x2, 3x3)
• Calculates total area utilization for each configuration
• Chooses configuration with highest area utilization
• Preserves aspect ratios in grid cells

Stage 3: Composite Generation

// Create composite image with proper dimensions
image_t *composite = image_create(terminal_width, terminal_height);
 
// Place each client's video in appropriate grid cell
for (int i = 0; i < source_count; i++) {
  int row = i / cols;
  int col = i % cols;
  int x = col * cell_width;
  int y = row * cell_height;
 
  // Scale and place frame in cell
  image_scale_and_place(source->frame, composite, x, y,
                        cell_width, cell_height);
}

Composite Features:

• Aspect ratio preservation in grid cells
• High-quality scaling (bilinear interpolation)
• Support for different grid sizes (1x1, 2x2, 3x3)
• Efficient memory usage via buffer pools

Stage 4: ASCII Conversion

// Convert composite to ASCII using client's terminal capabilities
ascii_frame_t *ascii_frame = image_to_ascii(composite, &client->capabilities,
                                             client->palette, client->color_mode);
 
// Apply client-specific rendering options
if (client->half_block_mode) {
  ascii_frame = apply_half_block_rendering(ascii_frame);
}
 
if (client->color_mode) {
  ascii_frame = apply_color_escape_sequences(ascii_frame,
                                              client->color_depth);
}

ASCII Conversion Features:

• Terminal capability awareness (color depth, UTF-8 support)
• Custom ASCII palettes (brightness-to-character mapping)
• Half-block rendering (2x vertical resolution)
• ANSI escape sequence generation for color output
• SIMD-optimized conversion for performance

Stage 5: Packet Generation

// Wrap ASCII frame in protocol packet
ascii_frame_packet_t *packet = create_ascii_frame_packet(ascii_frame);
 
// Queue packet for delivery via send thread
packet_queue_enqueue(client->video_packet_queue, packet, packet_size);

Packet Features:

• Protocol-compliant packet headers
• Frame metadata (dimensions, timestamp, checksum)
• Compression support for large frames
• Thread-safe queue operations

Per-Client Customization

Unlike traditional video mixing that generates one output, ascii-chat creates personalized frames for each client:

Terminal Capability Awareness

Color Depth:

• 1-bit (mono): Black and white ASCII only
• 8-color: Basic ANSI colors
• 16-color: Standard terminal colors
• 256-color: Extended palette
• 24-bit RGB: Full color support (when available)

Character Support:

• ASCII-only: Basic character set (portable)
• UTF-8: Box drawing characters for smoother frames
• Unicode: Special characters for enhanced rendering

Render Modes:

• Foreground color: Standard text coloring
• Background color: Block-based rendering
• Half-block: 2x vertical resolution using Unicode half-blocks

Custom Palettes:

• Brightness-to-character mapping
• Multiple predefined palettes (simple, extended, full)
• Custom palette support for client preferences

Frame Customization

Each client receives frames optimized for their terminal:

// Generate personalized frame for client
ascii_frame_t *frame = generate_personalized_ascii_frame(client_id);
 
// Frame is customized based on:
// - Client's terminal dimensions (width, height)
// - Client's color depth (1-bit, 8-color, 256-color, RGB)
// - Client's palette preference (simple, extended, full)
// - Client's UTF-8 support (ASCII-only vs box drawing)
// - Client's render mode (foreground, background, half-block)

Customization Benefits:

• Optimal quality for each client's terminal
• Efficient use of available terminal capabilities
• No wasted bandwidth on unsupported features
• Consistent rendering across diverse terminals

Performance Optimizations

Frame Buffer Management

Double-Buffer System:

• Each client uses double-buffer for smooth frame delivery
• Always provides latest available frame
• No frame caching complexity or stale data concerns
• Professional-grade real-time performance
• Memory managed via buffer pools

Buffer Overflow Handling: When clients send frames faster than processing:

• Aggressive frame dropping (keep only latest)
• Logarithmic drop rate based on buffer occupancy
• Maintains real-time performance under load
• Prevents buffer exhaustion

Memory Management

Buffer Pool Integration:

// All allocations use buffer pools
image_t *composite = image_create_from_pool(terminal_width, terminal_height);
 
// Automatic cleanup on return
buffer_pool_return(composite);

Zero-Copy Operations:

• Frame data is reference-counted (avoid copies)
• Double-buffer system eliminates need for frame copies
• Efficient memory usage via buffer pools
• Automatic cleanup on client disconnect

Memory Efficiency:

• Frame data is shared across clients where possible
• Buffer pool reduces malloc/free overhead
• Automatic cleanup prevents memory leaks
• Graceful degradation on allocation failures

Concurrency Optimizations

Reader Locks on Client Manager:

• Allows parallel frame generation across clients
• No blocking between client frame generation
• Minimal lock contention

Lock-Free Media Buffers:

• Audio buffers use lock-free ring buffers
• Video buffers use atomic operations for double-buffer
• Minimal synchronization overhead

Per-Client Isolation:

• Each client's frame generation is independent
• No shared state between frame generations
• Linear scaling performance

Integration with Other Modules

Integration with render.c

Called By:

• Video render threads call stream generation functions at 60fps
• generate_personalized_ascii_frame(): Main entry point for frame generation

Provides To:

• Personalized ASCII frames for each client
• Optimized frame generation per terminal
• Terminal capability awareness

Integration with client.c

Provides To:

• Client state access for frame generation
• Terminal capabilities from client state
• Frame queue for delivery

Uses From:

• Client video buffers (reads frames from all clients)
• Client terminal capabilities (for ASCII conversion)
• Client rendering preferences (palette, color mode)

Integration with protocol.c

Consumed By:

• Video frames stored by protocol handlers
• Frame data accessed by stream generation

Provides To:

• Frame generation uses stored video data
• Client capabilities set by protocol handlers

Integration with video/

Used By:

• ASCII conversion implementation (video/)
• SIMD-optimized conversion routines
• Palette management (palette.c)

Provides To:

• Composite images for ASCII conversion
• Terminal capabilities for conversion optimization
• Rendering preferences for palette selection

Grid Layout System

Supported Layout Types

Single Client (1x1):

• Full-screen video when only one client connected
• Maximum quality for single user
• No scaling overhead

Side-by-Side (2x1):

• Two clients side-by-side
• Equal width split
• Aspect ratio preserved in each half

Grid Layouts (2x2, 3x2, 3x3):

• Multiple clients in grid arrangement
• Optimal space utilization
• Aspect ratio preserved in each cell

Layout Calculation Algorithm

Algorithm:

1. Try all grid configurations from 1x1 to NxN
2. For each configuration, calculate total area utilization
3. Choose configuration with highest area utilization
4. Calculate cell dimensions with aspect ratio preservation

Area Utilization:

• Accounts for aspect ratio preservation
• Prefers larger frames over smaller frames
• Optimizes for terminal space usage
• Handles different source aspect ratios

ASCII Conversion

Conversion Process

RGB to Grayscale:

• Weighted conversion (red, green, blue weights)
• Configurable color weights for optimization
• SIMD-optimized for performance

Grayscale to ASCII:

• Brightness-to-character mapping
• Multiple palette options (simple, extended, full)
• Terminal capability awareness

Color Application:

• ANSI escape sequence generation
• Terminal color depth support
• Palette-based color selection

Rendering Modes

Foreground Color:

• Standard text coloring
• Efficient escape sequences
• Compatible with most terminals

Background Color:

• Block-based rendering
• Higher quality appearance
• Better color blending

Half-Block Rendering:

• 2x vertical resolution
• Uses Unicode half-block characters
• Requires UTF-8 support
• Professional-grade quality

Performance Optimizations

Linear Scaling:

• Each client gets dedicated render thread
• No shared bottlenecks between clients
• Performance scales linearly up to 9+ clients
• Real-time guarantees maintained per client

Frame Generation Speed:

• 60fps per client with precise timing
• SIMD-optimized ASCII conversion
• Efficient memory usage via buffer pools
• Minimal CPU overhead

Memory Usage:

• Per-client composite buffers (temporary)
• Frame data shared across clients
• Buffer pool reduces malloc/free overhead
• Automatic cleanup on client disconnect

Best Practices

DO:

• Always use latest available frame (aggressive frame dropping)
• Use buffer pools for all allocations
• Check client active status before operations
• Use snapshot pattern for client state access
• Optimize for terminal capabilities

DON'T:

• Don't cache frames unnecessarily
• Don't hold locks during frame generation
• Don't skip frame dropping under load
• Don't ignore terminal capabilities
• Don't allocate without buffer pools

See also

src/server/stream.c

src/server/stream.h

Server Overview

Render Threads

topic_video

topic_palette