Network

🌐 Network I/O, packet protocol, and audio/video streaming More...

Modules
	Compression Module
	📦 Network compression utilities for ascii-chat
	Packet Types
	📨 Network protocol packet type definitions and structures

Detailed Description

🌐 Network I/O, packet protocol, and audio/video streaming

Network operations, packet types, and audio/video networking.

Audio/Video Networking

Overview

The Audio/Video Networking module provides the network protocol implementation for audio, video, and ASCII frame packets in ascii-chat. The system handles packet serialization, compression, encryption integration, and message formatting for real-time media streaming over TCP.

Implementation: lib/network/av.h

Key Features:

• ASCII frame packet transmission (text-based video frames)
• Image frame packet transmission (raw pixel data)
• Audio packet transmission (single and batched)
• Protocol message handling (size, audio, text)
• Compression integration for large packets
• Cryptographic encryption support
• Error handling and validation

Packet Types

The system handles multiple packet types:

Frame Packets:

• ASCII frames: Text-based video frames for terminal display
• Image frames: Raw pixel data (RGB, RGBA, BGR, etc.)

Audio Packets:

• Single audio packets: Legacy single-packet format (PACKET_TYPE_AUDIO)
• Audio batch packets: Batched audio samples (PACKET_TYPE_AUDIO_BATCH) - efficient

Message Packets:

• Size messages: Terminal dimension notifications (PACKET_TYPE_SIZE_MESSAGE)
• Audio messages: Audio metadata announcements (PACKET_TYPE_AUDIO_MESSAGE)
• Text messages: Text chat messages (PACKET_TYPE_TEXT_MESSAGE)

Operations

Frame Operations

ASCII Frame Transmission:

// Send ASCII frame with compression support
int result = av_send_ascii_frame(sockfd, frame_data, frame_size);
if (result < 0) {
    log_error("Failed to send ASCII frame");
}

Image Frame Transmission:

// Send image frame with compression support
int result = av_send_image_frame(sockfd, image_data, width, height, format);
if (result < 0) {
    log_error("Failed to send image frame");
}

Audio Operations

Single Audio Packet (Legacy):

// Send single audio packet (legacy - less efficient)
int result = av_send_audio(sockfd, samples, num_samples);
if (result < 0) {
    log_error("Failed to send audio packet");
}

Audio Batch Packet (Efficient):

// Send batched audio with encryption support
int result = send_audio_batch_packet(sockfd, samples, total_samples,
                                      batch_count, crypto_ctx);
if (result < 0) {
    log_error("Failed to send audio batch");
}
 
// Or use convenience function without encryption
result = av_send_audio_batch(sockfd, samples, total_samples, sample_rate);

Benefits of Batching:

• Reduced packet overhead (75% fewer packets)
• Better bandwidth efficiency (80% reduction in overhead)
• At 44.1 kHz: 47 packets/second (was 188)

Message Operations

Terminal Size Message:

// Send terminal size notification
int result = av_send_size_message(sockfd, width, height);
if (result < 0) {
    log_error("Failed to send size message");
}

Audio Message:

// Send audio metadata announcement
int result = av_send_audio_message(sockfd, num_samples);
if (result < 0) {
    log_error("Failed to send audio message");
}

Text Message:

// Send text chat message
int result = av_send_text_message(sockfd, "Hello, world!");
if (result < 0) {
    log_error("Failed to send text message");
}

Message Parsing:

// Parse size message string
unsigned short width, height;
int result = av_parse_size_message("SIZE:160,45\n", &width, &height);
if (result < 0) {
    log_error("Failed to parse size message");
}

Protocol Integration

The Audio/Video Networking module integrates with:

Packet System:

• network/packet.h: Packet header system for framing
• network/packet_types.h: Packet type definitions

Compression:

• compression.h: Automatic compression for large packets
• Compression threshold-based decision making
• Decompression on receive path

Cryptography:

• crypto/crypto.h: Encryption/decryption support
• Automatic encryption when crypto context provided
• Encryption policy enforcement

Network Layer:

• network/network.h: Socket I/O operations
• platform/socket.h: Platform socket abstraction

Message Formats

Protocol messages use text-based formats for compatibility:

Size Message:

• Format: "SIZE:width,height\n"
• Example: "SIZE:160,45\n"
• Prefix: SIZE_MESSAGE_PREFIX ("SIZE:")
• Format string: SIZE_MESSAGE_FORMAT ("SIZE:%u,%u\n")

Audio Message:

• Format: "AUDIO:num_samples\n"
• Example: "AUDIO:1024\n"
• Prefix: AUDIO_MESSAGE_PREFIX ("AUDIO:")
• Format string: AUDIO_MESSAGE_FORMAT ("AUDIO:%u\n")

Text Message:

• Format: Plain text with packet headers
• No special formatting required

Compression

Large packets automatically support compression:

• ASCII frames: Compressed if size exceeds threshold
• Image frames: Compressed for large images
• Audio batches: Compressed for large audio data

Compression Decision:

• Size threshold-based (e.g., 100KB)
• Compression ratio check (only if >80% size reduction)
• Automatic compression/decompression

Encryption

All packet functions handle encryption when crypto context is provided:

Automatic Encryption:

// Send encrypted audio batch
send_audio_batch_packet(sockfd, samples, total_samples,
                        batch_count, crypto_ctx);

Encryption Policy:

• Handshake packets: Always unencrypted (plaintext)
• Session packets: Encrypted when crypto context provided
• Encryption enforcement: Optional or required based on policy

Error Handling

All functions return error codes:

• 0: Success
• -1: Error (check errno or use network_error_string())

Error Reporting:

if (av_send_ascii_frame(sockfd, frame_data, frame_size) < 0) {
    log_error("Frame send failed: %s", network_error_string());
}

Performance

Batched Audio Packets:

• 75% reduction in packet count (4 chunks per packet)
• 80% reduction in packet overhead
• At 44.1 kHz: 47 packets/second (was 188)
• Packet overhead: 846 bytes/sec (was 3.4 KB/sec)

Compression Benefits:

• Reduces bandwidth for large frames
• Automatic compression threshold-based decision
• Only compresses if beneficial (>80% size reduction)

Message Format:

• Text-based for compatibility
• Simple parsing (no complex binary formats)
• Maximum message length: 32 bytes (SIZE_MESSAGE_MAX_LEN, AUDIO_MESSAGE_MAX_LEN)

Best Practices

1. Use batched audio for efficiency:

   // Prefer batched audio
   av_send_audio_batch(sockfd, samples, total_samples, sample_rate);
   // Avoid single-packet audio (legacy)
   // av_send_audio(sockfd, samples, num_samples);

2. Handle compression automatically:

   // Compression is automatic - no manual intervention needed
   av_send_ascii_frame(sockfd, frame_data, frame_size);

3. Use encryption when available:

   // Use encryption context when available
   send_audio_batch_packet(sockfd, samples, total_samples,
                           batch_count, crypto_ctx);

4. Validate message formats:

   // Parse messages with validation
   if (av_parse_size_message(message, &width, &height) < 0) {
       log_error("Invalid size message format");
   }

5. Handle errors gracefully:

   if (av_send_ascii_frame(sockfd, frame_data, frame_size) < 0) {
       // Handle error - connection may be lost
       handle_network_error();
   }

Packet Details

ASCII Frame Packet (PACKET_TYPE_ASCII_FRAME):

• Contains ASCII frame data with metadata
• Metadata: width, height, compression status, checksum
• Payload: ASCII frame string (may be compressed)

Image Frame Packet (PACKET_TYPE_IMAGE_FRAME):

• Contains raw pixel data with metadata
• Metadata: dimensions, format, compression status, checksum, timestamp
• Payload: Pixel data (RGB, RGBA, BGR, etc.) - may be compressed

Audio Batch Packet (PACKET_TYPE_AUDIO_BATCH):

• Contains multiple audio chunks aggregated
• Metadata: batch count, total samples, sample rate, channels
• Payload: Float samples array (interleaved if stereo)

Message Packets:

• Size message: Terminal dimension notification
• Audio message: Audio metadata announcement
• Text message: Plain text chat message

See also

network/av.h

network/packet.h

network/packet_types.h

Network README

Overview

Welcome to the Network I/O module! This is where all the network communication happens—everything from basic socket operations to packet transmission. The network module is designed for real-time video streaming, which means it's optimized for low latency, reliability, and handling large data transfers smoothly.

What does the network module do?

The Network I/O module provides fundamental network operations that everything else builds on. Think of it as the foundation for all network communication. Here's what it handles:

• Socket I/O operations with timeout support (so operations don't hang forever)
• Connection management (connect, accept) with timeouts (critical for reliable connections)
• Socket configuration (keepalive, non-blocking mode, timeouts) to keep connections healthy
• Chunked transmission for large data transfers (so big video frames don't block the connection)
• Packet protocol with encryption and compression support (built on top of the basic I/O)
• Error reporting and diagnostics (so you know what went wrong when things break)

All operations support configurable timeouts, which is critical for real-time video streaming—we can't afford to wait forever when something goes wrong. The timeout system ensures that operations fail fast rather than hanging indefinitely.

Implementation: lib/network/network.h, lib/network/packet.h

Architecture

The network module is built around a few key concepts: timeouts to prevent hanging, socket configuration to keep connections healthy, and a packet protocol that handles the complexity of reliable data transmission. Let's walk through how everything fits together.

Timeout System

All I/O operations support configurable timeouts—this is probably the most important feature for real-time video streaming. Without timeouts, operations can hang forever when the network is flaky, which would freeze your video stream.

What timeouts do we have?

• Connection timeout: 3 seconds (CONNECT_TIMEOUT)—when you're trying to connect to a server, we'll give up after 3 seconds rather than waiting forever
• Send timeout: 5 seconds (SEND_TIMEOUT)—when sending data, we'll give up after 5 seconds if it's not going through
• Receive timeout: 15 seconds (RECV_TIMEOUT)—when receiving data, we'll wait up to 15 seconds (this is longer because network delays are more common on receive)
• Accept timeout: 3 seconds (ACCEPT_TIMEOUT)—when waiting for incoming connections, we'll check every 3 seconds rather than blocking forever

Why these values?

These timeouts are tuned specifically for real-time video streaming. They're short enough that you notice problems quickly, but long enough that normal network hiccups don't cause false positives. Chunked transmission prevents large data from blocking indefinitely—even if a single chunk times out, we can still make progress on the rest.

Socket Configuration

Socket configuration is about keeping your connections healthy and efficient. We configure sockets with several important settings:

What socket options do we set?

• TCP keepalive: This detects dead connections automatically—if the connection has been idle for a while, keepalive probes check if it's still alive. If the other end doesn't respond, we know the connection is dead and can clean it up. We use KEEPALIVE_IDLE, KEEPALIVE_INTERVAL, and KEEPALIVE_COUNT to tune this.
• Non-blocking mode: This enables asynchronous I/O patterns—instead of blocking and waiting for data, we can check if data is available and do other work while waiting
• Socket-level timeouts: We set SO_SNDTIMEO and SO_RCVTIMEO at the socket level, so even if the application doesn't explicitly check timeouts, the socket itself will time out
• Platform abstraction: All of this works cross-platform (Linux, macOS, Windows) through our platform abstraction layer

Packet Protocol

The packet protocol sits on top of the basic socket I/O and adds structure, validation, encryption, and compression. It's how we reliably transmit data even when the network is unreliable.

What does the packet protocol do?

The packet protocol provides:

• Packet header validation: Every packet starts with a magic number and CRC32 checksum, so we can detect corrupted packets immediately
• Secure packet transmission: Packets can be encrypted using the crypto context (automatic encryption/decryption)
• Secure packet reception: Packets are automatically decrypted and validated when received
• Automatic compression: Large packets are automatically compressed to save bandwidth
• Protocol compliance checking: We verify that packets follow the protocol specification, so malformed packets are rejected early

How are packets structured?

Packet Structure:

• Header: Contains a magic number (so we know it's really our packet), type (what kind of packet it is), length (how much data follows), CRC32 (to detect corruption), and client ID (which client this is from)
• Payload: The actual data, which can be encrypted and/or compressed
• Validation: The CRC32 checksum in the header lets us verify packet integrity before we even try to decrypt or decompress

This structure makes it easy to detect problems early—if the magic number is wrong or the CRC32 doesn't match, we know the packet is corrupted before we waste time trying to decrypt it.

Operations

Socket I/O Operations

Send/Receive Operations:

// Send data with timeout
ssize_t sent = send_with_timeout(sockfd, data, len, SEND_TIMEOUT);
if (sent < 0) {
    log_error("Send failed: %s", network_error_string());
}
 
// Receive data with timeout
ssize_t received = recv_with_timeout(sockfd, buffer, len, RECV_TIMEOUT);
if (received < 0) {
    log_error("Receive failed: %s", network_error_string());
}

Connection Operations:

// Connect to server with timeout
if (!connect_with_timeout(sockfd, &addr, addrlen, CONNECT_TIMEOUT)) {
    log_error("Connection failed: %s", network_error_string());
}
 
// Accept connection with timeout
int client_fd = accept_with_timeout(listenfd, NULL, NULL, ACCEPT_TIMEOUT);
if (client_fd < 0) {
    log_error("Accept failed: %s", network_error_string());
}

Packet Operations

Basic Packet I/O:

// Send a packet
asciichat_error_t err = packet_send(sockfd, PACKET_TYPE_PING, NULL, 0);
if (err != ASCIICHAT_OK) {
    log_error("Packet send failed");
}
 
// Receive a packet
packet_type_t type;
void *data;
size_t len;
err = packet_receive(sockfd, &type, &data, &len);
if (err == ASCIICHAT_OK) {
    // Process packet
    process_packet(type, data, len);
    free(data);  // Free allocated buffer
}

Secure Packet I/O:

// Send encrypted packet
int result = send_packet_secure(sockfd, PACKET_TYPE_ASCII_FRAME,
                                 frame_data, frame_size, crypto_ctx);
if (result < 0) {
    log_error("Secure send failed");
}
 
// Receive and decrypt packet
packet_envelope_t envelope;
packet_recv_result_t result = receive_packet_secure(sockfd, crypto_ctx,
                                                     true, &envelope);
if (result == PACKET_RECV_SUCCESS) {
    // Process decrypted packet
    process_packet(envelope.type, envelope.data, envelope.len);
    free(envelope.allocated_buffer);  // Free allocated buffer
}

Socket Configuration

Socket Setup:

// Set socket timeout
if (set_socket_timeout(sockfd, SEND_TIMEOUT) < 0) {
    log_error("Failed to set socket timeout");
}
 
// Enable TCP keepalive
if (set_socket_keepalive(sockfd) < 0) {
    log_error("Failed to enable keepalive");
}
 
// Set non-blocking mode
if (set_socket_nonblocking(sockfd) < 0) {
    log_error("Failed to set non-blocking mode");
}

Integration

The Network I/O module integrates with:

• platform/socket.h: Uses platform socket abstraction
• network/packet_types.h: Packet type definitions
• crypto/crypto.h: Encryption/decryption operations
• compression.h: Compression/decompression support
• network/av.h: Media packet sending
• crypto/handshake.h: Crypto handshake operations

Performance

Chunked Transmission:

• Enables progressive sending of large frames
• Prevents timeout issues with large data
• Allows progress tracking

Timeout Handling:

• Prevents indefinite blocking
• Tuned for real-time video streaming
• Platform-specific optimizations (select/poll/epoll)

Keepalive Settings:

• Efficiently detects dead connections
• No application-level heartbeats required
• Configurable idle time and probe count

Error Handling

Error Reporting:

// Get human-readable error string
const char *error_msg = network_error_string();
log_error("Network error: %s", error_msg);

Error Codes:

• Function return values: -1 indicates error
• asciichat_error_t codes: ASCIICHAT_OK on success, error codes on failure
• packet_recv_result_t codes: PACKET_RECV_SUCCESS, PACKET_RECV_EOF, etc.

Timeout Configuration

Timeout configuration is all about balance—too short and you get false positives on slow networks, too long and you delay error detection unnecessarily. We've tuned these values specifically for real-time video streaming.

What timeouts do we use?

• CONNECT_TIMEOUT (3 seconds): Reduced from the default for faster connection attempts. If a connection doesn't work, we want to know quickly so we can try again or report the error
• SEND_TIMEOUT (5 seconds): This gives enough time for timely delivery of video frames. It's long enough that network hiccups don't cause false positives, but short enough that stuck sends are detected quickly
• RECV_TIMEOUT (15 seconds): This is longer than send timeout because network delays are more common on receive. We want to give the network time to deliver data, but not so much time that we hang forever
• ACCEPT_TIMEOUT (3 seconds): This is balanced between responsiveness (we don't want to waste CPU checking too often) and quick detection of problems

⚠️ Important timeout notes:

Timeout values that are too short may cause false positives on slow networks—you might think the connection is dead when it's just slow. Values that are too long may delay error detection unnecessarily—you might wait too long before realizing something is wrong.

If you're running on a very slow network (like satellite internet), you might want to increase these timeouts. If you're running on a very fast network (like a local LAN), you could decrease them for faster error detection.

Chunked Transmission

Large data transfers are automatically chunked—this is how we send big video frames without blocking the connection. Instead of sending everything at once (which could cause timeouts or block other operations), we break large data into smaller chunks.

How does chunked transmission work?

• Prevents timeout issues: Large frames can be several megabytes—if we try to send it all at once, we might timeout before it finishes. Chunking lets us send the frame in pieces, so each piece can complete quickly
• Enables progress tracking: We can report progress as chunks are sent, so you know things are working even when sending large frames
• Handles partial sends automatically: Sometimes send() doesn't send everything you ask it to—chunking handles this gracefully by retrying until the chunk is sent
• Retries until complete: If a chunk fails to send, we retry until all data is sent or the timeout occurs

This is used automatically for large data—you don't need to do anything special. The system detects when data is large enough to benefit from chunking and handles it automatically.

Packet Security

The packet protocol supports:

• Encryption: Automatic encryption/decryption when crypto context provided
• Compression: Automatic compression for large packets
• Validation: Magic number and CRC32 checksum verification
• Policy Enforcement: Encryption requirement enforcement

Handshake Packets:

• Always sent unencrypted (plaintext)
• Use packet_is_handshake_type() to check before encryption
• Required for crypto handshake establishment

Best Practices

1. Always check return values:

   if (send_with_timeout(sockfd, data, len, timeout) < 0) {
       // Handle error
   }

2. Free allocated packet buffers:

   void *data;
   size_t len;
   packet_receive(sockfd, &type, &data, &len);
   // ... use data ...
   free(data);  // Always free

3. Use secure packet functions for encryption:

   send_packet_secure(sockfd, type, data, len, crypto_ctx);
   receive_packet_secure(sockfd, crypto_ctx, true, &envelope);

4. Enable keepalive for long-lived connections:

   set_socket_keepalive(sockfd);

5. Configure timeouts appropriately:

   set_socket_timeout(sockfd, SEND_TIMEOUT);

See also

network/network.h

network/packet.h

network/packet_types.h