📬 Thread-safe per-client packet queues More...

Files
file	packet_queue.c
	📬 Lock-free packet queue with per-client isolation and memory pooling

Detailed Description

📬 Thread-safe per-client packet queues

Packet Queue

Overview

Welcome! Let's talk about packet queues—the unsung heroes that keep ascii-chat's network communication smooth and organized.

Imagine you're at a busy restaurant. Orders are coming in from multiple tables, and the kitchen needs to process them one at a time. You don't want orders getting mixed up or lost, right? That's exactly what packet queues do for network data—they line up incoming packets so each client's data gets processed in order, without chaos.

Each client connected to the server gets their own dedicated packet queue. When packets arrive over the network, they get placed in the appropriate client's queue. Then, a separate thread can process packets at its own pace without blocking the network thread. It's like having a separate order ticket for each table—clean, organized, and efficient.

Implementation: lib/packet_queue.c/h

What makes packet queues useful?

Per-client queues: Each client has their own queue (no cross-contamination!)
Thread-safe: Multiple threads can safely interact with the queue
Flexible ownership: Choose whether the queue copies data or just references it
Buffer pool integration: Works with our buffer pool for efficient memory usage
Statistics tracking: See how well your queues are performing

Architecture

Queue Structure

// Single packet node in the queue
typedef struct packet_node {
    packet_header_t header;   // Type, length, CRC, client_id
    void *data;               // Payload
    struct packet_node *next; // Next packet in queue
} packet_node_t;
 
// Per-client packet queue
typedef struct packet_queue {
    packet_node_t *head;   // Front of queue (dequeue here)
    packet_node_t *tail;   // Back of queue (enqueue here)
    size_t size;           // Current packet count
    size_t max_size;       // Maximum allowed packets (0 = unlimited)
    bool owns_data;        // If true, queue frees packet data
    mutex_t mutex;         // Thread safety
 
    // Statistics
    uint64_t packets_enqueued;
    uint64_t packets_dequeued;
    uint64_t packets_dropped;  // Dropped due to max_size limit
    size_t peak_size;          // Peak queue depth
} packet_queue_t;

Data Ownership

Queues can either copy packet data or reference it:

Deep copy mode (owns_data = true):

Queue allocates its own copy of packet data
Safe for producer to free original data immediately
Queue frees data when packet dequeued
Higher memory usage, but safer

Reference mode (owns_data = false):

Queue stores pointer to original data
Producer must keep data alive until dequeued
Consumer responsible for freeing data
Lower memory usage, but requires careful lifetime management

// Deep copy mode (safer) - copy_data=true
packet_queue_t *q = packet_queue_create(100);
 
char *data = SAFE_MALLOC(1024, char*);
strcpy(data, "Hello");
packet_queue_enqueue(q, PACKET_TYPE_ASCII_FRAME, data, strlen(data), client_id, true);
SAFE_FREE(data);  // Can free immediately since copy_data=true!
 
// Reference mode (lower overhead) - copy_data=false
packet_queue_t *q2 = packet_queue_create(100);
 
char *data2 = SAFE_MALLOC(1024, char*);
strcpy(data2, "Hello");
packet_queue_enqueue(q2, PACKET_TYPE_ASCII_FRAME, data2, strlen(data2), client_id, false);
// data2 must stay alive until dequeued since copy_data=false!
 
// Dequeue returns a queued_packet_t pointer
queued_packet_t *packet = packet_queue_dequeue(q2);
if (packet) {
    process_packet(packet->data);
    packet_queue_free_packet(q2, packet);  // Free packet properly
}

API Reference

Creation/Destruction

// Create packet queue
packet_queue_t *queue = packet_queue_create(100, true);
if (!queue) {
    log_error("Failed to create packet queue");
    return NULL;
}
 
// Use queue...
 
// Destroy queue
packet_queue_destroy(queue);

Integration

Buffer Pool Integration:

Packet queues use buffer pool for efficient memory allocation
Node pool pre-allocates packet nodes
Data pool pre-allocates packet data buffers
Reduces malloc/free overhead in high-throughput scenarios

Network Integration:

Each client has a dedicated packet queue
Network receive thread enqueues packets
Client processing thread dequeues packets
Decouples network I/O from packet processing

Performance

Throughput:

Enqueue: ~1M packets/second (mutex-protected)
Dequeue: ~1M packets/second (mutex-protected)
Memory overhead: ~40 bytes per queued packet

Scalability:

Per-client queues eliminate contention between clients
Statistics tracking has minimal overhead
Buffer pool reduces allocation overhead

Thread Safety

Mutex Protection:

All queue operations are mutex-protected
Thread-safe for concurrent enqueue/dequeue
No lock-free optimizations (simplicity over speed)

Concurrent Access:

Multiple threads can enqueue concurrently
Only one thread should dequeue (per queue)
Statistics are updated atomically

See also: packet_queue.h; buffer_pool.h

Files