utf8.h File Reference

🔤 UTF-8 Encoding and Decoding Utilities More...

Go to the source code of this file.

Functions
int	utf8_decode (const uint8_t *s, uint32_t *codepoint)
	Decode a UTF-8 sequence to a Unicode codepoint.
int	utf8_display_width (const char *str)
	Calculate terminal display width of a UTF-8 string.
int	utf8_display_width_n (const char *str, size_t max_bytes)
	Calculate terminal display width of a UTF-8 string up to byte limit.
size_t	utf8_prefix_bytes_for_width (const char *str, size_t max_bytes, int max_width)
	Return the byte length of a prefix fitting a terminal column limit.
bool	utf8_output_enabled (void)
	Return whether terminal output should use UTF-8 according to –utf8.
bool	utf8_is_valid (const char *str)
	Check if a string is valid UTF-8.
bool	utf8_is_ascii_only (const char *str)
	Check if a string contains only ASCII characters.
size_t	utf8_char_count (const char *str)
	Count UTF-8 characters (not bytes).
size_t	utf8_to_codepoints (const char *str, uint32_t *out_codepoints, size_t max_codepoints)
	Convert UTF-8 string to array of Unicode codepoints.
int	utf8_next_char_bytes (const char *str, size_t max_bytes)
	Get byte length of next UTF-8 character.
const char *	utf8_strcasestr (const char *haystack, const char *needle)
	Case-insensitive substring search with full Unicode support.
void	utf8_write_lowercase (FILE *output, const char *text)
	Write UTF-8 string to file with ASCII lowercase conversion.

Detailed Description

🔤 UTF-8 Encoding and Decoding Utilities

This header provides simple, efficient UTF-8 validation and decoding without external dependencies. The implementation handles multi-byte UTF-8 sequences and validates encoding correctness.

CORE FEATURES:

• Multi-byte UTF-8 sequence decoding (1-4 bytes)
• Unicode codepoint extraction
• UTF-8 validation during decoding
• No external dependencies (pure C implementation)
• Safe handling of invalid sequences

UTF-8 ENCODING:

UTF-8 encodes Unicode codepoints using 1-4 bytes:

• 1 byte: ASCII characters (0x00-0x7F)
• 2 bytes: Latin-1 supplement, etc. (0x80-0x7FF)
• 3 bytes: Most CJK characters (0x800-0xFFFF)
• 4 bytes: Rare characters, emoji (0x10000-0x10FFFF)

VALIDATION:

The decoder validates UTF-8 sequences during decoding:

• Checks for valid byte sequences according to UTF-8 rules
• Detects overlong encodings (security feature)
• Detects invalid byte patterns
• Returns error on invalid sequences

Note

This is a minimal implementation for basic UTF-8 handling.

For full Unicode support, consider using a library like ICU.

Invalid sequences return -1 to indicate error.

Author

Zachary Fogg me@zf.nosp@m.o.gg

Date

October 2025

Definition in file utf8.h.

Function Documentation

utf8_prefix_bytes_for_width()

size_t utf8_prefix_bytes_for_width	(	const char *	str,
		size_t	max_bytes,
		int	max_width )

Return the byte length of a prefix fitting a terminal column limit.

Parameters

str	Input string or bounded slice (may be NULL)
max_bytes	Maximum readable bytes
max_width	Maximum terminal columns; negative values return zero

Returns

Prefix length in bytes, without modifying the input

Stops at NUL, invalid/incomplete UTF-8, or an incomplete/unsupported escape. Complete ANSI CSI sequences consume zero columns. Codepoints are never split; combining marks and trailing CSI sequences are included even at the width limit. Widths follow utf8proc_charwidth(), not grapheme-cluster or terminal-specific emoji shaping. A zero-column limit may include leading zero-width tokens.

Definition at line 93 of file utf8.c.

                                                                                     {
  int width;
  return utf8_prefix_scan(str, max_bytes, max_width, &width);
}

References width.

Referenced by ascii_create_grid(), and truncate_with_ellipsis_mode().