The UTF-8 codec: one call per character, reports included.
The UTF-8 codec: one call per character, reports included.The decoder is one call, bj_utf8_next: a decode remembers nothing between characters, so every other way in, the scan and the validator included, is a loop over it. This example reads a view both ways, then meets the failures the codec reports instead of hiding: a byte that cannot start a character, a sequence cut off by the end of the view, and the maximal-subpart policy that keeps a valid character after garbage.
(void)user_data;
valid ? "valid " : "replaced ",
valid ? report->
value : 0xFFFDu);
}
int main(
int argc,
char* argv[]) {
(void)argc; (void)argv;
size_t cursor = 0;
size_t count = 0;
++count;
}
bj_info(
"The pull door read %zu characters", count);
bj_info(
"The push door, same view:");
bj_info(
" %zu reports", report_count);
static const char bytes[] =
"\xC3\xA9\xE0\x80\x41";
bj_info(
"A damaged view, E9 then E0 80 41:");
size_t first_error = 0;
bj_info(
"The view is not valid UTF-8: first error at byte %zu",
first_error);
}
char encoded[4];
bj_info(
"U+00E9 writes %zu bytes, a surrogate writes %zu",
written_e9, written_surrogate);
return 0;
}
uint32_t bj_bool
Boolean type used throughout the Banjo API.
#define BJ_TRUE
Boolean true value (1).
#define bj_info(...)
Log a message using the BJ_LOG_INFO level.
struct bj_str bj_str_from_cstr(const char *c_string)
Create a view from a NUL-terminated C string.
A borrowed view of a sequence of bytes.
uint32_t value
Decoded code point, or the partial value for an error.
enum bj_utf8_state state
Result of the decoding attempt.
const char * bytes
The bytes covered by the report, borrowed from the view.
size_t n_bytes
Number of bytes covered by the report.
size_t byte_offset
Offset of the first byte in the view.
size_t bj_utf8_scan(struct bj_str view, bj_utf8_fn callback, void *user_data)
Process every UTF-8 character or decoding error in a view.
bj_bool bj_utf8_validate(struct bj_str view, size_t *first_error)
Check whether a view contains only valid UTF-8.
size_t bj_utf8_encode(uint32_t value, char bytes[4])
Encode a Unicode code point as UTF-8.
bj_bool bj_utf8_next(struct bj_str view, size_t *cursor, struct bj_utf8_report *report)
Read one UTF-8 character or decoding error from a view.
@ BJ_UTF8_STATE_VALID
A valid UTF-8 character was decoded.
Details about one UTF-8 decoding attempt.
Logging utility functions.
Portable main() replacement with platform-aware entry shim.
UTF-8 string views and owning strings.
Read, write and validate UTF-8 characters.
static bj_bool print_report(const struct bj_utf8_report *report, void *user_data)