Python and JSON Unicode: Keep International Text Intact

Published September 5, 2026 · Reviewed by the json2py editorial team

JSON text is Unicode, and Python 3 strings are Unicode too. In ordinary cases that makes international names, accents and emoji straightforward to handle. Problems usually appear at the boundaries: when reading a file with the wrong encoding, when an API declares the wrong character set or when escaped output is mistaken for damaged text.

Use UTF-8 deliberately at file boundaries

When opening a JSON file, specify encoding="utf-8" unless you have reliable information that the source uses a different encoding. UTF-8 is the standard choice for modern JSON interchange. Specifying it makes behavior consistent across computers instead of depending on a local operating-system default.

Escaped characters are still valid JSON

Python's json.dumps may represent non-ASCII characters as sequences such as \\u00e3 when ensure_ascii is left at its default. That output remains valid JSON and round-trips correctly. Set ensure_ascii=False when people need to read the resulting UTF-8 text directly and your destination accepts UTF-8.

Do not encode strings twice

A frequent bug is converting text to bytes too early, then trying to serialize those bytes as JSON. Keep values as normal Python strings until a network library or file writer needs encoded bytes. The library that sends an HTTP request should normally handle the final UTF-8 encoding when given structured JSON data.

Check headers and logs

For an HTTP response, the declared content type and character set help explain what the client should decode. If text looks corrupted, log a harmless sample using a representation that makes code points visible, then compare the raw bytes and declared encoding. Never place confidential payloads in diagnostic logs.

Test with real characters

An automated test that uses only ASCII cannot reveal an encoding regression. Include a small, non-sensitive sample with accents, a non-Latin character and an emoji where those are valid for your application. Confirm that the original and decoded JSON values compare equal.