Convert Big5 files to UTF-8

Where Big5 files still come from

• Product lists, order exports and invoices from suppliers in Taiwan or Hong Kong – Excel on a Traditional Chinese Windows saves a plain CSV (comma delimited) in Big5.
• Files saved as “ANSI” in Notepad on a Traditional Chinese Windows. Before Windows 10 version 1903 that was the default.
• Exports from banks, tax and accounting software that have to go into a system that only accepts UTF-8.
• Old websites, BBS archives, Chinese comments in source code and subtitle files (.srt, .ass) that show garbage on a TV – see fix subtitle encoding.

¥x¥_¥««H¸q°Ï ¹q¸Ü
¸ê®Æ½s½X ÀÉ®×

台北市信義區 電話
資料編碼 檔案

How to convert

1. Drag the file into the box above or choose it – CSV, TXT, SRT, JSON and other text formats work.
2. Mojibuster reads the file as Big5, Big5-HKSCS, GBK and UTF-8 and keeps the reading that produces real Chinese. The result shows which code page it used.
3. Wrong guess? Click Read as … in the result to re-read the file in another code page.
4. Click Download to save the UTF-8 file. For CSV files keep Save with BOM on so Excel opens them correctly.

How to recognize Big5 at a glance

• 中 is A4 A4 in Big5 and shows as ¤¤ in a Western program; in UTF-8 it is E4 B8 AD.
• 文 is A4 E5 and shows as ¤å.
• 你好 is A7 41 A6 6E and shows as §A¦n.

Symbols like ¤, § and ¦ mixed with Latin letters almost always mean Big5 text displayed as Windows-1252. The bytes are all still there, so the file converts without loss. Pairs of accented capitals like ÖÐÎÄ point to GBK instead, and 中文 is the opposite case: UTF-8 read as Windows-1252 – see what mojibake is.

Big5, CP950 and Big5-HKSCS

Traditional Chinese Windows uses CP950: Big5 plus a few Microsoft additions. Standard Big5 has about 13,000 characters and lacks Cantonese characters such as 嘅, 咗 and 嘢; Hong Kong’s Big5-HKSCS adds them. Mojibuster decodes with the browser’s Big5 table, which includes HKSCS, so these characters convert correctly too.

The backslash problem (許功蓋)

許 is B3 5C in Big5 – and 5C is the backslash \. Old programs and databases treated it as an escape character, so a \ appeared or the next character vanished; 功, 蓋, 餐 and 珮 are affected the same way. In UTF-8 no byte of a Chinese character overlaps with \, so the problem disappears after converting.

Converting on the command line

For many files or a script, standard tools work. On Linux, macOS and Git Bash: iconv -f BIG5 -t UTF-8 in.txt > out.txt; use -f BIG5-HKSCS for files with Hong Kong characters and -f CP950 for Microsoft’s additions.

In PowerShell 7: Get-Content in.txt -Encoding 950 | Set-Content out.txt -Encoding utf8. Windows PowerShell 5.1 only knows Default, which is the system code page – Big5 only on a Traditional Chinese Windows. More in fix PowerShell encoding. These commands don’t guess the code page; pick the wrong one and you get new garbage. Mojibuster checks the options for you.

What can’t be restored

Characters that have already become ? are gone: the information was lost when the file was saved. Characters outside Big5 – some rare characters in names – became ? the moment the file was saved as Big5. Characters from the Big5-UAO extension used on PTT are not in the browser’s Big5 and may come out as different characters. Mojibuster shows how many ? the file contains; why they appear is explained under question marks instead of characters.

Frequently asked questions

More guides

• Mojibake fixer for all kinds of text Repair garbled characters in text and files.
• Fix CSV encoding Repair special characters in CSV exports and save as UTF-8.
• Excel characters When Excel shows é instead of é after opening a CSV.
• Fix UTF-8 Remove mojibake from databases, emails and websites.
• Fix subtitles Repair SRT and VTT subtitles with broken characters.
• What is mojibake? Meaning, origin and causes of garbled text.
• Double-encoded When é first became é and then é.
• é instead of é Table of é, ñ, ü and more – and what’s behind them.
• Apostrophes & quotes ’, “, – and … instead of apostrophes, quotes and dashes.
• ANSI → UTF-8 Convert old ANSI files to UTF-8 without losing accented characters.
• Question marks instead of characters ? or � instead of letters – what can be rescued and what can’t.
• ‚Äô instead of ’ Mojibake from a Mac: ’, é – and é from the Windows console.
• URL-encoded characters %C3%A9 instead of é – make percent codes from links and forms readable.
• WordPress & database Repair SQL dumps and WordPress content showing é instead of é.
• MySQL & phpMyAdmin Set the charset of table, connection and import correctly.
• Outlook Fix broken characters in e-mails and contact exports.
• PowerShell -Encoding, console and script files in PowerShell 5.1 and 7.
• Python & pandas encoding in read_csv, to_csv, open() and requests.
• Notepad++ Choose or convert the encoding – the crucial difference.
• Customer data & CRM Repair customer lists from CRM and ERP without uploading them.
• Shift_JIS → UTF-8 Convert Shift_JIS (CP932) and EUC-JP files from Japan to UTF-8.
• EUC-KR → UTF-8 Convert EUC-KR and CP949 files from Korea to UTF-8.
• GBK → UTF-8 Convert GBK, GB2312 and GB18030 files from mainland China to UTF-8.
• Windows-1251 → UTF-8 Convert old Russian files (cp1251, KOI8-R, CP866) to UTF-8.
• Windows-1256 to UTF-8 Converts old Arabic files (cp1256, Arabic ANSI) to UTF-8.
• Windows-1253 to UTF-8 Converts old Greek files (cp1253, ISO-8859-7) to UTF-8.
• Windows-1255 to UTF-8 Converts old Hebrew files (cp1255, ISO-8859-8) to UTF-8.
• Windows-1250 → UTF-8 Converts old Polish and Central European files to UTF-8.
• Big5, GBK, Shift_JIS → UTF-8 Convert old Chinese, Japanese and Korean files to UTF-8.
• Hindi, Tamil, Bengali text Hindi, Tamil or Bengali text shows à¤, à® or à¦? Repair it here.
• Kruti Dev to Unicode Hindi typed in Kruti Dev shows up as fgUnh? Convert it to Unicode.
• Preeti to Unicode Nepali typed in Preeti shows up as g]kfn? Convert it to Unicode.
• Bijoy to Unicode Bengali typed in Bijoy shows up as evsjv? Convert it to Unicode.
• For businesses & data protection officers How Mojibuster protects data – and how to verify it yourself.