BOM

最新推荐文章于 2025-07-04 11:33:13 发布

原创最新推荐文章于 2025-07-04 11:33:13 发布 · 509 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#character #byte #deprecated #stream #processing #file

c/c++ 专栏收录该内容

4 篇文章

订阅专栏

byte order mark (BOM)

名词解释 2010-03-04 16:36:04 阅读20 评论0 字号：大中小

参考： http://en.wikipedia.org/wiki/Byte-order_mark

UTF-8

public final static byte [] UTF8_HEAD ={(byte) 0xEF,(byte) 0xBB,(byte) 0xBF};

fos.write(UTF8_HEAD);

The byte order mark (BOM) is a Unicode character used to signal the endianness (byte order) of a text file or stream. Its code point is de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">U+FEFFde>. BOM use is optional, and, if used, should appear at the start of the text stream. Beyond its specific use as a byte-order indicator, the BOM character may also indicate which of the several Unicode representations the text is encoded in.

Because Unicode can be encoded as 16-bit or 32-bit integers, a computer receiving Unicode text from arbitrary sources needs to know which byte order the integers are encoded in. The BOM gives the producer of the text a way to describe the text stream's endianness to the consumer of the text without requiring some contract or metadata outside of the text stream itself. Once the receiving computer has consumed the text stream, it presumably processes the characters in its own native byte order and no longer needs the BOM. Hence the need for a BOM arises in the context of text interchange, rather than in normal text processing within a closed environment.

Usage

In UTF-16, a BOM (de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">U+FEFFde>) is placed as the first character of a file or character stream to indicate the endianness (byte order) of all the 16-bit code units of the file or stream.

If the 16-bit units are represented in big-endian byte order, this BOM character will appear in the sequence of bytes as de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">0xFEde>followed by de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">0xFFde> (where "de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">0xde>" indicates hexadecimal);
if the 16-bit units use little-endian order, the sequence of bytes will have de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">0xFFde> followed by de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">0xFEde>.

The Unicode value de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">U+FFFEde> is guaranteed never to be assigned as a Unicode character; this implies that in a Unicode context the de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">0xFFde>, de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">0xFEde> byte pattern can only be interpreted as the de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">U+FEFFde> character expressed in little-endian byte order (since it could not be a de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">U+FFFEde> character expressed in big-endian byte order).

While UTF-8 does not have byte order issues, a BOM encoded in UTF-8 may nonetheless be encountered. A UTF-8 BOM is explicitly allowed by the Unicode standard^[2], but is not recommended^[3], as it only identifies a file as UTF-8 and does not state anything about byte order.^[4] Many Windows programs (including Windows Notepad) add BOMs to UTF-8 files by default. However in Unix-like systems (which make heavy use of text files for file formats as well as for inter-process communication) this practice is not recommended, as it will interfere with correct processing of important codes such as the shebangat the start of an interpreted script.^[5] It may also interfere with source for programming languages that don't recognise it. For example, gcc reports stray characters at the beginning of a source file, and in PHP, if output buffering is disabled, it has the subtle effect of causing the page to start being sent to the browser, preventing custom headers from being specified by the PHP script. The UTF-8 representation of the BOM is the byte sequence de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">EF BB BFde>, which appears as the ISO-8859-1 characters de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">???de> in most text editors and web browsers not prepared to handle UTF-8.

Although a BOM could be used with UTF-32, this encoding is rarely used for transmission. Otherwise the same rules as for UTF-16 are applicable. For theIANA registered charsets UTF-16BE, UTF-16LE, UTF-32BE, and UTF-32LE a "byte order mark" must not be used, an initial U+FEFF has to be interpreted as a (deprecated) "zero width no-break space", because the names of these charsets already determine the byte order. For the registered charsets UTF-16 and UTF-32, an initial U+FEFF indicates the byte order.

If the BOM character appears in the middle of a data stream, it should, according to Unicode, be interpreted as a "zero-width non-breaking space" (essentially a null character^{[citation needed]}). Its deliberate use for this purpose is deprecated in Unicode 3.2, however, with the "Word Joiner" character,de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">U+2060de>, strongly preferred. This allows de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">U+FEFFde> to be used solely with the semantic of BOM.

Unwanted BOMs

Some text editing software in a UTF-8 environment on MS Windows adds a BOM to the beginning of text files. If the page is displayed in Latin-1 (ISO-8859-1), the three bytes show up as ??? (i+umlaut, double right guillemet, inverted question mark).

Adding a BOM to the beginning of a PHP file (.php) results in the BOM being displayed as HTML, rather than getting processed as PHP, since it comes before the opening <?php tag. Besides being a visual nuisance, it can cause the default HTTP headers to be sent, therefore preventing the sending of customized HTTP headers. This happens because whatever HTTP headers are in the queue are sent out as soon as the first text goes out for the page. The only solution is to hunt down the infected PHP file(s) and manually remove the BOM characters with another editor.

[edit]Representations of byte order marks by encoding

Encoding	Representation (hexadecimal)	Representation (decimal)	Representation (ISO-8859-1)
UTF-8	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">EF BB BFde>^{[t 1]}	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">239 187 191de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">???de>
UTF-16(BE)	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">FE FFde>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">254 255de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">t?de>
UTF-16(LE)	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">FF FEde>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">255 254de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">?tde>
UTF-32(BE)	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">00 00 FE FFde>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">0 0 254 255de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">□□?tde> (□ is the ascii null character)
UTF-32(LE)	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">FF FE 00 00de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">255 254 0 0de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">t?□□de> (□ is the ascii null character)
UTF-7	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">2B 2F 76de>, and one of the following bytes: de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">[ 38 \| 39 \| 2B \| 2F ]de>^{[t 2]}	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">43 47 118de>, and one of the following bytes: de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">[ 56 \| 57 \| 43 \| 47 ]de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">+/vde>, and one of the following characters: de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">8 9 + /de>
UTF-1	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">F7 64 4Cde>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">247 100 76de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">÷dLde>
UTF-EBCDIC	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">DD 73 66 73de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">221 115 102 115de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">Ysfsde>
SCSU	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">0E FE FFde>^{[t 3]}	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">14 254 255de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">□t?de> (□ is the ascii "shift out" character)
BOCU-1	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">FB EE 28de> optionally followed by de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">FFde>^{[t 4]}	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">251 238 40de> optionally followed by de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">255de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">??(de> optionally followed by de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">?de>
GB-18030	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">84 31 95 33de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">132 49 149 51de>	de style="background-color: rgb(249, 249, 249); font-family: monospace, sans-serif; ">□1■3de> (□ and ■ are unmapped ISO-8859-1 characters)