Binary Log

/***************************************************************************** MySQL Binary Log log_event.h This log consists of events. Each event has a fixed-length header, possibly followed by a variable length data body. The data body consists of an optional fixed length segment (post-header) and an optional variable length segment. The events which really update data are Query_log_event, Execute_load_query_log_event and old Load_log_event and Execute_load_log_event events (Execute_load_query is used together with Begin_load_query and Append_block events to replicate LOAD DATA INFILE. Create_file/Append_block/Execute_load (which includes Load_log_event) were used to replicate LOAD DATA before the 5.0.3). Any @c Log_event saved on disk consists of the following three components. - Common-Header - Post-Header - Body The Common-Header, documented in the table @ref Table_common_header "below", always has the same form and length within one version of MySQL. Each event type specifies a format and length of the Post-Header. The length of the Common-Header is the same for all events of the same type. The Body may be of different format and length even for different events of the same type. The binary formats of Post-Header and Body are documented separately in each subclass. The binary format of Common-Header is as follows. <table> <caption>Common-Header</caption> <tr> <th>Name</th> <th>Format</th> <th>Description</th> </tr> <tr> <td>timestamp</td> <td>4 byte unsigned integer</td> <td>The time when the query started, in seconds since 1970. </td> </tr> <tr> <td>type</td> <td>1 byte enumeration</td> <td>See enum #Log_event_type.</td> </tr> <tr> <td>server_id</td> <td>4 byte unsigned integer</td> <td>Server ID of the server that created the event.</td> </tr> <tr> <td>total_size</td> <td>4 byte unsigned integer</td> <td>The total size of this event, in bytes. In other words, this is the sum of the sizes of Common-Header, Post-Header, and Body. </td> </tr> <tr> <td>master_position</td> <td>4 byte unsigned integer</td> <td>The position of the next event in the master binary log, in bytes from the beginning of the file. In a binlog that is not a relay log, this is just the position of the next event, in bytes from the beginning of the file. In a relay log, this is the position of the next event in the master's binlog. </td> </tr> <tr> <td>flags</td> <td>2 byte bitfield</td> <td>See Log_event::flags.</td> </tr> </table> Summing up the numbers above, we see that the total size of the common header is 19 bytes. The Post-Header has five components: <table> <caption>Post-Header for Query_log_event</caption> <tr> <th>Name</th> <th>Format</th> <th>Description</th> </tr> <tr> <td>slave_proxy_id</td> <td>4 byte unsigned integer</td> <td>An integer identifying the client thread that issued the query. The id is unique per server. (Note, however, that two threads on different servers may have the same slave_proxy_id.) This is used when a client thread creates a temporary table local to the client. The slave_proxy_id is used to distinguish temporary tables that belong to different clients. </td> </tr> <tr> <td>exec_time</td> <td>4 byte unsigned integer</td> <td>The time from when the query started to when it was logged in the binlog, in seconds.</td> </tr> <tr> <td>db_len</td> <td>1 byte integer</td> <td>The length of the name of the currently selected database.</td> </tr> <tr> <td>error_code</td> <td>2 byte unsigned integer</td> <td>Error code generated by the master. If the master fails, the slave will fail with the same error code, except for the error codes ER_DB_CREATE_EXISTS == 1007 and ER_DB_DROP_EXISTS == 1008. </td> </tr> <tr> <td>status_vars_len</td> <td>2 byte unsigned integer</td> <td>The length of the status_vars block of the Body, in bytes. See @ref query_log_event_status_vars "below". </td> </tr> </table> The Body has the following components: <table> <caption>Body for Query_log_event</caption> <tr> <th>Name</th> <th>Format</th> <th>Description</th> </tr> <tr> <td>@anchor query_log_event_status_vars status_vars</td> <td>status_vars_len bytes</td> <td>Zero or more status variables. Each status variable consists of one byte identifying the variable stored, followed by the value of the variable. The possible variables are listed separately in the table @ref Table_query_log_event_status_vars "below". MySQL always writes events in the order defined below; however, it is capable of reading them in any order. </td> </tr> <tr> <td>db</td> <td>db_len+1</td> <td>The currently selected database, as a null-terminated string. (The trailing zero is redundant since the length is already known; it is db_len from Post-Header.) </td> </tr> <tr> <td>query</td> <td>variable length string without trailing zero, extending to the end of the event (determined by the length field of the Common-Header) </td> <td>The SQL query.</td> </tr> </table> The following table lists the status variables that may appear in the status_vars field. @anchor Table_query_log_event_status_vars <table> <caption>Status variables for Query_log_event</caption> <tr> <th>Status variable</th> <th>1 byte identifier</th> <th>Format</th> <th>Description</th> </tr> <tr> <td>flags2</td> <td>Q_FLAGS2_CODE == 0</td> <td>4 byte bitfield</td> <td>The flags in @c thd->options, binary AND-ed with @c OPTIONS_WRITTEN_TO_BIN_LOG. The @c thd->options bitfield contains options for "SELECT". @c OPTIONS_WRITTEN identifies those options that need to be written to the binlog (not all do). Specifically, @c OPTIONS_WRITTEN_TO_BIN_LOG equals (@c OPTION_AUTO_IS_NULL | @c OPTION_NO_FOREIGN_KEY_CHECKS | @c OPTION_RELAXED_UNIQUE_CHECKS | @c OPTION_NOT_AUTOCOMMIT), or 0x0c084000 in hex. These flags correspond to the SQL variables SQL_AUTO_IS_NULL, FOREIGN_KEY_CHECKS, UNIQUE_CHECKS, and AUTOCOMMIT, documented in the "SET Syntax" section of the MySQL Manual. This field is always written to the binlog in version >= 5.0, and never written in version < 5.0. </td> </tr> <tr> <td>sql_mode</td> <td>Q_SQL_MODE_CODE == 1</td> <td>8 byte bitfield</td> <td>The @c sql_mode variable. See the section "SQL Modes" in the MySQL manual, and see sql_priv.h for a list of the possible flags. Currently (2007-10-04), the following flags are available: <pre> MODE_REAL_AS_FLOAT==0x1 MODE_PIPES_AS_CONCAT==0x2 MODE_ANSI_QUOTES==0x4 MODE_IGNORE_SPACE==0x8 MODE_NOT_USED==0x10 MODE_ONLY_FULL_GROUP_BY==0x20 MODE_NO_UNSIGNED_SUBTRACTION==0x40 MODE_NO_DIR_IN_CREATE==0x80 MODE_POSTGRESQL==0x100 MODE_ORACLE==0x200 MODE_MSSQL==0x400 MODE_DB2==0x800 MODE_MAXDB==0x1000 MODE_NO_KEY_OPTIONS==0x2000 MODE_NO_TABLE_OPTIONS==0x4000 MODE_NO_FIELD_OPTIONS==0x8000 MODE_MYSQL323==0x10000 MODE_MYSQL323==0x20000 MODE_MYSQL40==0x40000 MODE_ANSI==0x80000 MODE_NO_AUTO_VALUE_ON_ZERO==0x100000 MODE_NO_BACKSLASH_ESCAPES==0x200000 MODE_STRICT_TRANS_TABLES==0x400000 MODE_STRICT_ALL_TABLES==0x800000 MODE_NO_ZERO_IN_DATE==0x1000000 MODE_NO_ZERO_DATE==0x2000000 MODE_INVALID_DATES==0x4000000 MODE_ERROR_FOR_DIVISION_BY_ZERO==0x8000000 MODE_TRADITIONAL==0x10000000 MODE_NO_AUTO_CREATE_USER==0x20000000 MODE_HIGH_NOT_PRECEDENCE==0x40000000 MODE_PAD_CHAR_TO_FULL_LENGTH==0x80000000 </pre> All these flags are replicated from the server. However, all flags except @c MODE_NO_DIR_IN_CREATE are honored by the slave; the slave always preserves its old value of @c MODE_NO_DIR_IN_CREATE. For a rationale, see comment in @c Query_log_event::do_apply_event in @c log_event.cc. This field is always written to the binlog. </td> </tr> <tr> <td>catalog</td> <td>Q_CATALOG_NZ_CODE == 6</td> <td>Variable-length string: the length in bytes (1 byte) followed by the characters (at most 255 bytes) </td> <td>Stores the client's current catalog. Every database belongs to a catalog, the same way that every table belongs to a database. Currently, there is only one catalog, "std". This field is written if the length of the catalog is > 0; otherwise it is not written. </td> </tr> <tr> <td>auto_increment</td> <td>Q_AUTO_INCREMENT == 3</td> <td>two 2 byte unsigned integers, totally 2+2=4 bytes</td> <td>The two variables auto_increment_increment and auto_increment_offset, in that order. For more information, see "System variables" in the MySQL manual. This field is written if auto_increment > 1. Otherwise, it is not written. </td> </tr> <tr> <td>charset</td> <td>Q_CHARSET_CODE == 4</td> <td>three 2 byte unsigned integers, totally 2+2+2=6 bytes</td> <td>The three variables character_set_client, collation_connection, and collation_server, in that order. character_set_client is a code identifying the character set and collation used by the client to encode the query. collation_connection identifies the character set and collation that the master converts the query to when it receives it; this is useful when comparing literal strings. collation_server is the default character set and collation used when a new database is created. See also "Connection Character Sets and Collations" in the MySQL 5.1 manual. All three variables are codes identifying a (character set, collation) pair. To see which codes map to which pairs, run the query "SELECT id, character_set_name, collation_name FROM COLLATIONS". Cf. Q_CHARSET_DATABASE_CODE below. This field is always written. </td> </tr> <tr> <td>time_zone</td> <td>Q_TIME_ZONE_CODE == 5</td> <td>Variable-length string: the length in bytes (1 byte) followed by the characters (at most 255 bytes). <td>The time_zone of the master. See also "System Variables" and "MySQL Server Time Zone Support" in the MySQL manual. This field is written if the length of the time zone string is > 0; otherwise, it is not written. </td> </tr> <tr> <td>lc_time_names_number</td> <td>Q_LC_TIME_NAMES_CODE == 7</td> <td>2 byte integer</td> <td>A code identifying a table of month and day names. The mapping from codes to languages is defined in @c sql_locale.cc. This field is written if it is not 0, i.e., if the locale is not en_US. </td> </tr> <tr> <td>charset_database_number</td> <td>Q_CHARSET_DATABASE_CODE == 8</td> <td>2 byte integer</td> <td>The value of the collation_database system variable (in the source code stored in @c thd->variables.collation_database), which holds the code for a (character set, collation) pair as described above (see Q_CHARSET_CODE). collation_database was used in old versions (???WHEN). Its value was loaded when issuing a "use db" query and could be changed by issuing a "SET collation_database=xxx" query. It used to affect the "LOAD DATA INFILE" and "CREATE TABLE" commands. In newer versions, "CREATE TABLE" has been changed to take the character set from the database of the created table, rather than the character set of the current database. This makes a difference when creating a table in another database than the current one. "LOAD DATA INFILE" has not yet changed to do this, but there are plans to eventually do it, and to make collation_database read-only. This field is written if it is not 0. </td> </tr> <tr> <td>table_map_for_update</td> <td>Q_TABLE_MAP_FOR_UPDATE_CODE == 9</td> <td>8 byte integer</td> <td>The value of the table map that is to be updated by the multi-table update query statement. Every bit of this variable represents a table, and is set to 1 if the corresponding table is to be updated by this statement. The value of this variable is set when executing a multi-table update statement and used by slave to apply filter rules without opening all the tables on slave. This is required because some tables may not exist on slave because of the filter rules. </td> </tr> </table> @subsection Query_log_event_notes_on_previous_versions Notes on Previous Versions * Status vars were introduced in version 5.0. To read earlier versions correctly, check the length of the Post-Header. * The status variable Q_CATALOG_CODE == 2 existed in MySQL 5.0.x, where 0<=x<=3. It was identical to Q_CATALOG_CODE, except that the string had a trailing '/0'. The '/0' was removed in 5.0.4 since it was redundant (the string length is stored before the string). The Q_CATALOG_CODE will never be written by a new master, but can still be understood by a new slave. * See Q_CHARSET_DATABASE_CODE in the table above. * When adding new status vars, please don't forget to update the MAX_SIZE_LOG_EVENT_STATUS, and update function code_name ****************************************************************************/