How zpool history buffer is laid out in memory
The zpool history buffer is stored as a dmu object containing <packed record length, record nvlist> tuples. The buffer contains the 'zpool create' history in the beginning followed by the rest of the records. The 'zpool create' history is never overwritten, while the rest of the buffer is implemented as a ring. The details about the zpool history buffer layout can be fetched from the spa_history_phys_t structure which is stored in the bonus buffer of spa_history object. The structure has the following fields
1. sh_pool_create_len: This field represents the ending offset in bytes of where the 'zpool_create' record is stored.
2. sh_phys_max_off: This field represents the physical ending offset in bytes of the log. So, it gives us the total length of the buffer.
3. sh_bof: This is the logical beginning of file. In case some records have been overwritten, which can be got from the sh_records_lost field of the spa_history_phys_t structure, the spa_history_log_to_phys function is used to convert the logical value to the physical value. If the sh_records_lost=0, then the logical bof is same as the physical bof.
4. sh_eof: This is the logical end of file. In case some records have been overwritten, which can be got from the sh_records_lost field of the spa_history_phys_t structure, the logical end of file exceeds the sh_phys_max_off . In this case, the spa_history_log_to_phys function is used to convert the logical value to the physical value within the buffer.
5. sh_records_lost: The last field is sh_records_lost which tells us the number of records overwritten. The history buffer is implemented as a ring except the 'zpool create' history. When the end of buffer is hit, the previously stored records are overwritten starting from the sh_pool_create_len, the offset where the zpool create history ends. So, the zpool_create history is never overwritten.
The zpool history buffer exists in memory as a packed nvlist. The packed nvlist contains a nvs_header_t structure at the beginning which stores the host endian and the encoding_type. The header is followed by a stream containing <size of the nvpair, nvpair> tuples. There are two types of encoding methods:
1. Native encoding
2. XDR encoding
The format of the packed nvlist for the XDR encoding is :
- encoding methode and host endian (4 bytes)
- nvl_nvflag (4 bytes)
- encoded nvpairs, the format of one xdr encoded nvpair is:
- encoded size of the nvpair (4 bytes)
- decoded size of the nvpair (4 bytes)
- name string, (4 + sizeof(NV_ALIGN4(string))
a string is coded as size (4 bytes) and data
- data type (4 bytes)
- number of elements in the nvpair (4 bytes)
- data
- 2 zero's for end of the entire list (8 bytes)
The format is similar for native encoding wherin the first 4 bytes are occupied by the header followed by the nvpair tuples. Each nvpair tuple has the format <decoded size of nvpair( 4 bytes), packed nvpair> .
For decoding a packed nvlist, first the host endian and the encoding type is get fron the nvs_header_t structure in the beginning of the buffer, Then depending on the type of encoding, the appropriate functions are called to read the size of the nvpair from the nvstream and then read in the nvpair. This continues until the end of file is reached. In the case of zpool_history buffer, the encoding type used is native encoding.
