by Mark Charney
========================================================================================
XED is an acronym for X86 Encoder Decoder. It is pronounced like the (British) English "z". XED2 is the current implementation of XED, in C. (XED0 is written in C++).
XED is a software library (and associated headers) for encoding and decoding X86 (IA-32 instruction set and Intel® 64 instruction set) instructions. The decoder takes sequences of 1-15 bytes along with machine mode information and produces a data structure describing the opcode and operands, and flags. The encoder takes a similar data structure and produces a sequence of 1 to 15 bytes. XED is multi-thread safe.
XED was designed to be very fast and extensible.
XED compiles with the following compilers:
XED works with the following operating systems:
The XED examples (Examples of using XED) also include binary image readers for Windows PECOFF, ELF and Mac OS* X MACHO binary file formats for 32b and 64b. These allow XED to be used as a simple (not symbolic) disassembler. The XED disassembler supports 3 output formats: Intel, ATT SYSV, and a more detailed internal format describing all resources read and written.
========================================================================================
Table of Contents
========================================================================================
This section describes the requirements for compiling with XED and linking the libxed.a library.
To use XED your sources should include the top-most header file: xed-interface.h.
Your compilation statement must include:
-Ixedpath/include
Your Linux or Mac OS* X link statement must reference the libxed library:
-lxedpath/lib/libxed.a
(or link against libxed.lib for Windows).
XED uses base types from stdint.h when GCC is the compiler. These types have the names: uint8_t, uint16_t, uint32_t, uint64_t int8_t, int16_t, int32_t, and int64_t. When the Microsoft Visual Studio compiler or the Intel compiler on Microsoft Windows are used used to compile XED, we create these types using the underlying Windows standard types. XED also defines a "uint_t" type that is shorthand for 'unsigned int".
========================================================================================
X86 instructions are 1-15 byte values. They consist of several well defined components that total at most 15 bytes:
One specific opcode byte is used as an 'escape' to indicate that two opcode bytes are required. All two-byte opcodes have this escape as their first opcode byte.
Immediates and displacements are usually limited to 4 bytes, but there are several variants of the MOV instruction that can take 8B values. The AMD 3DNow ISA extension uses the immediate field to provide additional opcode information.
The encodings are very byte-oriented. Bit-fields in the REX, opcode, MODRM and SIB bytes are simple and self-contained.
The legacy prefix bytes are used for:
There are 11 distinct legacy prefixes. Three of them (operand size, and the two repeat prefixes) have different meanings in different contexts; Sometimes they are used for opcode refinement and do not have their default meaning. Less frequently. two of the segment overrides can be used for conditional branch hints.
There are also multiple ways to encode certain instructions, with the same or differing length.
========================================================================================
XED has two fundamental interfaces: encoding and decoding. Supporting these interfaces are many data structures, but the two starting points are the xed_encoder_request_t and the xed_decoded_inst_t . The xed_decoded_inst_t has more information than the xed_encoder_request_t , but both types are derived from a set of common fields called the xed_operand_values_t.
The output of the decoder, the xed_decoded_inst_t , includes additional information that is not required for encoding, but provides more information about the instruction resources.
The common operand fields, used by both the encoder and decoder, hold the operands and the memory addressing information.
The decoder has an operands array that holds order of the decoded operands. This array indicates whether or not the operands are read or written.
The encoder has an operand array where the encoder user must specify the order of the operands used for encoding.
========================================================================================
The FAR versions of 3 opcodes (really 6 distinct opcodes) are given the opcode names CALL_FAR, JMP_FAR and RET_FAR. The AMD documentation lists the far return as RETF. I call that RET_FAR to be consistent with the other far operations.
To distinguish the SSE2 MOVSD instruction from the base string instruction MOVSD, XED calls the SSE version MOVSD_XMM.
The instruction 0x90 is very special in the instruction set because it gets special treatment in 64b mode. In 64b mode, 32b register writes normally zero the upper 32 bits of a 64b register. No so for 0x90. If it did zero the upper 32 bits, it would not be a NOP.
There are two important NOP categories. XED_CATEGORY_NOP and XED_CATEGORY_WIDENOP. The XED_CATEGORY_NOP applies only to the 0x90 opcode. The WIDENOP category applies to the NOPs in the two byte table row 0F19...0F1F. The WIDENOPs take MODRM bytes, and optional SIB and displacements.
========================================================================================
For decode requests (xed_decoded_inst_t), the operands array is stored in the xed_inst_t strcture once the instruction is decoded. For encode requests, the request's operand order is stored in the xed_encoder_request_t.
There are several types of operands:
Each operand has two associated attributes: the R/W action and a visibility. The R/W actions (xed_operand_action_enum_t) indicate whether the operand is read, written or both read-and-written, or conditionally read or written. The visibility attribute (xed_operand_visibility_enum_t) is described in the next subsection.
The memory operation operand is really a pointer to separate fields that hold the memory operation information. The memory operation information is comprised of:
There are several important things to note:
There are 3 basic types of resource visibilites:
Explicit are what you think they are: resources that are required for the encoding and for each explicit resource, there is field in the corresponding instruction encoding. The implicit and suppressed resources are a more subtle.
SUPP operands are:
IMPL operands are:
The implicit resources are required for selecting an encoding, but do not show up as a specific field in the instruction representation. Implicit resources do show up in a conventional instruction disassembly. In the IA-32 instruction set or Intel64 instruction set, there are many instructions that use EAX or RAX implicitly, for example. Sometimes the CL or RCX register is implicit. Also, some instructions have an implicit 1 immediate. The opcode you chose fixes your choice of implicit register or immediate.
The suppressed resources are a form of implicit resource, but they are resources not required for encoding. The suppressed operands are not normally displayed in a conventional disassembly. The suppressed operands are emitted by the decoder but are not used when encoding. They are ignored by the encoder. Examples are the stack pointer for PUSH and POP operations. There are many others, like pseudo resources.
The explicit and implicit resources are expressed resources -- they show up in disassembly and are required for encoding. The suppressed resources are considered a kind of implicit resources that are not expressed in ATT System V or Intel disassembly formats.
The suppressed operands are always after the implicit and explicit operands in the operand order.
Immediates and displacements are different things in the ISA. They can be 1, 2, 4 or 8 bytes. Branch displacements (1, 2 or 4 bytes) and Memory displacements (1, 2, 4 or 8 bytes) refer to the signed constants that are used for relative distances or memory "offsets" from a base register (including the instruction pointer) or start of a memory region.
Immediates are signed or unsigned and are used for numerical computations, shift distances, and also hold things like segment selectors for far pointers for certain jump or call instructions.
There is also a second 1B immedate used only for the ENTER instruction.
XED will try to use the shortest allowed width for a displacement or immediate. You can control XED's selection of allowed widths using a notion of "legal widths". A "legal width" is a binary number where each bit represents a legal desired width. For example, when you have a valid base register in 32 or 64b addressing, and a displacement is required, your displacement must be either 1 byte or 4 bytes long. This is expressed by OR'ing 1 and 4 together to get 0101 (base 2) or 5 (base 10).
If a four byte displacement was required, but the value was representable in fewer than four bytes, then the legal width should be set to 0100 (base 2) or 4 (base 10).
========================================================================================
========================================================================================
XED0 was written in C++, and XED2 is written in C. The port from XED0 to XED2 is relatively, but not completely, mechanical.
Generally speaking, one takes the name of a class in XED0 and prepends it to the name of the XED0 method function to get the XED2 function.
If you are using C++, when you include the "xed-interface.h" header, you must wrap it:
extern "C" { #include "xed-interface.h" }
In XED0, there was a xed_common_fields_t class that was common to encode and decode. In XED2, the shared data structure is an array of xed_operand_values_t elements. The elements are basically integers accessible via the Operand storage fields interface.
In XED0 there was a "xed_decoded_resource_t" type. This type has been removed and is subsumed by the xed_operand_t operands array associated with each decoded instruction. The operands array is accessed from the xed_inst_t and the xed_inst_operand() function. Encode requests now have a separate encode order array updated by the xed_encoder_request_set_operand_order function.
If you used XED0's ostream operators they are gone. They were just simple wrappers for my "enum2str()" functions which continue to exist. So
xed_iclass_t iclass = xedd->get_iclass(); cout << iclass << endl;
xed_iclass_enum_t iclass = xed_decoded_inst_get_iclass(xedd); cout << xed_iclass_enum_t2str(iclass) << endl;
Here are some common changes one has to make.
XED:: -> (nothing) xed_iclass_t -> xed_iclass_enum_t xedregs_t -> xed_reg_enum_t XEDICLASS_ -> XED_ICLASS_ XEDREG_ -> XED_REG_ using namespace XED; -> (remove) #include "xed-interface.H" -> extern "C" {\n#include "xed-interface.h"\n}\n #include "xed-iclass.H" -> extern "C" {\n#include "xed-iclass-enum.h"\n}\n #include "xed-category.H" -> extern "C" {\n#include "xed-category-enum.h"\n}\n #include "xed-extension.H" -> extern "C" {\n#include "xed-extension-enum.h"\n}\n xedd->xed_get_base_reg(0) -> xed_decoded_inst_get_base_reg(xedd,0); xedd->xed_get_index_reg(0) -> xed_decoded_inst_get_index_reg(xedd,0); the XED_ROLE_* are generally replaced with corresponding XED_OPERAND_* but now instead of XED_ROLE_NORMAL you specify which register operand.
Far direct pointer storage has changed. In XED0, far direct pointers were stored in a 6B immediate on IA32. In XED2 there is a 4B displacement and 2B segment selector stored in the immediate.
Encoding immediates, branch displacements and memory displacements: In XED0, there were function calls for building a xed_immdis_t that incorporated a legal_widths bit mask. In XED2, while I support a C implementation of xed_immdis_t, I discourage its use. Instead, there are functions for finding the shortest legal width of a signed or unsigned number (xed_shortest_width_signed, xed_shortest_width_unsigned). Using that length, you can then call xed_encoder_request_set_memory_displacement, xed_encoder_request_set_branch_displacement, xed_encoder_request_set_simm, xed_encoder_request_set_uimm0, or xed_encoder_request_set_uimm1 .
========================================================================================
Send bugs and questions to mark.charney@intel.com. Complete bug reports that are easy to reproduce are fixed faster, so try to provide as much information as possible. Include: kit number, your OS version, compiler version. Try to reproduce the problem in a simple example that you can send us.
========================================================================================
The information in this manual is subject to change without notice and Intel Corporation assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association with this document. This document and the software described in it are furnished under license and may only be used or copied in accordance with the terms of the license. No license, express or implied, by estoppel or otherwise, to any intellectual property rights is granted by this document. The information in this document is provided in connection with Intel products and should not be construed as a commitment by Intel Corporation.
EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. Intel products are not intended for use in medical, life saving, life sustaining, critical control or safety systems, or in nuclear facility applications.
Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompat- ibilities arising from future changes to them.
The software described in this document may contain software defects which may cause the product to deviate from published specifications. Current characterized software defects are available on request.
Intel, the Intel logo, Intel SpeedStep, Intel NetBurst, Intel NetStructure, MMX, Intel386, Intel486, Celeron, Intel Centrino, Intel Xeon, Intel XScale, Itanium, Pentium, Pentium II Xeon, Pentium III Xeon, Pentium M, and VTune are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.
Other names and brands may be claimed as the property of others.
Copyright 2004-2007, Intel Corporation.