|Publication number||US7606932 B1|
|Application number||US 10/713,397|
|Publication date||Oct 20, 2009|
|Filing date||Nov 14, 2003|
|Priority date||Nov 14, 2003|
|Publication number||10713397, 713397, US 7606932 B1, US 7606932B1, US-B1-7606932, US7606932 B1, US7606932B1|
|Inventors||Robert E. Cypher|
|Original Assignee||Sun Microsystems, Inc.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (6), Referenced by (4), Classifications (11), Legal Events (3)|
|External Links: USPTO, USPTO Assignment, Espacenet|
1. Field of the Invention
This invention relates to the field of multiprocessor computer systems and, more particularly, to communication within multiprocessor computer systems.
2. Description of the Related Art
Multiprocessing computer systems include two or more processors that may be employed to perform computing tasks. A particular computing task may be performed upon one processor while other processors perform unrelated computing tasks. Alternatively, components of a particular computing task may be distributed among multiple processors to decrease the time required to perform the computing task as a whole.
Various components within a multiprocessing computer system may communicate with each other during operation. For example, various components may participate in a communication protocol that involves sending and receiving communications. A popular architecture in commercial multiprocessing computer systems is a shared memory architecture in which multiple processors share a common memory. In shared memory multiprocessing systems, a cache hierarchy is typically implemented between the processors and the shared memory. In order to maintain the shared memory model, in which a particular address stores exactly one data value at any given time, the communication protocol implemented in shared memory multiprocessing systems include a coherency protocol. Generally speaking, an operation is coherent if the effects of the operation upon data stored at a particular memory address are reflected in each copy of the data within the cache hierarchy. For example, when data stored at a particular memory address is updated, the update may be supplied to the caches that are storing copies of the previous data. Alternatively, the copies of the previous data may be invalidated in the caches such that a subsequent access to the particular memory address causes the updated copy to be transferred from main memory or from a cache.
Communications sent between components of a multiprocessing computer system may include the address of data being accessed. The range of addressable space within a multiprocessing system may be large, and thus addresses may contain a significant number of bits. This may, in turn, increase the size of the communications sent between devices.
Various embodiments of systems and methods implementing a variable-size mask format are disclosed. In one embodiment, a system may include a device configured to initiate a transaction by sending an address packet and an additional device configured to respond to the address packet. The additional device is coupled to receive the address packet from the device via an address network. The address packet includes a mask field configured to indicate which of N quantities of data are targeted by the transaction. The address packet also includes a granularity field configured to indicate a size of each of the N quantities of data. If the size is less than a maximum size, a portion of the granularity field stores additional address bits of an address of the N quantities of data. If the size is the maximum size, none of the granularity field stores any additional address bits of the address of the N quantities of data.
A better understanding of the present invention can be obtained when the following detailed description is considered in conjunction with the following drawings, in which:
While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present invention as defined by the appended claims.
Each of processing subsystems 142 and/or I/O subsystem 146 may communicate according to a communication protocol in order to perform activities such as accessing data at addresses mapped to memory subsystems 144 and/or I/O subsystems 146. Devices configured to perform accesses to memory subsystems 144 are referred to herein as “active” devices. Because each active device within computer system 140 may access data in memory subsystems 144, potentially caching the data, a communication protocol implemented by memory subsystems 144 and active devices such as processing systems 142 may include a coherency protocol in order to maintain coherency between processing subsystems 142 and memory subsystems 144. Each client in
Memory subsystems 144 are configured to store data and instruction code for use by processing subsystems 142 and I/O subsystem 146. Memory subsystems 144 may include dynamic random access memory (DRAM), although other types of memory may be used in some embodiments.
I/O subsystem 146 is illustrative of a peripheral device such as an input-output bridge, a graphics device, a networking device, etc. In some embodiments, I/O subsystem 146 may include a cache memory subsystem similar to those of processing subsystems 142 for caching data associated with addresses mapped to one of memory subsystems 144.
In one embodiment, data network 152 may be a logical point-to-point network. Data network 152 may be implemented as an electrical bus, a circuit-switched network, or a packet-switched network. In embodiments where data network 152 is a packet-switched network, packets may be sent through the data network using techniques such as wormhole, store and forward, or virtual cut-through. In a circuit-switched network, a particular client device may communicate directly with a second client device via a dedicated point-to-point link that may be established through a switched interconnect mechanism. To communicate with a third client device, the particular client device utilizes a different link as established by the switched interconnect than the one used to communicate with the second client device. Messages upon data network 152 are referred to herein as data packets.
Address network 150 accommodates communication between processing subsystems 142, memory subsystems 144, and I/O subsystem 146. Messages upon address network 150 are generally referred to as address packets. When an address packet references a storage location within a memory subsystem 144, the referenced location may be specified via an address conveyed within the address packet upon address network 150. Subsequently, data corresponding to the address packet on the address network 150 may be conveyed upon data network 152. In one embodiment, address packets may correspond to requests for an access right (e.g., a readable or writable copy of a cacheable coherency unit whose address maps to a memory subsystem 144) or requests to perform a read or write to a non-cacheable memory location (e.g., data whose address maps to an I/O subsystem 146). Such address packets may be sent by a device in order to initiate a transaction to access or gain an access right to data at a particular non-cacheable or cacheable address. In transactions for cacheable addresses, subsequent address packets may be sent as part of that transaction by other devices in order to implement the access right and/or ownership changes needed to satisfy the transaction. In the computer system 140 shown in
Address Packet Variable-Size Mask Format
An active device may generate an address packet and send that address packet on the address network in order to request access to data at a particular address. In one embodiment, an initiating device 142A may initiate a transaction requesting access to a particular quantity of data by sending an address packet specifying that data and the requested access right (for cacheable addresses) or type of access to be performed (for non-cacheable addresses). For example, an active device may initiate a transaction for a non-cacheable address mapped to an I/O device 146 by sending an address packet on the address network to the I/O device 146. The address network 150 may convey the address packet to one or more receiving devices. Note that the address packet sent by the initiating device 142A may be handled by one or more intervening devices (e.g., a memory subsystem 144 or directory) before being provided to a receiving device in some embodiments.
The address packet may include a command field indicating the requested type of access and/or access right (e.g., as defined within a coherency protocol) and an address field containing an address identifying the data to be accessed or to which access is requested. In response to such an address packet, a device that maps the specified address may respond by updating the value of the specified data and/or by sending one or more address and/or data packets on the address and/or data networks respectively.
Active devices within computer system 140 may be configured to generate address packets that include a mask format having a variable size. The mask format may include N bits and indicate which of N X-byte quantities, starting at a given address, to access (e.g., read or write for non-cacheable addresses) or for which an access right (e.g., read and/or write access for cacheable addresses) is requested. The value of X may be indicated by a granularity field within the address packet.
The command field 202 may indicate the type of transaction of which the address packet is part. The type of transaction may correspond to the type of access or access right requested by the initiator of the transaction. For example, in one embodiment, a command field may indicate any of various cacheable commands such as RTO (Read-to-Own, used to request write access to a cacheable coherency unit), RTS (Read-to-Share, used to request read access to a cacheable coherency unit), WB (Write Back, used to write a modified cacheable coherency unit back to memory) and non-cacheable commands such as RIO (Read Input/Output, used to read from a non-cacheable address) and WIO (Write Input/Output, used to write to a non-cacheable address).
The address field 204 may indicate the address of the data to be accessed or to which an access right is requested. The address field 204 may include fewer than all of the address bits needed to specify a particular quantity of data. The remaining address bits not included in the address field may be implicit (e.g., the value of the remaining address bits may be implicitly zero) for certain values of the command field. For example, in situations in which the smallest addressable data quantity is a byte, certain commands may operate on aligned 64-byte quantities, and thus the lowest six address bits may be implicitly zero for those commands. For other values of the command field, the variable-size mask may be used to select the data on which to operate.
If particular granularities of the variable-size mask are selected within the address packet 200, a portion of the command field 204 used to indicate the granularity of the mask may also include additional non-implicit address bits. These address bits may be implicit for other granularities and for commands that do not use the variable-size mask.
In some embodiments, for each different selectable granularity X of the variable-size mask, a different number of address bits may be implicit for each of the N X-byte quantities identified in the mask field. For example, if a 1-byte granularity is selected and there are eight bits in the mask field such that the mask field can select any of eight bytes, the value of the last three address bits may be implicit for each of the quantities identifiable by each bit in the mask field. For example, the lowest order bit of the N bits in the mask may correspond to the byte at the address ending in “000”, the next lowest order bit may correspond to the byte at the address ending in “001”, and so on. If instead an 8-byte granularity is selected, the value of the last six address bits may be implicit. When the granularity field indicates a selected granularity for which fewer address bits are implicit, a portion of the granularity field may be used to store additional non-implicit address bits. When instead the granularity field indicates a selected granularity for which more address bits are implicit, that portion of the granularity field may be used to specify the selected granularity.
Depending on the granularity indicated in MaskType[3:0], at least some of the bits included in MaskType[3:0] may be used as additional address bits instead of being used to specify a granularity, as shown in
In the illustrated embodiment, different granularities are indicated by the lowest bit within MaskType[3:0] that has a logical value of ‘1’. For example, when the lowest bit MaskType is set to ‘1’, it indicates that a one-byte granularity is selected and that the remaining bits [3:1] within MaskType are used as address bits. If MaskType is the lowest bit within MaskType that has a value of ‘1’, it indicates that a two-byte granularity is selected and that the remaining bits [3:2] within MaskType are used as address bits. When MaskType is the lowest bit within MaskType that has a value of ‘1’, it indicates that a four-byte granularity is selected and that the remaining bit MaskType is used as an address bit. When MaskType is the lowest bit within MaskType having a value of ‘1’, it indicates that an eight-byte granularity is selected and that none of the bits in MaskType are used as address bits.
Using a MaskType encoding such as the one shown in
Each of the N data quantities (here, N=8) of the specified granularity at the specified address may be selected by setting a corresponding bit in Mask[7:0].
When a two-byte granularity is selected, address bits 46-4 are available (bits 46-6 in the address field 204 and bits 5-4 in the granularity field 210). Each of the eight two-byte quantities addressable by Address[46:4] is associated with a respective one of the bits in the mask field 212. For example, as shown, the two-byte quantity at Address[46:4],0000-Address[46:4],0001 is associated with Mask.
If a four-byte granularity is selected, address bits 46-5 are available (bits 46-6 in the address field 204 and bit 5 in the granularity field 210). Eight four-byte quantities are addressable by Address[46:5], each of which is associated with a respective bit in the mask field 212. For example, the four-byte quantity at Address[46:5],0100-Address[46:5], 0111 is associated with Mask.
When an eight-byte granularity is selected, address bits 46-6 are available in the address field 204. In this embodiment, no address bits are available in the granularity field 212 when the largest granularity is selected. Each of the eight eight-byte quantities addressed by Address[46:6] is associated with a respective bit in the mask field 212. For example, the eight-byte quantity at Address[46:6],011000-Address[46:6],011111 is associated with Mask.
If one or more bits of the granularity field indicate that the granularity is less than the maximum granularity, as determined at 601, the remaining bits of the granularity field (other than the one or more that indicate that the granularity is less than the maximum) may indicate a portion of the address of the targeted data, as shown at 603. If the granularity is equal to the maximum granularity, all of the bits of the granularity field may indicate granularity and none of the granularity bits may store address bits, as indicated at 605. In one embodiment, the granularity may be indicated by the lowest-order non-zero bit in the granularity field. For example, if the lowest-order bit in the granularity field is set, the specified granularity is the minimum granularity. If the highest-order bit in the granularity field is the lowest-order non-zero bit, the maximum granularity is selected.
The granularity indicated by all or part of the granularity field indicates the size X, in bytes in this example, of the data quantity to which each bit in the mask field corresponds. Each bit in the mask field indicates whether an associated X-byte data quantity is included in the data to be accessed or to which an access right is requested in response to the address packet, as indicated at 607.
Processing subsystem 142A initiates the first transaction to gain access to a cacheable coherency unit by sending an address packet 200A on the address network. The address packet 200A includes a command field 202A indicating the type of transaction (e.g., RTS, RTO) being initiated and an address field 204A including at least a portion of the address of the cacheable coherency unit. The command field 202A includes neither granularity information nor an address mask. Processing subsystem 142B currently has an ownership responsibility for the cacheable coherency unit. In response to receiving the address packet 200A and having ownership of the cacheable coherency unit, processing subsystem 142B sends a data packet 700A to the initiating processing subsystem 142A as part of the transaction. For example, if the command field 202A of address packet 200A specifies an RTS transaction, processing subsystem 142B may return a data packet containing a copy of the cacheable coherency unit to the initiating processing subsystem 142A.
The second transaction initiated by processing subsystem 142A is initiated to access non-cacheable data mapped to I/O subsystem 146A. Processing subsystem 142A initiates this transaction by sending an address packet 200B on the address network. Address packet 200B includes a command field 202B and an address field 204B indicating at least part of the address of the non-cacheable data. In this address packet 200B, processing subsystem 142A encodes a command portion Cmd (e.g., indicating the type of non-cacheable transaction such as RIO (Read I/O) or WIO (Write I/O)), a granularity-field 210B, and a mask field 212B. Processing subsystem 142A encodes a value in the granularity field 210B indicative of the size of each data quantity selectable by the mask field 212B. For example, if there are eight bits in the mask field 212B and processing subsystem 142A is initiating the second transaction to access four eight-byte quantities, processing subsystem 142A may encode a granularity field value indicating that an eight-byte granularity is selected and selectively set the bits in the mask field 212 corresponding to the desired four eight-byte quantities. The granularity field 210B may not include any address bits.
In response to receiving address packet 200B and mapping the non-cacheable data indicated by the portion of the address included in address field 204B, I/O subsystem 146A may process the address packet 200B. I/O subsystem 146A may determine that the command field 202B of address packet 200B includes a mask field 212B and granularity field 210B in response to the value of the Cmd portion of the command field. The I/O subsystem may decode the granularity field 210B to determine the size of data quantity to which each bit in the mask field corresponds. The size indicated by the granularity field 210B may also indicate that the granularity field 210B does not include any additional address bits of the address of the non-cacheable data. I/O subsystem 146A may determine which of the N data quantities addressable by the address bits included in address field 204B are to be accessed by determining which of the N bits in the mask field 212B are set.
In response to processing the address packet 200B, the I/O subsystem 146A may return a data packet 700B to processing subsystem 142A. For example, if the Cmd portion of the command field 202B indicates an RIO (Read I/O) transaction, I/O subsystem 146A may return a data packet 700B including a copy of the portions of the data at the address in address field 202B selected by the value of mask field 212B. If the Cmd portion of the command field 202B indicates a WIO (Write I/O) transaction, I/O subsystem 146A may return a “pull” data packet 700B to processing subsystem 142A indicating that I/O subsystem 146A is ready to receive the write data. In response, processing subsystem 142A may send an additional data packet (not shown) including the new value of the portions of the data at the address indicated by the address field 202B selected by the value of mask field 212B.
Processing subsystem 142A may initiate a third transaction for different non-cacheable data mapped to I/O subsystem 146B by sending a third address packet 200C on the address network. Packet 200C includes a command field 202C and an address field 204C that includes at least a portion of the address of the non-cacheable data. Processing subsystem 142A encodes a Cmd portion of the command field 202C to indicate the type of transaction being initiated. For example, the Cmd portion may indicate that a RIO (Read I/O) transaction is being initiated. Processing subsystem 142A encodes a granularity field 210C in the command field 202C indicating the size of each data quantity associated with one of the bits in the mask field 212C. In this address packet 200C, the processing subsystem 142A encodes additional address bits into a portion of the granularity field 210C that is not needed to indicate granularity. For example, the granularity may be indicated by the lowest-order set bit in the granularity field 210C. Bits that are higher than the lowest-order set bit may not be needed to indicate granularity, and thus these bits may store additional address bits to be appended to the address bits included in address field 204C. The processing subsystem 142A encodes the mask field 212C to indicate which of a set of data quantities are involved in the transaction. For example, if the processing subsystem 142A is initiating a RIO for six out of eight one-byte quantities addressable by the address bits in the address field 204C and granularity field 210C, the processing subsystem 142A may set the six bits of the mask field 212C that correspond to the desired six bytes and clear the two bits of the mask field 212C associated with the other two bytes.
In response to mapping the non-cacheable data addressed by the address field 204C and receiving address packet 200C, I/O subsystem 146B may process address packet 200C. I/O subsystem 146B may determine the transaction to be performed from the Cmd portion of the command field 202C. Based on the granularity indicated in the granularity field 210C, I/O subsystem 146B determines that the granularity field 210C also includes additional address bits to be appended to the address bits in address field 204C. I/O subsystem 146B also identifies which of the N data quantities of the size indicated by the granularity field 210C addressed by the address bits in address field 204C and granularity field 210C are targeted in the transaction based on which of the N bits in the mask field 212C are set. In response to the address packet 200C, I/O subsystem 146B may return a data packet 700C to the initiating processing subsystem 142A. For example, if the Cmd portion of command field 202C indicates an RIO transaction, I/O subsystem 146B may return a data packet that includes a copy of each data quantity selected by mask field 212C.
As the example of
Note that all values herein refer to the logical values of signals or bits, and not to electrical levels. For example, a 1 value, or a signal that is asserted, refers to a signal with the given logical meaning, regardless of how that signal is implemented. Similarly, a set bit refers to a bit with a given logical meaning.
Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. For example, while the granularity field is described above as indicating granularity in bytes, other embodiments may indicate different granularities (e.g., words, megabytes, etc.). It is intended that the following claims be interpreted to embrace all such variations and modifications.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5873121 *||Nov 19, 1996||Feb 16, 1999||Advanced Micro Devices, Inc.||Efficient memory management system for minimizing overhead in storage of data transmitted in a network|
|US6122189 *||Oct 2, 1998||Sep 19, 2000||Rambus Inc.||Data packet with embedded mask|
|US6175884 *||Nov 3, 1998||Jan 16, 2001||Intel Corporation||Efficient communication of transaction types using separate and orthogonal attribute fields in packet headers transferred between hubs in a computer system|
|US7013157 *||Jun 3, 2002||Mar 14, 2006||Cisco Technology, Inc.||Method for multicast delivery with designated acknowledgment|
|US7069436||Nov 1, 2000||Jun 27, 2006||Sony Corporation||Information transmission system and method, transmitting apparatus, receiving apparatus, data processing device and data processing method, and recording medium|
|US7185103 *||Sep 10, 2002||Feb 27, 2007||Juniper Networks, Inc.||Rate-controlled transmission of traffic flow information|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8275972 *||Aug 23, 2006||Sep 25, 2012||Ati Technologies, Inc.||Write data mask method and system|
|US8412912||May 2, 2012||Apr 2, 2013||Ati Technologies Ulc||Write data mask method and system|
|US8732415||Feb 26, 2013||May 20, 2014||Ati Technologies Ulc||Write data mask method and system|
|US20080052474 *||Aug 23, 2006||Feb 28, 2008||Ati Technologies, Inc||Write data mask method and system|
|U.S. Classification||709/236, 370/470, 370/472, 370/476|
|Cooperative Classification||H04L45/00, H04L45/12, G06F15/17337|
|European Classification||H04L45/12, H04L45/00, G06F15/173D|
|Oct 12, 2010||CC||Certificate of correction|
|Mar 6, 2013||FPAY||Fee payment|
Year of fee payment: 4
|Dec 16, 2015||AS||Assignment|
Owner name: ORACLE AMERICA, INC., CALIFORNIA
Free format text: MERGER AND CHANGE OF NAME;ASSIGNORS:ORACLE USA, INC.;SUN MICROSYSTEMS, INC.;ORACLE AMERICA, INC.;REEL/FRAME:037304/0194
Effective date: 20100212