|Publication number||US6801970 B2|
|Application number||US 09/968,056|
|Publication date||Oct 5, 2004|
|Filing date||Sep 30, 2001|
|Priority date||Sep 30, 2001|
|Also published as||US20030065842|
|Publication number||09968056, 968056, US 6801970 B2, US 6801970B2, US-B2-6801970, US6801970 B2, US6801970B2|
|Inventors||Dwight D. Riley, Chris Pettey|
|Original Assignee||Hewlett-Packard Development Company, L.P.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (5), Non-Patent Citations (2), Referenced by (22), Classifications (12), Legal Events (6)|
|External Links: USPTO, USPTO Assignment, Espacenet|
1. Field of the Invention
The present invention is related to scheduling transactions on computer bus systems, and in particular, support for priority transaction scheduling on computer bus systems employing the high-performance extension (PCI-X) to the peripheral component interconnect (PCI) protocol.
2. Description of the Related Art
PCI and PCI-X are bus systems widely used in computers. PCI-X provides performance improvements over PCI because PCI-X operates at higher clock frequencies made possible by a register-to-register protocol and other protocol enhancements, such as an attribute phase and split transactions. PCI-X also is supposed to lead to follow-on technologies that include better performance capabilities. This is important because with current PCI-X, if the rate of servicing a device on the bus requiring immediate servicing were to drop below a threshold, the computer system would crash. The device, for example, could be the computer system's main microprocessor. One possible improvement could be to enhance stability from computer system crashes and enhance the servicing of transactions for such devices to benefit performance.
Current PCI capable devices have a Scheduler to implement transaction Ordering Rules to determine which transaction in a queue will be handled next. The Ordering Rules are rules for emptying the queues. They are specified to guarantee a consistent view of data by all devices in the system and rational behavior between multiple devices and their software drivers (if any). Conventional PCI Ordering Rules apply globally to all transactions without regard to the underlying communications semantics. The Relaxed Ordering attribute in PCI-X transactions allows certain ordering requirements to be indicated explicitly on a transaction-by-transaction basis, providing a tool to help system designers and software writers achieve better overall performance. The current PCI capable devices, however, have no way of determining the relative priority between transactions communicated to them from other devices. The current PCI-X protocol and scheduling is still subject to system crashes because a clear priority usage model is not provided for devices that should have priority for exclusive or early handling.
There are three types of queues that the Scheduler controls. These queues are the Posted Write, Completion, and Request queues, each of which handles a different transaction class. Certain classes of transactions are permitted to bypass other classes for handling next, and other transactions are not so permitted. For example, the PCI-X Relaxed Ordering attribute may be used to allow a memory write transaction to pass other memory writes and to allow a Split Read Completion to pass memory writes.
Therefore, for these and other reasons it is imperative to improve transaction handling priority without adding cost to the system. A better transaction handling scheme may predicate the longevity of PCI-X.
Embodiments of the present invention feature support for controlling transaction priority for PCI-X. These embodiments provide indicia of priority to communicate to PCI-X-to-PCI-X bridges that a transaction tagged by the indicia should be scheduled, using a special handling instruction, ahead of any other transaction not invoking this special handling instruction. The special handling instruction allows the priority transaction to be scheduled first. The indicia of priority can be implemented by setting an unused bit(s) in a PCI-X attribute field or multiplexed with another used signal bit(s) to schedule the associated transaction as the priority transaction over the other transactions that do not have their corresponding bit set. The present invention is useful for interrupt messaging, audio streams, video streams, isochronous transactions or for high performance, low bandwidth control structures used for communication in a multiprocessor architecture across PCI-X.
Embodiments of the present invention feature a technique, in a computer system, of providing indicia of priority for PCI-X transactions. According to the technique, a plurality of PCI-X phases is created in the computer system. A transaction priority mechanism for PCI-X is provided in the computer system by inserting priority indicia into an unused attribute portion (e.g., at the location of an unused bit(s)) of the plurality of PCI-X phases.
A better understanding of the present invention can be obtained when the following detailed description of the preferred embodiment is considered in conjunction with the following drawings, in which:
FIG. 1 is an exemplary schematic block diagram of a computer system in accordance with an embodiment of the invention;
FIG. 2 is an exemplary schematic diagram of a printed circuit motherboard of the computer system of FIG. 1;
FIG. 3 illustrates a prior art PCI-X Memory Write Block (32-bit or 64-bit) or Split Completion transaction identifying all the PCI-X transaction phases, in accordance with an embodiment of the invention;
FIG. 4 illustrates Burst Transaction Requester Attribute Bit assignment in accordance with an embodiment of the invention; and
FIG. 5 illustrates DWORD Transaction Requester Attribute bit assignment in accordance with an embodiment of the invention.
PCI-X is described in the PCI-X Addendum to the PCI Local Bus Specification, Revision 1.0 (the PCI-X specification) and in U.S. Pat. No. 6,266,731B1 to Riley et al., entitled High Speed Peripheral Interconnect Apparatus, Method and System, which are incorporated by reference herein in their entireties. PCI is described in the PCI Local Bus Specification, Revision 2.2 (the general PCI specification), which also is incorporated by reference in its entirety. When PCI is mentioned herein, it is meant to include all of PCI, including PCI-X, and when PCI bus, the bus operating in PCI mode, or PCI device are mentioned herein, they are meant to include PCI-X. Full PCI protocol refers to the PCI protocol that includes PCI-X.
The present invention provides a technique for enhancing the operation of computer system buses that use the extensions to the peripheral component interconnect specification (i.e., PCI-X), as well as logic circuits and signal protocols thereof. The technique is fully backward compatible with current PCI and PCI-X systems or devices, and such systems or devices that do not recognize the signals and techniques provided for in the present invention will simply ignore them. For illustrative purposes, embodiments of the present invention are described herein for computer systems using Intel Corporation microprocessor architectures, and certain terms and references are specific to such processor platforms. PCI-X and the enhancements described herein according to the present invention, however, are hardware independent, and may be used with any host computer designed for this interconnect standard. As will be appreciated by those skilled in the art of computer systems, the present invention may be adapted and applied to any computer platform using the PCI-X standard and other standards.
Referring to FIG. 1, an exemplary schematic block diagram of a computer system according to the present invention is illustrated. The computer system is generally indicated by the numeral 100 and includes central processing unit(s) (CPU) 102, core logic 104 (“North Bridge”), system random access memory (RAM) 106, a video graphics controller 110, a local frame buffer 108, a video display 112, a PCI/SCSI bus adapter 114, a PCI/EISA/ISA bridge 116 (“South Bridge”), a PCI/IDE controller 118, and, optionally, a network interface card (NIC) 122. Single or multilevel cache memory (not illustrated) may also be included in the computer system 100 according to the current art of microprocessor computer systems. The CPU 102 may be a plurality of CPUs 102 in a symmetric or asymmetric multi-processor configuration.
The CPU 102 is connected to the core logic 104 through a CPU host bus 103. The system RAM 106 is connected to the core logic 104 through a memory bus 105. The core logic 104 includes a host-to-PCI bridge between the host bus 103, the memory bus 105 and a PCI bus 109. More than one PCI bus is contemplated herein as well as PCI-X-to-PCI-X bridges (not illustrated), and is within the scope and intent of the present invention. The local frame buffer 108 is connected between the video graphics controller 110 and the PCI bus 109. The PCI/SCSI bus adapter 114, PCI/EISA/ISA bridge 116, PCI/IDE controller 118 and the NIC 122 are connected to the PCI bus 109. Some of the PCI devices, such as the Video controller 110 and NIC 122, may plug into PCI connectors on the computer system 100 motherboard (FIG. 2).
Hard disk 130 and tape drive 132 are connected to the PCI/SCSI bus adapter 114 through a SCSI bus 111. The NIC 122 may be connected to a local area network 119. The PCI/EISA/ISA bridge 116 connects over an EISA/ISA bus 113 to a ROM BIOS 140, non-volatile random access memory (NVRAM) 142, modem 120, and input-output controller 126. The modem 120 connects to a telephone line 121. The input-output controller 126 interfaces with a keyboard 146, real time clock (RTC) 144, mouse 148, floppy disk drive (FDD) 150, serial port 152, and parallel port 154. The EISA/ISA bus 113 is a slower information bus than the PCI bus 109, but its interfacing cost is less.
When the computer system 100 is first turned on, start-up information stored in the ROM BIOS 140 is used to begin operation thereof. Basic setup (BIOS) instructions are stored in the ROM BIOS 140 so that the computer system 100 can load more complex operating system (OS) software from a memory storage device, such as the disk 130. Before the operating system software can be loaded, however, certain hardware in the computer system 100 is configured to properly transfer information from the disk 130 to the CPU 102. In the computer system 100 illustrated in FIG. 1, the PCI/SCSI bus adapter 114 is configured to respond to commands from the CPU 102 over the PCI bus 109 and transfer information from the disk 130 to the CPU 102 via buses 109 and 103. The PCI/SCSI bus adapter 114 is a PCI device and remains platform independent. Therefore, separate hardware independent commands are used to setup and control any PCI device in the computer system 100. These hardware independent commands, however, are located in PCI BIOS contained in the computer system ROM BIOS 140. The PCI BIOS is firmware that is hardware specific but meets the general PCI specification. Plug and play and PCI devices in the computer system are detected and configured when a system configuration program is executed. The results of the plug and play and PCI device configurations are stored in the NVRAM 142 for later use by the startup programs in the ROM BIOS 140 and the PCI BIOS that configure the necessary computer system 100 devices during startup. Also during startup a built-in-self-test (BIST) may perform diagnostic testing of components, such as PCI devices, in the computer system.
Referring to FIG. 2, a schematic diagram of an exemplary computer system motherboard according to FIG. 1 is illustrated. The computer system motherboard 200 includes printed circuit board 202, on which components and connectors are mounted. The printed circuit board 202 includes conductive printed wiring used to interconnect these components and connectors. The conductive printed wiring (illustrated as buses 103, 105 and 109) may be arranged into signal buses having controlled impedance and signaling characteristics. Illustrated on the printed circuit board 202 are the core logic 104, CPU(s) 102, RAM 106, embedded PCI/ISA/EISA bridge 116, embedded PCI/SCSI bus adapter 114, and PCI connectors 206 a, 206 b (connectors are the same for PCI and PCI-X). The motherboard 200 may be assembled into a case with a power supply, disk drives, etc. (not illustrated), which form the computer system 100 of FIG. 1.
The present invention provides support for priority transactions on the PCI bus, which can be implemented in the computer system of FIGS. 1 and 2, as well as in a variety of other computer systems and computer buses. For example, the present invention could be implemented in a computer system employing a serial point-to-point bus, such as 3GIO by Intel Corporation, or another one employing a parallel bus, such as HyperTransport™. In accordance with embodiments of the invention, indicia of a priority transaction are provided to communicate to PCI-X-to-PCI-X bridges and the Completer device that the transaction is an exclusive priority transaction that should be scheduled by the Scheduler using a special handling instruction. The priority transaction is scheduled ahead of any other transaction that does not have this indicia set, thereby allowing priority transactions to be scheduled first or early. Embodiments of the present invention implement these indicia by setting a bit in a PCI-X attribute field of the transaction that schedules the associated transaction as the priority transaction ahead of other transactions that do not have the corresponding bit set. In a sense, the present invention implements a simplistic quality of service technique across the interconnect. The present invention can be used for interrupt messaging, audio streams, video streams, isochronous transactions, or for high performance, low bandwidth control structures used for communication in a multiprocessor architecture across PCI-X.
FIG. 3 illustrates a prior art PCI-X Memory Write Block (32-bit or 64-bit) or Split Completion transaction identifying all the PCI-X transaction phases, in accordance with an embodiment of the invention. A PCI-X bus transaction or transfer consists of one (or more) address phases, an attribute phase, and any number of data phases transferred across an interconnect to a device, for example, a peripheral device, within or without the computer system. The PCI-X transaction is a combination of address 300, attribute 302, clock (target response) 304, and data 306 phases. The transaction shown in FIG. 3 also includes a bus turn-around 308 phase. The transaction is signaled by a single assertion or activation of FRAME# on the bus, such as the PCI-X bus 109 of FIGS. 1 and 2, as will be appreciated by those skilled in the art. The attribute phase 302 is the clock after the address phase 300, as shown in prior art FIG. 3. The attribute phase 302 uses a 36-bit attribute field contained on Address and Data lanes AD[31:0] and Bus Command and Byte Enables lanes C/BE[3:0]# (i.e., the lower bus halves) during the attribute phase of a PCI-X transaction, as schematically shown in FIG. 3.
The attribute field defines and describes each PCI-X bus transaction in more detail than the conventional PCI specification allows. It contains several bit assignments that include information about the size of the transaction (byte count), bus number, ordering of transactions, cache snooping requirements, sequence number, identity of the transaction initiator or device addressed by the transaction, and other transaction handling instructions. The attributes appear in the clock immediately following the address phase on the AD bus, as indicated in FIG. 3. According to the PCI-X specification, the initiator of every transaction drives attributes on the C/BE[3:0]# and AD[31:0] buses in the attribute phase. The attribute phase is always a single clock regardless of the width of the data transfer or the width of the address (single or dual address cycle).
The upper bus halves (AD[63:32]) and (C/BE[7:4]#) of 64-bit devices are reserved and driven high (RDH) during the attribute phase 302, as will be appreciated by those skilled in the art. These upper bus halves are unused bits in PCI-X that are available for the present invention. Some other bits also may be unused (e.g., reserved bits) and available in the lower bus halves (AD[31:0]) and (C/BE[3:0]).
FIG. 4 illustrates a BURST Transaction Requester Attribute Bit assignment in accordance with an embodiment of the invention. As will be appreciated by those skilled in the art, the Requester is the original initiator of a transaction. FIG. 4 is similar to a conventional PCI-X bit assignment except for the use of an unused bit(s) in the attribute phase 302. According to the present invention, the bit assignment for the attribute phase 302 includes priority (PR) indicia or tag 400, for example, in the lower bus half (AD[31:0]) of the AD lanes., as shown in FIG. 4. The indicia in general consist of a bit(s) chosen and designated from the unused bits. In this exemplary embodiment, the indicia 400 are located at bit 31, which is normally reserved in conventional PCI-X. The indicia 400 are set to communicate to a PCI-X-to-PCI-X bridge or other PCI-X compatible device, or other bridge or device (not shown) receiving the transaction information in (or outside) the computer system (e.g., the computer system 100 of FIG. 1) that the transaction in which it is set is tagged for exclusive priority to be scheduled for immediate handling ahead of any other transaction not having the analogous indicia set. The location of the indicia 400 is merely exemplary in FIG. 4. In other embodiments, it could instead replace any other unused bit(s) (e.g., reserved, such as RDH), or be multiplexed with any other bit(s) signals already being used in the attribute phase 302, as will be appreciated by those skilled in the art.
FIG. 5 illustrates DWORD Transaction Requester Attribute bit assignment in accordance with an embodiment of the invention. The bit assignment in FIG. 5 includes indicia 500 (e.g., a bit(s)) at the position of conventionally reserved bit 31 in the lower bus half of the AD lane. Indicia 500 are like the indicia 400 of FIG. 4, but for a DWORD transaction rather than a burst transaction. Again, FIG. 5 is merely exemplary and the specific location of indicia 500 can be different, as discussed above.
In certain other embodiments according to the invention, the tagged priority transaction instead could be handled by the Scheduler earlier than some other transactions not having the indicia 400 or 500 set (i.e., untagged), but not necessarily earlier than all untagged transactions. This would be implemented according to the special handling instruction used by the Scheduler.
In yet other embodiments of the invention, the tagged transaction could be implemented similarly, but not equivalently, to a PCI-X Relaxed Ordering transaction that uses the set indicia 400 or 500 as a Relaxed Ordering bit (or relaxed ordered attribute bit) to invoke another special handling instruction allowing certain ordering requirements to be indicated explicitly on a transaction-by-transaction basis. For example, the PCI-X Relaxed Ordering attribute may be used to allow the tagged transaction to pass memory writes or other transactions, as will be appreciated by those skilled in the art. In other embodiments of the invention, the tagged transaction, by setting the indicia 400 or 500, could be implemented to meet the PCI-X Ordering Rules that use the Scheduler to determine which transaction in a queue will be handled next, but handling the tagged transaction as soon as possible. Alternatively, in another embodiment, the tagged transaction could be implemented as a sort of combination of the previous two embodiments just discussed in which the indicia 400 or 500 is used in conjunction with the Relaxation Ordering bit. In this embodiment, if the indicia 400 or 500 is set for a tagged transaction but the Relaxation Ordering bit is not set for that transaction, then the tagged transaction is handled as an exclusive priority transaction to be handled next immediately by the PCI-X-to-PCI-X bridge. On the other hand, if both the indicia 400 or 500 and the Relaxation Ordering bit are set, then the tagged transaction is handled only as a Relaxation Ordering transaction as soon as possible, as above.
Still other embodiments of the invention require that another attribute be created and carried in the attribute phase 302 in FIG. 3. Alternatively, one or more reserved bits in conventional PCI-X could be replaced with indicia (e.g., a bit(s)) to create and carry this new attribute in the attribute phase 302. Conventional PCI-X supports various classes of commands with their own set of attributes, including Burst, DWORD, Split Completion/Exception Message, and Configuration. These commands are carried in the C/BE# lanes. For the present embodiments, a new class of command would be created having its own attribute. The new class of command would invoke another special handling instruction for the Scheduler to handle the transaction as a priority transaction. This additional attribute could be created by a command in the C/BE# lane RDH bits, or multiplexed with another bit in the C/BE# lanes. This represents a deviation from the PCI-X specification. If the priority transaction is a DWORD transaction, another possibility is for this new command to be covered in an unused bit(s) in the DWORD transaction's byte count field, which is not used in DWORD transactions, or the indicia 500 itself could be located and set in a priority DWORD transaction's unused bit(s) in its byte count field, as will be appreciated by those skilled in the art. This new attribute could be RDH throughout, but have indicia like the indicia 500 of FIG. 5 (e.g., a bit(s)) that could be set so that the transaction becomes a priority transaction for special handling as in any of the other embodiments described herein using an unused bit(s).
The present invention does not guarantee the rate of delivery, but for a priority transaction, it only places the tagged transaction at the head of the queue for service by the PCI-X-to-PCI-X bridge and the Completer or other device. The present invention can be implemented in 32-bit or 64-bit extensions to the PCI-X protocol.
The foregoing disclosure and description of the preferred embodiment are illustrative and explanatory thereof, and various changes in the components, circuit elements, circuit configurations, signals, and signal connections, as well as in the details of the illustrated circuitry and construction and method of operation may be made without departing from the spirit and scope of the invention.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5625824 *||Mar 3, 1995||Apr 29, 1997||Compaq Computer Corporation||Circuit for selectively preventing a microprocessor from posting write cycles|
|US5987555 *||Dec 22, 1997||Nov 16, 1999||Compaq Computer Corporation||Dynamic delayed transaction discard counter in a bus bridge of a computer system|
|US6175889 *||Oct 21, 1998||Jan 16, 2001||Compaq Computer Corporation||Apparatus, method and system for a computer CPU and memory to high speed peripheral interconnect bridge having a plurality of physical buses with a single logical bus number|
|US6247089 *||Sep 16, 1998||Jun 12, 2001||Advanced Micro Devices, Inc.||Efficient data loading scheme to minimize PCI bus arbitrations delays and wait states|
|US6279087 *||Dec 22, 1997||Aug 21, 2001||Compaq Computer Corporation||System and method for maintaining coherency and improving performance in a bus bridge supporting write posting operations|
|1||*||"A flexible compatible PCI interface for nuclear experiments" by Saleh, H.; Engels, R.; Reinartz, R.; Reiinhart, P; Rongen, F. (abstract only).|
|2||*||"HARTIK 3.0: a portable system for developing real-time applications" by Lamastra, G.; Lipari, G.; Buttazzo, G.; Casile, A.: Conticelli, F. (abstract only).*|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7120722 *||Dec 17, 2002||Oct 10, 2006||Intel Corporation||Using information provided through tag space|
|US7213094 *||Feb 17, 2004||May 1, 2007||Intel Corporation||Method and apparatus for managing buffers in PCI bridges|
|US7366808 *||Nov 23, 2005||Apr 29, 2008||Hitachi, Ltd.||System, method and apparatus for multiple-protocol-accessible OSD storage subsystem|
|US7447975 *||Sep 12, 2002||Nov 4, 2008||Hewlett-Packard Development Company, L.P.||Supporting cyclic redundancy checking for PCI-X|
|US7454540 *||Jan 17, 2006||Nov 18, 2008||Ricoh Company, Ltd.||Data transferring system and electronic apparatus|
|US7536491||Mar 11, 2008||May 19, 2009||Hitachi, Ltd.||System, method and apparatus for multiple-protocol-accessible OSD storage subsystem|
|US8006017 *||Aug 23, 2011||Intel Corporation||Stream priority|
|US8060680 *||Sep 16, 2002||Nov 15, 2011||Hewlett-Packard Development Company, L.P.||Method of allocating memory|
|US8108584 *||Oct 15, 2008||Jan 31, 2012||Intel Corporation||Use of completer knowledge of memory region ordering requirements to modify transaction attributes|
|US8205026||Jun 19, 2012||Intel Corporation||Stream priority|
|US8307144||Dec 22, 2011||Nov 6, 2012||Intel Corporation||Use of completer knowledge of memory region ordering requirements to modify transaction attributes|
|US8347011 *||Jan 1, 2013||Intel Corporation||Stream priority|
|US20040054839 *||Sep 16, 2002||Mar 18, 2004||Lee Terry Ping-Chung||Method of allocating memory to peripheral component interconnect (PCI) devices|
|US20040054955 *||Sep 12, 2002||Mar 18, 2004||Compaq Information Technologies Group, L.P.||Supporting cyclic redundancy checking for PCI-X|
|US20050182886 *||Feb 17, 2004||Aug 18, 2005||Edirisooriya Geetani R.||Method and apparatus for supporting multi-function PCI devices in PCI bridges|
|US20060136639 *||Dec 21, 2004||Jun 22, 2006||Intel Corporation||Stream priority|
|US20060171300 *||Jan 17, 2006||Aug 3, 2006||Koji Oshikiri||Data transferring system and electronic apparatus|
|US20070118676 *||Nov 23, 2005||May 24, 2007||Yoshiki Kano||System, method and apparatus for multiple-protocol-accessible OSD storage subsystem|
|US20080209090 *||Mar 11, 2008||Aug 28, 2008||Hitachi, Ltd.||System, method and apparatus for multiple-protocol-accessible OSD storage subsystem|
|US20100095032 *||Oct 15, 2008||Apr 15, 2010||David Harriman||Use of completer knowledge of memory region ordering requirements to modify transaction attributes|
|US20120042106 *||Oct 27, 2011||Feb 16, 2012||Futral William T||Stream priority|
|CN102981984B *||Oct 14, 2009||Apr 6, 2016||英特尔公司||使用完成者对存储器区域排序要求的知识来修改事务属性|
|U.S. Classification||710/105, 370/402, 710/107, 710/306, 710/313|
|International Classification||G06F13/42, G06F13/40, G06F13/36, G06F9/46|
|Cooperative Classification||G06F2213/0024, G06F13/423|
|May 12, 2004||AS||Assignment|
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS
Free format text: CHANGE OF NAME;ASSIGNOR:COMPAQ INFORMATION TECHNOLOGIES GROUP LP;REEL/FRAME:014628/0103
Effective date: 20021001
|Apr 7, 2008||FPAY||Fee payment|
Year of fee payment: 4
|Apr 14, 2008||REMI||Maintenance fee reminder mailed|
|Apr 5, 2012||FPAY||Fee payment|
Year of fee payment: 8
|Nov 9, 2015||AS||Assignment|
Owner name: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP, TEXAS
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.;REEL/FRAME:037079/0001
Effective date: 20151027
|Mar 24, 2016||FPAY||Fee payment|
Year of fee payment: 12