US 7984242 B2
A barrier for synchronizing program threads for a plurality of processors includes a filter configured to be coupled to a plurality of processors executing a plurality of threads to be synchronized. The filter is configured to monitor and selectively block fill requests for instruction cache lines. A method for synchronizing program threads for a plurality of processors includes configuring a filter to monitor and selectively block fill requests for instruction cache lines for a plurality of processors executing a plurality of threads to be synchronized.
1. A system for synchronizing program threads for a plurality of processors comprising:
a filter configured to be coupled to a plurality of processors executing a plurality of threads to be synchronized; and
in which said filter is configured to monitor and selectively block fill requests for instruction cache lines corresponding to the threads to be synchronized.
2. The system of
3. The system of
4. The system of
5. The system of
6. The system of
7. The system of
8. The system of
9. A method for synchronizing program threads for a plurality of processors comprising:
configuring a filter to monitor and selectively block fill requests for instruction cache lines for a plurality of processors executing a plurality of threads to be synchronized.
10. The method of
11. The method of
12. The method of
13. The method of
14. The method of
15. The method of
16. The method of
17. The method of
18. A filter for synchronizing program threads for a plurality of processors comprising:
said filter configured to be coupled to a plurality of processors executing a plurality of threads to be synchronized, where said filter is configured to selectively halt execution of any of said threads until said threads have been synchronized.
19. The filter of
20. The filter of
This application is a continuation of, and claims the priority under 35 U.S.C. §120 of U.S. patent application Ser. No. 11/271,535 filed Nov. 10, 2005, entitled “Program Thread Synchronization,” now U.S. Pat. No. 7,555,607, which is incorporated herein by reference in its entirety. That application is also related to U.S. application Ser. No. 11/272,197, filed on Nov. 10, 2005, which is now U.S. Pat. No. 7,587,555, the entire contents of which are hereby incorporated by reference.
A computer system may include two or more interconnected processors that execute multiple program threads concurrently. Such a multiprocessor computing system allows a computing task to be completed more quickly by dividing the task into smaller tasks which are performed concurrently by the various processors.
A multiprocessor system may include a number of processors formed on separate computer chips which are then assembled into a multiprocessor system. Such systems are also referred to as parallel computing systems. Increasingly, computer chips are being manufactured that include multiple processors (or “cores”) on a single computer chip. Such single-chip multiprocessors tend to occupy less space and to better facilitate communication between the processors than traditional multiprocessor systems.
In either case, synchronization is typically required among the threads executed by a multiprocessor system. For example, interdependencies among the threads may require that one or more threads produce a result before one or more other threads make use of the result. Thus, it may be necessary to utilize a synchronization barrier in which each thread to be synchronized is allowed to execute until it reaches a barrier, the thread is stalled at the barrier until the other threads reach the barrier; and, then, the threads are allowed to resume execution.
Conventional synchronization techniques may require that a counter is shared by all of the threads to be synchronized. The counter is incremented by each thread that arrives at the barrier. Each thread repeatedly tests whether the counter has reached a given value to determine whether the other threads have also reached the barrier. Because each thread contends for access to it, the counter may become a hot spot. To lessen contention for the counter, the threads may test less often; however, this may increase the time required for each thread to learn that the barrier has been reached.
Alternatively, a synchronization mechanism can be implemented by specialized hardware. However, this can complicate the design and manufacture of multiprocessor system that includes such specialized hardware and can limit flexibility of the system.
The present specification provides a method of and system for synchronizing program threads executing on one or more computer processors. Execution of multiple threads is synchronized by stalling execution of each of the program threads at a barrier until all of the threads to be synchronized reach the barrier. Each thread is stalled by invalidating the cache line for a next instruction in instruction cache (I-cache) for the processor executing the thread.
To continue stalling each thread after the cache lines are invalidated, a barrier filter monitors I-cache fill requests and blocks the provision of selected cache lines to the processors. Once all of the threads to be synchronized reach the barrier, the invalidated cache lines are validated and fill requests for the cache lines are allowed. As a result, the threads resume execution. Execution can be resumed quickly because reading the validated instruction can be accomplished quickly, typically in one clock cycle.
An interface 110 interfaces the processors 102A-D, I-caches 104A-D and filter 108 to a system bus 112. The system bus 112 allows the processors 102A-D to communicate with additional portions of the system 100, such as program memory, one or more outer cache levels and I/O devices (not shown). In an embodiment, the processors 102A-D, I-caches 104A-D, filter 108 and interface 110 are included in a multi-core integrated circuit chip 114. Alternatively, processors of the system 100 may be implemented by multiple integrated circuit chips. For example, the processors 102A-D may each be implemented as a separate integrated circuit chip.
Each of the processors 102A-D receives program instructions that it is to execute from its respective I-cache 104A-D. For example, the processor 102A receives its instructions from the I-cache 104A; the processor 102B receives its instructions from the I-cache 104B; and so forth. While four processors 102 are shown, it will be apparent that a multiprocessor system could have more or fewer processors.
Each I-cache 104A-D may have an associated I-cache controller (not shown). The I-cache controllers load instructions into the I-caches 104A-D by retrieving the instructions from program memory or an outer cache level (e.g., level-2 cache). As shown in
When threads are to be synchronized, each is stalled at a barrier while it awaits the arrival of the other threads to the barrier. As mentioned, to stall a thread executing on a processor, the cache line for a next instruction to be executed by the processor is invalidated. The striped lines 107A-D in the I-caches 104 of
The filter 108 monitors, and selectively blocks, I-cache fill requests. This may be accomplished by the filter 108 monitoring the interconnect 106 for specified addresses contained in the fill requests. When a fill request is identified by the filter 108 as one that is to be blocked, the filter 108 blocks that request. This may be accomplished by the filter 108 preventing the I-cache from loading the identified cache lines or by failing to pass the identified fill request to level-2 cache.
In an embodiment, the filter 108 is informed of the presence of a barrier in advance of any of the threads reaching the barrier. Upon reaching the barrier, each of the threads to be synchronized may invalidate its own cache line. The filter 108 blocks fill requests for the invalidated cache lines until all of the threads reach the barrier. In another embodiment, the first thread that arrives at a barrier initiates synchronization by invalidating the appropriate cache lines for all of the threads to be synchronized. This thread may also notify the filter 108 to commence blocking fill requests for the invalidated cache lines.
The locations of barriers in each thread to be synchronized are determined in order to synchronize the threads. The locations will depend upon tasks to be performed by the various processors and interdependencies of the tasks. Accordingly, the locations of barriers may be determined when the threads are programmed or compiled. In an embodiment in which threads are capable of invalidating their own cache line, a barrier procedure call is placed in each thread to be synchronized at one or more appropriate synchronization points. The barrier filter 108 is informed of the presence of these barriers.
The program text of the barrier procedure 200 is preferably aligned with cache lines in I-cache 102 and begins at an address given as A, which is the starting address of the head 202 of the barrier. The size of the I-cache lines is given as L, which is typically smaller than that of outer cache levels. Preferably, the second line of the program text of the barrier procedure in I-cache contains the tail 204. Thus, the address for the second line may be given as A+L.
As shown in
The head 202 may also include an instruction to discard pre-fetched instructions. This purges any copies of the invalidated cache line that may have been pre-fetched and kept internally to the processor 102. As shown in
The tail 204 of the barrier procedure 200 informs the filter 108 that the current thread is exiting the barrier. This is so that the filter 108 can block any future fill requests for address A+L coming from this thread. Blocking future fill requests avoids one thread running ahead of the other threads such that it reaches and surpasses another barrier (or another instance of the same static call to the barrier procedure) before the other threads have exited the current barrier. As shown in
As is also shown in
The filter table 300 may be initialized at the beginning of an application. Thus, support of the operating system may be required to provide the physical addresses of the head and tail parts of the barrier, that is, the addresses A and E.
In the blocking state 402, the filter 108 processes each incoming cache fill request by checking to see whether the target address is blocked. This may be accomplished by the filter 108 comparing the target address for fill requests to its address A entries in column 302 of the table 300. The fill requests will target the address immediately following the address A (i.e. address A+L) though the filter 108 may identify these requests based on its knowledge of the address A. If the target address is blocked, the filter 108 does not pass the requested cache line to the corresponding I-cache. Instead, the filter 108 may obtain the requested cache line from memory and store it in the corresponding cache line content field 312 of the table 300. The filter 108 may then set the bit in the pending vector (field 308) that corresponds to the requesting processor 102 to indicate that the processor 102 has reached the barrier and has a pending request. Otherwise, if the target address is not blocked, the filter 108 passes the requested cache line to the corresponding processor 102.
In addition to monitoring fill requests in the blocking state 402, the filter 108 monitors invalidate instructions issued by the processors 102. For each invalidate that targets address A+L, the filter 108 increments the corresponding counter C (field 306) in the table 300. The filter 108 remains in the blocking state until the counter C reaches N, the number of threads to be synchronized. Once the counter C reaches this number, this indicates that all of the threads to be synchronized have reached the barrier. At this point, the automaton for the barrier moves to the service state 404 and resets the valid bit (field 312) to indicate that the cache line content is valid. Thus, the filter 108 monitors invalidate requests to determine when each thread has reached the barrier. It will be apparent that the filter 108 could monitor other messages from the processors to determine when each thread has reached the barrier. For example, the filter 108 could monitor stores to specific locations.
In the service state 404, the pending fill requests are serviced. New fill requests arriving while in the state 404 are serviced as well. As in the blocking state 402, the filter 108 processes each incoming cache fill request by checking to see whether the target address is blocked. However, in the service state for a particular barrier, the corresponding address A+L is not blocked. Thus, those fill requests are serviced. In the service state 404 the filter 108 monitors invalidate instructions targeting the address E for the barrier. The filter 108 identifies which processor issues each invalidate targeting the address E and sets corresponding bit in the exited vector (field 310) to indicate which threads have exited the barrier. Once the bit in the exited vector corresponding to a processor 102 is set, the filter 108 will not service any further fill requests from that processor 102 that target the address A+L for the barrier.
When setting the corresponding bit in the exited vector (field 310) would cause the vector 310 to have all of its bit equal to one, this indicates that all of the N threads have been synchronized and have exited the barrier. Accordingly, the automaton for the barrier returns to the blocking state 402. In addition, the counter C (field 306), the pending vector (field 308) and the exited vector (field 310) are cleared. The cache line content (field 312) need not be cleared since it may be reused. This process of selectively blocking fill requests (in the blocking state 402) and then servicing the requests (in the service state 404) is repeated for each barrier instance encountered as the threads are executed.
In another embodiment, the first thread that arrives at a barrier initiates synchronization by causing the appropriate cache lines to be invalidated for all of the threads to be synchronized. This thread also notifies the filter 108 to commence blocking fill requests for the invalidated cache lines. Threads that reach the barrier subsequent to the first thread already have their cache lines invalidated so that when they attempt to read this cache line, they stall at the barrier. Preferably in this embodiment, threads that reach the barrier after the first thread do not attempt to invalidate the cache lines of the other threads.
So that a thread can determine whether it is the first thread to reach the barrier, each thread may request exclusive access to a memory location in the system 100 (
For this embodiment, the barrier procedure may be modified from that shown in
In an embodiment of the invention, the OS of the system 100 provides support for implementing the barriers. In this case, the OS accesses the filter 108 so as to save and to restore the state of the filter 108, possibly including the table 300, in the event of context changes, such as when an application is de-scheduled or a page holding program text is swapped out of memory. In case the OS does not provide such support, compensating actions may be performed elsewhere, such as by the filter 108 itself. For example, the filter 108 may monitor the OS to identify context changes by associating process identifications of the threads to contexts. The filter 108 may then associate the process identifications with the barriers in its table 300. In this way, the filter 108 may invalidate cache lines and block fill requests for each processor according to the process identification for the thread currently executing on the processor. So that the filter 108 is informed of the process identification for each thread, the invalidate instructions of the barrier procedure (e.g., procedure 300 of
In accordance with embodiments described above, it is assumed that a barrier is used to synchronize one thread per processor. Thus, there are N threads to be synchronized, one for each of N processors. In this case, the pending vector 308 and the exited vector 310 may each have N bits. Also, the counter C (field 306) indicates that all of the threads have reached the barrier when it reaches a count of N. Having one thread per processor also allows the barrier filter 108 to use the processor ID to identify threads that enter or exit a barrier. Thus, the processor ID may be carried with the invalidate request issued by threads exiting a barrier to enable the filter 108 to identify which thread it should stop servicing requests from. Accordingly, allowing one thread per processor simplifies the thread tracking performed by the filter 108. In other embodiments, there may be more or fewer threads to be synchronized than processors.
In an embodiment in which there are fewer threads to be synchronized than processors, a bit vector which indicates the threads to be synchronized for each barrier may be added to each barrier entry in the filter table 300. This vector may be referred to a participation vector. For example, where there are eight processors, but for a particular barrier, only the first five and the last one of the processors execute threads to be synchronized, the participation vector may be given as 11110001. Thus, a column may be added to the table 300 in which the participation vector entered may be different for different barriers in the table 300 so as to indicate which threads and corresponding processors participate in each barrier.
A number of bits set in the participation vector indicates the number of threads that are to participate in the corresponding barrier. This number may be used by the filter 108 to determine the count required to be reached by the counter C (field 306) to indicate that the required number of threads have arrived at the barrier. Thus, when the counter C for a barrier reaches the particular number of threads participating in the barrier, this indicates that all of the threads to be synchronized have reached the barrier. Also, the participation vector indicates which bits in the exited vector (field 310) are needed to indicate that all of the threads have exited the barrier. Thus, the filter 108 may monitor the bits of the exited vector which have been set to one, and when a thread exiting the barrier would cause the exited vector to match the participation vector, this indicates to the filter 108 that all of the threads being synchronized have exited the barrier.
In another embodiment, one or more of the processors may execute multiple threads that to be synchronized. Thus, when the first of the threads of such a processor reaches the barrier, that thread is stalled; however, the processor continues to execute each additional thread until it also reaches the barrier. To accomplish this, invalidated I-cache lines are uniquely associated with each of the threads. Thus, each thread stalls on an invalidated I-cache line that is unique to the thread. Upon exiting the barrier, each thread may invalidate a unique cache line that contains dead code to identify itself to the filter 108 and to indicate that it is exiting the barrier.
Request coalescing might otherwise occur if two or more threads executing on the same processor attempted to invalidate the same I-cache line address. This is because the system 100 may perceive all but one of the requests as superfluous and may drop them or merge the requests. Thus, using distinct cache lines for the threads defeats request coalescing and enables the filter 108 to identify the individual threads even if executing on the same processor.
So that each thread executing on the same processor invalidates a unique I-cache line, the threads may each execute a unique version of the barrier procedure 300 (
Alternatively, rather than each thread executing a unique version of the barrier procedure 300 (
However the addresses A and E are determined for each thread, the filter 108 monitors the invalidate requests issued by the threads. The filter 108 selectively blocks fill requests for invalidated I-cache lines based on the invalidate requests issued by the threads when executing the head of the barrier procedure and that target each unique address A+L. Because fill requests are blocked selectively for each thread based on its unique address A+L, a thread may be stalled from executing on a processor while another thread may continue to execute on the same processor until it reaches the barrier. Also, because each thread issues an invalidate targeting a unique address E when exiting a barrier, the filter 108 is able to determine which threads have exited the barrier even when multiple threads are executed by the same processor.
In an embodiment, the filter 108 identifies invalidate requests as belonging to a particular barrier based on the addresses A+L and E targeted by the request (in addition to identifying the particular thread, as explained above). This allows the filter 108 to determine when all of the threads to be synchronized have arrived at a particular barrier and when they have exited the particular barrier. This may be accomplished by the filter 108 storing a bit pattern or mask for the addresses A+L and E for each barrier in the fields 302 and 304 of the table 300 rather than the addresses themselves. The pattern for the field 302 may be given as AP, while the pattern for field 304 may be given as EP. Thus, given a predetermined mask M, an invalidate request targeting address X is identified as belonging to the barrier if M AND X equals AP, where “AND” is a logical AND operation.
The mask M may be selected such that higher order bits of X specify the particular barrier and the lower order bits specify the particular thread. Assume that addresses can be encoded on 8 bits, and that the size of cache lines is four bytes. Therefore, the first four addresses 00000000 through 00000011 are contained in the first cache line, and 00000100 is the address of the beginning of the second cache line. In general, the beginning address of a cache line has its two lowest-order (rightmost) bits equal to 0. We can also decide that cache lines participating in the same barrier have the same high-order bits. For example, an 8-bit address can have a structure as shown in
A valid mask M may be 11110000. Now assume the addresses A+L targeted by two different threads are: (A+L)1=11010000 and (A+L)2=11010100. In this case, M AND (A+L)1 yields 11010000, as does M AND (A+L)2. Thus, the first four bits of the addresses (A+L)1 and (A+L)2, which are 1101, are the same and, thus, specify the same barrier. In this case, the value AP of field 302 for this particular barrier is 1101, with the lower order bits being “don't cares.” The next two bits of the targeted address specify the particular thread. Thus, the thread that issued the invalidate request for address (A+L)1 or (A+L)2 can be distinguished by the filter 108 examining bits five and six (bits “yy”). For address (A+L)1 these bits are 00 and for address (A+L)2, these bits are 01. Bits 7 and 8 can be ignored by the filter 108.
Invalidate requests for address E may be handled similarly. For example, the value EP for field 304 for this same barrier may be given as 1100. This value differs from the value AP so that the filter 108 is able to distinguish between invalidates that target the address A+L and those that target the address E. However, bits five and six may still be used to identify the particular thread. Thus, in the example, an invalidate request targeting address 11000100 can be identified as an E invalidate issued by thread 01 for the same barrier as in the example above. It will be apparent that different bits of the targeted addresses A+L and E may be used to identify the particular barrier and the particular thread.
The system 600 of
As shown in
The system 600 may also include memory controllers 118A-D which control accesses to memories 120A-D. Similarly to the system 100 of
In systems with multiple multi-core chips, several techniques for large-scale barrier synchronization are presented. In one embodiment, filter hardware (which implements filter 108) is present within each chip, as explained above, and conventional software barrier techniques are used for synchronization between chips. In this case, the barrier is broken down into two stages. Once the participating cores on one chip have reached the first stage, one core on the chip executes a software barrier with other threads each on different chips by inter-chip communications among the participating chips. Once this software barrier is complete, the threads on each chip are signaled to proceed.
In an alternative embodiment, barriers are processed hierarchically by a combination of intra- and inter-chip communication. One chip's filter is designated as a home filter for a given barrier address, in the way that memory addresses in a NUMA multiprocessor have home locations. Non-home filters participating in a barrier aggregate their arrivals at the barrier before communicating with the home filter. For example, if a chip has eight cores participating in a barrier, but is not the home node, its filter can wait until each of its 8 cores has reached the barrier (using techniques described above) before communicating to the home filter. The home filter requires state bits corresponding to the summarized status of each other processor chip in the shared memory system. This can be faster than the software method described above, but may be less scalable for systems with very large numbers of chips.
In yet another embodiment, inter-chip synchronization may be performed with conventional hardware synchronization networks, such a global AND/OR tree networks. This results in a hybrid hardware mechanism.
It will be apparent that computer-readable media may be provided having stored thereon computer code for performing any of the functions described herein. For example, the computer code may be loaded from the computer-readable media to the system 100 or 600 for causing the system 100 or 600 to operate in accordance with the teachings herein.
While the foregoing has been with reference to particular embodiments of the principles described herein, it will be appreciated by those skilled in the art that changes in these embodiments may be made without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims.