US 7876900 B1
In one embodiment, a hybrid backplane coding scheme transmits data using lengthy sequences of scrambled data, separated by 8b/10b control character sequences that prepare the receiver for the next scrambled sequence and permit realignment if necessary. Advantageously, the sender of the scrambled data can be changed during the control character sequence. The hybrid backplane coding scheme can be designed such that the power spectral density of scrambled data and control character sequences are similar, which permits good performance with high-speed electrical differential receivers. Other embodiments are described and claimed.
1. A method of transmitting digital data sequences from a transmitting device to a receiving device across a serial channel, wherein the receiving device recovers the timing of the digital data sequences from the digital data sequence itself, the method comprising:
scrambling multiple digital data sequences;
sequentially transmitting the scrambled digital data sequences across the serial channel;
transmitting unscrambled channel control character sequences, each sequence comprising at least twenty bits, across the channel between the scrambled digital data sequences; and
measuring DC imbalance in one of the scrambled digital data sequences and selecting and inserting into the scrambled digital data sequence an additional unscrambled control character that counters the measured DC imbalance.
2. The method of
3. The method of
4. The method of
5. The method of
6. The method of
7. The method of
8. The method of
9. The method of
10. The method of
11. The method of
12. The method of
13. The method of
14. The method of
15. The method of
16. The method of
17. The method of
18. The method of
19. The method of
20. The method of
21. The method of
22. A distributed packet switch comprising:
an electrical backplane comprising a plurality of differential signal trace pairs and a plurality of line cards and at least one switch fabric card coupled to the electrical backplane, each line card coupled to the at least one switch fabric card by at least two of the differential signal trace pairs, each of the differential signal trace pairs used for unidirectional communication either from a line card to a switch fabric card or from a switch fabric card to a line card, the line cards and switch fabric cards communicate digital data sequences across the differential signal trace pairs in a series of epochs, each epoch on a differential signal trace pair that is communicating a corresponding digital data sequence during that epoch comprising an unscrambled channel control character sequence of at least twenty bits and a scrambled version of the corresponding digital data sequence;
wherein the line cards comprise transmitters that measure DC imbalance in one of the scrambled digital data sequences and select and insert into the scrambled digital data sequence an additional unscrambled control character that counters the measured DC imbalance.
23. The distributed packet switch of
24. The distributed packet switch of
25. The distributed packet switch of
26. The distributed packet switch of
27. The distributed packet switch of
28. The distributed packet switch of
1. Field of the Invention
This invention relates generally to high-speed digital data transmission, and more particularly to a coding and framing method for digital data transmission.
2. Description of Related Art
Digital data is often transmitted from one device to another device as a digital sequence. In its simplest form, a digital sequence is a transmission pattern formed using a time sequence made up of “pulses” of two signal types, representing two binary symbols (a zero and a one). In an electrical transmission system, the sender may send one voltage to represent zero, and another voltage to represent one, each bit represented by the appropriate voltage during its corresponding bit time. Such a system can also be differential, i.e., a pair of electrical conductors is used, with one of the pair sending the opposite signal as the other of the pair, both conductors active during each bit time. By detecting and differentiating between the two signal types sent by the sender and recovering the timing of a transmitted sequence, a receiver can reconstruct the original digital data sequence.
Since from a flexibility standpoint the digital data input sequence is preferably unconstrained as to its content, some allowable input sequences may cause problems at the receiver if sent without modification. For instance, if the receiver is to recover timing information from the transmitted data, a certain number of data transitions (one to zero or zero to one) per unit time are required. A long string of zeros or ones in the input data may allow the receiver timing to drift, causing bit errors. Also, if an electrically transmitted signal is not dc-balanced (i.e., an equal numbers of zeros and ones are transmitted over time), the transmitter and receiver cannot be dc-coupled, or else the signal levels representing a zero and a one will drift. Further, were the channel susceptible to any of these conditions, a malicious sender could intentionally attack the transmission system by sending a data sequence with a pattern designed to confuse the receiver.
To solve these and other problems (such as how to detect and/or correct transmission errors at the receiver), many digital data transmission systems employ transmission coding. The transmission code adds some overhead bits and/or characters to the input sequence, and manipulates the input sequence to avoid one or more of the aforementioned problems.
One transmission code in wide use today is the 8b/10b transmission code as disclosed by Franaszek and Widmer in U.S. Pat. No. 4,486,739, “Byte Oriented DC Balanced (0,4) 8b/10 b Partitioned Block Transmission Code.” The 8b/10b transmission code (“8b/10b code”) accepts an input sequence of eight-bit blocks, and transforms each eight-bit block from into a ten-bit codeword. Each ten-bit codeword contains either six ones and four zeros, five ones and five zeros, or four ones and six zeros, with a maximum design spacing between adjacent zero-to-one and one-to-zero transitions in each codeword to allow the receiver to maintain timing. Each input value that can be coded with a codeword having six ones and four zeros can also be coded with a codeword having four ones and six zeros. One of those two codewords is selected for transmission by a rule that nulls any running zero-to-one disparity in the transmitted sequence. By following the same rules as the transmitter, the receiver decodes the codewords to recover the eight-bit blocks of the input sequence.
The 8b/10b code also provides some ten-bit codewords that represent channel control characters. Some of these characters contain a “comma” property, i.e., they contain a bit pattern that can only occur within such a codeword and not across any two other consecutive characters. These comma characters are useful for instantaneous byte synchronization at the receiver. One pair of such control words are commonly referred to as “K” and “R”, representing even and odd idle characters. Other typical control words include an align character “A”, which can be used to align data transmitted on multiple parallel bit lanes.
One drawback associated with 8b/10b coding is its high coding overhead. For every eight-bit input value, ten bits are transmitted across the channel. Factoring in the added control words, more than 20% of the channel's physical bit capacity is consumed by the 8b/10b code overhead.
For 10 Gb/s (gigabits/second) Ethernet, a different coding scheme is used to avoid the high coding overhead of the 8b/10b code. This coding scheme, commonly referred to as 64b/66b coding, is described by Walker et al. in European Patent Application EP1133124A2, entitled “Coding for Packetized Serial Data.” 64b/66b coding achieves a more efficient use of the channel than 8b/10b coding—for each 64-bit input block, only 66 bits are transmitted. The 66 bits comprises a two-bit block type code and 64 bits of scrambled data and control words.
When a 64-bit block contains one or more control words, such as block B2, coding and decoding complexity is increased. The block could contain all control words, or could contain mixed data and control words, as block B2 illustrates. A finite number (10) of different mixed data/control symbol arrangements are allowed, one of which (a single data symbol D9 followed by a packet termination symbol T and a string of other control symbols) is illustrated. In such a circumstance, type pattern generator 26 produces an eight-bit frame type code FT to frame composer 22 and a two-bit block type code T2 with a value “10”—opposite the code used for information-data only. The frame type code FT describes the ordering of information and control words in the 64-bit block, and also describes at least one control word. In this particular example, the frame type code FT describes the mixed data/control sequence consisting of one data word, followed by a termination symbol, followed by other control words. Frame type code FT is coded as the first eight bits of the 64-bit formatted frame F2 sent by frame composer 22 to scrambler 24. The second eight bits of F2 contain the single data word D9. The termination symbol is understood from the value of FT. The remaining six control words are coded as seven-bit control words and placed in specific locations in the remaining 48 bits of frame F2. Scrambler 24 scrambles block B2 to produced scrambled frame S2. Transmitter 28 prepends block type code T2 (“10”) to scrambled block S2 and place the 66-bit frame on channel 29. The receiver interprets the “10” two-bit block type code as indicating that the block will require additional decoding after descrambling. The type code from the descrambled data is interpreted, allowing the receiver to restore the original 64 bits of control and information words from the descrambled data.
One difficulty with the 64b/66b code is that because 64 of every 66 bits are scrambled—and the 64 scrambled bits include the control code characters—the 64b/66b code has no “comma” control character that allows easy block synchronization of transmitter and receiver. As a result, it may take long periods of time involving hunting and guessing to synchronize or resynchronize the channel should the channel become desynchronized.
The invention may be best understood by reading the disclosure with reference to the drawings, wherein:
The following disclosure describes embodiments of a channel coder, decoder, system, and method that can be used instead of an 8b/10b, 64b/66b, or similar coding scheme. These embodiments overcome the coding inefficiency of an 8b/10b system by using a scrambler, instead of an 8b/10b coder, to prepare digital data for transmission. In some preferred embodiments, however, 8b/10b unscrambled codewords are inserted between segments of the scrambled digital data, and used to align the receiver and help maintain proper channel timing and DC balance. Unlike the simple two-bit toggled block type codes used in 64b/66b coding, the unscrambled codeword groups are long enough to be matched with a scrambler in preferred embodiments to provide similar average power spectral density, whether the channel is transmitting the unscrambled codewords or scrambled digital data. This provides a distinct advantage for high-speed electrical serdes (serializer/deserializer) operation, since these serdes units are generally highly sensitive to changes in received signal spectra and may produce more bit errors when the spectral density of a received signal shifts rapidly.
In one application described herein, high-speed signals traversing an electrical backplane are coded using a hybrid channel coder according to an embodiment of the present invention.
Backplane 100 has slots with electrical connectors that accept packet input/output cards, also known as “line cards,” an RPM (route processing module) card, and switch fabric cards. When such cards are connected to backplane 100, the backplane supplies signals, and preferably power, to each card. Although two line cards, four switch fabric cards, and one RPM card are shown in
Line card LC1 is responsible for packets received at input ports 60 and transmitted from output ports 62, e.g., from/to other network devices. The ports themselves may be configured to accept different optical and/or electrical connectors and signaling formats. A packet processor (not shown) on LC1 determines an appropriate output line card and port for received packets, tags the packets with a tag header describing this internal switch destination, and forwards the packets to an ingress traffic manager (ITM) Mi1. Ingress traffic manager Mi1 stores the tagged packets in queues, each queue storing packets bound for a particular line card egress traffic manager (ETM). Queue occupancy information is transmitted across backplane 100 on scheduler bus 74, to a scheduler 72 on an RPM card 70.
Scheduler 72 receives similar queue occupancy information from other line card ITMs as well, e.g., an ITM Mi2 on a line card LC2. Scheduler 72 arbitrates between the line card ITMs based on some criteria, and arrives at an epoch-based schedule. As used herein, an epoch is a known switch time interval, during which the switch fabric cards will remain in a given switch fabric configuration. Typical selected epoch durations are sufficient for each ITM to transmit 10,000 to 100,000 bytes of stored packet data. Thus the epoch-based schedule grants permission, at each epoch, for some number of ITMs to transmit multiple queued packets to some number of ETMs. The epoch grants are transmitted by the scheduler to the ITMs using scheduler bus 74.
When ITM Mi1 has packets queued for a specific ETM, e.g., ETM Me2 on line card LC2, scheduler 72 will eventually grant ITM Mi1 an epoch in which to transmit the packets to ETM Me2. When ITM Mi1 receives such a grant, it retrieves tagged packets from one or more queues in which it holds packets bound for E™ Me2, and submits the packets as an epoch data frame to a backplane ingress port Bi1.
Backplane ingress port Bi1 splits the epoch data frame into strands S1 i 1, S2 i 1, S3 i 1, and S4 i 1, where the strands taken together form a “port pipe” PPi1 for transmitting epoch data from backplane ingress port Bi1 to the switch fabric. Each strand transmits a portion of the epoch data across backplane 100, e.g., on a differential trace pair, to a corresponding switch fabric port interface Pi1 on a corresponding switch fabric card SF1, SF2, SF3, or SF4.
Referring back to
One attractive feature of this design is that each backplane ingress and egress port pipe strand always uses the same transmitter and receiver. Accordingly, each link remains bit-synchronized by the idle character sequences generated by each transmitter while the switch matrix on each card reconfigures to connect different ingress port pipes to different egress port pipes. Because K and R have a comma property, the links also remain character-aligned during reconfiguration.
Switch fabric egress ports Pe1 and Pe2 on each switch fabric card operate similarly to backplane ingress port Bit, except the 8b/10b epoch frame data is received, already 8b/10b formatted, from switch matrix 80. Referring to switch fabric egress port Pe2 and an egress port pipe PPe2, the switch fabric cards transmit an egress frame across backplane 100 to backplane egress port Be2. Backplane egress port Be2 detects the epoch start from the KRKRKS pattern, and then begins decoding eight-bit strand data from each ten-bit character strand. The decoded strand data is aligned and presented to egress traffic manager Me2 for eventual transmission on an output port 66.
In summary, each port pipe strand uses a relatively short inter-epoch control character sequence (e.g.,
The scrambled frames 514 and 520 can achieve 100% coding efficiency, as a one-to-one correspondence can exist between input data bits and coded data bits. In the described electrical backplane utilizing high-speed differential signaling pairs, however, a variety of other considerations exist, many of which differ from conditions found on other links.
First, the high-speed differential receivers that receive each port pipe strand require a certain number of bit transitions per unit time to maintain phase lock with the transmitter. In the prior art 8b/10b coding scheme, extremely frequent transitions are guaranteed, no matter what data is presented to the coder. With a scrambler, however, it is possible that a given epoch data segment could produce an extended “run” of zeros or ones on the channel, causing the receiver to lose phase lock with the transmitter. Therefore, a preferred scrambler for this use would have an extremely low probability of generating runs long enough to cause significant timing error drift. Also, in the unlikely occurrence of such a run, an inter-epoch comma sequence such as the one provided in
Secondly, even when runs aren't long enough to defeat timing lock, high-frequency differential receivers can be extremely sensitive to shifts in the “eye pattern”—the characteristic time-voltage separation between differential signals plotted between successive bit transitions—for received data. Should the eye pattern shift substantially between a scrambled frame and the inter-epoch control sequence, this receiver sensitivity may cause a receiver to generate bit errors after a channel transition from control to scrambled data, or vice versa. Accordingly, it has now been discovered that in a hybrid coding scheme it is preferable to use a scrambler and a control character sequence that have similar power spectral densities. Generally, this can be achieved with a scrambler that has a similar distribution of run length probabilities as those occurring in the control character sequence.
As a third consideration, it has now been discovered that a scrambler that produces channel patterns with strong characteristic frequencies can disrupt the group delay for bit transitions traversing the backplane, again causing variations in the received eye pattern that can result in received bit errors. Furthermore, such patterns can increase electromagnetic interference (EMI), which is generally undesirable for a number of reasons. Accordingly, preferable scramblers present a scrambler output with a fairly continuous spectral content.
As a fourth consideration, particularly when the hybrid code is transmitted through an electrical backplane, it is preferable that the hybrid code be DC-balanced. DC balancing allows the transmitters and receivers to be AC-coupled to the backplane, e.g., using DC blocking capacitors. Such capacitors allow hot-swapping of cards, and may be required when the backplane distributes primary power to the cards. Generally, for a continuous scrambled transmission that could run to 75,000 bits or more, the scrambler itself should be closely DC-balanced in order to maintain DC balance on the hybrid coded channel.
Finally, as a practical consideration, it is preferably that the scrambler not require long registers.
Conceptually, scrambler 600 comprises 28 bit-delay registers X0 to X28 chained together, and two bit XOR operators 610 and 620. Each delay register contains a parallel load input for initializing the scrambler with a seed value. Once the seed value is loaded, at each clock cycle k delay register Xn receives the value stored in register X(n−1) during clock cycle k−1. The value stored in registers X18 and X28 are XORed by XOR operator 620, and the output of XOR operator 620 and a serial input I(k) are XORed to produce a serial output O(k). The serial output O(k) is also stored in register X0 as X0(k). In other words,
Scrambler 600 has been experimentally found to have a transition density of around 30%, good DC balance, a fairly smooth power spectral density, a run length distribution similar to that found in 8b/10b sequences, and a maximum run length that rarely exceeds 25 bits (and has a probability of less than 10−18 of reaching 58 bits). As an added advantage, since the output of scrambler 600 statistically approximates the power spectral density of 8b/10b code, it can be used with the inter-epoch control sequences used in prior art 8b/10b epoch coding, allowing hybrid-coded epoch data to be transmitted, if desired, through prior art switch fabric cards. Furthermore, the seed value for the scrambler can be set experimentally to provide a smooth run-length transition at the beginning of each epoch, for typical data appearing at the head of an epoch.
Some embodiments possess hardware capable of performing scrambler bitwise shifts and XOR operations at line rate. It is noted, however, that various hardware approaches exist for parallelizing the bit-serial computation of a scrambler. Such an approach can be used in embodiments of the present invention to scramble multiple bits of a parallel input stream at less than line rate, followed by serialization of a multiple bit scrambler output at line rate.
Packets are received from and transmitted to remote packet network devices using optics 130 and accompanying serializer/deserializers (serdes) 140. Serdes 140 and a packet processing engine PPE exchange data on one or more parallel interfaces. Packet processing engine PPE processes ingress packets before forwarding them to ITM Mi1, and processes egress packets received from ETM Me1. A packet classification engine PCE and an attached CAM (content addressable memory) lookup information needed by packet processing engine PPE to direct ingress packets to the appropriate ITM queue.
In this example, nine switch fabric cards are used (eight active plus a backup), and thus nine switch fabric ingress strands Si0 to Si8 and nine switch fabric egress strands Se0 to Se8 are present. Three serdes 160, 162, and 164 handle serialization/deserialization for the strands, with serdes 160 handling ingress and egress strands 0-3, serdes 162 handling ingress and egress strands 4-7, and serdes 164 handling ingress and egress strands 8.
All strands are handled similarly. As an example, ITM Mi1 supplies parallel strand 0 data Pi0 to serdes 160 using eight bitlines and a clock, with the clock running up to 195 MHz and the data supplied at Double Data Rate (DDR, meaning new data is supplied on each negative and positive transition of the clock). Serdes 160 produces the inter-epoch 8b/10b control sequence, scrambles the epoch data strand Pi0, merges the two with appropriate timing, and serializes the hybrid data stream for transmission on a differential serial channel Si0 at 3.125 Gbps.
Serdes 160 handles the received serial strand Se0 is analogous fashion to create a parallel strand Pe0 for E™ Me1.
In one embodiment, each serdes is capable of detecting 8b/10b idle character sequences in each incoming port pipe strand, and aligning to those sequences. The serdes make no attempt to descramble or adjust bit alignment while scrambled data is transmitted. Instead, each strand is parallelized and submitted in a DDR format to switch matrix 80 on ten bit lines, with a corresponding 156.25 MHz clock for each strand. Switch matrix 80 acts as a crossbar to route the ingress port pipe strands to egress port pipe strands according to a switch configuration supplied by a scheduler through a backplane scheduler interface 220. The switched strand data is supplied by switch matrix 80 back to the serdes, each strand submitted in a DDR format on ten bit lines with a corresponding 156.25 MHz clock. As the egress strand data supplied to each serdes may switch sources between epochs, the serdes align the KR sequences for the last epoch and the current epoch during the inter-epoch gap. This alignment makes the transition from one sender to the next appear seamless at the egress port pipe strand receivers on the line cards. Throughout this entire switching process, the scrambled epoch data is handled as 10-bit characters, just as if it comprised 8b/10b channel-coded characters.
In the embodiment described above, line card serdes 160, 162, and 164 are responsible for generation and decoding of hybrid-coded epoch data.
Referring first to
Further explanation of the internal operation of hybrid channel coder 900 will refer also to
A clock CCLK has a clock rate equal to 1/10 the line rate of the serdes transmitter 970 output, e.g., a 312.5 MHz clock rate for 3.125 Gbps serial transmission. Twenty-two positive CCLK edges, 0-21, and corresponding negative clock edges, are illustrated in
Controller 910 is responsible for placing 8b/10b control character sequences and scrambled epoch data on the coder output at appropriate times during each epoch. At CCLK 0 in
The SOE signal is generated by another switch module (the RPM), and is therefore not synchronous with CCLK. Controller 910 may therefore receive the SOE signal at any time in a KR cycle. In
On the falling edge of CCLK 1, controller 910 initiates its start-of-epoch control sequence. One aspect of this control sequence is to drive a predetermined number of additional KR character pairs on OUT, followed by a KS character pair where S is an 8b/10b start character. Accordingly, with the predetermined number of KR pairs set to four, controller 910 commands K character types for CCLKs 2, 4, 6, 8, and 10, R character types for CCLKs 3, 5, 7, and 9, and an S character type for CCLK 11.
While the start-of-epoch control characters are generated, controller 910 also initializes the scrambler and causes it to begin scrambling epoch data. On the negative edge of CCLK 1, controller 910 asserts a RESET signal to scrambler 600 and 32:10 gearbox 940. It is assumed that by this time an ITM has begun filling FIFO 920 with epoch data for the current epoch, represented as 32-bit FIFO data words D1, D2, etc. in
Gearbox 940 and scrambler 600 coordinate, e.g., by handshaking signals (not shown), the transfer of scrambled 32-bit words from scrambler 600 to gearbox 940. The scrambler is capable of generating a scrambled 32-bit word every three CCLK cycles or less, and the 32:10 gearbox consumes five scrambled 32-bit words every 16 clock cycles.
After the 8b/10b start character S has been transferred to serdes transmitter 970, controller 910 toggles MUX select (at the positive edge of CCLK 12) to switch OUT from AOUT to GOUT. MUX select can also be provided to the gearbox, or the gearbox can use internal timing to determine when to start placing scrambled data on GOUT. In
At a switch fabric card, the hybrid bit sequence of
When the expected KRKS sequence is received, controller 1010 issues a reset to 10:32 gearbox 1040 and to a descrambler 600. Gearbox 1040 begins accumulating scrambling data, formats the data into 32-bit words, and forwards the words to descrambler 600. Descrambler 600 resets by loading its bit registers with the value stored in a seed register 1030, which must match the seed used by seed register 930 to compose the sequence. Descrambler 600 outputs descrambled data words to a FIFO 1020, which transmits strand data to an ETM.
It has been determined that the hybrid coding scheme is generally more easily received, at least for data transmission using electrically conductive traces, when the scrambler power signal density closely resembles the control sequence power spectral density.
Many additional features can be used in an embodiment of the present invention. For instance, a hybrid coder can be designed to insert additional control characters within the scrambled data portion, either at fixed intervals or after a given number of bits without a bit transition. Such additional control characters can be selected to adjust any DC imbalances in the scrambled data. The inter-epoch control character sequences can also be varied to adjust for DC imbalances in a just-transmitted scrambled epoch. As previously mentioned, the number of control characters appearing between epochs can be varied, e.g., to adjust timing when switching between different scrambled data sources.
The epoch length and control character length can be varied depending on application. In the packet switch example described, epoch lengths are generally set such that 30 to 80 thousand bits are serialized and transmitted during each epoch. Of these bits, roughly 600 (60 8b/10b control characters) are generally transmitted during the inter-epoch period. Other embodiments can, of course, use different values for epoch length and unscrambled control character length. It is believed that workable embodiments require at least 20 bits representing unscrambled control characters per epoch.
Although specific signaling formats and speeds have been used in exemplary embodiments, the concepts described herein can be applied to other situations. Hybrid coding using 8b/10b control coding and an x29+x19+1 scrambler has been used in other embodiments at signaling speeds up to 12.5 Gbps. 8b/10b control coding has some attractive features, but its use herein does not preclude a hybrid coding scheme using other control character sequences or coding schemes, including characters that have no “comma” property considered separately and characters that may not be considered control characters in other applications. Although conventional two-level differential/NRZ (non-return to zero) serial signaling has been described, other serial signaling approaches, such as PAM4 (Pulse Amplitude Modulation-4 level) NRZ, or optical signaling methods, could be used with an embodiment of the present invention. The scrambler can operate in the serial data stream portion of a serdes, with an appropriate serial bypass path for unscrambled control characters.
The described switch fabric cards pass scrambled data through without descrambling that data. In other embodiments intermediate serdes on a link could descramble incoming data and re-scramble it. In such a case, different seeds could be used on each half-link, and/or the seed on each half-link would not need to be reloaded each epoch.
One of ordinary skill in the art will recognize that the concepts taught herein can be tailored to a particular application in many other advantageous ways. For instance, although a specific implementation example shows usage of hybrid coding across a backplane, such codes could also be employed on a link that connects two network platforms. The particular functional block descriptions and data widths described herein apply to some embodiments, and could be advantageously modified for use in different systems.
Although the specification may refer to “an”, “one”, “another”, or “some” embodiment(s) in several locations, this does not necessarily mean that each such reference is to the same embodiment(s), or that the feature only applies to a single embodiment.