PCIe deep dive: TS1/TS2, LTSSM link training, TLP types and transaction ordering rules

You are interviewing for a PCI Express (PCIe) verification role, and the interviewer goes deep on the protocol: from link training at the physical layer up to transaction types and ordering at the transaction layer. Answer each part as you would at a whiteboard, and for each behavior say how you would check it in a testbench. ### Clarifying Questions - Which data rates should I assume: 8b/10b encoding (2.5 and 5.0 GT/s) or 128b/130b encoding (8.0 to 32.0 GT/s)? Ordered-set formats and equalization differ. - Is the design under test a Downstream Port (for example a root port or a switch downstream port) or the Upstream Port of an endpoint? - Should the ordering discussion assume a single traffic class, or several traffic classes mapped to different virtual channels? ### Part 1 — TS1 and TS2 ordered sets What are TS1 and TS2 ordered sets? What do they carry, which problems do they solve while a link trains, and why does the protocol need two kinds instead of one? ```hint Follow one field Pick the Link Number or Lane Number field and trace how its value changes from the start of training until the link is up. The purpose of the ordered sets follows from that trace. ``` ```hint Heard versus agreed Think about how a port can tell that its partner has not only received its proposal but also accepted it, and is ready to move on. ``` #### What This Part Should Cover - Where training ordered sets sit in the protocol layers, and when they are sent - The main fields and what each one negotiates - The different jobs of TS1 and TS2 in the training handshake - Lane-level problems solved during training: lock, polarity, width and lane order ### Part 2 — LTSSM and link training Walk through the Link Training and Status State Machine (LTSSM) from reset to L0. Then explain why a link that is already up enters Recovery, and how a link changes speed. ```hint Main road first Name the top-level states on the way to L0 before any substates, then ask what event leads into each of the remaining states. ``` ```hint What can change on a live link List what can go wrong, or need to change, while a link is running, and map each item to a state transition. ``` #### What This Part Should Cover - Detect, Polling, Configuration and L0, with the condition that ends each state - The different roles of the Downstream and Upstream Ports when link and lane numbers are negotiated - Recovery: what triggers it, speed changes and equalization - Low-power states and their exits, and what happens above the physical layer once the link is up ### Part 3 — TLPs: Posted, Non-Posted and Completion What is a Transaction Layer Packet (TLP), and what does its header carry? Classify memory, I/O, configuration, message and atomic requests as Posted or Non-Posted. Explain how a completion finds its way back to its requester, and how a large read can be answered. ```hint Who waits for an answer For each request type, ask whether the requester needs anything back, data or just a confirmation, and what that implies for buffering and flow control. ``` #### What This Part Should Cover - Header fields that matter for routing, ordering and integrity - A correct classification of every request type - Completion routing and matching, completion status, and split completions - Separate flow-control credits per category, and why they exist ### Part 4 — When may transactions reorder? Within one traffic class, when may a later transaction pass an earlier one? Give the rule for each pair of Posted Request, Non-Posted Request and Completion, explain why each rule exists, and describe how you would check ordering in a testbench. ```hint Two forces The rules balance two goals that pull in different directions: keeping software's producer–consumer assumptions true, and making sure the fabric can never deadlock. ``` ```hint Header attributes Some attribute bits in the TLP header relax the default rules. Consider which bits, and which pairs they affect. ``` #### What This Part Should Cover - The must-not-pass, must-be-able-to-pass and may-pass relationship for every pair - The producer–consumer reason for the strict entries, and the deadlock reason for the must-pass entries - Relaxed Ordering and ID-Based Ordering, and completions that belong to the same request - An ordering checker, plus stress scenarios that exercise the must-pass rules ### What a Strong Answer Covers - Correct layering: ordered sets and the LTSSM in the physical layer, TLPs in the transaction layer, and the data link layer between them - Accurate protocol mechanics rather than vocabulary, at the generation the interviewer asked about - The reasons behind the rules (handshakes, producer–consumer ordering, deadlock avoidance), not only the rules - A verification view throughout: checkers, functional coverage and error injection ### Follow-up Questions - How would you write functional coverage for the LTSSM, and what would you cross it with? - A link that should train to x8 comes up as x4. How would you debug it? - How do flow-control credits interact with the ordering rules when one category runs out of credits? - How would a scoreboard handle a Memory Read that is answered by several completions?