Recently while reviewing a test bench architecture with a group of verification engineers, an inveterate though interesting debate started. The engineer proposing the verification plan had ‘scoreboard’ as well as ‘checkers’ as two different components in his test bench. The debate was on the clear definition of these 2 terminologies as every engineer had his own understanding of these words. Though various verification methodologies have termed the test bench components differently, still names like driver, bfm, generator and sequencer etc. reveal a lot about their functionality with the exception of scoreboard. Engineers across the globe have varied opinion on the history of this term and its origin as a verification component is untraceable.
Here is a quick preview of the various definitions of ‘what is a scoreboard’?
A scoreboard is a large board in a ballpark, sports arena, or the like for publicly displaying the score in a game or match and often other statistics, such as the count of the balls and strikes etc.
A scoreboard is a centralized hardware mechanism (initially used in CDC 6600 – 1964) used to keep a track on the activity (instructions being fetched, issued, executed etc. and the resources they use) inside a system that helps in determining certain decisions (dynamically scheduling instructions). Here the activities are tracked by flipping bits in scoreboard that keep a score of what is going on in the system.
These definitions are relevant to the scoreboard usage in verification. There are examples of primitive test environment where testing as an activity was maintained using an array of registers (scoreboard) representing tests to be executed. The registers would maintain if a test ran and the status of the result. On completion of the test suite the scoreboard would help generate the statistics and determine next steps. Scoreboard however has evolved from tracking & decision making into multiple avatars wherein it can be as simple as a data structure or as complicated as having the transfer function (predictor) and compare function included. So far in my experience I haven’t seen scoreboards performing protocol compliance checks so I believe my colleague was right in proposing scoreboard as an entity, predicting and comparing while the protocol checks to be handled by a separate component he referred to as checker.
These definitions are relevant to the scoreboard usage in verification. There are examples of primitive test environment where testing as an activity was maintained using an array of registers (scoreboard) representing tests to be executed. The registers would maintain if a test ran and the status of the result. On completion of the test suite the scoreboard would help generate the statistics and determine next steps. Scoreboard however has evolved from tracking & decision making into multiple avatars wherein it can be as simple as a data structure or as complicated as having the transfer function (predictor) and compare function included. So far in my experience I haven’t seen scoreboards performing protocol compliance checks so I believe my colleague was right in proposing scoreboard as an entity, predicting and comparing while the protocol checks to be handled by a separate component he referred to as checker.
Later that day while reading an intriguing article on context aware applications I came across this definition of context which states that –
“Context is any information (the parts of a written or spoken statement that precede or follow a specific word or passage, usually influencing its meaning or effect) that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.”
With the dilemma on the real intention of the user who coined this word (scoreboard) in verification domain or where it was first used I could only conclude that the definition of scoreboard is relative to the context it is used in. The context could be a methodology that defines this term clearly or the test bench architecture where it is used. If you know the functionality of the other components in the test bench (which you can fairly conclude by their names) the rest can be assumed to be taken care by scoreboard J
Check out what Scoreboard means in UVM @ verification academy.
Drop in a comment to share your experiments with the scoreboard definitions!
In "Comprehensive Functional Verification", Wile, Goss and Roesner define scoreboard as "a temporary holding location for information that the checker will use to perform its function. It can receive its information in various ways: an input monitor component, a stimulus component, or loaded at the beginning of a simulation in a golden vectors environment."
ReplyDeleteThere are cases for which scoreboards are not likely to perform the required check, for example, an illegal configuration. Nevertheless, scoreboards are usually better, especially if you reuse them throughout your test benches.
ReplyDeleteAgree with Alan...in terms of OVM/UVM scoreboard is a place where expected data prepared by checkers/tasks (sub components/instances in SB) without timing involvement. Monitor ensures on timing event corresponding checker/task called in expected flow.
ReplyDeleteLinkedInGroup: DVClub Silicon Valley
ReplyDeleteAs mentioned in his cookbook Mark Glasser defines the term scoreboard as a generic term for a wide range of component types whose function is to answer does-it-work questions. The essential characteristic of a scoreboard is that it collects data about the operation of the DUT as the simulation proceeds and compares it with a reference of some sort to determine if the DUT is functioning correctly.
Posted by Jamal Elhaitout
LinkedIn Group: Design Verification Professionals
ReplyDeleteIn my experience, scoreboard is the entity that holds predicted data and DUT data and comparing between them. hence it holds a functionality of a checker.
the prediction is made by a different entity (reference model, golden model) but I've met people who mix it all together (scoreboard, checkers and reference model) and call in all a scoreboard.
Posted by Lior Mizrahi
Ideally A scoreboard should sample input data and score them off when it matches with the output data
ReplyDeleteLinkedInGroup: Think Verification
ReplyDeleteNot entirely true. I use scoreboard which store the time, and sometimes even compare it within giving limits. A nice article, though.
Posted by Asi Lifshitz in response to Rohitash
Linkedin Group: Design Verification Professionals
ReplyDeletevery nice blog, thanks for link.
Posted by Ivan Rezki
Linkedin Group: Design Verification Professionals
ReplyDeleteSomewhere i read these lines about scoreboard and checker
A Scoreboard is used to keep track of how many transactions were initiated, how many finished and how many are pending and whether a given transaction passed or failed.
A checker is used to check whether a given transaction has taken place correctly . This may include data correctness and correct signalling order.
Ohh ..you have already described it in your website.
Thanks
Posted by Sunny Paul
Checker: Term typically used for assertions in the test environment to check the protocol behaviour, ofcourse can be used for data integrity. Protocol here is the one used by master to communicate with target DUT.
ReplyDeleteScoreboard: It mainly compared the data going to DUT and data came out from DUT, in simple words. Other capacity of UVM scoreboards are that one scb can be used to provide comparision of data for multiple agents whereas checkers are written in monitors therfore agent specific.