Marlin & Dory way of 'Finding Bugs'

On our return after watching ‘Finding Dory’, my son asked, “Dad, if you were to find Dory would you be able to do that”? I said, “Ofcourse”! Next, came HOW? I reminded him that my job is to Find Bugs and so I know the tricks of the game already. That made him super excited and wanting to know more about it. Given that this time the reference was picked by him, I decided to continue the same to explain him further.

In the movie, Marlin and Nemo were finding Dory inside the Marine Life Institute (MLI), likewise, we find bugs inside the design called as System on Chip (SoC). The SoC has a lot of similarity to MLI in the sense that it is big and complex. As MLI had different sections, our SoC has different blocks where Dory (Bug) can be found. Also it is not only the sections but the inter-connections that are equally important. When we look for Dory (Bug) inside these blocks we call it IP verification and when our focus is on the inter-connections we call it Integration or SoC Verification.

Image Source : http://www.socwall.com/desktop-wallpaper/7462/dory-and-marlin-by-ryone/

We start off our quest using the Marlin way i.e. “Assess the situation, evaluate, and plan it out”. We call it the Directed Verification approach wherein we understand the design, prepare a plan on where and how we would look around for Dory (Bug) and then execute accordingly. During this process we also keep asking (reviews) around (designers & peers) to let us know if we are missing out on anything. So if Dory is somewhere around, there is a chance we may sight her. But since Dory doesn’t think much before acting, that makes her unpredictable. There is always a possibility that we may not find her as per our plan.

My son’s eyeballs zoomed…. THEN?

Then we also do what Marlin & Nemo did i.e. follow “What would Dory do”? My son jumped, "She wouldn’t think twice and be random". Yes! We pick the Dory way and we call it Random Verification. We search randomly everywhere in an unplanned sequence and guess what? The chances that we would find Dory (Bug) increase. To make it more effective, we define weights and constraints to the randomness so as to improve our luck of finding her further. The approach now becomes Constrained Random Verification (CRV).  While following this random pattern we also take a note (coverage) of where all we have visited to avoid repeating same place again and save time. Now we can find her faster. Tracking coverage on top of CRV is called Coverage Driven Verification (CDV). So if we missed finding Dory (Bug) using the Marlin way (Directed verification) we still have an option to find her the Dory way (CRV).

That settled my son for a while till he pointed again saying, “Dad, maybe you should seek help from Bailey, the beluga whale who can find Dory faster than anyone using echo location”. I smirked and told him that we have our Bailey too and we call it Formal Verification. But then, Bailey was dependent on the whale voice between Dory & Destiny, the whale shark without which he couldn’t be of much help. Similarly, in Formal we are dependent on the assertions that connect the tool to the bug in the design. The effectiveness of this approach is purely dependent on the quality of voice (assertions) and the connect (covering all parts of design) between Dory & Destiny. But yes, if that is in place, it is really fast & effective.

Now convinced that his dad would be able to find Dory, my son asked, “So once you have found Dory, what do you do next”?

I laughed and told that we don’t have to find only 1 Dory (Bug). There are many of them and the address and architecture of the institute (new SoC) also keeps changing. So we just keep Finding Dory (Bug)!!!

Disclaimer: "The postings on this blog are my own and not necessarily reflects the views of Aricent"

Learning Verification with Angry Birds

What do you say when your little one asks you, “Dad! What do you do”? Well I said, “I am an engineer”. For his age, he knew who is a driver, a doctor and a policeman. So the next question was “What does an engineer do”? I pointed him to different man made stuff around to explain him what all an engineer does. As an inquisitive kid he wanted to know if I build them all. That is when I tried to explain different engineering functions building different artifacts. So the question came back as to, “What do you do”? Finally, I told him that, “I find BUGS in the designs”. The next one was HOW? Given that he watched The Angry Bird movie recently & loves to play that game so I picked from there to explain what a verification engineer really does.

Figure 1 : Labeled screenshot of The Angry Birds game 

As in the figure above, the screenshot of the game is called TESTBENCH for us. The target that is seen on the right is called the DESIGN UNDER TEST or DUT in short. Our goal is to hammer the DUT with minimum iterations such that all the BUGS inside it like the pigs above get kicked out. On the left you see a series of angry birds waiting to take the leap. We refer to them as the PACKETS or SEQUENCE ITEMS. They are all from the same base class “angry_birds” i.e. have certain characteristics in common while some different features in each one so as to ensure we hit the DUT differently. We sequence these birds (sequence items) in such a way so as to generate different scenarios to weed out the pigs (bugs). This scenario is called a TEST CASE. The catapult shown is known as the DRIVER in our testbench. It takes the angry bird (sequence item) and throws (drives) it on to the DUT at different points known as INTERFACES of the DUT.  Once the angry bird (sequence item) hits the DUT, there is an inbuilt MONITOR in the game (testbench) that confirms if the flight taken is useful or not & if it is, how much? If the hit resulted in correct outcome the SCOREBOARD gives a go ahead and this leads to the scores that we get and we call it COVERAGE. The high score is the maximum coverage achieved with this test case.  When we are able to kill all the pigs (here bugs) hidden in different parts of the DUT, we are all set to move to another screen i.e. new test case targeting another part of the DUT. Once all tests at a given level pass, we move to the next level which is a little tougher. We can call it moving vertically i.e. block to subsystem to SoC/Top OR moving horizontally within a given scope i.e. more complex test scenarios or stress tests. Usually when we have passed all levels, by that time another version of the game is released and we move to that one i.e. next PROJECT.

After explaining it to my son, I felt he would be fascinated with my work. He thought about it and said, “Dad, so you don’t really work, you go to office and play”!!!

All I could tell him was, “Become a Verification Engineer and you can play too at work”!!!

Disclaimer: "The postings on this blog are my own and not necessarily reflects the views of Aricent"