Negative Testing in functional verification!!!

Imagine someone on an important call and the mobile device reboots suddenly! The call was to inform that the devices installed at the smart home seems to be behaving erratically with only elderly parents & kids to provide any further details. On booting up, the smartphone flashes that there has been a security breach and data privacy has been compromised. Amidst this chaos, the car’s cruise control didn’t respond to pressing of the pedals!!! Whew!!!.... nothing but one of the worst nightmares in the age of technology we live in! But what if some of it could be true someday? What if the user has little or no idea about that technology?

The mobile revolution has enabled a common man to access technology and use it for different applications. The data from Internet world statistics suggest that internet adoption worldwide has increased from 0.4% of world population in 1995 to 54.4% in 2017. Related data also indicate that a sizable portion of the users are aged & illiterate. The ease of use has potentially driven this adoption further with the basic assumption that devices would be functioning correctly 24x7 even if used incorrectly out of ignorance. The same assumptions are seamlessly getting extended to safety critical domains such as Medical & Auto introducing several unknown risks for the user.

So how does this impact the way we verify our designs?

Traditionally, verification is assumed to be ensuring that the RTL is an exact representation of the specifications. Given that the state space based on the design elements is so very huge, a targeted verification approach covering positive verification has been in practice all throughout. Here, Proof of no bug is assumed to be equal to No proof of bug! The only traces of anything beyond this approach include –

- Introducing asynchronous reset during the test execution to check that the design boots up correctly again.

- Introducing stimulus triggering exceptions in the design.

- Simulating architecture or design deadlock scenarios.

- Playing around with key signals per clock for low power scenarios and reviewing the corresponding design response.

But as we move forward with security and safety becoming key requirements of the design, is this good enough? There is a clear need to redefine the existing approach and bring Negative testing to mainstream! Negative testing ensures that the design can gracefully handle invalid inputs, unexpected user behavior, potential security threats or defects such as structural faults introduced while the device is operational. Amidst shrinking design schedules, negative testing really requires creative thinking coupled with focused effort. 

To start with, it is important to question the assumptions used while defining the verification plan for the design. Validating those assumptions itself can lead to a set of scenarios to be verified under this category. Next, review the constraints applied while generating stimulus to list out potential illegal inputs of interest. Caution should be taken in defining this list as the state space would be large. Reviewing it in the context (Context Aware Verification) of end application would surely help in narrowing down this illegal stimulus set. Further to this, faults need to be injected at critical points inside the DUT using EDA tools or innovative testbench techniques. This is important for safety critical applications where the design needs to respond to random faults and exit properly while notifying about the fault or even correct it. Of course not to forget that appropriate coverage needs to be applied to measure the reach of this additional effort.

As we step into an era of billions of devices empowering humans further, it is crucial that this system of systems is defect free especially when it touches safety critical part of our life. Negative testing is a potential way forward ensuring reliability of designs for such applications. As is always said – 

Better safe than sorry!

Portable Stimulus : redefining verification play field yet again!

In the last 3+ decades, verification has come a long way! Right from quick testing by designers to dedicated verification teams moving from directed testing to constrained random and adding elements of formal apps at times, it has been an eventful journey! Standardization of UVM enabled a common framework with which the fraternity marched forward in sync with each other. Horizontal reuse viz. at the IP level experienced maximum benefits of UVM while vertical resume viz. from IP to SoC level observed limited returns. Apart from the methodology, verification has also proliferated beyond simulation or emulation into virtual prototyping, FPGA validation, post silicon functional validation & HW SW co-verification. Today, the reuse aspects are not limited from IP to SoC or across projects but between platforms too. It is extremely important to realize reuse of efforts at any level across different vehicles enabling first silicon success. The challenge however is that each of these vehicles involve multiple stakeholders like architects (Chip , SW, system), SystemC modelling engineers, RTL designers, verification engineers, prototyping experts, post silicon debuggers and SW developers, each defining & driving the stages of SoC design cycle. Different goals focusing on a specific stage, different approaches in solving these problems and different means of representing solutions has made the task of reuse across the platforms – a convoluted puzzle!!!

To solve this problem, Accellera initiated a task force called Portable Stimulus Working Group (PSWG) that reviewed the concern & potential solutions. After long & regular sessions of intense activity in last couple of years, the group has come up with a proposal in form of a standard.  Beta release of the preliminary version of the Portable Test and Stimulus Standard is now opened for public review.

The basis of the solution relies upon taking the stimulus definition across platforms to an abstraction layer where the focus is on what is needed instead of how it shall be implemented? The idea is to understand & represent the action/outcome i.e. the behavior of the DUT when the test runs. While representing these actions, the focus would be on what would be the inputs and outputs, what resources are required in the DUT, and their relationship with each other? The how part i.e. the implementation will be left to a hidden layer (read EDA tool) to generate the required stimulus for the target platform based on the scenario definition. The actions referred above are all declarative and not procedural or executable by themselves. A set of these static actions can be used to construct a scenario, analyzed for coverage to determine the paths to be traversed & dumped into a test format using the hidden layer.  

To represent these actions, the PSWG has proposed 2 formats - Domain Specific Language (DSL) that is close to System Verilog and a restricted C++ library. Both these formats have equivalent semantics that are completely interchangeable such that for each line in the DSL there is an exactly equivalent C++ library entry. If one defines the actions of an IP in DSL and another IP in C++ library, both can be read together to generate a SoC level scenario for the target platform.

While the road of moving from developing each testcase to a testbench that generates tests has been long, it’s time to take another step in the direction of standardizing the stimuli! This would feed the testbenches for IP/SoC verification & be reused by other workhorses of verification & validation in the SoC design cycle. Remember, reuse is one of the keys to keep up with Moore's law!!!

The public review period for this proposal is open until Friday, March 30, 2018. Download & review NOW!!!

Moana of Verification!!!

Dear Readers!!!! Welcome back!!!

During this lull period of sharing thoughts, I realized that even blogging faces the same dilemma of verification on “How much is enough”. Finally, as a verification engineer should think, I decided to continue as much as I can with the hope to improve blogging frequency than before – wish there was  Moore’s law for bloggers too 😊!!! Since this is the first one of this year (Happy New year folks!!!) & by this time of the year the intent, action and discussions on new year resolutions would have died down mostly, let’s start from there.  While speaking to budding DV engineers on their new year resolutions around verification, I discovered that somehow focus on the end goal is missing & they seem to be trending into all directions. Reason? Well! Possibly many –

- The never ending gap between industry expectations & Academia.

- Missing core elements in the Job description of a verification engineers.

- Overwhelming solutions that enable verification….. etc.

Still confused on what I mean? Let me explain!

When I started my career as a verification engineer with legacy directed verification approaches, all I learnt was that, to be successful you need to have - a Nose for Bugs! In a way, that was also because one needed to maximize returns from each test so, you handcraft each one to really mine bugs. With rising silicon complexity, verification domain has been experiencing consistent advancements enabling us verify designs faster & better. During this shift, the expectations from verification engineers also kept changing i.e. demanding experience on new flow, language or methodology. The rise of SV & UVM further accelerated this shift giving a taste of elaborate & sometimes exotic code development opportunities for the DV engineers. While this continued, reuse gained momentum on the design & eventually on verification too. Due to this the code development gave way to reuse & with latest verification flow/methodologies, the verification engineers started spending more time on derivative designs that demand debug capabilities over development. 

Coming back to our budding engineers discussion, learning a new flow/methodology would end up having expectations on development work which may not really be needed always. This trend has often lead to the confusion of multiple directions where the engineer ends up settling on the means losing focus on the end goal.  As a DV engineer, our goal is to find bugs! Approaches like Directed/Constrained Random/Formal or languages, methodologies & platforms like simulator/emulator etc. are all means to hunt bugs. While expertise in many or all of these are important and occasionally may even lead to a career, the expectations from verification engineer is really to catch bugs! 

How does this relate to the topic of the blog?

Well! Moana is a film on the story of a girl named Moana, daughter of a chief of a Polynesian village. She is chosen by the ocean to reunite a mystical relic with a goddess. When a blight strikes her island, Moana sets sail in search of Maui, a legendary shapeshifting demigod, in the hope of returning the heart of Te Fiti and saving her people. In this journey of hers, she discovers that her tribe were ‘voyagers’ who have forgotten their virtue and settled as villagers on an island which would die down soon. She not only saves the island & her people but leads them back to their original self – a journey which is more exciting & enriching!

Similarly, UVM, formal apps and emulation etc. are all means to find bugs in your verification journey. Don’t just settle for the means which might be short-lived sometimes but shoot for the end goal & be the Moana of Verification!!! A worthy resolution to pursue 😊!!!    

What was your resolution on verification on this New Year???

Quick chat with Ravi Subramanian : Keynote speaker DVCon India 2017

Dr. Ravi Subramanian

For many decades, the semiconductor industry followed Moore’s law, transforming what we called as a discrete chip carrying a function on silicon into a small IP inside the SoC on silicon today. As we continue to debate beyond Moore, more than Moore or stagnation of this law and step in the world of IoT, we realize that the system is no more only a single SoC, but instead, it is a conglomeration of multiple tiny & large systems working in tandem producing interesting use cases & enhancing user experience. But are we as the verification engineering workforce ready with the required skills along with the right arsenal of tools and efficient workhorses to ride through this new challenge?

Dr. Ravi Subramanian, Vice president and General manager of Mentor’s IC Verification Solutions Division shares a holistic view on this subject in his opening keynote on Day 2 at DVCon India 2017. The talk titled Innovations in Computing, Networking, and Communications: Driving the Next Big Wave in Verification, dives into convergence of different technologies and its impact on verification. A quick chat with Ravi, revealed the excitement that we all can look forward to in his talk as well as the future that lies ahead for all of us. Read on!!!

Ravi your keynote focusses on drivers to the next big wave in verification. Tell us more about it?

Yes, my talk will focus on the amazing innovations our industry is developing with respect to computing, networking, and communications. These include the changing nature of computing, the dramatic changes in networking and storage, and the disruptive effect of new broadband communications. Yet, the next big wave in design is actually the convergence of these technologies, which is driving today’s internet-of-things and autonomous systems revolution. A common theme across these emerging systems is the need for low power, security, and safety—whether you are talking about devices on the edge or high-availability systems in the cloud. These new challenges have opened innovation opportunities for us to rethink the way we approach verification

IoT is driving the convergence of different technologies. How would it affect the way we verify the systems today?

To answer your question, I first want to step back in time to provide a framework for today’s challenges. In the 1990’s the concept of separation of concerns was introduced into engineering. Essentially, the idea is that verification would become more productive if we focused on verifying orthogonal concerns or requirements of the design separately versus trying to verify multiple concerns combined. For example, during this period of time, we learned that it is more efficient to verify functional concerns and physical concerns in separate simulation runs. This approach to verification worked well up to about 10 years ago. The emergence of mobile devices introduced new low-power requirements that made it difficult to separate concerns. For example, today we see that physical concerns (such as low power management) now can directly affect functional behavior of a device. Hence, these concerns need to be verified together. Bringing together physical, electrical, and functional has become mandatory.

The key point is that convergence of computing, networking, and communication, which is driving IoT, has introduced new layers of verification requirements that did not exist years ago, and the interaction of these requirements has had a profound effect on the way we must verify systems today.

What are the solutions that the EDA industry is driving to enable this next big wave in verification?

One contributing factor to growing verification complexity is the emergence of new layers of verification requirements, as I previously mentioned. For example, beyond the traditional functional domain, we have added clock domains, power domains, mixed-signal domains, security domains, safety requirements, software, and then obviously, overall performance requirements. Hence, we see the next big wave in verification on multiple fronts:

Continuing introductions of focused solutions optimized for specific verification concerns. Examples of these focused solutions include: formal apps focused on  verifying security features within a design or power apps used to provide complete RTL power exploration and accurate gate-level power analysis within emulation.

Emerging system-level analysis solutions, which provide new metrics and insight into the fully integrated SoC. This becomes essential for system-level performance analysis. The IoT SoC, for example, is a different beast than today’s state-of-the art networking SoC.

Greater convergence across multiple verification engines (e.g., simulation, emulation, and FPGA prototyping), which will improve productivity. The new Accellera Portable Stimulus standard will help facilitate this convergence and foster the introduction of new verification solutions.

Q4: Do you see domain specific solutions like automotive or machine learning etc. getting enabled for verification?

Yes, in fact there are multiple opportunities to leverage big data analytics to solve many system-level analysis problems. Machine learning is only one approach used today for big data analytics; however, there are others. Now, concerning domain-specific solutions in the automotive space, formal technology is being leveraged to improve productivity related to safety fault analysis.

Do you expect all workhorses (Simulation, Emulation & Formal) playing a critical role in verifying these new converged system level designs?

Obviously, this depends on the design. A project developing sensors for an IoT edge solution has different verification requirements than a project developing an automotive SoC containing multiple CPU and GPU cores, a coherent fabric, and multiple complex interfaces. Nonetheless, with increased design integration, multiple verification engines are required today that address the growing volume of verification requirements.

This is the 4^th edition of DVCon in India. What are your expectations from the conference?

DVCon, in general, is recognized as the premier conference on the application of languages, tools, methodologies and standards for the design and verification of electronic systems and integrated circuits. And DVCon India is no exception, which has continued to grow in both attendance and exhibitor participation. I expect DVCon 2017 will continue to deliver high-quality technical content and provide valuable networking opportunities for its attendees. It is the premier venue to share state-of-the-art developments and connect the creative minds working on these developments.

Thank you Ravi!

Join us on Day 2 (Sep 15) of DVCon India 2017 at Leela Palace, Bangalore to attend this keynote and other exciting topics.

Disclaimer: “The postings on this blog are my own and not necessarily reflect the views of Aricent”

Quick chat with Alok Jain : Keynote speaker, DVCon India 2016

Alok Jain

All of us have heard the story of a woodcutter and the importance of the quote “Sharpen your axe”. It applies well to everything we do including verification. Two decades back, the focus of a verification engineer was predominantly on “What to Verify”. As complexity grew “How to Verify” became equally important. To enable this, EDA teams rolled out multiple technologies & methodologies. As we try to assimilate & integrate these flows amidst first time silicon & cost pressure, it is important for us to sharpen our axe through continuous learning, applying the right tool for the right job and applying it effectively.

Alok Jain, Senior Group Director in the Advanced Verification Division at Cadence would be discussing on similar lines as part of his DV track keynote on Day 1 at DVCon India 2016. With 20+ years of industry experience, Alok leads the Advanced Verification Division at Cadence India. Having associated with different technologies around verification in the past 2 decades, Alok candidly shared his views on the challenges beyond complexity that verification teams need to focus on. Here is a curtain raiser for his talk "Verification of complex SoCs" –

Alok your keynote topic focuses on challenges in verification beyond the complexity resulting from Moore’s law. Tell us more about it?

The keynote is going to focus on challenges and potential solution for verification of complex SoCs. Verifying a complex SoC consisting of tens of embedded cores and hundreds of IPs is a major challenge in the industry today. One of the big challenges is performance and capacity. Given the size and complexity of modern SoCs, tests can run for 18-24 hours or even more. One has to figure out how to get the best verification throughput. Another challenge is generation of test benches and tests. The test benches have to be developed in a way which can achieve good performance in both simulation and hardware acceleration. Tests have to be created that stress the SoC under the application use cases, low power scenarios, and multi-core coherency scenarios. The tests have to be re-usable across pre-silicon and post-silicon verification and validation platforms. Yet another challenge is coverage. One has to measure verification coverage across formal, simulation, and acceleration platforms at the SoC level to know when you are done. The final challenge is how to effectively debug across RTL, test bench, and embedded software on multiple verification platforms.

In the last decade, advancements in verification was focused primarily on unifying HVL(s) & methodologies. What changes do you foresee in verification flows ‘Beyond UVM’?

UVM is very well suited for IP, Sub-system and some specific aspects of SoC verification. However, UVM is not the best approach for general SoC verification. UVM is essentially developed for “bottom-up” verification where the focus is on trying to exhaustively verify IP/sub-systems. SoCs require a more “top-down” verification where the focus is on stressing the SoC under important application use cases. There is a need to reuse SoC content across simulation, emulation, FPGA and post-silicon. UVM is optimized for simulation and is too slow and heavy for high speed platforms. Finally, there is a need to drive software stimulus on CPUs in coordination with hardware interfaces. It is difficult in UVM to drive and control software and hardware interfaces. All this is asking us to explore options beyond UVM. The keynote will cover some more insights into options beyond UVM.

The rise of IoT is stretching the design demands to far ends i.e. server class vs edge node devices. How do you see verification flows catering to these demands?

Several of the requirements for IoT verification are similar to the ones for complex SoCs. But then there are some unique additional requirements from the IoT world. The first is simply the cost of verification. For complex SoCs, the cost of verification has been steadily rising. For IoT applications, one has to consider alternative methods and flows that can reduce the cost. One option is to use some form of a correct by construction approach where the design is specifically done in a way to enable a simpler form of verification. Another approach is to put much more emphasis on reuse. This includes horizontal reuse which is portability across multiple platforms and vertical reuse which is reuse from IP to sub-system to SoC. Another requirement is verification throughput for design with considerably more analog, mixed signal and low power content. Finally, one has to devise verification techniques and flows that can cater to the security and safety requirements of modern IoT applications.

Formal took a while to become mainstream. The rise of Apps in Formal seems to have accelerated this adoption. What’s your view on this?

Yes, I do agree that Apps has considerably accelerated the pace of adoption of formal. Traditionally, formal tools have been developed and used by formal PhDs and experts. The main charter and motivation of these experts was to solve the coolest and hardest problems in formal verification. It was only after some time that both sides (developers and users) started realizing that formal can be used in a much more practical and usable way by engineers to solve specific problems. This lead to the development of various formal apps which greatly enabled the mainstream usage of formal.

This is the 3rd edition of DVCon India. What are your expectations from the conference?

I am expecting to attend keynotes, technical papers and panel discussions that give me an understanding of some the latest work in the domain of design and verification of IPs, sub-systems and SoCs. In addition, I am looking forward to the opportunity to network with some of my peers from the industry and academia.

Thank you Alok!

Come join us in this exciting journey to contribute, collaborate, connect & celebrate @ DVCon India 2016!

Disclaimer: “The postings on this blog are my own and not necessarily reflect the views of Aricent”

Quick chat with Sushil : Keynote speaker, DVCon India 2016

Sushil Gupta

A very famous urdu verse that translates  translates to “When I started I was alone, slowly others joined and a caravan formed” truly describes the plethora of challenges in SoC verification that continues to abound as the design complexity marches north. It started with growing logic on the silicon and moved to performance before power took over. While we still juggle up to handle the PPA implications, time to market pressure with cost effective secure customized solutions further add enough spice to the problem.

Sushil Gupta, Group Director in the Verification group at Synopsys covers these problems & potential solutions in his keynote titled “Today’s SoC Verification Challenges: Mobile and Beyond” on Day 2 of DVCon India 2016. Sushil joined Synopsys in 2015 as part of acquisition of Atrenta. He has 30 years of industry experience which spans various roles in engineering management and leadership in EDA and VLSI Design companies. Here is a quick excerpt of the conversation with Sushil around this topic –

Sushil your keynote topic focuses on challenges in verification associated with the next generation of SoCs. Tell us more about it?

We have seen the chip design industry shift its focus from computers and networking into System on Chips (SoC) for mobility – smartphones, tablets, and other consumer devices. The next wave of SoCs go beyond mobility into IoT, automotive, robotics, etc. These SoCs integrate hundreds of functions into a single chip and a complete software stack with drivers, operating system, etc.. The result is 10X increase in verification complexity in continually shrinking market windows. My talk focuses on these challenges and how verification solutions must scale to address them effectively.

Reuse of IP/Subsystems is the key trend with SoCs today. Do you think that reuse from third party add to challenges in verification? If yes, how?

IP/sub-system reuse (both third party and in-house) helps accelerate the integration of multiple functions into a single chip. However, these IP/sub-systems can come from multiple sources with heterogeneous design and verification flows. The resulting SoCs are extremely complex with  millions of lines of RTL and testbench, protocols, assertions, clock and power domains, and billions of cycles of OS boot.

Do you think progress in verification methodologies & flows have reached to a point where consolidation is key to allow verification engineer use the best of each? Any specific trends that you would like to highlight on this?

Integrated verification platforms are key to verification convergence. Verification now extends beyond functional verification into low power verification, debug automation, static  and formal verification, early software bring-up and emerging challenges with safety, security and privacy. This requires not only best-in-class verification tools and engines, but also native integrations between the tools to enable seamless transitions and faster convergence.

Sushil you have had a significant stint with formal at Atrenta. What are your thoughts on adoption of Formal coming to mainstream? How does the trend looks moving forward?

Formal is fast becoming mainstream because it can catch bugs that are otherwise very difficult to detect. Advancements in performance, debug and capacity of formal verification tools has enabled formal to become an integral part of a comprehensive SoC verification flow. The emergence of formal ‘Apps’ for clock and reset domains, low power, connectivity, sequential equivalence, coverage exclusions, etc. has enabled a broad range of design and verification engineers to benefit from formal verification without the need to be a formal “expert”.   

This is the 3rd edition of DVCon India. What are your expectations from the conference?

Speaking from my own experience having started my career with TI India in 1986, India has a very rich design and verification expertise. I hope to learn about the latest challenges and innovations in verification and look forward to working with our customers and partners on new breakthroughs.

Thank you Sushil!

Join us on Day 2 (Sept 16) of DVCon India 2016  at Leela Palace, Bangalore to attend this keynote and other exciting topics.

Disclaimer: “The postings on this blog are my own and not necessarily reflect the views of Aricent”

Marlin & Dory way of 'Finding Bugs'

On our return after watching ‘Finding Dory’, my son asked, “Dad, if you were to find Dory would you be able to do that”? I said, “Ofcourse”! Next, came HOW? I reminded him that my job is to Find Bugs and so I know the tricks of the game already. That made him super excited and wanting to know more about it. Given that this time the reference was picked by him, I decided to continue the same to explain him further.

In the movie, Marlin and Nemo were finding Dory inside the Marine Life Institute (MLI), likewise, we find bugs inside the design called as System on Chip (SoC). The SoC has a lot of similarity to MLI in the sense that it is big and complex. As MLI had different sections, our SoC has different blocks where Dory (Bug) can be found. Also it is not only the sections but the inter-connections that are equally important. When we look for Dory (Bug) inside these blocks we call it IP verification and when our focus is on the inter-connections we call it Integration or SoC Verification.

Image Source : http://www.socwall.com/desktop-wallpaper/7462/dory-and-marlin-by-ryone/

We start off our quest using the Marlin way i.e. “Assess the situation, evaluate, and plan it out”. We call it the Directed Verification approach wherein we understand the design, prepare a plan on where and how we would look around for Dory (Bug) and then execute accordingly. During this process we also keep asking (reviews) around (designers & peers) to let us know if we are missing out on anything. So if Dory is somewhere around, there is a chance we may sight her. But since Dory doesn’t think much before acting, that makes her unpredictable. There is always a possibility that we may not find her as per our plan.

My son’s eyeballs zoomed…. THEN?

Then we also do what Marlin & Nemo did i.e. follow “What would Dory do”? My son jumped, "She wouldn’t think twice and be random". Yes! We pick the Dory way and we call it Random Verification. We search randomly everywhere in an unplanned sequence and guess what? The chances that we would find Dory (Bug) increase. To make it more effective, we define weights and constraints to the randomness so as to improve our luck of finding her further. The approach now becomes Constrained Random Verification (CRV).  While following this random pattern we also take a note (coverage) of where all we have visited to avoid repeating same place again and save time. Now we can find her faster. Tracking coverage on top of CRV is called Coverage Driven Verification (CDV). So if we missed finding Dory (Bug) using the Marlin way (Directed verification) we still have an option to find her the Dory way (CRV).

That settled my son for a while till he pointed again saying, “Dad, maybe you should seek help from Bailey, the beluga whale who can find Dory faster than anyone using echo location”. I smirked and told him that we have our Bailey too and we call it Formal Verification. But then, Bailey was dependent on the whale voice between Dory & Destiny, the whale shark without which he couldn’t be of much help. Similarly, in Formal we are dependent on the assertions that connect the tool to the bug in the design. The effectiveness of this approach is purely dependent on the quality of voice (assertions) and the connect (covering all parts of design) between Dory & Destiny. But yes, if that is in place, it is really fast & effective.

Now convinced that his dad would be able to find Dory, my son asked, “So once you have found Dory, what do you do next”?

I laughed and told that we don’t have to find only 1 Dory (Bug). There are many of them and the address and architecture of the institute (new SoC) also keeps changing. So we just keep Finding Dory (Bug)!!!

Disclaimer: "The postings on this blog are my own and not necessarily reflects the views of Aricent"

Designers should not verify their own code! REALLY?

Around 2 decades back, the demands from designs were relatively simple and the focus was on improving performance. Process nodes had longer life, power optimization wasn’t even discussed and time to market pressure was relatively less given that the end products enjoyed long life. In those days, it was the designer who would first design and later verify his own code usually using the same HDL. Over the years, as complexity accelerated, a new breed of engineers entered the scene, the DV engineers! The rational given was that there is a need for an independent pair of eyes to confirm if the design is meeting the intent! Verification was still sequential to the design in early days of directed verification. Soon, there was a need for constrained random verification (CRV) and additional techniques to contain the growing verification challenge. The test bench development now started in parallel to the design, improving the size & need of verification teams further. With non HDLs i.e. HVLs entering the scene the need for DV engineers was inevitable. All these years, the rational of having an additional pair of eyes continued to be heard to an extent that we have started believing that designers should not verify their own code.

In my last post I emphasized the need for collaboration wherein designers and verification engineers need to come together for faster verification closure. Neil Johnson recently concluded in his post on designers verifying their own code. My 2 cents to whether designers should not or shall I say do not verify their own code –

To start with, let’s look at what all involves verification? The figure adjoining is a summary of efforts spent in verification based on the study conducted by Wilson research in 2012 (commissioned by Mentor). Clubbing some of the activities it is clear that ~40% of the time is spent in Test planning, Test bench development and other activities. The rest ~60% of the effort is spent on Debug & creating + running tests. The DUT here can be an IP or SoC. 

When an IP is under development or SoC is getting integrated, the DV engineers would be involved into the 40% of the activities mentioned above. These are the tasks that actually fall in line with the statement of additional pair of eyes validating design intent. They need to understand the architecture of the DUT and come up with a verification plan, develop verification environment and hooks to monitor progress. At this level, the involvement of the design team starts with activities like test plan review, code coverage review, inputs to corner cases and tests of interest. 

So, once the design is alive on the testbench, do the designers just sit & watch the DV team validate the representation of spec developed by them? NO!

Debugging alone is a single major activity that consumes an equal amount or sometimes more efforts from the designers to root cause the bug. Apart from it, there is significant involvement of the design team during IP & SoC verification.

For IPs, CRV is a usual choice. The power of CRV lies in automating the test generation using the testbench. A little additional automation enables the designers to generate constrained tests themselves. Assertions are another very important aspect in IPs. With introduction of assertion synthesis tools, the designers work on segregating the generated points into assertions or coverage. For SoCs, apart from reuse of CRV, directed verification is an obvious choice. Introduction to new tools on graph based verification help designers to try out tests based on the test plan developed by the DV engineer. Apart from this, corner case analysis and usecase waveform reviews are another time consuming contributions put in by designers for verifying the DUT.

Coming back to the rational on having an independent pair of eyes verify the code, the implication was never that the designers shall not verify their own code. Infact there is no way for the DV team to do it in a disjoint fashion. Today the verification engineer himself is designing a highly sophisticated test bench that is actually equivalent to a designer’s code in complexity.  So it would be rather apt to say that it is the two designs striking each other to enable verification under the collaboration between design & verification teams!

What is your take on this? Drop a note below!

The 4th C in Verification

The 3C’s of verification i.e. Constraints, Checkers & Coverage have been playing an important role enabling faster verification closure. With growing complexity and shrinking market windows it is important to introduce the 4^th C that can be a game changer in actually differentiating your product development life cycle. Interestingly the 4^th C is less technical but highly effective in results. It is agnostic to the tool or flow or methodology but if introduced and practiced diligently would surely result in multi-fold returns. Since verification claims almost 70%of the ASIC design cycle, it is evident that timely sign off on DV would be the key to faster time to market of the product. Yes, the 4^th C I am referring to is Collaboration!

UVM demonstrates a perfect example of collaboration within the verification fraternity to converge on a methodology that benefits everyone. Verification today spreads beyond RTL simulations to high level model validation, virtual platform based design validation, analog model validation, static checks, timing simulations, FPGA prototyping/emulation and post silicon validation. What this means is that we need to step out and collaborate with different stakeholders enabling faster closure.

The first & foremost being the architecture team, RTL designers & analog designers who conceive the design and realize it in some or the other form and many a times fall short of accurate documentation. The architecture team can help to a large extent in defining the context under which theverification needs to be carried out thereby narrowing down the scope. With a variety of tools available, the DV teams can work closely with designers to clean the RTL removing obvious issues that otherwise would stall simulation progress. Further, assertion synthesis and coverage closure would help in closing the verification at different levels smoothly. Working with analog designers can help tune the models and their validation process wrt the circuit representation of the design. This enables faster closure of designs that see increased scope of analog on silicon.

Next are the tools that we use. It is important to collaborate with the EDA vendors in not just being the user of the tool but working closely with them in anticipating the challenges expected in the next design and be early adopters of the tools to flush the flows and get ready for the real drill. Similarly, joining hands with the IP & VIP vendors is equally crucial. Setting up right expectations with the IP vendors on the deliverables from verification view point i.e. coverage metrics, test plans, integration guide, integration tests etc. would help in faster closure on SoC verification. Working with VIP vendors to define how best to leverage the VIP components, sequences, tests & coverage etc. at block and SoC level avoids redundant efforts and help in closing verification faster.

The design service providers augment the existing teams bringing the required elasticity to the project needs or take up ownership of derivatives and execute them. These engineers are exposed to a variety of flows and methodologies while contributing to different projects. They can help in introducing efficiency to the existing ways of accomplishing tasks. Auditing existing flows and porting the legacy environment to better ones is another way these groups can contribute effectively if partnered aptly.

Finally the software teams that bring life to the HW we verify. In my last blog I highlighted the need for HW & SW teams to work more closely and how verification teams acts as a bridge between the two. Working closely with the SW teams can improve reusability and eliminate redundancies in the efforts.

Collaboration today is the need of the hour! We need to be open to recognize the efforts put in by different stakeholders from the ecosystem to realize a product. Collaboration improves reuse and avoids a lot of wasted efforts in terms of repeated work or incorrect understanding of intent. Above all, the camaraderie developed as part of this process would ensure that any or all these folks are available at any time to jump in the hour of need to cover for unforeseen effects of Murphy’s law.

Drop in your experiences & views with collaboration in the comments section.

DVCON India : Journey of Verification (preview)

It’s hard to tell whether it’s been 30+ years or more since the initial footsteps were taken towards this journey of verification. Whenever or wherever it took off, the caravan continues to grow. For sure it has been an exciting journey so far! The start of DVCON in India this year marks an important milestone for the troops settled in different geographies and connected with this common link we call Verification (Sid'dha-karana in the local language hindi). As an acknowledgement towards the countless efforts by each one in this journey and to celebrate the contributions of each verification engineer towards silicon success, yours truly is presenting a talk on Journey of Verification at DVCON India 2014 on Sept 25 in Bangalore.

The 2 day event touches all aspects of verification with keynotes from industry luminaries, tutorials from subject matter gurus, invited talks from domain leaders, best of the pick papers /posters, opportunity to network with the who’s who and extended exhibition hours. In this mixed bag covering all technical areas on system development and DV, I plan to share a recap of everything around verification. The story starts off introducing the basic ingredients of verification and peeps into the the need for a verification engineer, verification team and an arsenal full of tools.

After introducing the star cast, the story would move touching areas like directed verification, constrained random, the language wars, methodology wars, format wars and the conclusions on them. We quickly review the IP verification of today, aspects of SoC verification and the trends on where are we heading. The story winds up revisiting some myths that I come across talking to verification engineers and managers as part of my job. I hope this discussion would help you remember those nostalgic experiences, learn something that was always considered implicit and absorb all of it with humour.

It is an attempt to present my 2 cents on this magical world of Verification that gives us an opportunity to start out hunt for bugs everyday and try out different arms in this quest. I hope to see you join us on Sept 25-26 at Bangalore.

- If you haven’t yet registered for DVCON India 2014, here is the link.
- If you are attending DVCON India, let’s meet!
- If you want to share your exciting moments in this journey of verification drop in a comment or email me. Everyone has a story to tell.... let’s share it here!

Suggested Reading -

Recap from the PAST - the last decade!

Living the PRESENT - the current decade!

Predictions for future - the next decade!

Taking a pulse of what's going on in Verification!

The design complexity today is marching forward at an accelerated speed and its effect on verification is witnessing equal leaps and bounds. The world of verification has grown multi-fold in past 5 years across all fronts covering what all to verify, how to verify and who all required to verify. Today, conferences hosted by EDA partners are not just focusing on a simulator, a language, a methodology or a new tool and instead bringing up discussions on a plethora of topics. It seems to address this growth, Cadence decided to have the 2^nd day of CDNLive India 2014 mostly targeting the DV community hosting multiple tracks on verification. With hundreds of footfalls, a mixed bag of papers on all aspects of verification, an extended exhibitor pavilion for partners along with lunch and tea sessions busy with networking, the event was truly in sync with the theme, connect... share... inspire!

A decade back all such events focussed on sharing the upgrades in tools particularly the simulator. The technology update included introduction to new features, added support for language and improved performance. Today no one talks about the simulator and instead everything around it getting integrated under a hood with features like –

• A broad VIP catalog with added support for easy bring up of test benches for IP/SoC, compliance test suite with ready made coverage/assertions. A new class of accelerated VIPs that interface easily with the emulators giving further boost to the overall productivity.
• Updates to power aware verification involving multiple tools (formal, simulator & emulator) with support for different power formats/versions, auto generation of relevant assertions and different aides to ease debugging these scenarios.
• How to model analog blocks to achieve high performance with electrical accuracy as part of the AMS support. Different languages supporting model development, coverage driven verification to validate these models and reuse of the environment when integrated with digital blocks.
• A portfolio of formal apps that can accomplish the job of static checks related to connectivity, power, registers, X-prop, protocol compliance checks etc.
• Improved support for IPXACT based flows to enable register modelling, register verification and interface connectivity promoting IP reuse with minimum issues.
• Integrated support for coverage collection and merging across different levels (IP, sub system & SoC) of verification involving different tools or flows used to achieve verification closure.
• Verification management featuring executable verification plans, regression management, triaging and analysis with different views based on user’s role in the project.
• Added debug tools and support to view the transactions, filter or play around back and forth with the simulation logs.
• Hardware accelerators with improved capacity, performance and features that enable detailed debugging, power aware support, assertions and coverage.
• Prototyping platforms/emulators and how they enable boot up with android etc. much before the silicon arrives.
• Improved Virtual platforms and models in sync with the above to enable early software development thereby shifting the whole product development cycle to the left.
• Performance analysis of SoC confirming architecture stability or assisting in exploring alternates promptly.

Yes! Verification has evolved into a GODZILLA beyond the control of one HVL, one methodology and one simulator. The rules of the game have changed and the industry is responding faster than ever to this change. Observing all these changes I recollect the topic EDA360 introduced by Cadence to the industry back in 2010. While the terminology might have lost its steam, the essence of the idea seems to have realized quite a lot since then. For those who missed reading about it please refer to the below posts summarizing EDA360. Believe me it’s worth reading!

EDA360 : Realizing what it is?

EDA360 : Realizing the Realizations

EDA360 Realizations : Verification perspective

Do you agree that the current state is in sync with EDA360? Leave a comment!

If you have missed out to any of these events, don’t miss to attend DVCON India 2014 on Sept 25, 26. Registrations now open.

DVCON goes GLOBAL!

Design and Verification Conference popularly known as DVCON is beginning to flex its muscles and move out of Silicon Valley to reach your friendly neighborhood this year. Yes, DVCON will be hosting the conference at Bangalore, India and Munich, Germany in 2014 on top of the already concluded conference at San Jose, California. This blog post is a quick refresher about the event for fellow engineers who have been busy solving the most intricate verification challenges in the trenches unknowingly that a platform does exist to discuss those puzzles and hear out how the community is handling them while sharing their solutions. It’s an avenue promoting collaboration among the design and verification community without a bias for a particular tool, flow or methodology. It’s a forum where everyone, a novice or a guru, a student or a professional, a beginner or an expert come together to discuss the takeaways that can be implemented as soon as they hit their work stations. It’s a place where you learn, discuss, network, collaborate and connect.....and this time it’s HERE (Bangalore & Munich)!

HISTORY of DVCON

It’s been 25 years in the spirit and 10 years to the name for this conference. The origin can be traced back to late 80s when VHDL started picking up; the user community started meeting twice a year under VUG (VHDL users group) leading to a conference by the name VIUF (VHDL International User Forum). Around the same time Verilog too gained user traction leading to IVC (International Verilog Conference) in early 90s. While the two events continued to serve the respective communities, they joined hands in 1997 giving way to IVC/VIUF conference later termed as HDLCon. Finally in 2003, HDLCon became DVCON giving it a legacy of 25 years and a brand that continues to evolve for a decade now. In 2014, it extends the reach to India and Europe.

DVCON India

ISCUG (Indian SystemC User’s Group) has been hosting an event with focus to accelerate adoption of SystemC as an open source standard for ESL design for past 2 years. This platform now morphs into DVCON India, a 2-day conference in Bangalore on Sept 25-26 2014 running Design Verification and ESL track in parallel.  In words of the committee, DVCon India is an attempt to bring a vendor-neutral international Design and Verification conference closer to home for the benefit of the broader engineering community. An excellent platform to share knowledge, experience and best practices covering ESL, Design & Verification for IP and SOC, VIP development and Virtual Prototyping for Embedded Software development and debug. The conference provides multiple opportunities to interact with industry experts delivering keynote speeches, invited talks, tutorials, panel discussions, technical paper presentations, poster sessions and exhibits from ecosystem partners.

Call for abstracts – submission deadline JULY 21^st, 2014.

Registrations soon to be opened.

DVCON Europe

A new chapter for the Design and Verification community in Europe starts at Munich, Germany this year on Oct 14-15, 2014. The focus of the conference would be on Electronic System Level (ESL), Verification & Validation, Analog/Mixed-Signal, IP reuse, Design Automation, and Low Power design and verification. To know more about what you’ll see at the conference hear it in the words of Martin Barnasconi, General Chair DVCON Europe here.

Call for abstracts – submission deadline over.

Registrations – Open.

This new move from Accellera Systems Initiative opens up opportunities for thousands of engineers who have benefited from the proceedings of DVCON for a long time to get involved, contribute and learn for a better tomorrow. As Benjamin Franklin rightly said,

Tell me and I forget.

Teach me and I remember.

Involve me and I learn.

An excellent opportunity for all of us to get involved & learn!