Negative Testing in functional verification!!!

Imagine someone on an important call and the mobile device reboots suddenly! The call was to inform that the devices installed at the smart home seems to be behaving erratically with only elderly parents & kids to provide any further details. On booting up, the smartphone flashes that there has been a security breach and data privacy has been compromised. Amidst this chaos, the car’s cruise control didn’t respond to pressing of the pedals!!! Whew!!!.... nothing but one of the worst nightmares in the age of technology we live in! But what if some of it could be true someday? What if the user has little or no idea about that technology?

The mobile revolution has enabled a common man to access technology and use it for different applications. The data from Internet world statistics suggest that internet adoption worldwide has increased from 0.4% of world population in 1995 to 54.4% in 2017. Related data also indicate that a sizable portion of the users are aged & illiterate. The ease of use has potentially driven this adoption further with the basic assumption that devices would be functioning correctly 24x7 even if used incorrectly out of ignorance. The same assumptions are seamlessly getting extended to safety critical domains such as Medical & Auto introducing several unknown risks for the user.

So how does this impact the way we verify our designs?

Traditionally, verification is assumed to be ensuring that the RTL is an exact representation of the specifications. Given that the state space based on the design elements is so very huge, a targeted verification approach covering positive verification has been in practice all throughout. Here, Proof of no bug is assumed to be equal to No proof of bug! The only traces of anything beyond this approach include –

- Introducing asynchronous reset during the test execution to check that the design boots up correctly again.

- Introducing stimulus triggering exceptions in the design.

- Simulating architecture or design deadlock scenarios.

- Playing around with key signals per clock for low power scenarios and reviewing the corresponding design response.

But as we move forward with security and safety becoming key requirements of the design, is this good enough? There is a clear need to redefine the existing approach and bring Negative testing to mainstream! Negative testing ensures that the design can gracefully handle invalid inputs, unexpected user behavior, potential security threats or defects such as structural faults introduced while the device is operational. Amidst shrinking design schedules, negative testing really requires creative thinking coupled with focused effort. 

To start with, it is important to question the assumptions used while defining the verification plan for the design. Validating those assumptions itself can lead to a set of scenarios to be verified under this category. Next, review the constraints applied while generating stimulus to list out potential illegal inputs of interest. Caution should be taken in defining this list as the state space would be large. Reviewing it in the context (Context Aware Verification) of end application would surely help in narrowing down this illegal stimulus set. Further to this, faults need to be injected at critical points inside the DUT using EDA tools or innovative testbench techniques. This is important for safety critical applications where the design needs to respond to random faults and exit properly while notifying about the fault or even correct it. Of course not to forget that appropriate coverage needs to be applied to measure the reach of this additional effort.

As we step into an era of billions of devices empowering humans further, it is crucial that this system of systems is defect free especially when it touches safety critical part of our life. Negative testing is a potential way forward ensuring reliability of designs for such applications. As is always said – 

Better safe than sorry!

Quick chat with Apurva Kalia : Keynote speaker DVCon India 2017

Apurva Kalia

The advancements in semiconductor industry starting picking up with the rise in performance of processors driving the computer industry. Next, the mobile segment opened floodgates when the PC market stagnated & then low power with smaller dimensions on top of performance drove the innovation in silicon implementation. The industry today is at cross roads once again awaiting the next big thing. Automotive is one of the key areas to get the ball rolling yet again. But then, each domain has its characteristics that needs to be aligned to!

Apurva Kalia, Vice President of R&D focusing on Automotive solutions at Cadence picks on an interesting topic for his DV track keynote on Day 1 at DVCon India 2017. With the auto industry shifting gears into autonomous cars, the question worth asking is – Would you send your child to school in an autonomous car? Yes, that’s the theme of Apurva’s keynote and here’s a sneak peek on this topic.

Apurva your keynote focusses on ‘autonomous cars’ – the talk of the town these days. Tell us more about it?

Well, there is major inflection point coming up in automotive electronics. We all know that Moore’s Law driven advances in cost per transistor and capacity have been holding up for many years. Complex chips are now possible within a cost factor that was not possible earlier. Moreover advances in algorithms, especially Machine Learning, now enables much more complex processing, especially vision based processing, to be done in real time. Both these trends coming together with advances in sensor technology has enabled systems to be created which can detect their environment quite accurately and in real time. This is the basis of autonomous driving. Also, as we know, every few years the semiconductor industry is looking for the next big trend which will drive the fab capacity. The above factors are pushing autonomous driving to be the talk of the town.

Security & Safety are emerging areas resulting from this topic. How does this change the way we verify our designs?

As I described above, with autonomous driving really taking off, these systems are becoming mission critical for the automobile. This means that the system needs to be safe and secure. It is inconceivable for a car to stop working at 80 kmph on a highway! Also, with the car needed to be connected to other cars and even to infrastructure and internet, this opens the system to attacks and makes it vulnerable. Therefore, these systems needs to make safe and secure to ensure safety and security of the automobile.

What are the solutions that the EDA industry is driving to enable ISO 26262 requirements from process & product perspective?

ISO26262 is the main standard that defines the safety requirements for automobiles. It is a very comprehensive standard which places requirements on all automotive systems. In fact edition 2 of the standard – coming out in Jan 2018 – will focus specially on semiconductors. Given the excitement around automotive electronics and autonomous systems, EDA industry needs to retool rapidly to address this need. Ensuring safety in these designs requires additional design and verification flows, methodologies and tool changes. The EDA industry needs to step up to define and create these flows, methodologies and tools required.

What are your views on the couple of accidents that happened in the US with autonomous cars? What could have been done better?

We are at early stages of this technology. Unfortunately as with any new technology, technology will take time to stabilize. In the meantime, during this stabilization time, unfortunate things like these accidents could happen. Organizations and individuals who are early adopters of these technologies take these risks, but they also contribute in a big way for advancement of these technologies. However, with the proper use of tools, implementation of standards, and focus on new solutions, we can avoid these kind of accidents.

How do you observe the adoption of autonomous cars across the globe & in India?

Autonomous cars are here to stay. They are solving real problems in real environments. We already have examples of autonomous cars on real roads – driving very safely. In fact, there are statistics which show that autonomous cars will actually cut down on accidents and fatalities – the most of which are caused by human error. Last year, I saw an engineering college in Delhi demonstrate an autonomous vehicle in Govindpuri – one of the most congested areas of Delhi. So this technology is real and works. I think it is just a matter of a few years when we will see this mainstream.

Do you see all workhorses (Simulation, Emulation & Formal) playing a critical role in realizing Auto grade designs?

Yes – all current EDA technologies – not just verification technologies, but even implementation technologies – need to be upgraded to support safety and security design and verification. All engines will need enhancements and special features to support these new requirements and flows.

This is the 4^th edition of DVCon in India. What are your expectations from the conference?

I have seen DVCon India grow from humble beginnings to an excellent conference today. I think this conference provides a very good platform to share and discuss new trends in design and verification. I look forward to stimulating conversations on new flows and technologies. This conference attracts many design companies and all EDA vendors in India – what better assemblage of the right people for these discussions.

Thank you Apurva!

Join us on Day 1 (Sep 14) of DVCon India 2017 at Leela Palace, Bangalore to attend this keynote and other exciting topics.

Disclaimer: “The postings on this blog are my own and not necessarily reflect the views of Aricent”