Thursday, December 27, 2012

Evolution of the test bench - Part 1

Nothing is permanent except change and need constantly guides innovation. Taking a holistic view with reference to a theme throws light on the evolution of the subject. In a pursuit to double the transistors periodically, the design representation has experienced a shift from transistors  à gates à RTL and now to synthesizable models. As a by-product of this evolution, ensuring functional correctness became an ever growing unbounded problem. Test bench is the core of verification and has witnessed radical changes to circumvent this issue.
 
While the initial traces of the first test bench is hard to find, the formal test benches entered the scene with the advent of HDLs. These test benches were directed in nature, developed using Verilog/VHDL and mostly dumb. The simulations were completely steered by the tests i.e. the stimuli and checker mostly resided inside the tests. Soon variants of this scheme came wherein the HDLs were coupled with other programming languages like C/C++ or scripting languages like perl & tcl.  Typical characteristics of that period were, stretched process node life, multiple re-spins a norm, elongated product life cycle and relatively less TTM pressure.  For most part of the 90’s these test benches facilitated verification and were developed and maintained by the designers themselves. As design complexity increased, there was a need for independent pair of eyes to verify the code. This need launched the verification team into the ASIC group! Directed verification test suite soon became a formal deliverable with no. of tests defining the progress and coverage. After sustaining for almost a decade, this approach struggled to keep pace with the ramifications in the design space. The prime challenges that teams started facing include –
 
- The progress was mostly linear and directly proportional to the complexity i.e. no. of tests to be developed
- The tests were strictly written wrt clock period and slight change in design would lead to lot of rework
- Maintenance of the test suite for changes in the architecture/protocol was manual & quite time consuming
- Poor portability & reusability across projects or different versions of the same architecture
- High dependency on the engineers developing the tests in absence of a standard flow/methodology
- Corner case scenarios limited by the experience and imagination of the test plan developer
- Absence of a feedback to confirm that a test written for a specific feature really exercised the feature correctly
 
Though burdened with a list of issues, this approach still continues to find its remnants with legacy code.
 
In the last decade, SoC designs picked up and directed verification again came to the fore front. With processor(s) on chip, integration verification of the system is achieved mainly with directed tests developed in C/C++/ASM targeting the processor.  Such an approach is required at SoC level because –
 
- The scenario should run on the SoC using the processor(s) on chip
- Debugging a directed test case is easier given the prolonged simulation time at SoC level
- The focus in on integration verification wherein test plan is relatively straight forward
Constraints required for CRV needs to be strict at SoC vs block level and fine tuning them would involve a lot of debugging iterations which is costly at SoC level
 
While the above points justify the need for a directed approach to verify SoC, the challenges start unfolding once the team surpass the basic connectivity and integration tests. Developing scenarios that mimic the use-cases, concurrence behaviour, performance and power monitoring is an intricate task. Answers to this problem are evolving and once widely accepted they will complement the SoC directed verification approach to further evolve into a defined methodology.
 
Directed verification approach isn't dead! It has lived since the start of verification and the Keep It Simple Silly principle would continue to drive its existence for years to come. In the next part, we check out the evolution of CRV, its related HVLs and methodologies.
 
If you enjoyed reading this post, you would also appreciate the following –
 
Wish you a Happy and Prosperous 2013!!!
 
KEYWORD - HISTORY

Sunday, October 21, 2012

HANUMAN of verification team!

While Moore’s law continues to guide innovation and complexity in semiconductors, there are other factors that are further accelerating it. From iPod to iPhone5 via iPad, Apple has redefined the dynamics of this industry. New product launches include one from Apple every year, equally competitive and multiple products from other players and from Chinese cell phone makers every 3 months. There is an ever increasing demand of adding multiple functions to the SoCs while getting it right the 'first time' in 'limited time'. IP and sub system reuse has controlled the design lag of these SoCs significantly. While IP is assumed to be pre-verified at block level, the total time required to verify the SoC still is significant. A lot of this attributed to –
 
- Need for a staged approach to verify integration of the IPs on SoC
- Need to run concurrency tests, performance tests and use case scenarios
- Prolonged simulation time due to large design size
- Deploying multiple techniques to converge on verification closure
- Extended debug due to test bench complexity, test case complexity and all of the above
 
These challenges coupled with Murphy’s law conspire to pose a question on the verification schedule that claims a significant portion of the SoC design cycle. In the blog post Failure and Rescue, the author points to an interesting fact,things can and will go wrong. Yet some have a better capacity to prepare for the possibility, to limit the damage, and to sometimes even retrieve success from failure”. This directly applies to SoC verification too. Verification leads and managers are expected to build teams and ensure that the tools and processes deployed will diminish perceived risks while reduce unforeseen ones. The process involves bringing in engineers, tools and processes that match the project requirements. For effective management and resiliency, one needs HANUMAN to bring in balance to execution amidst uncertainty.
 
Who is Hanuman? HANUMAN is a Hindu deity, an ardent devotee of Lord Rama. With the commencement of festive season in India, number of mythological stories gain prominence. Hanuman is a central character in the Indian epic Ramayana, and also finds mentions in several other texts. He is also referred to as ‘Sankat Mochan’ i.e. SAVIOR who helped Lord Rama in precarious circumstances during the fight against evil. Last season, we correlated these epics with Verification here.
 
So where does HANUMAN find relevance in Verification?
 
SoCs today cannot be verified just with a team of verification engineers and a simulator. The process demands much more viz
 
- Meticulous planning on what to verify, how to verify, who will verify what and when we are done.
- Architecting the verification infrastructure to address verification plan development & tracking, test scenario generation, VIP sourcing and integration, assertion identification, coverage collection, power aware simulations, acceleration or emulation, regression management and automated triaging.
- Engineers, who can use the infrastructure efficiently, are experts in protocols and methodology, strong with problem solving and debugging.
 
Handling complexity amidst dubiety demands a RESCUER i.e. HANUMAN. The stakes are high and so are the challenges. Multiple intricate situations arise during the course of verification to decelerate the schedule. The RECOVERER from such situations can be an engineer, a tool or a methodology and that entity at that instance is a manifestation of HANUMAN.
 
Sit and recall your past projects...
...if you delivered in such situations, feel proud to be one
...if you identify someone who did it, acknowledge one
...if you haven't till now then be one!
 
May these avatars of HANUMAN continue to drive your silicon ‘right the first time and every time’.
Happy Dussehra!

Sunday, October 7, 2012

Verifying with JUGAAD

The total effort spent on verification is continuously on the rise (click here). This boost can be attributed to multiple reasons such as –
- Rising complexity of the design further guided by Moore’s law
- Constrained random test benches coupled with complex cross cover bins
- Incorporating multiple techniques to confirm verification closure
- Debugging RTL and the Verification environment
 
A study conducted by Wilson Research Group, commissioned by Mentor Graphics revealed that, mean time a design engineer spends in verification has increased from an average of 46% in 2007, to 50% in 2010. It also confirmed that debugging claims most part of verification engineer's bandwidth. While the effort spent on RTL debugging may rise gradually with the design size and complexity, TB debugging is showing up frequent spikes. Absence of a planned approach and limited support of the tools to enable this further adds up to the woes.  Issues in the verification environment arise mainly due to –
- Incorrect understanding of the protocol
- Limited understanding of the language and methodology features
- First timers making silly mistakes
- ‘JUGAAD’ (Hindi word for workaround)
 
Unlike design, the verification code was never subjected to area and performance optimization and the verification engineers were liberal in developing code. If something doesn’t work, find a quick workaround (jugaad) and get it working without contemplating the impact on testbench efficiency. Market dynamics now demand the faster turnaround of product and if verification is sluggish that impacts the product development schedule considerably. Below is one such case study picked from past experience wherein a complex core with parallel datapaths culminating into the memory arbiter (MA) block was to be verified.
 
GIVEN
 
CRV with Vera+RVM used to verify MA and block (XYZ) feeding MA. 100% functional coverage was achieved at block level for both modules. XYZ used complete frame structure to verify so average simulation time of test was 30 mins while MA used just a series of bytes & words long enough to fill FIFOs and simulation time was <5 mins. To stress MA further with complete frames of data and confirm it works fine with XYZ, CRV was chosen for XYZ+MA as a DUT. The rest of the datapath feeding XYZ was left to directed verification at top level as the total size of the core was quite large.
 
EXECUTION
 
The team quickly integrated the two environments and started simulating the tests. But this new env was taking ~16X more time as compared to XYZ standalone environment thereby impacting the regression time severely. This kicked off the debugging process of analyzing the bottleneck. First approach was to comment out the instances of MA monitor & scoreboard in the integrated env and rerun. If simulation time reduces then uncomment the instances and its tasks one by one to root cause the problem. On rerunning with this change there was no drop in simulation time. Damn! How was that possible?
 
Reviewing the changes, the team figured out that instead of commenting out the instances, the engineer had commented out the start of transactions. He claimed that just having an instance in the env shouldn’t affect as long as no transactions are getting processed by MA components. Made sense! But then why this Kolaveri (killer instinct)?
 
To nail down the problem multiple approaches like code review, increasing verbosity of logs and profiling were kicked off in parallel.
 
ROOT CAUSE
 
The MA TB had 2 mistakes. A thread was spawned from the new () task of scoreboard for maintaining the data structure and this code had a delay(1) to it. This was added by the MA engineer while debugging standalone env at some point in time as a JUGAAD.
 
task ma_xyz :: abc()
{
     variable declarations…
     while(1)
     {
        
        delay(1);         
     }
}
task new()
{
   
    fork
      abc();
    join none
}
 
Since this thread was spawned from new(), even though the start_xactor task was dormant this thread was still active causing the delay. Replacing this delay by posedge(clock) solved the issue and to respect guidelines this task was moved to a suitable place in the TB.
 
Lesson learnt – 'Jugaad' in the verification env of yesteryears doesn’t work so very well with modern day verification environment. Think twice while fixing you verification code or else the debugging effort on your project would again overshoot beyond average!
 
I invite you to share your experiences with such goofups! Drop an email to siddhakarana@gmail.com

Saturday, September 15, 2012

Communicating BUGS or BUGgy Communication

A few decades back when the designs had limited gate count, designers used to verify their code themselves. With design complexity increasing, the verification engineers were introduced to the ASIC teams. As Moore’s law continues to drive the complexity further, IP reuse picked up and this ensured that the engineers are spread all around the globe and communication across geographies is inevitable.
 
The reason for introducing the verification engineer was BUGS. A lot has been written and discussed on verification but reference to bugs is limited. With IPs getting sourced from different parts of the world and companies having extended teams everywhere, communicating the BUGs effectively becomes all the more important. “Wrong assumptions are costly” and in semiconductor industry this can throw a team out of business completely. Recently, in a mid-program verification audit for a startup with teams sitting between US & India, I realized that putting a well defined structure on communicating bugs could improve the turnaround time tremendously. Due to different time zones and working style, there was a lot of back & forth communication between the team members. Having a well defined format for communicating bugs helped a lot.
 
BUG COMMUNICATION CYCLE
 
BUG reported à BUG reproduced by designer à BUG fixed à FIX validated à BUG closed à revisit later for review, reproducing when required or data mining.
 
APPROACH
 
Before the advent of CRV, directed verification approach was common. Communicating bugs was simple and required limited information.  Introduction of CRV helped in the finding the bugs faster but sharing and preserving information around bugs became complicated. With a well defined approach, this problem can be moderated. Here is a sample format on what should be taken care of at each stage of the above cycle –
 
BUG reporting
 
Defining mnemonics for blocks or data path or scenarios etc. that can be appended along with the bug title helps. This enables categorizing the bugs at any time during project execution.
 
While the tool enforces adding the project related information, severity, prioritization and contact details of all concerned to broadcast information, the detailed description section should include –
 
- Brief description of the issue
- Information on diagnosis based on the debug
- Test case(s) used to discover this bug
- Seed value(s) for which the bug is surfaced
- Command line option to simulate the scenario and reproduce the issue
- Testbench Changelist/TAG on which the issue was diagnosed
- RTL Changelist/ TAG on which this issue was diagnosed
- Assertion/Cover point that covers this bug
- Link to available logs & dump that has the failure
 
BUG fixing
 
After the designer has root caused and fixed the bug, the bug report needs to be updated with –
 
- Root cause analysis and the fix
- Files affected during this fix
- RTL changelist/TAG that has the fix
 
FIX Validation
 
After the BUG moves to fixed stage, the verification engineer needs to update the TB if required and rerun the test. The test should pass with the seed value(s) required and then with multiple random seeds. The assertion/cover point should be covered in these runs multiple times. With this, the report can be updated further with –
 
- RTL changelist/TAG used to validate the bug
- Testbench changelist/TAG on which the issue was validated
- Pointer to the logs & waveforms of the validated test (if required to be preserved)
 
BUG Closed
 
After validating the fix, the bug can be moved to closed state. The verification engineer needs to check if there is a need to move this test to the smoke test list/mini regression and update the list accordingly.
 
Following the above approach would really help to communicate, reproduce and preserve the information around BUGs. The rising intricacies of the design demand a disciplined approach to attain efficiency in the outcome. Communicating bugs is the first step towards it!
 
HAPPY BUG HUNTING!!!

Wednesday, August 15, 2012

Laws and Verification

It’s Independence Day in India and I decided to use the freedom of applying some of the common laws to “Sid’dha-karana” aka Verification" :)

LAW means “a generalization that describes recurring facts or events in nature” and engineering is full of laws. Though laws are formulated for a particular aspect of engineering, most of these are widely applicable to parallel areas too. The idea here is to salute the heroes who articulated these postulates and derive conclusions applicable to our field of interest.

The Laws

In semiconductor industry the most widely referenced one is Moore’s Law formulated by Intel co-founder Gordon E. Moore, who described the trend in his paper (1965) later generalized as – The number of transistors on a chip will double approximately every two years.

Another common adage quite popular is Murphy’s Law named after Edward A Murphy based on his conclusion after experiments in 1949 stating "If there's more than one way to do a job, and one of those ways will result in disaster, then somebody will do it that way." The law is more generalized since then as – Anything that can go wrong, will go wrong.

Next set of laws form the basis for classical mechanics popularly known as Newton’s law of motion compiled by Sir Isaac Newton in 1687. The laws are –
First Law - Every object in a state of uniform motion tends to remain in that state of motion unless an external force is applied to it.
Second Law – The relationship between an object's mass m, its acceleration a, and the applied force F is F = ma. Acceleration and force are vectors and the direction of the force vector is the same as the direction of the acceleration vector.
Third Law – For every action there is an equal and opposite reaction.

Another law commonly discussed with reference to multiprocessing is Amdahl's law named after computer architect Gene Amdahl. It states that the performance improvement to be gained from using some faster mode of execution is limited by the fraction of the time the faster mode can be used.

Finally the law of Natural Selection formulated by Charles Darwin in support of the theory of evolution stating that, some individuals survive and reproduce more often than others, and as a consequence their heritable characteristics become more common over time.

Applying to Verification

Moore’s law – Amount of code to be verified doubles every 2 years.

Murphy’s law – Any code that isn’t verified will not work.

Based on the above two laws, we can further derive a conclusion that total Verification effort doubles every 2 years where effort = Engg time + Tools + Flows + CPU cycles.

Newton laws
First Law - Once the bug rate stabilizes, every code continues to be stable unless there is a change in external force. This force can be exerted by modifying constraints, introducing another methodology like assertions, formal etc.
Second Law - The improvement in rate of finding bugs is directly proportional to the net force applied and is inversely proportional to the total lines of code. This means, the effort, approach and outcome all depend on the F=ma relationship wherein, force is same as described in First law, m = lines of code and a is improvement in bug rate. In comparison to IPs, for large designs like SoCs, the effort is more while modifying the approach and the rate of finding bugs slows down for a given time.
Third Law - Every attempt to break the design results into more stable and mature design.

Amdahl’s law – When multiple verification techniques are applied, the overall gain (i.e. stability of code) is limited by the fraction of the time these techniques can be used e.g. CRV over directed, Formal over CRV, Manual tests vs auto generation of tests etc.

Law of Natural Selection – Change is inevitable and during the process of verification evolution, multiple approaches show up but only few find their footprint across design tape outs.

Drop in a comment with other laws you find applying to verification!
Happy Independence Day!!!

Sunday, April 29, 2012

Verification claims 70% of the chip design schedule!

Human psychology points to the fact that constant repetition of any statement registers the same into sub-conscious mind and we start believing into it. The statement, “Verification claims 70% of the schedule” has been floating around in articles, keynotes and discussions for almost 2 decades so much that even in absence of any data validating it, we believed it as a fact for a long time now. However, the progress in verification domain indicate that this number might actually by a "FACT".

20 years back, the designs were few K gates and the design team verified the RTL themselves. The test benches and tests were all developed in HDLs and sophisticated verification environment was not even part of the discussions. It was assumed that the verification accounted for roughly 50% of the effort.

Since then, the design complexity has grown exponentially and state of the art test benches with lot of metrics have pre empted legacy verification. Instead of designers, a team of verification engineers is deployed on each project to overcome the cumbersome task. Verification still continues to be an endless task demanding aggressive adoption of new techniques quite frequently.

A quick glance at the task list of verification team shows following items –
- Development of metric driven verification plan based on the specifications.
- Development of HVL+Methodology based constrained random test benches.
- Development of directed test benches for verifying processor integration in SoC.
- Power aware simulations.
- Analog mixed signal simulations.
- Debugging failures and regressing the design.
- Add tests to meet coverage goals (code, functional & assertions).
- Formal verification.
- Emulation/Hardware acceleration to speed up the turnaround time.
- Performance testing and usecases.
- Gate level simulations with different corners.
- Test vector development for post silicon validation.

The above list doesn’t include modeling for virtual platforms as it is still in early adopter stage. Along with the verification team, significant quanta of cycles are added by the design team towards debugging. If we try to quantify the CPU cycles required for verification on any project, the figures would easily over shadow any other task of the ASIC design cycle.

Excerpts from the Wilson Research study (commissioned by Mentor) indicate interesting data (approximated) –
- The industry adoption of code coverage has increased to 72 percent by 2010.
- The industry adoption of assertions had increased to 72 percent by 2010.
- Functional coverage adoption grew from 40% to 72% from 2007 to 2010.
- Constrained-random simulation techniques grew from 41% in 2007 to 69% in 2010.
- The industry adoption of formal property checking has increased by 53% from 2007 to 2010.
- Adoption of HW assisted acceleration/emulation increased by 75% from 2007 to 2010.
- Mean time a designer spends in verification has increased from an average of 46% in 2007 to 50% in 2010.
- Average verification team size grew by a whopping 58% during this period.
- 52% of chip failures were still due to functional problems.
- 66% of projects conitnue to be behind schedule. 45% of chips require two silicon passes and 25% require more than two passes.

While the biggies of the EDA industry are evolving the tools incessantly, a brigade of startups has surfaced with each trying to check this exorbitant problem of verification. The solutions are attacking the problem from multiple perspectives. Some of them are trying to shorten the regressions cycle, some moving the task from engineers to tools, some providing data mining while others providing guidance to reduce the overall efforts.

The semiconductor industry is continuously defining ways to control the volume of verification not only by adding new tools or techniques but redefining the ecosystem and collaborating at various levels. The steep rise in the usage of IPs (e.g. ARM’s market valuation reaching $12 billion, and Semico reporting the third-party IP market grew by close to 22 percent) and VIPs (read posts 1, 2, 3) is a clear indicative of this fact.

So much has been added to the arsenal of verification teams and their ownership in the ASIC design cycle that one can safely assume the verification efforts having moved from 50% in early 90s to 70% now. And since the process is still ON, it would be interesting to see if this magic figure of 70% still persist or moves up further!!!

Saturday, March 31, 2012

Choosing the right VIP

In past few months, while interacting with customers, I came across a couple of cases where the VIP played a spoilsport. In one case, the IP & VIP were procured from the same vendor during the early phase of standard protocol evolution. One of the key USPs of the product was this IP and the demonstration of the complete reference design was anticipated at a global computing event. The limitations of the VIP were revealed quite late during SoC verification. The team struggled hard to manage the bug rate in the IP itself during the SoC verification phase. Finally the product got delayed to an extent where the company (~startup) couldn’t manage to sustain and went for an asset sale. The problem wasn’t selecting the IP & VIP from the same vendor but lack of home work and maybe a wrong business decision during the selection process.

In another case, an in house developed IP having seen multiple tapeouts was getting validated on a new SoC architecture targeting a different application. The VIP was third party which had been used to verify the IP throughout. After integration testing at SoC level, while developing usecase scenarios, the limitation of the VIP came forward. Since the schedule didn’t allow the liberty to add features to the VIP and validate this scenario, the team went ahead with the tapeout. Unfortunately the application didn’t work on silicon and the root cause analysis revealed a bug in the IP itself. Result, re spin required.

Showstopper bugs, Project cancellations, Tapeout delays etc. all point to missing the market window and the post mortem may point figure to the VIP!!!

Selecting the right VIP demands a detailed evaluation process along with a comprehensive checklist to compare the solution from different vendors. To find available solutions in the market, www.chipestimate.com and www.design-reuse.com are quite useful. After identifying the set of applicable vendors, it is important to  collect relevant information for initial analysis –

- Detail documentation i.e. reference manual (architecture of VIP) and user manual (how to use it). These documents should list out all aspects of the VIP functionality including parameterizable sequences, component configurations, APIs, parameters, coverage, error messages etc.
- Understanding the process of VIP development from vendor i.e. how is the VIP validated, the maturity process, how many and what kind of design it has verified. The release process, bug reporting mechanism, turn around time for fixing bugs, commitment and plans to support the evolving standards.
- Vendor overall VIP portfolio, languages & methodology used, membership & participation to the relevant standards, development/support staff availability and domain expertise, level of onsite support possible.
- Standard compliance, compliance coverage report, items not supported in the current release, list of assertions, code & functional coverage report.
- Support across methodologies, simulators, accelerators, formal verification engines and ESL verification.
- Exiting customer references.

A detailed comparative study of the above data will help in narrowing down the search to final 1 or 2 vendors. The next phase involves qualitative analysis to validate the data provided by the vendor and the assumptions made during first phase by the evaluator. Here the engineer(s) needs to play around with the VIP using the examples provided by the vendor / test in target env / create a short circuit env with master-slave agents of VIP and try different scenarios to evaluate –

- Ease of use in terms of configuring the components, using the sequences, portability from block to system level, how would it work when simulated in parallel with other VIPs in the SoC verification environment.
- Does it meet the requirements of the metrics planned for determining verification completion and to what extent does the VIP inherently reports it by itself.
- APIs provided to develop tests for customized functionality of the IP, developing real life scenarios, directed tests, stress testing, errors injection & detection capability, debug and observability.

Interestingly the experience with the vendor during the evaluation phase indicates the level of support that can be expected further. The key to this two phase evaluation process is the checklist. VSIA provides a comprehensive QIP metrics (Final VSI edition) that provides a jump start to this evaluation process.

As it is said - "the devil is in the details", an additional effort during the start without overlooking the finer points can prevent from serious problems later. 

It would be interesting to learn your VIP evaluation experience. Drop a comment now!

Related posts -
Verification IP : Changing landscape - Part 1
Verification IP : Changing landscape - Part 2
 

Sunday, February 26, 2012

Verification IP : Changing landscape - Part 2

In the part one of this document we discussed about VIP and the changing landscape. While the increasing demand of VIPs would drive more participants to enter this space, there are significant challenges to come up with a decent VIP portfolio.

Challenges for new entrants

- Which protocol(s)/standard(s)? SoC design houses are well aware of their requirements, but any other entity wanting to invest in developing the VIPs to license further needs to engage with potential customer(s) and build the team with right skill set accordingly.
- Which HVL and Methodology? Starting with an early customer helps in resolving this. However the risk is that the other potential customers may want a different HVL & Methodology. The current trends indicate that development with SV (IEEE 1800) and UVM is a safe bet.
- Expertise with domain/protocol and VIP development? Building a team without a design partner would mean bringing in domain experts along with VIP developers. Further the team needs to be around after implementation so as to work with customers for VIP deployment, fix bugs, modify based on customer requirement and upgrade as the standards evolve.
- How to validate the VIP? Initial clean up may involve short circuiting the Master & Slave agents of the VIP. To prove it golden, either it has to be validated with an already available golden VIP in the market (added cost) or working with a strategic customer who is willing to try out this solution on the already available golden IP or IP under development. Note that it takes significant cycles for the VIP to mature.
- Inter operability with tools? Once the VIP is working with one set of tools, it is important to evaluate it with other simulators. Porting of the VIPs to work on the emulators and formal tools (if applicable) should be planned next. Note that all tool vendors have alliance programs to facilitate such requirements.
- Additional cost? Apart from engineering & tool cost, membership to the standard bodies contributes to additional cost. Members are required to pay yearly fees for adopting the standard to develop a commercial product.

If new comers have a challenge to enter this arena, what should the existing partners of this ecosystem do?

SoC Integrators

When the focus is on rolling out SoCs, investing into in-house development of a VIP for standard protocol(s) doesn’t help in adding any differentiation to the end product. VIP development would incur cost towards tool licenses and engineers during and after development as resources would be required to support multiple projects deploying the VIP, fix bugs and upgrade the VIP as standards evolve at a fast pace. For big design houses getting a good deal with vendors would clearly direct ‘buy’ decision. For startups, utilizing resources to get the product out faster is more important than investing into VIP development. Getting into strategic engagement with VIP providers is a better solution.

EDA Vendors

Investing into VIP makes perfect sense for the EDA vendors for multiple reasons.
- The cost of development reduces with the tool sets available in house.
- The licensing model for the VIP is same as that of tools and complements the product portfolio quite well.
- The sales channel can be reused since the customer base is same.
- If the solution has limited competition it can open up sockets with new customers while making sure that the design houses sticking to vendor specific flow are maintained.
- The VIP(s) can be used as a test vehicle for validating the tools.
- With cloud computing picking up, a strong VIP portfolio will help in offering cloud based verification services quite easily.

Design Service Providers

For design service providers, it is good to invest in VIP development as it complements the services offered. Since they work with both SoC integrators and EDA vendors it makes sense for them to get into partnerships with either or both of these parties instead of developing a solution standalone. Sub system as an IP shall pick up soon. This is where the design services companies should invest by building a complete verification environment and test suite to verify these sub systems while partnering with the IP & VIP providers.

Changing landscape ....

Verification has been the forte for India. Whether to get the right skill set or reduce cost, the verification engineers from India have been involved at all levels i.e. VIP development, IP verification or SoC verification. Geographically, the changing landscape on the VIPs point the efforts to be converging from SoC integrators, EDA vendors and Design service providers in India.

Related posts -
Verification IP : Changing landscape - Part 1
Choosing the right VIP

Monday, February 13, 2012

Verification IP : Changing landscape - Part 1

For decades, EDA industry has been working out options to improve their offerings and ensure silicon success for the semiconductor industry. A few decades back, while the EDA giants were unknown, design automation was exercised individually in every organization developing a product. Gradually these tools moved out of the design houses and the ‘make vs buy’ decision carved a new league with EDA front runners. Talking specifically about verification, while the simulation tools were procured from outside, the in house CAD groups were still required to develop a flow for automation, self checking, regression management and data analysis. In the last decade, all of this came up as a package from EDA vendors thereby further squeezing the CAD teams in the design houses. So, what next?  Well, the recent acquisitions in this space i.e. Cadence acquiring Denali, Synopsys acquiring nSys and ExpertIO indicate where we are heading. Initially the VIP providers were able to sustain with licensing and basic verification support model but now the pressure is surmounting as VIPs get commoditized. The VIP only vendors will have to identify themselves either with the EDA vendors or with Design service providers wherein the VIPs complement other offerings. Before moving ahead let’s discuss what is a VIP?
What is a VIP?
A Verification IP is a standalone ‘plug and play’ verification component that enables the verification engineer in verifying the relevant DUT (design under test) module at block, sub system and SoC level. Based on the need, the VIP can act as a BFM to drive DUT signals or MONITOR the signals and VALIDATE them for correctness and data integrity. It may have a set of protocol CHECKERS and test scenarios to confirm compliance with the standards (if any) or COVER groups identifying corner cases and test completeness. VIPs are developed using HDLs or HVLs with standard methodology or as a set of assertions. It needs to have enough APIs providing flexibility for modifying the internal units as per DUT. User should be able to extend the available monitors, scoreboard and checker hooks while developing a verification environment around this VIP. Ease of developing scenarios at different levels using available functions and sequences is important in minimizing the verification turn around cycle. Quick integration with other tools like coverage, simulate, debug and transaction analysis is critical in successfully deploying the VIP.  Finally the most crucial is the support provided by the VIP vendor to the user in responding to issues and upgrading the VIP as the standards evolve.
The common VIPs available include –
- MIPI protocols like DSI, CSI, HSI, SlimBus, Unipro, DigRF & RFFE.
- Bus protocols like AXI, AHB, APB, OCP & AMBA4.
- Interfaces like PCIexpress, USB2.0, USB3.0, Interlaken, RapidIO, JTAG, CAN, I2C, I2S, UART & SPI.
- Memory models & protocol checkers for SD/SDIO, SATA, SAS, ATAPI, DDR2/DDR3, LPDDR etc.

Companies providing the above VIPs fall into one of the below categories -
- EDA vendors
- VIP only providers (some of them offering IPs also)
- Design services companies

Although, there is a lot of IP development still continuing with the design houses, the motivation to procure VIP from outside is much higher for obvious reasons. 

In the next post we will discuss the challenges for new entrants that raise barriers to entry and role of Soc Integrators, EDA companies and design services organizations in this changing landscape.

Related posts -
Verification IP : Changing landscape - Part 2
Choosing the right VIP
 

Monday, January 23, 2012

Constrained Random Verification and Simulators

In response to my earlier post on CRV, Daniel Payne raised an interesting question on Semiwiki – “Is there any difference in choosing a VHDL, Verilog, System C or System Verilog simulator from Cadence, Synopsys or Mentor and then using the CRV approach”?
Before answering this, let’s quickly check the CRV approach. Constrained Random Verification includes choosing an HVL, defining the test bench architecture and developing constraints to generate legal random stimuli. When this test bench is used to simulate the DUT, a random seed value and a simulator become part of the verification environment. The seed helps in reproducing the failure only if the other inputs i.e. the test bench architecture (components hierarchy) and the set of constraints are constant. Any change to these inputs may lead to different results with the same seed value also. The random seed value and the constraints are fed to the constraint solver integrated in the simulator to generate random values.
Let’s assume the HVL used is System Verilog (IEEE 1800). The standard provides details on the data types, how to randomize, how to define constraints and output of expressions. It does not specify the algorithms involved and the order in which the constraint solver should generate the outcome. The implementation of the solver is left to the sole discretion of the simulator R&D teams. The available simulators from different vendors may also support multiple solvers that tradeoff between memory usage and performance. Therefore, a given input may not give the same results when run on –
- Simulators from different sources OR
- Different versions of the same simulator OR
- Using different solver from the same simulator.  
All the constraint solvers claim to solve standard puzzles like Sudoku, N-Queen problem etc. Some of them even claim to solve these standard problems in limited or minimum iterations. Although these claims are correct but its reproducibility entirely depends upon how the constraints are defined. The capability of any constraint solver to return results in a given time primarily depends upon the representation of constraints. For a given equation, the solver may sometimes time out, but if the constraints typifying the equation are restructured it gives results instantly.
So, what to look for in the simulator when implementing CRV approach?
All the industry standard simulators have stable solvers varying on performance and there is no standard way to benchmark them. However one should consider other important aspects too -

- Support of the data types planned for implementing the test bench.
- Ease of debugging the constraint failures.

Next important part is how to define clean and clear constraints that would aid the solver in giving out results quickly. Following the KISS (Keep It Simple Silly) principle sets the right ground. Split the constraints to smaller equations, particularly if the expressions involve multiple mathematical operators and data types. This will provide better control on the constraints blocks and help in solving these constraints easily. Defining easy to solve constraint would improve the predictability of the outcome i.e. the generated value to be in a legal range and how fast it can be generated.

Coming back to the main question, the answer is YES, different simulators will give different results (values and fastness) while using CRV approach. This will affect the turnaround time of regressions and coverage closure too. However, there is no standard benchmarking available on the constraint solvers from these simulators. It is up to the verification engineer how best he/she can define the constraints and turn the output to be solver independent.

NOTE – This doesn’t mean that solver will never hit a bottleneck. Even with well defined constraints, the constraint solvers fail to deliver the results sometimes.