Showing posts with label PREDICTIONS. Show all posts
Showing posts with label PREDICTIONS. Show all posts

Sunday, March 3, 2013

Over-verification : an intricate puzzle

For verification, it was an eventful week. DVCON 2013 kept everyone busy with record attendance at the sessions and by following the tweets & blogs that resulted from them. The major highlight of this year’s conference was release of the latest update to System Verilog standard, IEEE 1800-2012 and free PDF copies made available, courtesy - Accellera.
 
With verification constantly marching to increase its claim on ASIC design schedule while retaining its position as a major factor for silicon re-spins, verification planning was a hot topic of discussion at DVCON. Some of the interesting points that came out of a panel discussion on verification planning were –
 
- Verification plan is not just a wish list. You have to define how you’re going to get there.
- Problem is not over-planning, but over-verifying designs because there has not been enough planning.
- Biggest objection we hear is we don’t have time to capture a verification plan. But you'll lose more time if you don't.
- What’s useful in verification planning is “ruthless prioritization.” You can never get it all done.
- My biggest challenge is getting marketing input into my verification plan.
- Failure to plan means planning to fail.
 
Last week, I guest blogged on a similar topic based on a recent survey conducted by Catherine & Neil. Clearly, the issue of poor planning gets highlighted in all areas of product development.
 
Traditionally, the verification plans were just a list of features to be verified, addressing ‘what to verify’. With the emergence of CRV, the plans started including the second aspect i.e. ‘how to verify’. Further, to bring focus to this never ending verification problem, CDV was adopted. The verification plans now started including target numbers in terms of coverage (code, functional and assertions) to define ‘when are we done’. With a given set of resources, when the ASIC design schedule is imposed on the verification plan, meeting the goals is a challenge. There arises a need to prioritize verification in terms of the features. Remember, Any code that is not verified will not work!
 
To enable this “ruthless prioritization”, collaboration is required among marketing, software and hardware groups to align to the design objectives. Everyone needs to understand the potential end applications and preferential ways in the design to achieve them. In case of IPs, this could mean that the initial releases target a limited set of applications based on the customers on board. Once that is achieved, ‘over-verifying’ can take over to further close on the grey areas. In case of SoC, it is a tough call. While design cost continues to increase with diminishing dimensions, the break even may not happen with limited applications (as a result of limited verification) of the SoC. The problem gets aggravated further with the specifications changing on the fly. A platform based approach could be a potential solution where variants of SoC are churned out frequently, but again defining the platform and prioritizing the features boils down to the same problem. A tough nut to crack!
 
The whole point of over-verifying comes to the fore front because verification is the long pole in the schedule. What if the designs could be ‘over verified’ within the timelines? What does it take to achieve that? Are tools like intelligent test benches, formal verification, hardware acceleration or cloud computing a solution? If yes, what is the associated cost and how does it affect?
 
There is no easy answer to any of these questions. In words of Albert Einstein, “The world we have created is a product of our thinking. It cannot be changed without changing our thinking.”
 
Probably when an answer comes out, we will be on the road of commoditizing the hardware. Till then 'over-verification' is what we have to live with!

Saturday, September 15, 2012

Communicating BUGS or BUGgy Communication

A few decades back when the designs had limited gate count, designers used to verify their code themselves. With design complexity increasing, the verification engineers were introduced to the ASIC teams. As Moore’s law continues to drive the complexity further, IP reuse picked up and this ensured that the engineers are spread all around the globe and communication across geographies is inevitable.
 
The reason for introducing the verification engineer was BUGS. A lot has been written and discussed on verification but reference to bugs is limited. With IPs getting sourced from different parts of the world and companies having extended teams everywhere, communicating the BUGs effectively becomes all the more important. “Wrong assumptions are costly” and in semiconductor industry this can throw a team out of business completely. Recently, in a mid-program verification audit for a startup with teams sitting between US & India, I realized that putting a well defined structure on communicating bugs could improve the turnaround time tremendously. Due to different time zones and working style, there was a lot of back & forth communication between the team members. Having a well defined format for communicating bugs helped a lot.
 
BUG COMMUNICATION CYCLE
 
BUG reported à BUG reproduced by designer à BUG fixed à FIX validated à BUG closed à revisit later for review, reproducing when required or data mining.
 
APPROACH
 
Before the advent of CRV, directed verification approach was common. Communicating bugs was simple and required limited information.  Introduction of CRV helped in the finding the bugs faster but sharing and preserving information around bugs became complicated. With a well defined approach, this problem can be moderated. Here is a sample format on what should be taken care of at each stage of the above cycle –
 
BUG reporting
 
Defining mnemonics for blocks or data path or scenarios etc. that can be appended along with the bug title helps. This enables categorizing the bugs at any time during project execution.
 
While the tool enforces adding the project related information, severity, prioritization and contact details of all concerned to broadcast information, the detailed description section should include –
 
- Brief description of the issue
- Information on diagnosis based on the debug
- Test case(s) used to discover this bug
- Seed value(s) for which the bug is surfaced
- Command line option to simulate the scenario and reproduce the issue
- Testbench Changelist/TAG on which the issue was diagnosed
- RTL Changelist/ TAG on which this issue was diagnosed
- Assertion/Cover point that covers this bug
- Link to available logs & dump that has the failure
 
BUG fixing
 
After the designer has root caused and fixed the bug, the bug report needs to be updated with –
 
- Root cause analysis and the fix
- Files affected during this fix
- RTL changelist/TAG that has the fix
 
FIX Validation
 
After the BUG moves to fixed stage, the verification engineer needs to update the TB if required and rerun the test. The test should pass with the seed value(s) required and then with multiple random seeds. The assertion/cover point should be covered in these runs multiple times. With this, the report can be updated further with –
 
- RTL changelist/TAG used to validate the bug
- Testbench changelist/TAG on which the issue was validated
- Pointer to the logs & waveforms of the validated test (if required to be preserved)
 
BUG Closed
 
After validating the fix, the bug can be moved to closed state. The verification engineer needs to check if there is a need to move this test to the smoke test list/mini regression and update the list accordingly.
 
Following the above approach would really help to communicate, reproduce and preserve the information around BUGs. The rising intricacies of the design demand a disciplined approach to attain efficiency in the outcome. Communicating bugs is the first step towards it!
 
HAPPY BUG HUNTING!!!

Sunday, February 26, 2012

Verification IP : Changing landscape - Part 2

In the part one of this document we discussed about VIP and the changing landscape. While the increasing demand of VIPs would drive more participants to enter this space, there are significant challenges to come up with a decent VIP portfolio.

Challenges for new entrants

- Which protocol(s)/standard(s)? SoC design houses are well aware of their requirements, but any other entity wanting to invest in developing the VIPs to license further needs to engage with potential customer(s) and build the team with right skill set accordingly.
- Which HVL and Methodology? Starting with an early customer helps in resolving this. However the risk is that the other potential customers may want a different HVL & Methodology. The current trends indicate that development with SV (IEEE 1800) and UVM is a safe bet.
- Expertise with domain/protocol and VIP development? Building a team without a design partner would mean bringing in domain experts along with VIP developers. Further the team needs to be around after implementation so as to work with customers for VIP deployment, fix bugs, modify based on customer requirement and upgrade as the standards evolve.
- How to validate the VIP? Initial clean up may involve short circuiting the Master & Slave agents of the VIP. To prove it golden, either it has to be validated with an already available golden VIP in the market (added cost) or working with a strategic customer who is willing to try out this solution on the already available golden IP or IP under development. Note that it takes significant cycles for the VIP to mature.
- Inter operability with tools? Once the VIP is working with one set of tools, it is important to evaluate it with other simulators. Porting of the VIPs to work on the emulators and formal tools (if applicable) should be planned next. Note that all tool vendors have alliance programs to facilitate such requirements.
- Additional cost? Apart from engineering & tool cost, membership to the standard bodies contributes to additional cost. Members are required to pay yearly fees for adopting the standard to develop a commercial product.

If new comers have a challenge to enter this arena, what should the existing partners of this ecosystem do?

SoC Integrators

When the focus is on rolling out SoCs, investing into in-house development of a VIP for standard protocol(s) doesn’t help in adding any differentiation to the end product. VIP development would incur cost towards tool licenses and engineers during and after development as resources would be required to support multiple projects deploying the VIP, fix bugs and upgrade the VIP as standards evolve at a fast pace. For big design houses getting a good deal with vendors would clearly direct ‘buy’ decision. For startups, utilizing resources to get the product out faster is more important than investing into VIP development. Getting into strategic engagement with VIP providers is a better solution.

EDA Vendors

Investing into VIP makes perfect sense for the EDA vendors for multiple reasons.
- The cost of development reduces with the tool sets available in house.
- The licensing model for the VIP is same as that of tools and complements the product portfolio quite well.
- The sales channel can be reused since the customer base is same.
- If the solution has limited competition it can open up sockets with new customers while making sure that the design houses sticking to vendor specific flow are maintained.
- The VIP(s) can be used as a test vehicle for validating the tools.
- With cloud computing picking up, a strong VIP portfolio will help in offering cloud based verification services quite easily.

Design Service Providers

For design service providers, it is good to invest in VIP development as it complements the services offered. Since they work with both SoC integrators and EDA vendors it makes sense for them to get into partnerships with either or both of these parties instead of developing a solution standalone. Sub system as an IP shall pick up soon. This is where the design services companies should invest by building a complete verification environment and test suite to verify these sub systems while partnering with the IP & VIP providers.

Changing landscape ....

Verification has been the forte for India. Whether to get the right skill set or reduce cost, the verification engineers from India have been involved at all levels i.e. VIP development, IP verification or SoC verification. Geographically, the changing landscape on the VIPs point the efforts to be converging from SoC integrators, EDA vendors and Design service providers in India.

Related posts -
Verification IP : Changing landscape - Part 1
Choosing the right VIP

Saturday, December 31, 2011

2011 : Bidding Adieu

The year 2011 had its own set of twists and turns. A plethora of applications riding over interesting set of electronic gadgets became part of our lives. Consumption of electronics devices continued to rise and market dynamics were unpredictable more than ever. New process nodes stabilized thereby promising to shrink designs  and reduce power consumption further. The tsunami added to the woes but Japan defied the odds and showcased its ability to bounce back quickly. The sad demise of Steve Jobs was a big loss to the electronics world. Amongst all of this we experienced that market window for products is constantly tightening which means the turnaround time for designs needs to reduce incessantly. Since verification claims most of the cycles in ASIC design, the community took one step further in multiple areas to bridge this gap. Some of the highlights include -

- UVM : Final release happened with quick adoption.
- Steps taken to bring together the differences in UPF & CPF.
- Tools for generating assertions automatically adopted widely.
- Cloud computing discussions moved from closed rooms to open forums.
- Hardware acceleration jumped to mainstream.
 
Some interesting facts also came up. While we keep talking about next generation of verification technologies, the primitive techniques continue to dominate in parallel. For example, at ‘Siddhakarana’ the most popular post was “Gate Level Simulations” with more than thousand hits and still getting visitors on daily basis. This essentially means there are areas that need attention before they become a bottleneck in getting the designs shipped faster.

On this note, I thank you for your valuable suggestions and readership throughout 2011. I look forward for the same in 2012.

Wish you and your family "HAPPY & PROSPEROUS 2012".

:)

Sunday, October 3, 2010

Predictions for future - The next decade!

Constraints & limitations with available resources lead to innovative thinking... to inventions... to evolution...

As we step into the next decade, the verification fraternity is all set to reap the benefits of the investments made in the present decade. Listed below are technologies (in no specific order) that would define the future course of verification. [Watch out for details on each one of them in future posts.]

1. UVM - The UVM standard committee is all set to roll out the UVM1.0 standard. This methodology sets up a new base for reusability, coverage driven verification, supporting different HVLs and curtailing the confusion of (which one's better).

2. Formal verification - Faster simulation, exhaustive scenario generation, handling complexity, you name it and formal [hai na] is there for it. Improved engines shall handle larger database and provide a defined way of moving from unit to chip level with formal. The APP based approach further brings in the required focus & results.

3. Hardware accelerators - Running real time applications at pre-silicon database with faster turnaround is the need of hour. Hardware accelerators would be able to suffice this demand enabling all domains to exercise real life data before moving to back end implementation.

4. Unified coverage model - Slowly creeping into main stream the work group is making progress to unify the coverage data base across different sources and simulators thereby improving reusability across many levels.

5. AMS - With analog claiming bigger chunk of the mixed signal die, verification is not limited to address correct integration only, instead measurement of analog performance while harmonics from digital operates in tandem would be more prevalent.

6. Low power designs - Orientation to go GREEN brings in new techniques to curb power. While verification ensures correct implementation it shall move a level up & assist in estimating power figures with different application scenarios (use cases) thereby defining optimum system architecture.

7. Electronic System Level (ESL) - Dealing with complexity is easier at higher abstraction levels. System modeling is gaining traction and so is system verification. Standards like TLM2.0 opened up gates for support of transaction modeling in different languages assisting in both design & verification.

8. Cloud computing - Problems associated with verification aren't only in developing test benches, debugging or reusing. Efficient management of compute infra structure to get maximum return on investments made in hardware & software is equally important. Cloud computing defines a new approach to deal with this chronic techno-business problem.

9. Virtual platforms - A complete platform that engages all players of the eco system at the same time i.e. application developers, embedded designers, ASIC design, verification & implementation teams to aid faster time to market will be the key to next generation designs. Verification holds the key to defining & developing such platforms.

10. Courses - With so much going in Verification, don't be surprised if it emerges as a separate stream for specialization in masters degree and adding more PHDs in this domain :)

The above list is what I forsee as the key to NEXT GEN verification.
If you have any other exciting stuff that should unfold in next decade do drop in your comments.

Previous -
'Recap from the past - the last decade'
'Living the present - the current decade'