Product

How We Learn From Your Bug History

9 min read James Okafor
Abstract data pattern visualization representing machine learning on historical engineering data

Every semiconductor team re-discovers the same categories of bugs. A new project starts, the design methodology looks solid on paper, and somewhere around the third simulation run the team is staring at a CDC hazard that looks almost exactly like the one they fixed in the previous chip — just in a different module, with a slightly different synchronizer topology, at a clock domain boundary that got added late in the design cycle.

This isn't a skills problem. It's a knowledge transfer problem. The engineer who fixed the previous chip's CDC bug has it encoded in their head; the new project's RTL was written by someone who joined after that tape-out. The institutional memory lives in simulation log files, bug databases, and post-silicon errata documents that nobody has time to fully read before starting the next project.

We built Fpgawright's learning layer around the premise that this institutional knowledge should be extracted, structured, and fed back into the static analysis tool automatically — so the lessons from your last bug are active in your current checkin flow, not sitting in a JIRA ticket from 18 months ago.

What We Ingest

The learning pipeline starts with three data sources that most IC design teams already have:

Simulation failure logs. VCS, Questa, Xcelium, and other major simulators produce structured failure output — assertion failures, X-propagation reports, timeout signals, functional test failures with VCD snapshots. We parse these logs to extract the failure type, the RTL hierarchy path of the failing signal, the test scenario that triggered the failure, and the timestamp. Over a project lifetime, this becomes a signal-frequency map: which parts of the design fail most often, under what stimulus conditions.

Bug database exports. Jira, Bugzilla, internal bug trackers — most have CSV or API export. We ingest bug reports and extract the root-cause category (when the team recorded it), the RTL module implicated, and whether the bug was found pre-silicon or post-silicon. The pre/post-silicon distinction matters for weighting: a class of bug that made it to silicon once is weighted higher than a class that's always caught in simulation, because it carries higher respin risk.

Waivers and suppressions from previous lint runs. When an engineer waives a lint warning, they're making an explicit statement about whether that warning class is relevant to their team's design style. A sustained pattern of waiving a particular rule class is a signal to de-weight it. A sustained pattern of acting on a particular rule class is a signal to up-weight it.

The Feature Extraction Step

Raw log files and bug reports aren't directly usable for building a weighting model — they need to be mapped to violation classes that the static analysis tool understands. This is the hard part, and it's where a lot of the engineering work in the learning pipeline lives.

Consider a simulation failure log that says: "assertion fpw_assert_0x3f4 failed at time 1235.4 ns, path top.ddr_phy.dll.lock_sync.q." To extract a signal from this, we need to:

  1. Identify that lock_sync.q is a synchronizer flip-flop output in a CDC crossing from the DLL clock domain to the system clock domain.
  2. Map that CDC crossing pattern to a static analysis rule category (in this case, a two-flop synchronizer with insufficient hold margin).
  3. Record that this module/rule combination produced a real failure in simulation under the given stimulus.

Step 1 requires having the RTL hierarchy available during ingestion, which means the learning pipeline runs against the actual RTL tree, not just the log files in isolation. Step 2 requires a mapping from CDC path characteristics to rule categories, which we built as a classification layer trained on labeled examples from EDA tool documentation and published post-silicon errata reports. Step 3 is bookkeeping.

For bug database entries, the extraction is less structured — we use an NLP classifier to map free-text root-cause descriptions to rule categories. "Metastability at DDR interface" maps to CDC hazard. "Setup violation in output path of multiply-accumulate" maps to timing depth violation. "Simulation X-propagation from uninitialized reset" maps to reset strategy violation. The classifier isn't perfect — it runs at roughly 78% precision on our test set of labeled historical bugs — but at the volumes we're working with, 78% precision on weak signal data is useful when combined with the higher-confidence signals from simulation logs and waiver patterns.

Building the Team-Specific Weighting Model

The output of the ingestion pipeline is a feature vector per rule category, per team: how often has this rule class fired in the past, how often has it predicted a real bug, how often has it been waived, and has any instance of this rule class ever made it to silicon (the highest-weight signal).

These features feed into a gradient-boosted classifier that produces a priority score for each rule category. The priority score isn't the default severity — it's a team-specific adjustment factor. A rule that's high-severity by default but has been consistently waived by this team for two years gets its effective priority reduced. A rule that's medium-severity by default but has predicted three silicon bugs for this team gets elevated to high-priority in their queue.

We're not rewriting rule logic — the underlying detection engine doesn't change. What changes is the ordering and prominence of results. High-priority rules appear at the top of the report with escalated visual weight. Low-priority rules are still reported but appear in the lower section with reduced prominence. Engineers can always see everything; the model just reshapes the queue.

Cold Start and Warm-Up Period

A question we get often: what does the tool do for a team with no historical data — a team starting their first project on Fpgawright, or a new team that hasn't taped out anything yet?

During cold start, the tool falls back to a baseline model trained on aggregated data from published post-silicon errata reports, EDA conference papers on common RTL bug classes, and the Synopsys and Cadence lint rule documentation sets. This baseline model isn't team-specific, but it reflects the empirical distribution of violation classes that have caused real problems across the industry — CDC hazards and reset synchronization issues are weighted high, minor style violations are weighted low.

The warm-up period is roughly 3-6 months of simulation runs and lint sessions for the team-specific model to diverge meaningfully from the baseline. We visualize this as a confidence band in the UI — you can see how much the team-specific weights differ from baseline, which grows over time as the model accumulates evidence.

We're not saying the cold-start model is as good as a mature team-specific model. It's not. But it's meaningfully better than uniform rule weights, and the gap closes predictably as the team generates data.

What the Model Doesn't Learn

There's a class of bugs the learning model can't help with: novel violation patterns that have never appeared in the team's history or in the training data. A new clock domain topology, a new bus protocol, a synthesizable construct that's valid in one tool and produces incorrect optimization in another — these produce violations that have no historical precedent to draw from.

For these, the model produces no elevation signal, which means they appear at baseline priority. This is correct behavior: the model makes no false confidence claims about violation classes it has no data on. The risk is that a genuinely novel violation gets under-prioritized because it looks like low-confidence noise.

Our partial mitigation is the rule category coverage display — we show engineers which rule categories have strong historical backing and which are running purely on baseline weights. If a rule category has no team-specific data and it fires in a novel context, the engineer is at least informed that this is an uncharted area for the model. That transparency is more useful than false confidence in either direction.

Data Isolation

One practical question: does learning from one team's data affect other teams' models? No. Team models are fully isolated — we don't do cross-team learning, and we don't aggregate team-specific data across organizations. The baseline model was trained on public data (published errata, open-source EDA papers, EDA tool documentation), not on any customer data. Each team's historical bug and simulation data is used only to build that team's model, and it stays within their account.

This is a deliberate design choice, not just a privacy requirement. Bug patterns are often architecture-specific and team-specific in ways that would produce noise if cross-pollinated. A team building high-speed PHY interfaces has completely different CDC risk profiles from a team building low-power state machine logic. Cross-team learning would dilute signal, not amplify it.