[paper] NetlistBench: SPICE Netlist Evaluation LLM Reliability

Ma, Jiarui, Jianghan Wang, Yuheng Ma, Ziyi Zhuang, and Xiaoguang Liu

"NetlistBench: Evaluating LLM Reliability in SPICE Netlist Recognition and Manipulation" 

In Proceedings of ACM, New York, NY, 8 pages

arXiv: 2608.12197 (2026)

* Southern University of Science and Technology, School of Microelectronics Shenzhen (CN)

Abstract: Large Language Models (LLMs) are increasingly used in circuit design workflows, yet their reliability on simulator-facing SPICE netlist recognition and manipulation remains poorly understood and is rarely separated from high-level design reasoning. Although netlists are textual, they encode structured circuit objects through topology and parameters. We present NetlistBench, a structure verified benchmark for SPICE netlist recognition and manipulation. NetlistBench contains 2,342 cases across 24 task families, covering parameter and connectivity recognition and edits, hierarchical operations, equivalence judgment, and long-horizon compound editing. Model outputs are evaluated by a deterministic structure aware oracle. Across six non-thinking LLMs, performance varies substantially with operation-level structural complexity. Simple local edits reach 96%–100% accuracy, while device addition drops to 41%–83% and equivalence judgment to 49%–90%. Enabling reasoning substantially improves weaker models but does not eliminate structure-preservation failures, with performance still degrading sharply as the edit horizon increases. NetlistBench identifies netlist reliability as a distinct bottleneck for trustworthy LLM-based circuit design automation.

Fig: SPICE netlists as a common representation layer in LLM-based circuit design workflows.

[paper] From RTL to Prompt Coding Chip Design

Lukas Krupp∗, Matthew Venn† and Norbert Wehn∗

From RTL to Prompt Coding: Empowering the Next Generation of Chip Designers through LLMs

arXiv:2601.13815v1 [cs.AR] 20 Jan 2026

∗RPTU University of Kaiserslautern-Landau, Kaiserslautern, Germany

†Tiny Tapeout

Abstract: This paper presents an LLM-based learning platform for chip design education, aiming to make chip design accessible to beginners without overwhelming them with technical complexity. It represents the first educational platform that assists learners holistically across both frontend and backend design. The proposed approach integrates an LLM-based chat agent into a browser-based workflow built upon the Tiny Tapeout ecosystem. The workflow guides users from an initial design idea through RTL code generation to a tapeout-ready chip. To evaluate the concept, a case study was conducted with 18 high-school students. Within a 90-minute session they developed eight functional VGA chip designs in a 130 nm technology. Despite having no prior experience in chip design, all groups successfully implemented tapeout-ready projects. The results demonstrate the feasibility and educational impact of LLM-assisted chip design, highlighting its potential to attract and inspire early learners and significantly broaden the target audience for the field.

Fig: Overview of the proposed idea-to-GDSII learning workflow integrating the LLM-based chat agent for the RTL implementation, VGA simulation tool, and GitHub-driven backend flow.

Acknowledgments: This paper was funded by the German Federal Ministry of Research, Technology and Space (BMFTR) as part of the “Chipdesign Germany” project under grant number 16ME0890.