{"version":"1","article":{"id":"qtestbench-test-framework-llm-quantum-code","slug":"qtestbench-test-framework-llm-quantum-code","language":"en","url":"https://haberpaper.com/en/news/qtestbench-test-framework-llm-quantum-code","api_url":"https://haberpaper.com/api/v1/articles/qtestbench-test-framework-llm-quantum-code?lang=en","title":"QTestBench measures AI-written quantum-code tests against independent checkers","summary":"Researchers affiliated with NYU Abu Dhabi present a framework for testing whether model-written quantum software tests distinguish correct from faulty solutions.","category":"business","region":"world","topics":["ai"],"published_at":"2026-10-13T20:11:52+03:00","modified_at":"2026-10-13T20:11:52+03:00","publication_basis":"publication-record","event_period":"2026-10-09","event_period_basis":"event-or-reference-period","sources":[{"name":"arXiv","url":"https://arxiv.org/html/2610.13811v1","publishedAt":"2026-10-09","checkedAt":"2026-10-13"}],"translations":[{"language":"tr","url":"https://haberpaper.com/tr/news/qtestbench-test-framework-llm-quantum-code","api_url":"https://haberpaper.com/api/v1/articles/qtestbench-test-framework-llm-quantum-code?lang=tr"},{"language":"en","url":"https://haberpaper.com/en/news/qtestbench-test-framework-llm-quantum-code","api_url":"https://haberpaper.com/api/v1/articles/qtestbench-test-framework-llm-quantum-code?lang=en"},{"language":"ar","url":"https://haberpaper.com/ar/news/qtestbench-test-framework-llm-quantum-code","api_url":"https://haberpaper.com/api/v1/articles/qtestbench-test-framework-llm-quantum-code?lang=ar"},{"language":"fr","url":"https://haberpaper.com/fr/news/qtestbench-test-framework-llm-quantum-code","api_url":"https://haberpaper.com/api/v1/articles/qtestbench-test-framework-llm-quantum-code?lang=fr"},{"language":"es","url":"https://haberpaper.com/es/news/qtestbench-test-framework-llm-quantum-code","api_url":"https://haberpaper.com/api/v1/articles/qtestbench-test-framework-llm-quantum-code?lang=es"}],"data_types":[],"data":null,"data_compilations":[],"data_downloads":[],"content":{"paragraphs":["The 9 October arXiv preprint QTestBench tests the tests written by large language models. A test file merely running is not treated as sufficient: the framework also measures whether generated tests accept correct solutions and catch faulty ones, using independent benchmark checkers as the reference. The paper’s first page lists the authors’ affiliations with NYU Abu Dhabi’s eBRAIN Lab and its Center for Quantum and Topological Systems. The work is presented as a preprint, not as a peer-reviewed publication.","The authors evaluate five models on eight tasks from PennyLane QHack and a fixed set of ten Qiskit HumanEval tasks. They report that the best test generator reached a test-quality score of 0.71 on QHack and 0.34 on Qiskit HumanEval; these scores apply to the paper’s tasks and scoring protocol, rather than representing general quantum-programming success rates. The study also uses checker-verified synthetic changes for three Qiskit tasks where naturally occurring faulty solutions were absent. Its contribution is an evaluation method showing why executable generated tests alone do not establish that those tests are trustworthy."],"key_points":[],"sections":[]},"text":"Researchers affiliated with NYU Abu Dhabi present a framework for testing whether model-written quantum software tests distinguish correct from faulty solutions.\n\nThe 9 October arXiv preprint QTestBench tests the tests written by large language models. A test file merely running is not treated as sufficient: the framework also measures whether generated tests accept correct solutions and catch faulty ones, using independent benchmark checkers as the reference. The paper’s first page lists the authors’ affiliations with NYU Abu Dhabi’s eBRAIN Lab and its Center for Quantum and Topological Systems. The work is presented as a preprint, not as a peer-reviewed publication.\n\nThe authors evaluate five models on eight tasks from PennyLane QHack and a fixed set of ten Qiskit HumanEval tasks. They report that the best test generator reached a test-quality score of 0.71 on QHack and 0.34 on Qiskit HumanEval; these scores apply to the paper’s tasks and scoring protocol, rather than representing general quantum-programming success rates. The study also uses checker-verified synthetic changes for three Qiskit tasks where naturally occurring faulty solutions were absent. Its contribution is an evaluation method showing why executable generated tests alone do not establish that those tests are trustworthy.","corrections":[],"editorial_illustration":{"kind":"symbolic-editorial-illustration","documentary":false,"scope":"section","asset_id":"business","url":"https://haberpaper.com/assets/illustrations/business-1536.webp","width":1536,"height":1024,"alt":"Symbolic industrial workshop with machinery, a warehouse and a crane.","language":"en"},"section_illustration":{"kind":"symbolic-editorial-illustration","documentary":false,"scope":"section","asset_id":"business","url":"https://haberpaper.com/assets/illustrations/business-1536.webp","width":1536,"height":1024,"alt":"Symbolic industrial workshop with machinery, a warehouse and a crane.","language":"en"}}}