About

Heldout Labs is an independent lab that tests AI models, agents and developer tools and publishes the results with full methodology and raw data. We are not paid by the companies we evaluate.

symbatix

Engineer working on AI infrastructure. Runs Heldout Labs and writes under a pen name.

"symbatix" is a pen name. The work is meant to be judged on its methodology and data, which are always published.

Elsewhere: Hacker News · GitHub · Reddit

Disclosures

The author works in AI infrastructure. When a test touches a product category with a conflict of interest, the page says so explicitly.

Contact

Corrections, data requests and press: [email protected]