I build and maintain RunLog: a free toolkit for comparing AI models, estimating costs and checking agent behavior. It brings together working software, measured experiments and the decisions behind them.
JavaScript & TypeScript · Python · AI evaluation · Testing & deployment
Prompt Pricer turns token and output assumptions into a cost estimate. I use the same JavaScript estimator for the browser and offline CLI to keep the calculations aligned. Shared settings preserve numeric assumptions without including prompt text.
JavaScript estimator · CLI implementation · Calculation regressions
Loop-Guard checks repeated tool requests, cycles and returned usage inside an application. The integration tests exercise real pinned SDKs through controlled transports. Unknown usage is not treated as free usage, and post-response budget checks are not advertised as hard spending caps.
Python core · TypeScript API · SDK integration tests · Supported modes
Ship-Check evaluates recorded tool traces against step, cost and tool-use assertions. Its offline Python example includes both passing and failing runs. The CI example keeps local checks separate from optional signed verification; neither replaces a trustworthy test harness.
The public benchmark combines nine practical work samples into quality, price and speed measurements. Sanitized task descriptions explain what is tested; exact prompts, fixtures, reference answers, model responses and project attribution remain private. Automated ratings can be wrong and do not establish broad capability. Output allowances varied where more room was needed to finish, and code was judged as text rather than executed inside engines.
The site runs on Cloudflare Workers with D1-backed benchmark publication. The public monitoring implementation combines external HTTP probes, persisted incident state and a missing-heartbeat watchdog. Tests cover failure and recovery behavior. Monitoring needs maintenance and an operator; it is not an uptime guarantee.
Monitoring architecture · Worker implementation · Incident and recovery tests
I use AI to assist with implementation and review. I check the resulting work through deterministic tests, browser journeys, SDK compatibility checks and measured experiments. I take responsibility for choosing the constraints, inspecting failures and maintaining what is released.
Cost estimates, measured provider usage and quality judgments are different kinds of evidence. I keep those distinctions visible instead of treating a fluent answer or a passing test as proof of everything.
The public toolkit is free and needs no account. The local tools have public MIT-licensed source, versioned downloads and checksums; the hosted site's full implementation is not published in those repositories. Try the installation examples.
Earlier account experiments are preserved separately for existing users. Accounts, Studio, private workspaces and payments remain deferred, with no launch date or paid plan promised. Running local tools with model providers may incur your own provider charges.
RunLog Systems operates the service. For a role, project discussion, feedback or a bug report, email hello@runlog.cc. Any custom work is agreed separately from the free toolkit.
