SpiderGate now scores 221 of its 1,216 catalogued AI models by capability

SpiderGate has opened capability scoring to every account, so teams can rank its model catalog by what each model is good at rather than reading benchmark tables by hand. 221 of the 1,216 models in the catalog now carry a score across nine categories, refreshed daily by an automated enrichment agent.

"Most model comparisons quietly average numbers that were never on the same scale," said Martin Shein, Founder of SpiderIQ. "We would rather publish a percentile against the models we can actually measure, and mark everything else not scored yet, than hand someone a confident number we cannot defend."

The nine categories are Coding, Design/Frontend, Reasoning, Math, Vision, Research, Tool Use, Chat and Translation. A model's score in a category is a within-category percentile rather than an average of raw benchmark results, because scores from different benchmarks are not on a common scale. Behind them sit 454 benchmark-to-category signals drawn from 4,821 published benchmark results, sourced from LLM-Stats and DesignArena. Opening any score shows the contributing benchmarks, with vendor-reported figures flagged as vendor-reported.

The catalog is maintained by an enrichment agent that runs every day. It pulls model specifications and pricing from OpenRouter's public API, benchmark results from LLM-Stats, and model lineage from Wikidata. Every field it writes carries a provenance stamp naming its source and the time it was fetched, and the agent never overwrites a record a human has curated.

Coverage is stated openly rather than rounded up. 221 of 1,216 models, about 18 percent, carry scores today; the remainder are marked not scored yet rather than ranked last. A model-evaluation layer that aggregates automated runs and human ratings is live but just getting started, and currently holds very little data.

Shein added, "Publishing the gaps is the part that earns trust. Anyone can rank the models they happen to have numbers for."

The same rankings are available to AI agents through the gateway's public leaderboard endpoint, so a model chosen in code matches the one a person would choose in the dashboard.