Reporting dashboard that broke at scale: the data architecture problem behind a slow interface
A reporting dashboard worked fine for small accounts but degraded badly with real production data. The team assumed it was a frontend problem. It wasn't — or not entirely. The slow interface was a symptom of an unoptimised data layer underneath: queries that fetched more than needed, no aggregation for common views, and an API that made several round trips where one would do. We addressed the data layer first, then the rendering. Both mattered.
The Challenge
The dashboard served accounts with tens of thousands of records. Loading a report triggered multiple sequential API calls, each returning full dataset responses that were then filtered client-side. There was no pre-aggregation for the summary views that users opened most often — those were computed fresh on every request. The frontend then tried to render everything at once. The team had added pagination as a workaround, but the underlying queries were still running against the full dataset regardless. Power users — the ones with the largest accounts and the most influence over renewals — had started exporting to spreadsheets instead.
The Solution
We started with the data layer. We profiled the most-used report queries and identified where full-table scans were happening unnecessarily. We introduced pre-aggregated summary tables for the five most common views, updated the API to serve these directly rather than computing on the fly, and consolidated the multi-call sequence into a single request with a structured response. Once the data layer was returning faster and leaner responses, we addressed the rendering: virtual scrolling for the large tables, deferred chart initialisation for sections below the fold. The changes were incremental — each step was independently deployable and measurable.
“We spent two sprints trying to fix it on the frontend and saw marginal improvement. When the data layer was fixed, the difference was immediate. We should have profiled the queries first.”