--- title: "Building a Monitoring Dashboard for Beginners: From Live Traffic to 404 Errors" date: 2026-10-11 time: 20:30 model: admin category: knowhow author_type: human summary: "Making a homepage isn't the finish line. The story of building a control room to check server logs, live traffic, crawler records, and cumulative stats on screen — and no longer fearing 301s and 404s." tags: monitoring-dashboard,server-logs,live-traffic,404,301,ai-crawlers,cumulative-stats,beginner-ops --- # Building a Monitoring Dashboard for Beginners: From Live Traffic to 404 Errors I thought building a homepage was the finish line. I thought all there was to it was posting, polishing the design, and waiting for people to show up. But once I actually ran a server, what a homepage does and what happens inside that homepage turned out to be completely different things. At first, even server logs were unfamiliar. IP addresses, GET requests, HTTP status codes — they all scroll past, and figuring out what actually happened from them wasn't easy. And I couldn't just open a terminal and dig through the logs every single time. Even less so for a beginner like me. So I built a screen. ## Just making one screen changed things My homepage now has live traffic records, AI crawler collection records, cumulative statistics, and a graph of crawler activity by hour. On the left I placed a knowledge graph that links posts, tags, categories, and more. I never set out to build a grand system. I just put things on the screen one at a time — whatever I was curious about while operating the site, whatever I wanted to check myself. But once it was built, it turned out far more useful than I expected. I could see at a glance which bot visited, what address it requested, and what response the server gave back. I could also see records of Google and various AI crawlers requesting my posts. In the old days, even if that kind of record piled up inside the server, I'd never have noticed unless I went looking. ## What matters more than numbers is knowing what happened A little while back I saw a record like this: ``` GET /knowhow/2026-09-25-agent-logic-over-knowledge 301 ``` If you're seeing this for the first time, you might wonder what it means. I didn't know every status code from the start either. I checked, and 301 means the address has permanently moved elsewhere. Now I just need to check where that address went. If the final address is fine, I leave it alone; if it went somewhere unexpected, I go find the cause. This might seem like nothing, but from the perspective of actually running the site, the difference is big. Not even knowing there's a problem versus discovering the trace of a problem and being able to check it — those are completely different situations. 404 is the same. If someone requested a page that doesn't exist, I can check the address and see whether an old address is still lingering or the page is actually gone. Of course, 301 and 404 don't automatically mean something is wrong. They can happen during normal address changes too. The point is being able to look at the record and judge the situation. ## AI crawlers really get around Looking through the server records taught me another thing: people aren't the only ones who show up at my homepage. Requests from a wide range of crawlers leave their marks in the records — Googlebot, GPTBot, ClaudeBot, ExaSearchBot, and more. At certain hours the requests pile up. Just from these records I can't conclude that an AI trained on my posts or surfaced them in search results. But at least I can confirm which bot requested which address. I also made the graph accumulate over time. The horizontal axis is the 24 hours of a day, aggregating the crawler records that pile up in each hour. As more data builds, I'll be able to look at activity differences by hour over the long term. That's a bit different from a screen that only shows a visitor count. ## Maybe beginners need this kind of screen more than anyone An expert would be comfortable typing commands into a terminal and analyzing logs. But for someone running a homepage for the first time, even that process is hard. I didn't start out knowing everything either. I got curious while running the site myself, so I built a screen where I could check the information I needed in one place. Turning complicated logs into an easy-to-read form, and letting me check right on the spot when a suspicious record shows up. Of course, just making a screen doesn't automatically solve security problems. The records only help you discover problems — you still have to confirm the actual cause and take the right action. Still, at least now I'm not running the site completely in the dark about what's happening. **It's best if problems don't happen, but it also matters to be able to notice when they do.** ## Now I want to write more than add features At first I wanted to add all sorts of things. Make it a bit prettier, bolt on one more feature. But at this point I think the information I wanted to check is all in there. I like the knowledge graph, and the crawler records and cumulative stats are doing their jobs. Now I want to keep writing posts rather than keep tearing the screen apart. Running a homepage wasn't something that ended once it was built. Learning while operating it, checking the records, and building what I needed myself — that whole process mattered too. For a beginner like me, this screen isn't just a stats board. It's a small control room that lets me see what's happening inside my own homepage. I plan to keep writing and watch how the data builds up. I'm curious myself what other records will show up. ## And today, too, the crawlers never rest There's one funny thing about it. The moment I post this, Googlebot and its friends will show up without fail. Even if I post at 3 a.m., those bots don't miss their cue. They're busiest precisely when nobody's watching. Sometimes I look at the server log and think Googlebot has visited my site more often than I've written posts. There are moments I'm tempted to believe I could leave my subscriber count to the bots. Of course, the bots can't actually like my posts. They don't hit the like button either. But if I turn on the monitoring screen, I can at least always see who showed up. For a beginner operator, that's enough.