A Day at the Bot Buffet — The AI Guests Kept Showing Up

We had a group booking at the homepage today.

Lately these big parties keep turning up and I honestly don't know what to do with myself. lol

If only they were actual paying customers...

I opened the server logs today and saw 4,422 requests for the day. Excluding duplicates, that's 891 unique URLs. Crawlers accounted for 71.6% of total traffic.

I mean, I know the place is free and all, but did word really get around that it's a great free buffet? Why does everyone keep showing up?

They never buy anything. They just swarm in and browse.

Like stopping by a supermarket for the free samples and then leaving, I guess. They never, ever buy. lol

These guests are something else.

The moment the doors open, they pile in. They scan every shelf, poke through every corner, and rummage through the back room. And then they buy nothing.

They look around to their heart's content and vanish without a word. hahaha

Of course, this isn't really a story about a shop that sells products. I'm comparing my website's posts and data to merchandise.

The guests don't just read posts and leave. They poke through the API, request graph data, and hunt down images and JavaScript files.

A human guest would read a few posts and go. These guests inspect every corner of the place.

Even with thousands of requests coming in, the server load never visibly spiked. Total transferred was around 102.4MB.

My site is centered on text and JSON data. It's structurally different from a place packed with heavy images. Nginx handles the static files, and rate limiting is active, so it's held up fine so far.

That said, just because it held up today doesn't mean it'll shrug off whatever traffic comes next. If search or database work spikes, things could get interesting.

But where's the line on free sampling?

This is where it gets a little more serious.

The site operator writes posts, organizes information, and pays for the server. The AI gathers that information and builds answers for its users.

So the question becomes:

If AI takes my content and puts it to use, what comes back to the person who made it?

There's no easy answer right now. Blanket-blocking crawlers probably isn't the answer, and neither is handing over everything with no limits.

For now, I've decided to watch the logs, protect the server, and keep a record of who shows up and how often.

The bot guests paid another visit today.

They didn't buy anything. They just swarmed in, browsed the shelves, and left.

Still haven't settled the bill. hahaha

Wonder who'll show up next?