lighthouses-bot

My bot won almost every game. The problem was who it played against.

For a contest I built a Rust bot for a territory-control game, and put it through eight generations. The ride taught me something good about complexity: I tried the flashy stuff —MCTS-style tree search, reinforcement learning, Thompson Sampling to pick a strategy— and almost none of it survived. The explicit heuristic won, the one that decides in under a millisecond. The clearest moment was a version I rewrote to be more sophisticated, with more layers and more nuance: it won 34% of its games, and the previous, simpler one won 53%. I reverted the change and kept the lesson that complexity adds failure modes faster than it adds value. The other lesson from back then wasn’t about AI but about reading: the rulebook said one thing and the engine did another, and several edges came from reading the code instead of the docs.

Then I measured it against real bots, from other competitors, pulled off GitHub. Fourth out of five, and the one in front scored four times as much. The eight generations had been competing against each other the whole time, and I had mistaken that ranking for a measure of quality.

So I threw the bot away and started another from scratch in Python, this time against the real field from day one. The next day it was already first out of seven, ahead of the ones that had beaten me. A couple of days later I froze the version and called it done. I didn’t hold out for long: two days after that I added one more tweak, worth a 6.8% gain, with the statistical significance to back it.

A week later the organiser’s package showed up with the tournament’s real conditions. It ran for 1,200 rounds and up to sixteen players; I had measured everything with three hundred rounds and seven. The map I used as a reference I had rebuilt by eye. And that 6.8% tweak, the last thing I had added, turned out to do 24% of damage on one of the official maps. I stopped applying it everywhere and moved it to a per-map table: on only where it was measured not to hurt, off on any map it didn’t know.

I came fourth. The lesson I take isn’t about bots: the scope of what you measure is the scope of what you can conclude, and not one decimal more. I had hundreds of games backing me up and still wasn’t measuring what I thought, twice in a row and for different reasons.

---

How it’s built — Rust and Python · explicit heuristics · no trained models