Perception · Lesson p2
Drive at what you can see
This is the written walk-through of the lesson: the idea, stage by stage, plus the deeper asides. The interactive version, where you write real Python in the browser and drive a simulated robot, lives on the lesson page.
A picture with two axes, and a blank one
Same camera, more of it. Instead of one row of floor you now get a grid: 20 cells across, 8 rows deep, reaching about 100px ahead of you and 90px either side. It arrives as one flat tuple in row major order, which is the convention nearly every image format in the world uses: the first 20 values are the nearest row, the next 20 are the row behind that, and so on. Cell r * n + i is column i of row r.
There is one crate out there and no flag to steer at. Press Run and watch the patch: it is completely blank. The crate is real, but it is further away than the camera can reach, and a blank picture is not a malfunction. It is the honest answer to the question you asked. Your find_line handled that case by returning 0.0, which quietly meant “dead ahead”. Here that lie would be expensive, so find_blob returns None instead, and the caller creeps forward until there is something to see.
Average the bright cells into a place
Finding the crate is the same weighted average you already wrote, run twice over the same cells. Walk every cell, skip the dark ones, and accumulate two running totals instead of one: the sideways position and the distance ahead, each weighted by that cell’s brightness. Divide both by the total weight at the end.
The two coordinates come from the cell’s address. Row r sits 2 + r * spacing px ahead of you. Column i sits at half_width - 2 * half_width * i / (n - 1) px to the side, the identical formula from the last lesson, and it keeps the identical sign: positive means the crate is off to your left.
Get the sideways number right and the wiring below does the rest: it steers proportionally toward it, exactly like following a line. When the robot turns to face something it could not see a moment ago, that is the whole lesson in one move.
Use the second coordinate to stop
The second coordinate is what keeps you from driving through the thing you were looking for. seek_crate takes the distance to your detection and stops once it drops under stop_at. If your ahead value is wrong, or you never computed it, the robot either parks far too early or arrives at speed and bounces off.
Watch the patch as you close in. The nearest row is the top one, so the crate first appears along the bottom edge, then grows and climbs upward row by row as the gap closes. Drive too far and it slides off the top, because the camera cannot see the floor underneath the robot. Your detection is only ever of the part still in frame.
Go deeper: Nothing in view is a measurement too
It is tempting to make a detector always return something. An answer is easier to work with than a maybe, and every caller gets simpler if it can assume one. But a detector that invents a reading when it sees nothing has stopped being a sensor and started being a guess, and the code downstream has no way to tell the two apart.
This is why real perception stacks carry a confidence alongside every detection, and why the interesting engineering is usually in what happens when confidence is low. Here the policy is the simplest one that works: no detection means creep forward and keep looking. A self-driving stack facing the same blank frame has to decide between coasting, braking, and handing back control, and it has to decide in the time it takes to read one more frame.
Notice too that your answer is biased while the crate is still arriving. The patch reaches only as far as its last row, so a crate beyond that is cut off at the far edge: you average the near slice of it and nothing else, and the centroid you get reads short. The bias fades as the crate closes, and it is gone once the whole crate sits inside the patch, well before the robot parks. Every real camera has this problem at the edge of its frame, and the usual fix is not better arithmetic, it is knowing the bias is there.
Ready to build it? The interactive lesson is where you write the code and watch the robot run.