No, just need one GBA (and the adapter to connect to the Gamecube). The other 1-3 people are on the tv with regular controllers. The person on the GBA is Pac-Man. The people on the tv are the ghosts. Pac-Man can see the whole playing board on the GBA and looks like he's playing just like a normal game. But the ghosts can only see a small part of the board where they are. Pac-Man does leave a fading trail behind him so the ghosts have some hint of where he is.

Does seem like a pretty fun party game.

I've got a GBA and a cable (got the cable so I could do the extra stuff in Zelda: Windwaker). I'll bring it along next gameday and we can play some PacMan.