"There's nothing magnetic in flash memory, so [a magnet] won't do anything," says Bill Frank, executive director of the CompactFlash Association. "A magnet powerful enough to disturb the electrons in flash would be powerful enough to suck the iron out of your blood cells," says Frank.
To have any change of damaging a flash drive you would need a rapidly changing magnetic field that could induce a significant current. A static magnetic field wouldn't do anything unless it were powerful enough to actually pull any iron components of the flash drive enough to damage it.
Also, I may be wrong, but I think the only significant ferromagnetic component in most flash drives is the steel sheath around the USB contacts. Some might have steel cases or other non-electronic parts. A few low profile ones don't even have the sheath or have a plastic one. Most of the functional conductors in the drive are copper, which is very weakly repelled by magnets.