Buoyancy Bandits