We do not know if David went to Jerusalem with his entire army, including men from the tribes of Israel, or only the men who had followed him while he was in exile. The response of the Jebusites to his approach suggests that David went with a smaller number.
6 And the king and his men went to Jerusalem against the Jebusites, the inhabitants of the land, who said to David, “You will not come in here, but the blind and the lame will ward you off” thinking, “David cannot come in here.”