The fugitives, including chief suspect Jamal al-Badawi, had been jailed in the port city of Aden since shortly after the destroyer was bombed, killing 17 American sailors.
Officials at Aden's governor's office would not say how the men escaped early Friday. But they quoted intelligence sources as saying security forces were out in force in a major search operation.
Photographs of the men were distributed to police and houses of the escaped men's relatives were searched, the officials said on condition of anonymity.
The Oct. 12, 2000, attack was blamed on Osama bin Laden's al Qaeda network.
Al-Badawi allegedly helped buy the dinghy used by the two suicide bombers, who rammed the destroyer as it was refueling in Aden.
The 10 men, some of whom are believed to be linked to al Qaeda, were part of a 17-man group arrested after the Cole bombing.
Officials said that the men might have left Aden and headed to al-Qaida strongholds in the northern province of Shabwah.
Last July, Walid Abdullah Habib, a Yemeni member of al-Qaida who was arrested while trying to enter the country illegally, escaped from prison.
Habib was arrested this year in a desert area near the Oman-Yemen border and handed over to Yemeni authorities. Habib is from Shabwah.
Yemen, the ancestral home of bin Laden, has been a hotbed of terrorist activity.
Supporters of al Qaeda have claimed responsibility for several bombings targeting security officials and government offices in the past few months.
Yemen committed itself to joining the war on terrorism following the Sept. 11 attacks in America and has allowed U.S. forces to enter the country and train its military.