Caramoan was founded in 1619 by a 60-year old zealous Spaniard missionary, Fray Francisco de la Cruz y Oropesa, who penetrated the thick virgin forest of the Caramoan Peninsula and discovered a small settlement in a place called Balwarte (an earth-work, rampart), In 1696, the said settlement was turned over to the administration of Sagrada Mitra. A parish dedicated to St. Michael the Archangel was also built.

In the summer of 1818, the Moros had once again sacked and burned the towns of Sagay and Indan, On October 26 of the same year, the provincial armadillo of Albay, led by 80-year old Pedro Estevan defeated a large Muslim fleet led by the son of the Sultan of Maguindanao, Prince Nune, in Tabgon Bay, off the Caramoan Peninsula. After this loss, the Muslims were not able to successfully attack a Bicol peninsular town again.