The men were among eight crew members aboard a vessel when it was boarded by the notorious kidnap-for-ransom militants -- who have distant links to the Islamic State group -- off Borneo in January 2020.
They were held in Sulu province in the southern Philippines -- the group's stronghold -- and during the ordeal, three were freed and one died.
On Thursday, three of the men and one of their captors were found on a beach in the nearby province of Tawi Tawi after their motorboat capsized in bad weather, police Lieutenant General Guillermo Eleazar said.
A fourth fisherman is missing while four militants travelling in a separate boat "were lost due to strong winds and big waves", Eleazar said.
A search operation was underway as were efforts to identify the victims and locate their families, he added.
Abu Sayyaf is a group of self-proclaimed militants based in the southern Philippines who have engaged in bombings as well as kidnappings of Western tourists and missionaries for ransom since the early 1990s.
In recent years, the group has also preyed on cargo ships, tugboats and fishing vessels in poorly policed waters of the region near the sea borders with Malaysia and Indonesia.