"No life should be lost in my name and also the ANC should never be divided in my name," he said during a nationally televised political address Wednesday. "I have therefore come to the decision to resign as President of the republic with immediate effect."
After his resignation announcement, Zuma said he disagreed with the decision of his political party and that he has always been a "disciplined member of the ANC."
"As I leave I will continue to serve the people of South Africa as well as the ANC, the organization I have served all of my life in," Zuma said.
Zuma, 75, survived multiple attempts by opposition parties to oust him during his more than eight years in power, earning him the nickname of the "Teflon president."
The leader of the South African opposition party, Mmusi Maimane, said it was about time Zuma resigned and that the former President should see his day in court on corruption charges.
Speaking to CNN immediately after Zuma's resignation, Maimane said the Democratic Alliance will put forth its own candidate in the national assembly on Thursday and if needed will contest the election of acting President Cyril Ramaphosa.
"It's about time that some form of accountability is afflicted on Jacob Zuma and I think that he has been a one-man wrecking ball to our economy, to our country," Maimane said.
Last year, South Africa's Constitutional Court ordered Zuma to repay millions of dollars in public funds spent on refurbishing his private homestead.
He also faces more than 783 allegations of corruption relating to a 1990s arms deal. Zuma denies all the corruption allegations against him.
The ANC had been trying to push Zuma out for months. It dumped him as party president in December, narrowly electing Ramaphosa, a 65-year-old millionaire former union leader, over Zuma's preferred successor, his ex-wife and former cabinet minister, Nkosazana Dlamini-Zuma.
Ramaphosa said Sunday that the uncertainty over Zuma's future had damaged the ANC.
But despite signs that the party was losing electoral support over the protracted drama, internal divisions forced Ramaphosa to tread cautiously. For years, the ANC -- which led South Africa out of apartheid -- put party unity first, even as it became apparent that Zuma was an electoral liability.
The ANC said it wanted to "salute the outstanding contribution he has made and express its profound gratitude to him for the role he has played in the African National Congress spanning over 60 years of loyal service."
It acknowledged mistakes were made but praised his "legacy of delivery."
The party said it was accepting his resignation and expects each ANC member to cast a vote for Ramaphosa as its candidate for president.
Ramaphosa was a close ally of late President Nelson Mandela and widely predicted to be his heir. However he lost out to Thabo Mbeki, an anti-apartheid leader who returned to South Africa from exile.
Ramaphosa left government soon afterward, turning instead to business. Since returning to public life he has spoken out against corruption in the country and has found significant support in urban areas, in the business community and among ANC stalwarts.
His election as leader in December had raised hopes that he'd be able to restore voters' faith in the party which had been eroded under Zuma's reign.
But Maimane of the Democratic Alliance said his party thinks further change is necessary.
"We don't believe Cyril Ramaphosa is the solution for South Africa," he said. The Democratic Alliance wants Parliament to be dissolved so the people of South Africa can issue a fresh mandate. "We think that the ANC has been complicit in defending Jacob Zuma and without doubt we will put forward a candidate."