"Today is a day that the history books will look back on as a good day for New York City," Council Speaker Corey Johnson said. "This is a step forward, this is progress, this is the right thing to do."
The jail is also notorious for its violence and poor conditions. The US attorney's office for the Southern District of New York reached a settlement with the city in 2015 after a multi-year investigation found adolescent inmates were not protected from "the rampant use of unnecessary and excessive force by New York City Department of Correction staff and violence inflicted by other inmates."
The balcony in the city council chamber was cleared prior to the vote after protestors chanted, "No new jails."
The majority leader claimed that something had been thrown from the balcony, noting "throwing things from the balcony compromises the safety of every member of this body."
A protestor responded "your safety is not in jeopardy, the safety of young black and brown people is in jeopardy."
"The era of mass incarceration is over. It's over," Mayor Bill de Blasio said in a press conference after the vote.
"This is about valuing our people, no longer condemning people and sending them on a pathway that only made their lives worse and worse."
According to de Blasio, New York's jail population has declined by nearly half in the last six years. The jail population has declined from 11,000 in 2014 to about 7,000 today, and is projected to be approximately 3,300 by 2026, de Blasio added in a press release.
CNN's Evan Simko-Bednarski contributed to this report.