All of Chile's 33 active bishops offered their resignations last month in an extraordinary act as part of the fallout from the scandal.
Francis appointed 61-year-old Barros as bishop of Osorno in 2015, when victims' allegations against him were already widely known.
"Cardinal O'Malley (who delivered the letter) called me after the Pope's visit here in Philadelphia and he told me, among other things, that he had given the letter to the Pope - in his hands", Juan Carlos Cruz said.
Despite a strong Catholic tradition, Chile is witnessing a growing rift between the people and the Church, sharpened by the string of sexual abuse scandals.
Victims accused Barros of knowing about what they said was Karadima's abusive behavior but doing nothing to stop it or report it to church authorities.
Besides Barros, the pope also agreed to the departures of Cristián Caro Cordero, bishop of Puerto Montt, and Gonzalo Duarte García de Cortázar, bishop of Valparaiso.
Pope Francis remains in the spotlight, following recent comments about climate change with pleads to oil executives that call for clean fuel.
"The band of delinquent bishops ... begins to disintegrate today".
They questioned Barros' suitability to lead given he had been a top lieutenant of Chile's most notorious predator priest and had been accused by victims of witnessing and ignoring their abuse by that priest.
The Vatican in 2011 sentenced Karadima, a powerful preacher close to Chile's elite, to a lifetime of penance and prayer for his sex crimes.
During a trip to Chile in January, the pope staunchly defended Barros, saying he believed he was innocent, and that accusations against him were "slander" until proven otherwise.
Barros, who is a central figure in the controversy, has been accused of covering up abuse committed by another priest in the 80s and 90s.
An global family rally the Catholic Church is hosting in Ireland will feature workshops on hot-button issues facing Catholic families, including protecting children from clergy sexual abuse, weathering divorce and ministering to lesbian and gay faithful.
He also chose to host three Chilean sex abuse survivors at his home in the Vatican so he could apologise to them personally and hear their recommendations for change.
He has since received two groups of Karadima's victims at the Vatican.
After receiving their 2,300-page report detailing the extent of clergy sexual abuse in Chile, the Pope apologised. Francis said he had misjudged Juan Barros and the events in Chile because he hadn't been given "truthful and balanced information".