Members of the POA, the trade union for prison staff, were told to return to work by 1pm following “meaningful engagement” with prisons minister Rory Stewart.
Mr Stewart “recognised that our concerns are justified and need addressing” following Friday’s protest, General secretary Steve Gillan said.
He said he was “confident a deal is a deal” after the prison service “backed down” over seeking an injunction against the demonstrators.
They have been demonstrating outside prisons in England and Wales from 7am over “unprecedented” levels of violence and safety concerns.
But Justice Secretary David Gauke branded Friday’s action “wrong” and “irresponsible”, adding that it “does nothing” to help reduce levels of violence.
He told reporters: “I agree with those who say that the level of violence is unacceptably high and we are determined to bring it down.
“But I think action of this sort does nothing to help that process, and locking prisoners up for 24 hours a day, which may be the consequence of what the POA are doing, only increases the risk of violence.
“It doesn’t help us address it.”
The action had knock-on effects on court cases, with some defendants in custody unable to be transported to hearings.
The union will hold talks with the prison service on Monday, Mr Gillan said.
He told the Press Association the Justice Secretary risked “inflaming” the situation after an agreement had been reached.
“The protest can’t have made things worse because his minister has recognised that our concerns are justified and need addressing. That’s why we called the protests off,” he said.
“And so while I understand the secretary of state will always say ‘no-one should ever protest, we should rely on negotiation and consultation’, unfortunately when nobody’s listening to you sometimes you’ve got to demonstrate that you don’t think it’s right or proper that 25 officers every day are being assaulted when they go to work.”
He added: “It couldn’t get any worse than it already was and what we now need is positive action to improve the safety of prisons.”
Thousands of prison staff took part in the demonstrations, the POA said, which Mr Stewart called “unlawful” earlier on Friday.
Mr Stewart said after the protests ended: “I am pleased that all parties have been able to bring a swift resolution to this action which, as I have made clear, was irresponsible and placed fellow staff and prisons at risk.
“The priority now must be to continue our constructive dialogue with the safety of our hard-working prison officers at its absolute heart. Ultimately our aims are the same – to see safe, secure and decent establishments that provide a positive environment for staff and prisoners.
“I have demonstrated my absolute commitment to bringing about that improvement but it will only happen if all sides work together.”
The walk-out was triggered by a damning report which warned of a “dangerous lack of control” at HMP Bedford, the union said.
Around 50 officers were outside the prison on Friday, with members recalling how one colleague’s arm was broken with a pool cue while another had his head stamped on.
Richard Gilbert, who has been an officer for 14 years at the facility, said he was suffering with PTSD and depression after a group of inmates repeatedly kicked him in the head.
On Thursday, Chief Inspector of Prisons Peter Clarke raised the alarm over the potential for a “complete breakdown” in order and discipline at HMP Bedford.
It was the fourth urgent notification the Government has issued since the scheme was introduced less than a year ago.
Standards across the prison estate have come under intense scrutiny in recent years amid a slew of highly critical reports and a deterioration in safety measures.
In his annual report for 2017/18, Mr Clarke warned staff and inmates have become “inured” to conditions unacceptable in 21st-century Britain.
He highlighted how thousands of inmates are living in squalid and overcrowded cells, locked up for nearly 24 hours a day.
Official figures published in July revealed that assault and self-harm incidents were continuing to rise, both reaching new record highs.
Overcrowding remains a key issue, with the prison population forecast by the MoJ to “steadily” rise by more than 3,000 over the next five years, reaching roughly 86,400 places in March 2023.
The MoJ said it doubled the prison sentence for anyone who assaults prison officers on Thursday.