Some 30,000 criminals per year in England and Wales, including burglars and most shoplifters, could be spared jail under the proposals, the Daily Telegraph reported.
Exceptions would be made for violent or sex offenders, Rory Stewart told the paper.
Mr Stewart said the move would ease pressure on prisons and that “very short” jail terms were “long enough to damage you and not long enough to heal you”.
He told the Daily Telegraph Magazine: “You bring somebody in for three or four weeks, they lose their house, their job, their family, their reputation.
“They come (into prison), they meet a lot of interesting characters (to put it politely) and then you whap them on to the streets again.
“The public are safer if we have a good community sentence … and it will relieve a lot of pressure on prisons.”
Since the early 1990s, the prison population has doubled, rising from around 40,000 to more than 80,000 in 2018, official figures show.
More than half of the 86,275 offenders sentenced to immediate custody in England and Wales in 2017 were handed sentences of six months or less, according to a Parliamentary response from Mr Stewart to shadow justice secretary Richard Burgon last month.
The Telegraph said Mr Stewart cited research indicating that criminals given short jail terms were more likely to re-offend than those given community sentences.
A Ministry of Justice spokeswoman said: “As we have said previously, short sentences are too often ineffective, provide little opportunity to rehabilitate offenders and lead to unacceptably high rates of reoffending.
“That’s why we are exploring potential alternatives but this work is ongoing and we have reached no conclusions at this time.”
Mark Leech, Editor of The Prisons Handbook for England and Wales said the call was “bizarre”.
Mr Leech said: “I am absolutely in agreement with Rory Stewart on the substance of this, there are better ways of punishing non-violent, non-sexual, offenders other than very short sentences that clog up the system and cost a fortune.
“But Rory is a Minister of the Crown, if he wanted to introduce legislation curbing the courts’ ability to pass short sentences in such cases he could start that ball rolling tomorrow.
“It’s bizarre that he doesn’t just do it.”