Saviors are way too dependent on ruling through fear / intimidation, and spread too thin. Rick's group took out what looked to be like a relatively well-armed and supplied outpost with ease (minus the Maggie/Carol hiccup). And that was with far fewer numbers. Now, Rick's group has been established as pretty resourceful and badass and all, but as long as the apocalypse has been going on I find it difficult to believe there weren't other groups, especially in that region, who were smart enough to engage in some guerrilla warfare against the saviors. For that matter, I haven't seen too many booby traps in general. If there's few enough roads for the saviors to set up an ambush and sit around waiting for Rick (presumably with people scouting and following them), then sure the Saviors have enough numbers to maintain...in a limited capacity. But it doesn't take much to dig a punji pit, and if you know they're scouting on foot you could dig several a night until their camp was surrounded by a veritable minefield of pit traps. Taking out vehicles (fuel supplies actually lasting that long notwithstanding) is also a relatively simple matter. Some spike strips, homemade caltrops, etc along their egress routes from their camp? A tripwire linked to a tree trunk to block the road? Or even just digging under the road and weakening the ground so that foot traffic can cross but vehicles collapse the pavement. How about fertilizer explosives? Where's all the construction and farm equipment? One guy with access to a welding shop could build tank traps that would completely block off the Saviors' supply routes. Hell, just source a few bags of concrete from somewhere and make a shitty wall in the middle of the road overnight, then dig ditches to the sides of it so they can't drive around.
Any halfway-decently supplied group could accomplish this easily I think. Wouldn't even take that long. Set up a sniper or two to take a few potshots and hide every time they try and clear a road and you could whittle them down pretty quickly. Their society would quickly turn on itself and Negan.