29" wheels were meet with similar scrutiny and derision. 20 years later they're practically THE MTB standard.
But look at what else has changed / advanced away from accepted standards in that time: head tube diameters, bottom brackets, 1x drivetrains, gear boxes, dropout spacing, axle choices, internally-routed dropper posts, wireless shifting.
Well they are the standard for two reasons. 1, you can't buy anything else now lol, and 2, they are better offroad because of their ability to rollover stuff better. But they have downsides, they are harder, or slower, to change direction. But all in all they are a good choice, a nice refinement. But on-road is another matter and the advantages are not so clear. As for the other stuff you listed, yes I agree, they are vast improvements, modern suspension probably being one of the real game-changers, but you didn't need to change wheel size to get all that did you?
My latest 'road' bike is an MTB, has a dropper post and full suspension, thru axles and a 1x drive train too, but a lot of that is irrelevant for road use. I only bought it for the suspension, to counter the rough blacktop roads without the bobbing of old suspension systems. I got all the rest thrown in because it's the industry standard. MTB trail bashing is a different world all together though, perhaps they'll benefit from 32 inches, or perhaps they'll be falling into the ravines more often, time will tell. Just because something is *new* doesn't automatically imply it's better.
Take modern hiking boots and work boots, most now have a polyurethane midsole which makes them a bit more comfortable, but it rots out in a few years and you have to throw them away. That's the industry standard now. New boot sales every 3 to 4 years
BTW, Vibram traditionally made the best soles in the world, all the major hiking boot companies use them. Zamberlan use them, I have a pair of 20 year old Zamberlan's that are still sound.
My old ones
The pair I bought 5 years ago for $500 and hardly ever wore. Boot menders won't touch them, they are throwaways. Both have Vibram soles.