The first ebike we bought, I was concerned about the ability to pedal it w/o power, but it is a geared hub motor, which has minimal drag, so it pedals as well as any other 55 pound bike. It takes little power to keep a bike cruising at 12-14 mph on flat land with no wind, and I often turn off or turn down power assist when cruising. When riding with my wife on her e-bike, we go slow enough that I often never turn on my power. Yes, you feel the extra weight accelerating quickly or going uphill.
I have converted several regular bikes to electric, adding motor/battery combinations that weigh between 9 and 15 pounds. These bikes weigh between 44 and 56 pounds now. They ride much like they did before conversion, After 18 months of e-bike riding and using power conservatively, my own weight is down over 20 pounds, which makes up for the battery/motor weight. When I get onto my regular GT Transeo, it feels a lot lighter than its 35 pounds.
My experience comes from either mid drive motors, or geared hubs, which have clutches to disengage the motors when it's not running. A direct drive hub motor has no clutch, so more force is needed to overcome magnet drag when not powered. How much, I don't know. (Edit: per Dunbar and Douglas, it's a lot. I thought so.)
I would expect most e-bike riders never want to be without power anyway, but it happens to me occasionally, mainly because I went out with a low battery. Once I had to go 18 mile when a connector broke.