I finally got around to removing the controller. It was definitely the source of the rattle. After I removed it. I put a piece of foam (the sheet type used for packing) on the bottom of the downtube where the controller sits, and also attached a piece to both sides of the controller. Put it all back together and no more rattle.
I have ridden a few times on some rough single track since and the rattle hasn't returned. So hopefully that fixes it. Nice to be able to ride the bike without that supper annoying rattling.