The battery is the only significant fire concern. If a battery has been designed properly to keep the cells within voltage, current and temperature limits, it shouldn't matter with bike or device you connect it to. The pack should protect itself. UL battery testing should ensure that the protections function reliably. So no need to buy a new bike. UL battery testing also covers mechanical damage to some extent with crush testing and vibration testing.
Having said that, having cells pushed outside their safety limits (voltage, current, temperature again) and mechanical damage is only one way that fires start. Many of the battery failures that I've seen arise are due to cell internal defects and water intrusion. UL 2271 is somewhat weak on both.
Internal cell defects are very difficult to detect so it's understandable that it would be hard to design a certification test to eliminate this problem.
The only defense against a cell going into thermal runaway (TR) is to design a pack to protect against fire propagation. A PPR test shows whether a pack can contain a cell that has gone into TR or if it eventually causes the whole pack to burn. There is no PPR test in UL 2271.
Maybe Ibrahim can correct me, but for ebike batteries that mount to the frame or a rear rack, UL 2271 only requires an IPX3 rating for water/dust ingress, which only covers water drops from certain directions. The recent
Specialized battery recall was due to fires caused by water intrusion. UL certification would not have changed this.