The OEM chargers that came with my bikes all get quite warm. Not too hot to touch but close. The cooling fan failed on one of them and it started to emit a smell of hot plastic.
Eventually, the high temp cutoff shut it down. Not all chargers have this feature and unfortunately, quality varies greatly among manufacturers. Since I often charge overnight or while unattended, the incident of the failed cooling fan caused me to spend some $$ on a quality Grin Satiator charger. It's a sealed unit with no cooling fan.
If it smokes, smells, or gets too hot to handle, replace it. Otherwise, it should be ok. It helps to charge in an open area with good air flow.