All the modems you'll find will communicate with the SLC Channel 0, and only get their power from the backplane. The SLC backplane is data-only and won't route communications like the serial port will.
I've used the phone-line DLM and the spread-spectrum radios from Data-Linc. You can't go wrong with them or with Miille Applied Research.
If your serial port is already in use, then you're probably going to need a 1747-KE module to translate DF1 Full Duplex (suitable for modems) to DH485 (suitable for 5/01, 5/02 and 5/03 controllers).