Ken:
If you are switching a DC load, you will have to make sure that the load polarity is right - unlike the contacts of a mechanical relay where it doesn't matter. A solid-state relay is, in essence, an SCR (a.k.a. a "thyristor"). Think of it as a diode that can be switched on or off. It goes without saying that a diode conducts only in one direction.
On the same token, if you are to switch an AC load, you must use an AC SSR, which is a combination of two SCRs, connected in parralel in opposite polarity. Of course, all this a simplification - there is more in industrial SSR than just one or two SCRs. But it describes its behavoir pretty well.
One other inherent property of an SSR is that the load current should not be too low, otherwise it will never switch on. This information may be found in manufacturer's datasheets. Another difference is that, like all the solid-state components, it doesn't handle overcurrents well and tends to break down first, before any fuses would blow. The failure of an SSR would invariably mean that your load will stay energized all the time; think about potential dangers for your application if this ever happens (and it will, sooner or later
).
Make sure that your load inductance and capacitance do not exceed the SSR specifications, since SSRs are much more "picky" about any kind of transient effects. They work much better with purely resistive loads.
Generally, I would use an SSR for a fast-switching load, or a pulsing one - a heater is a classical application. Don't forget that an SSR is typically more expensive than a mechanical relay of the same load capacity. There must be a good justification to replace everything at once.