This is entirely dependent on the PLC and possibly on the settings within the PLC.
Default, is commonly everything starts at zero, although the logic results of the first scan may quickly change that.
Other PLCs may be set to start on their last remembered states. Some have some memory areas saved to battery backup, and some can have all areas saved to backup.
You also have to check what happens if power is lost on the CPU and not on the IO, this can most commonly happen when you use a bus system such as Profibus. Most IO will switch off, but some can be set to remember their state.
The point of this is that each individual PLC (and IO) has different possible behaviours, so you should read the manuals, and if in doubt, bench test the behaviour.