The encoder input signals gotta have something to be referenced to, so if you're using an independent power supply for the encoder you need to connect its DC COM to [CORRECTION] the A-/B-/Z- terminals.
What power supply voltage does this encoder use ? 5, 9, 15, 24, something else ?
I can't find that specific wiring diagram in my quick search; which document is it from ?