It would be helpful to put some real numbers on this, particularly torque and tension.
What is the desired tension? And what torque does this translate to in the tensioning roll? What is torque then on the output of the gearbox? From that, we can calculate the torque at the input to the gearbox. Now, how many poles is the motor? or if you will, what is the base speed? From that, we can determine the motor's full load torque and, by comparing it to the torque required on the input of the gearbox, we can find the percent of load torque the tension represents on the motor.
If the tension is somewhere in the range of 25-100% of motor torque, then we stand some chance of getting the drive to regulate it smoothly. If, on the other hand, it is down in the noise, say, under 10% of motor torque, then regulation becomes very difficult.
It might be useful to do the same exercise with the older 4/1 gearbox.
Let us know what some of these values are. We can draw some conclusions about the drive if we have them.
Thanks