We have added an amount of unbalance as indicated by the machine. still the machine shows substantial unbalance and the angle also got shifted. What is the problem?

Unbalance is a vector. To compensate that you need to compensate at the exact location. In case, you have compensated the right quantity, but at a different place (Angle), you will get the resultant of the residual unbalance and its location.

The above three figures indicate for the same unbalance value and compensation values, the residual unbalance is indicated. So, in essence, both amount of unbalance and location of unbalance determines the effect of balancing.