Hi
No, there would not be any problem as such. Provided you maintain details of all such credits to your account and have explanations to prove the genuineness in case and query arises.
On a safer side, you should file NIL income tax returns for these periods.