I don't think they can ask for such huge amount of penalty.
I think they are charging 100% value of goods which is charged in case the transporter doesn't carry proper documents.
In this case you can quote Kerala High court judgement in the case of Rai Prexim India Private Ltd where it was held that in case of any mistake due to human error no penalty can be charged if proper tax has been paid by the assessee.
Hope you find the information helpful if you do please rate it 5 and provide your valuable feedback for my improvement.
Thank you