Is there any penalty for late GST registration in Singapore?

Yes, there is a penalty for late GST registration. The amount of the penalty will depend on the length of time that you were late in registering.
If you were more than three months late in registering, the penalty will be 10% of the tax due. If you were late in registering by less than three months, the penalty will be 5% of the tax due.