Nose cartilage damage can be pretty serious. My friend got punched so hard in the nose that he had to have surgery to fix it.
Blood collected around the damaged cartilage, and the surgeon said that this might cause part of his nose to die and collapse. The surgery prevented that from happening.
You know a punch has to be pretty intense to break cartilage! That stuff is fairly flexible.