Some of you guys don't know much about space, physics, and the potential for damage just like Norton explained. IF and asteroid was moving fast enough (and there are some in our own Solar System that are) can punch a hole in a Star Destroyer and keep on trucking like it ran into a wall of rain. So this kind of damage is feasible and realistic. Great idea Norton!
BTW - I'm an amateur astronomer and I know a thing or two about celestial objects, the physics involved, especially the physics behind momentum and forces in space. A rock, the size of a baseball - travelling at 32000mph in space (not uncommon), can rip through 10 feet of steel and keep moving as if it hit soft butter. Might have a few cracks in it, but it'll still be a rock - the size of a baseball, moving at 32000mph - give or take a few hundred mph for the previous impact.