I'd definitely rebuild it, if it's missing parts it can be very unsafe to drive.
My column had those 4 screws loose, I could move it in all directions, only thing it didn't do was telescope! I used blue loctite and torqued to spec those 4 bolts, I hope to never re-visit the inside of that column again.