Pop rivets ,I know you tried ,but how about this , Knock the metal pin out of the rivet ,push the rivet in from the inside, clamp 1/2 inch or 3/8 drive extension in the vice in the vertical, place flat head of rivet on to extension use ball pain end of hammer and knock down rivet .
that's my take on the problem .
David
HI Mark