To understand who is the holder of record, and thus who retains the voting rights, you just need to follow the shares. Initially, the shares are held by one of the three sources. Whichever source initially held the shares was also the holder of record. When the shares were used in the short sale transaction, the initial source lost its voting rights as it was no longer the holder of record. Even the margin account customer who holds the shares long will lose his or her voting rights in this situation - this is part of the margin account agreement.
The shares are then sold in the market, and the investor who purchases these shares becomes the holder of record for these shares, thus controlling the voting rights. The investor going short does not get the voting rights. When this investor closes his or her short position, the shares are returned to the brokerage firm, and the voting rights return to the initial owner whose shares were used in the short sale.