Don't buy silver because Buffet bought silver. He may have bought it as a hedge against other investments he holds. He may have bought it because he can make money off it that you can't (by lending it to short-sellers, and such). He may have bought at a price much lower than you'd get because of *when* he bought, or even how much. He may have sold it already. His time window may differ substantially from yours. He may lack the opportunities you have, etc.
You and only you have to determine which investments make sense for you, your life, your goals, and your portfolio.