Greatly enhanced upconversion luminescence was demonstrated by integrating the core-shell upconversion nanorods with the Ag nanogratings. Both the Ag nanogratings and upconversion nanorods were fabricated/synthesized in a facile, cost-effective, high-throughput way. Experimental results showed that the upconversion luminescence intensity of Er in the core-shell upconversion nanorods can be well tuned and enhanced by changing the shell thickness and the period of the Ag nanograting. The underlying physical mechanism for the upconversion luminescence enhancement was attributed to the plasmonically enhanced near infrared broadband absorption of the periodic Ag nanograting and the localized surface plasmon resonance of Ag nanocrystals. The maximum enhanced factors of 523 nm, 544 nm (green emission), and 658 nm (red emission) of Er ions excited at 980 nm are 3.8-, 5.5-, and 4.6-folds, respectively. Our fabrication approach and results suggest that such a simple integration is potentially useful for biosensing/imaging and anti-counterfeiting applications.