Many cancer cells demonstrate a high rate of glucose consumption via glycolysis to provide intermediates for macromolecule biosynthesis. To accomplish this metabolic change, the expression of pyruvate dehydrogenase kinases (PDKs) is rapidly increased in cancer cells. Inhibition of PDKs could promote the function of mitochondria by increasing the oxidative metabolism of pyruvate, resulting in the death of cancer cells. In this review, we provide an overview of the structural information available for PDKs and their connections to known therapeutic effects. We then describe the development of small molecule PDK inhibitors in medicinal chemistry with particular emphasis as anticancer agents. Finally, directions for further development of PDK inhibitors as potential anticancer agents are discussed.