Chronic myelogenous leukemia (CML) is a malignant myeloproliferative disease. According to the American Cancer Society's 2021 cancer data report, new cases of CML account for about 15% of all leukemias. CML is generally divided into three stages: chronic phase, accelerated phase, and blast phase. Nearly 90% of patients are diagnosed as a chronic phase. Allogeneic stem cell transplantation and chemotherapeutic drugs, such as interferon IFN-α were used as the earliest treatments for CML. However, they could generate obvious side effects, and scientists had to seek new treatments for CML. A new era of targeted therapy for CML began with the introduction of imatinib, the first-generation BCR-ABL kinase inhibitor. However, the ensuing drug resistance and mutant strains led by T315I limited the further use of imatinib. With the continuous advancement of research, tyrosine kinase inhibitors (TKI) and BCR-ABL protein degraders with novel structures and therapeutic mechanisms have been discovered. From biological macromolecules to classical target protein inhibitors, a growing number of compounds are being developed to treat chronic myelogenous leukemia. In this review, we focus on summarizing the current situation of a series of candidate small-molecule drugs in CML therapy, including TKIs and BCR-ABL protein degrader. The examples provided herein describe the pharmacology activity of small-molecule drugs. These drugs will provide new enlightenment for future treatment directions.