Clinically traditional Chinese medicine (TCM) has been proven to possess obvious anti-tumor effects. It is critical to further explore the effective components and the corresponding mechanism of the TCM against target cells, which also has great significance for developing novel nano-TCM formulations for clinical treatment of tumor. This paper systematically reviews the anti-tumor effects of Chinese herbal compound and the anti-tumor mechanism of single herb. We also summarized the progress in the current traditional nano-TCM preparations. Taking the shikonin in Herba Arnebiae as an example, using the nano-material self-assembly technology, we discussed the design of novel nano-macromolecule TCM formulations while considering the mechanism of single herb and the clinical obstacles.