The performance of organic photovoltaic (OPV) device has been improved rapidly benefiting from the development of non-fullerene acceptors (NFAs). In these new systems, the mechanism models developed in conventional OPV systems with fullerene acceptors are not fully valid for describing the generation and recombination processes of photo charges. Here, we summarize recent studies on the excited carrier dynamics using ultrafast spectroscopy. In the model systems, an intra-moiety state arising from the molecule interaction mediates the hole transfer channel, allowing charge separation at a small interfacial driving energy. Moreover, the bimolecular recombination plays an important role in voltage loss in OPV blends with NFAs, which can be strongly suppressed by manipulating the alignment of interfacial and local triplet states. The energy landscape of key levels, susceptible to the molecular interaction, is essential for charge generation and recombination, which can be manipulated for future device optimization.