He was delivering a speech in the western city of Nara, public broadcaster NHK said.
Abe, 67, appeared to be in a state of cardiac arrest, the network said and Kyodo news agency. Shots were heard and a white puff of smoke was seen as Abe made a campaign stump speech outside a train station, NHK said.
An NHK reporter on the scene said they could hear two consecutive bangs during Abe’s speech.
Abe served two terms as prime minister to become Japan’s longest-serving premier before stepping down in 2020 citing ill health.
But he has remained a dominant presence over the ruling Liberal Democratic party (LDP) party, controlling one of its major factions.
His protege, Prime Minister Fumio Kishida, faces an upper house election on Sunday in which analysts say he hopes to emerge from Abe’s shadow and define his premiership.
Abe first took office in 2006 as Japan’s youngest prime minister since World War Two. After a year plagued by political scandals, voter outrage at lost pension records, and an election drubbing for his ruling party, Abe quit citing ill health.
He became prime minister again in 2012.
ALSO READ:Boris Johnson, British Prime Minister, Resigns
Abe hails from a wealthy political family that included a foreign minister father and a great-uncle who served as premier.