Located at a distance of 5 km from Vijayawada, Krishna district Mogalrajapuram Caves actually are three cave temples that date back to the 5th century. Though in ruins, these caves present a wonderful sight. During 5th century AD, Vishnukundins who ruled Krishna district excavated cave temples at Mogalrajapuram. It has five rock cut sanctuaries and of these only one is in good condition.
The Mogalarajapuram caves having three cave temples, enshrines the idols of Lord Nataraja, Vinayaka and other deities. However, the Ardhanareeshwara Murthy is considered only one of its kinds in Deccan India.
One of the caves has an overhanging cornice with artificial windows. The idols of Lord Nataraja and Vinayaka in one cave are still in good condition compared to the other worn out images of deities.
Source:
http://www.tourism4india.in/Vijayawada/Spots/Mogalrajapuram-Caves.aspx