The caves are believed to be once meditation place for the Buddha. The first Buddhist Council lead by Mahakashyapa, one of Buddha’s main disciples, was held here and the texts on Buddha’s teachings were put to writing. This event took place after about six months of Buddha’s Mahaparinirvana.

There are important Jain Temples on the way to the Saptaparni caves. A partially ruined ancient Shiva temple is also found here. There are signs of unexcavated mounds nearby. The pillars of the temple are plain and octagonal. The front roof had been destroyed and the Shiva lingam is enshrined in the inner chamber.

Source:

http://www.bihartourism.gov.in/districts/nalanda/Satparni%20Hills.html