Dashwood and his wealthy friends were all members of the Hellfire Club, whose adopted motto seemed to be Rabelais' "Do what you will." Members included Thomas Potter (son of the Archbishop of Canterbury), John Wilkes, William Hogarth, the Earl of Bute, the Marquis of Granby and the Prince of Wales. Benjamin Franklin was also a member. Meetings of The Hellfire Club were originally held in London's George and Vulture Inn and then at Medmanham Abbey. Rumors of the questionable activities of the members caused Dashwood's group to move into the West Wycombe caves. It is believed they continued to perform mock religious rituals and participated in sexual debauchery wearing masks and gothic robes. The only actual sightings of the Hellfire Club's activities occurred several times on the River Thames. Dressed as monks, Dashwood and his friends boarded a boat with prostitutes dressed as nuns to engage in sexual activities under the full moon.

Some accounts claimed that Dashwood and his Hellfire Club were actually members of a Masonic group, while others claimed that the Dashwood and his brethren performed satanic rituals and sexual debauchery regularly, citing that even the design of the cave system itself is a sexual portrayal. It's known for certain that Dashwood was a man of much notoriety whose drunkenness and public religious pranks earned him an infamous reputation. Other than that, the jury is still out on what really went on during meetings of The Hellfire Club.

Inside the hand-carved West Wycombe caves, pick axe marks are still visible on the chalk walls. The marks provide the visitor with an eerie link to the past but also an appreciation for the incredible amount of energy that went into fashioning the caves.

As creepy as the place proves to be, it's no wonder that visitors have had unnerving experiences in the caves. People have witnessed a man, thought to be Paul Whitehead, walking around the tunnels as if looking for something — according to legend he's looking for his heart, which he gifted to Dashwood upon his death. (The box is still there but his heart went missing long ago). Visitors most frequently claim that their heads are touched. A woman in white apparently haunts the feasting hall as well. The story associated with this woman tells of a note that was written, asking her to meet a man in the caves and she was to wear a wedding dress. Once inside the caves, she met three boys who had written the letter as a prank. Her anger caused her to throw a rock, which hit the wall, bounced back and hit her in the head. Apparently she died in the caves on the same day. Lights have been seen moving through the caves at night before closing.

Paranormal investigations of the caves have resulted in EVP evidence and several personal experiences that are consistent with the stories. Figure-like shadows have also been seen and strange lights have been caught on camera.