Passford House was known as The Mount in the early part of the 20th century when it was occupied by Lord Arthur Cecil (half-brother of the Marquis of Salisbury) who was well-known for his work on improving the breed of New Forest ponies.
The house was requisitioned during World War II as a convalescent home for troops. In 1946 the Cecil family sold the house, which was converted into a hotel.