Pennington House probably had its origins as a farmhouse. It was certainly in existence in 1781, when it was owned by Henry Brownrigg. By 1794 it was the home of Robert Man, a Rear Admiral of the Red Squadron. He purchased a barn and 14 acres of land in that year. He died at Pennington House in 1813. He was one of three local admirals who fought in the Revolutionary and Napoleonic Wars and is buried at Milford Church.
In 1826 Col John Shedden bought the house using his wife’s fortune from slave plantations in Jamaica, for his daughter Adelaide, who married Major Pringle Taylor. Despite efforts to sell the house, it remained in the family during the 19th century though it was frequently let.
In 1901 Keppel Pulteney of St Austins bought the house, though he let it to the 4th Marquis of Headfort whose cousin Mary Fitzpatrick had married William Cornwallis-West of Newlands. The Marquis caused a stir when he married the actress Rosie Boote whom he had seen perform on stage in ‘The Messenger Boy’.
In 1909 the house was bought by Frederick Grotrian who was a keen gardener and laid out the gardens. In 1916 the house was bought by the Hon Francis Brownlow whose famous greyhound ‘Master McGrath’ is commemorated with a statue in the grounds.