The original house was built by Edwin Inman (1853-85) in about 1878 and later sold to Thomas Joseph Davis Rawlins, a Lymington banker, who greatly enlarged it. Rawlins married Annie Maria Torah at Lymington in 1890. The house was occupied by a retired Royal Naval Commander William George Le Cocq after Edwin Inman’s death. He and his wife Mary had four servants: a cook, parlourmaid, housemaid and nurse.
Under TJ Rawlins the house quadrupled in size, but by 1935 it and the garden had badly deteriorated, having been empty for several years. In about 1937 it was bought by speculators who divided it into two and painted it yellow. The 10-bedroomed house, in 5½ acres, had been offered to the council at a favourable price but they refused and so the West Hayes estate was developed in the grounds, designed by Philip Tilden.