The village of Garton is pleasantly situated about one and a half miles from the sea, and ten miles south of Hornsea. The area of Garton with Grimston, which was made into a civil parish for local government purposes, has been much reduced by coastal erosion over the years.
Most of the area is covered by boulder clay and lies more than 15 metres above sea level, rising to 22 metres near the coast giving an unbroken line of steep cliffs down to the sea. The coast road from Hornsea to Withernsea runs through the west part of the village. The village has a linear plan, and is surrounded by open fields and countryside. The main street is formed by the Aldbrough road, with a side road to Grimston.
The Grade 1 Listed Church of St Michael is built of boulders with ashlars' dressings, and consists of chancel with south chapel, nave with south aisle, porch, and west tower. Parts survive from the 12th century, and it has been added to and parts rebuilt over the centuries. It is now in good repair and services are supported by a small congregation.
The Garton manor house recorded from 1718 was presumably the current Blue Hall, which was so called because of the colour of its roof tiles. The house is of red brick and was built in the later 17th century with an L-shaped plan. It is a Grade II listed building, and to the front there were three storeys and five bays with a central entrance. Over the years the top storey and the bays have been removed. The interior retains a 17th century staircase with heavy turned balusters, and panelled rooms with other fittings of the period. There are remains of a possible moat and walled forecourt to the front. The house was once owned by the Constable family, and Henry Constable and his sister Margaret both lived there, dying within a few days of each other in 1701.
The main street of Garton continues eastward to the sea and Moat Farm, the former site of Grimston Garth, mentioned in the 15th century when it was the seat of the Grimston family. The manor house, or a replacement on its moated site, was later known successively as Grimston Garth Farm, and Moat Farm. It was said to have burnt down during the lifetime of William Grimston, but a substantial house survived in 1772. It was not used by the family as they stayed at nearby Hilston when visiting Holderness. The old house is now derelict but the moat is still there with some fishponds, and a smaller moat near the surrounding earthwork called the Mount.
The new Grimston Garth was built by Thomas Grimston about 1781 and was used as a summer residence by the family, who all lived at Kilnwick. It is castellated and was designed by John Call of York. It is built of brick with stone dressings, all of which were originally colour washed. The house has been restored to its former splendour and is lived in today. The main block has a circular tower at each corner. The main reception rooms are hexagonal and are on the two first floors.