A complex, large and interesting building, Crichton Castle consists of ranges of buildings from the 14th to 16th centuries, enclosing a small courtyard. The oldest part is a 14th century keep, formerly of three storeys. The basement was vaulted and had a pit-prison. A stair led up the hall and entrance on the first floor, and another turnpike stair led to the floors above. The castle had a small courtyard.
In the 15th century a new gatehouse of three storeys was added, then further ranges enclosing the courtyard. Another block was added in the 16th century, Italian Renaissance in style with an arcaded diamond-faced façade.
Outside the castle are the roofless stables, which are said to be haunted by the ghost of William Crichton.
The castle was the property of the Crichtons and probably first built about 1370. Sir William Crichton, Chancellor of Scotland, entertained the young Earl of Douglas and his brother before having them murdered at the 'Black Dinner' in Edinburgh Castle in 1440. John Forrester slighted the castle in retaliation. Crichton, however, founded the nearby Collegiate Church wherein priests were to pray for his salvation - he needed all the help he could get.
The Crichtons were forfeited for treason in 1488, and the property later passed to Patrick Hepburn, Lord Hailes, who was made Earl of Bothwell. One of the family was James Hepburn, 4th Earl, third husband of Mary, Queen of Scots. In 1559 the castle was besieged and captured by the Earl of Arran and, after the Earl of Bothwell was forfeited, given to Francis Stewart, who added the Renaissance range. Mary, Queen of Scots, attended a wedding here in 1562. Francis Stewart was such a wild and unruly fellow that in 1595 he was also forfeited and he had to flee abroad. Crichton passed through many families, and became a romantic ruin. Turner painted the castle, and Walter Scott included it in 'Marmion'.
The castle is said to be haunted by a horseman, who enters the castle by the original gate, which is now walled up.