Standing dramatically on a small island, Castle Stalker is a tall, massive and simple keep, rectangular in plan, of four storeys and a garret. A flush parapet was remodelled in the 16th century, and a gabled caphouse crowns the stair. The thick walls are pierced by shot-holes.
There are two entrances. The first, at ground level, is defended by a machiolation at parapet level and leads to the vaulted basement, which has a prison in one corner. A small mural stairway leads to the first-floor hall. The main entrance, reached by an external stone stair, is at first floor level and also protected by a machiolation. Private chambers occupied the storeys above the hall. A small courtyard has a landing place for boats.
Castle Stalker was built by Duncan Stewart of Appin, who was made Chamberlain of the Isles for his part in helping James IV destroy the MacDonald, Lord of the Isles. It is believed to have been used by James IV as a hunting lodge. The assassination of Campbell of Cawdor in 1592 started a feud between the family and the Campbells. The Stewarts of Appin fought at the Battle of Inverlochy, under the Marquis of Montrose, in 1645 against the Campbells. In 1620 the castle was sold to the Campbells, but the Stewarts retrieved it after a long siege in 1685. A Stewart garrison surrender to William and Mary's forces in 1690. In 1715 the clan fought for the Jacobites at Sherrifmuir during the Jacobite Rising. The 9th Chief did not support the 1745 Rising, but the clan were led by Stewart of Ardshiel. The 9th Chief sold his estates in 1765. The castle was abandoned in about 1780, was roofless in 1831, but restored from ruin in the 1960s.