The objects found on the castle hill prove continuous settlement since the Neolithic Age. Around 1000 BC. a fortified hillfort was built on the castle hill, as evidenced by the intact settlement areas. The Early Iron Age was the time of the migration of the Celts to the east, who also left behind some remains (graves). The prehistoric settlement was replaced by Roman fortifications, followed by a late antique refuge, followed by an early medieval Old Slavic hillfort and burial ground. In the 9th century, there was a defensive tower of the Salzburg church lords on this site.

The oldest written sources mention that the Salzburg Archbishop Konrad I, who held this position from 1106 to 1147, had a fortress built on the site of the remains of older medieval buildings in the years 1125 to 1130. The well in the courtyard, which is 54 m deep, probably dates from the 11th or 12th century, while the current wreath was built in the Borok period, or the second half of the 17th century. Around 1220, the Salzburg Archbishop Eberhard II. and the Styrian Duke Leopold VI. established a mint, which was supposedly located in the castle near the Romanesque palace and Konrad's Tower. The mint ceased to operate in 1230. The Ptuj knight Herman is mentioned in 1235, and in the same century there are several other mentions of buildings related to Ptuj. In 1247, a "small" castle (castrum minus) is mentioned, which suggests that there was also a large castle ("castrum maius"). In 1260, a castle fee (pro custodia castri) is mentioned, which would have amounted to 200 marks. Around 1376, two castles (beide burg und stet) are mentioned in an anonymous chronicle. In 1433, the Ptuj castle lords Hartnid and Friderik wrote about the Salzburg fief in Ptuj: the castle, the castle tower, the castle hills and 12 guard estates around it. The last medieval record of the Ptuj castle is from 1487, in the travel diary of Paolo Santonin, a companion of the Aquileian visitor. It is written that the castle was strong and impregnable.
The Ptuj castle was very important in the Middle Ages. It was the center of the feudal possessions of the Salzburg Church in Podravje, the Lords of Ptuj and its good strategic location. Ptuj Castle, besides Stubenberg and Wildon, was the only one in Styria at that time that had to provide provincial defense with 200 guardsmen if necessary.