The yellow in the middle of the picture is Max II castle, Schloss Hohenschwangau. Ludvig II spent his childhood here. It was also here Ludvig II met Richard Wagner who gave him the inspiration to Schloss Neuschwanstein.
Castle Hohenschwangau was built in the 12th century but was renovated by Max II in the 1830's.