The plunger will not stay in unless you have time on the Limit Timer clock. If you have time on the clock, the Limit Timer may be burned up.
If you look at the back of the Kiln Sitter faceplate, you will see a small spring attached to a locking slide. The spring pulls the locking slide downward so that it engages in the groove on the Kiln Sitter button. If the slide doesn't engage, the button won't stay pressed in.
That spring has very little tension. The smallest resistance will prevent the spring from pulling the locking slide downward.
The most common reason the locking slide doesn't engage is that one of the wires in the switch box is interfering with the locking slide. If you ever replace an element and remove the switch box, you should be aware that wires can interfere with the movement of the locking slide.
If the spring breaks, the locking slide won't engage. But that is rare. Sometimes the spring will rust and become weak.
Also, dust and corrosion can prevent the locking slide from moving freely.