This paper introduces Optimizing Directional Stimulus Prompting Through Human Feedback (oDSP-HF), a structured approach to AI-powered scaffolding that enhances LLM-driven educational support. By refining ‘Directional Stimulus Prompting’ (DSP) through user interaction, oDSP-HF enables LLMs to generate adaptive, reflective hints rather than direct answers. This approach was applied in the system prompting of two AI agents—Aiza and Alice, designed to support Academic English writing and computational thinking, respectively—demonstrating its practical applications in education.