Generative AI in medical education remains promising but its effects on learner performance are unclear. Our pilot randomised controlled trial studied the impact of ChatGPT-assisted learning on anaphylaxis in undergraduate medical students. The intervention group outperformed the control in learning outcomes, highlighting potential AI dependency concerns. While ChatGPT aided knowledge retrieval, its efficacy in complex tasks was limited. Further research is crucial to understand AI-human interactions and mitigate biases.