These panels have detached at speed before.
I recall these panels clip into small rectangular holes molded into part of the front fender.
If your panel is not in place have a close look at these holes. There may be one that is damaged or deformed that prevents the clips on the small panel from attaching properly.
A close-up photo of your mounting bracket would be helpful for us to diagnose the cause of these detachments.
On mine if I lightly press the upper edge (with the rubber lip seal), there is some 1/16" of movement. Perhaps some foam mounting tape is required inside for a more secure mounting.
Cheers, Steve