LOOKING TO LEARN: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling