信息工程大学学报2026，Vol.27Issue(3)：371-378,8.DOI:10.3969/j.issn.1671-0673.2026.03.017
视觉—语言协同微调LLaVA的遥感图像目标检测方法
Remote Sensing Image Object Detection Method Based on Vision-Language Collaborative Fine-Tuning LLaVA
摘要
Abstract
To address the challenge that remote sensing image object detection heavily depends on ex-tensive annotated data and that existing efficient fine-tuning methods for general parameters exhibit in-sufficient spatial perception when adapted to remote sensing tasks,a vision-language collaborative effi-cient fine-tuning method for multimodal large models is proposed.Firstly,the existing dataset is recon-structed to construct instruction data suitable for large language and vision assistant(LLaVA)fine-tun-ing.Secondly,combining the multi-cognitive visual adapter(Mona)and the quantized low-rank adapta-tion(QLoRA)method,the LLaVA model is efficiently fine-tuned on the instruction fine-tuning dataset.Finally,the target category and coordinate information are obtained in the inference stage.Experiments on the NWPU VHR-10 dataset show that compared with the pre-trained LLaVA model,the collabora-tive fine-tuning model has greatly improved in F1 score,accuracy,and mIoU.Its mAPs on both NWPU VHR-10 and DIOR datasets are comparable to those of traditional detection models.This method still performs well in low-annotation scenarios,providing a new technical approach for the efficient applica-tion of multimodal large models in remote sensing vision tasks.关键词
遥感图像/目标检测/LLaVA模型/多认知视觉适配器/指令数据集/量化低秩适配Key words
remote sensing image/object detection/LLaVA model/Mona/instruction dataset/QLoRA分类
信息技术与安全科学引用本文复制引用
张珈懿,陈琦,屈丹,司念文,于佳珺..视觉—语言协同微调LLaVA的遥感图像目标检测方法[J].信息工程大学学报,2026,27(3):371-378,8.基金项目
河南省科技攻关项目(252102211040) （252102211040）
河南省自然科学基金(252300420990) （252300420990）
