Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation
[Sztuczna inteligencja] Autorzy: Guangyuan Dong, Ziwei Hong, Xuehao Zhou i 6 więcej
Medical question answering spans specialties and clinical operations that may benefit from different adaptation directions. We propose ARBOR, a parameter-efficient method that selects rank-one components from a shared low-rank basis for each question. An additive gate combines question representations, specialty tags, operation tags, and their interaction; a learned coefficient scales the adapter residual. An illustrative separation under orthogonal, equiprobable subtasks shows how conditional selection can avoid an approximation floor faced by a fixed update with the same active rank. This result motivates the design without asserting a corresponding bound for medical corpora. On Qwen3-8B across CMB, CMExam, MedQA, and MedMCQA, five-seed experiments yield 69.69% mean accuracy across benchmarks, exceeding LoRA r16 and MoELoRA by 1.26 and 1.30 percentage points, respectively. The reporte…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja