Objective To explore a method to improve the accuracy and generalization ability of medical diagnostic neural network models under conditions of small data volumes, and to address the issue of poor neural network model performance in computer-aided diagnosis of nail diseases due to limited training data. Methods A dual-model strategy integrating instance segmentation with fine-grained feature classification was proposed. The neural network model based on dual-model strategy was trained using the dataset of Image-Based Intelligent Diagnosis of Nail Disease Model task of the first National Digital Health Innovation Application Competition & Health and Medical Big Data Theme Competition. This dataset covered 8 types of nail diseases, including nail matrix nevi, paronychia, nail psoriasis, onychomycosis, subungual hemorrhage, melanonychia, periungual warts, and nail melanoma, with class imbalance present. The diagnostic performance of the dual-model strategy was evaluated and compared with single-model strategies (image classification models ［ResNet50 and Swin Transformer］ and target detection model based on faster region-based convolutional neural network ［Faster R-CNN］) under the same hardware and software training conditions. Results The dataset included 1 048 samples, including 210 cases of nail matrix nevi, 186 cases of paronychia, 69 cases of nail psoriasis, 203 cases of onychomycosis, 149 cases of subungual hemorrhage, 71 cases of melanonychia, 93 cases of periungual warts, and 67 cases of nail melanoma, with 90% used for training various models and 10% for evaluation. The micro F1 score was 0.324 in the image classification model based on ResNet50, 0.381 in the image classification model based on Swin Transformer, 0.572 in the target detection model based on Faster R-CNN, and 0.714 in the dual-model strategy model Mask R-CNN＋Swin Transformer. The accuracy rates for diagnosing different nail diseases in the dual-model strategy were: nail matrix nevi 80.95% (17/21), paronychia 89.47% (17/19), nail psoriasis 100.00% (7/7), onychomycosis 70.00% (14/20), subungual hemorrhage 73.33% (11/15), melanonychia 14.29% (1/7), periungual warts 55.56% (5/9), and nail melanoma 42.86% (3/7). The micro F1 score for evaluating the dual-model strategy on a test set of 1 000 cases was 0.844. Conclusion The dual-model strategy can effectively combine models with different functions to well accomplish the task of intelligent diagnosis of nail diseases under small data volume training conditions.