ObjectiveBenefiting from their high-resolution spectral information and large-scale spatial information, hyperspectral images (HSIs) have demonstrated exceptional capabilities in numerous remote sensing applications. Over the past few decades, hyperspectral image classification (HIC) has attracted considerable research attention. Many machine learning-based HIC methods have been proposed; however, these approaches require a sufficient number of labeled samples to achieve ideal classification accuracy. Unfortunately, the high cost and effort associated with labeling HSIs often results in a scarcity of labeled data for many newly acquired HSIs. Therefore, researchers have introduced the cross-domain HIC (CD-HIC) mechanism, which uses a hyperspectral image with sufficient labeled samples (source domain) to assist in classifying a hyperspectral image with limited labeled samples (target domain). However, cross-domain classification is one of the major challenges in HIC due to feature and class distribution differences between source and target domains. The cross-domain few shot learning (CDFSL) methods, which integrated domain adaptation with few-shot learning, have been widely applied to the CD-HIC problem. Owing to the difficulty of spectral sequence encoding and the spectral similarity between classes, most existing CDFSL methods use convolutional neural network (CNN) or other remarkable spatial feature extractors to obtain spatial information, thereby improving classification accuracy. However, extracting spatial features often leads to distortion in the distribution of ground objects and their class boundaries. Aiming to address this issue, a lightweight Res-3D-CNN with embedded Transformer layers (LRCT) has been designed for feature extraction in CD-HIC. LRCT effectively captures long-term dependencies of the spectrum while simultaneously extracting spatial information, thereby notably improving the performance of spectral feature-based methods.MethodIn this study, a simple and effective deep learning network is proposed for feature extraction from HSIs. In CNNs, the convolution (Conv) captures high-frequency features of images by employing a weight-sharing mechanism within local receptive fields. In contrast, Transformers model long-range dependencies between features using self-attention mechanisms and adaptively focus on key areas. Moreover, the Transformer exhibits low-pass filtering characteristics, which primarily captures the low-frequency global information of images. Considering the complementary characteristics of Conv and Transformer, the Transformer layer is embedded into Res-3D-CNN to establish a lightweight dual-stream feature extraction network to perform feature extraction on the source and target domains. Furthermore, the CDFSL method is adopted to learn general information from the extracted features of source class data, which helps in target class data prediction with only very few or no labeled data. This approach helps achieve outstanding classification performance in the subsequent CD-HIC. The LRCT-CDFSL method comprises four main aspects as follows: 1) Data preprocessing: using a mapping layer to combine the dimensions of the original his; 2) Feature learning based on lRCT: employing a deep neural network for feature learning to enhance the representation capability for HSI data; 3) Few-shot learning: enhancing intra-class compactness and inter-class separability by calculating the Euclidean distance between labeled and unlabeled samples, thereby effectively adapting to scenes with limited samples; 4) Domain adaptation: using domain adaptation techniques to enable the feature extractor to generate highly generalized features, thereby improving the generalization capability of the model across different domains.ResultUsing Chikusei data as the source domain, and Indian Pines, Salinas, and Pavia University data as the target domains, extensive experiments are conducted to validate the performance of the proposed LRCT-CDFSL model and ensure fair comparison with advanced methods. Moreover, the effectiveness of the proposed LRCT-CDFSL is respectively tested by using the Indian Pines, Salinas, and Pavia University datasets as the target domain. The experimental results demonstrate that the LRCT-CDFSL method achieves faster and more accurate classification performance compared to existing methods. When only five labeled samples per class are available, the LRCT-CDFSL method achieves overall accuracy (OA) scores of 71.01% and 92.06%, and 84.14% on the respective target domain datasets. Compared to current mainstream cross-domain few-shot HIC methods, the LRCT-CDFSL method shows superior classification performance across various target domain datasets. Specifically, LRCT-CDFSL improves OA by 7.57%, 3.35%, and 2.77%, respectively, and reduces training time by 36%, 37%, and 30%, respectively.ConclusionA deep transfer learning network called LRCT network is introduced by embedding Transformer layer into a residual three-dimensional CNN (Res-3D-CNN). In Res-3D-CNN, a 1 × 1 convolution kernel is incorporated to adjust the number of channels in the feature map, reduce the model parameters, and accelerate the training. In addition, after
![]()
![]()
convolutional kernels,
![]()
![]()
convolutional kernels are then introduced to expand the number of channels of the feature map, thereby addressing the problem of reducing the feature map area caused by the convolution kernel. Additionally, the network shows excellent performance in few-shot learning and domain adaptation using convolutional kernels of different scales for feature extraction. Meanwhile, the Transformer layer is used to extract the local-global semantic information of HSI. Experimental results reveal that LRCT effectively captures spatial-spectral features through a combination of local-global information, fully representing the local-global semantic information of HSIs and enabling strong classification performance in the subsequent CD-HIC task. However, within the LRCT-CDFSL framework, the metric-based few-shot learning method, which emphasizes the relationships between samples, has not yet been fully explored. Aiming to further enhance performance in CD-HIC tasks, future studies may explore the integration of cross-attention learning techniques into the LRCT-CDFSL architecture. This enhancement is expected to improve the generalization capability and adaptability of the model across diverse domains.… …
相似文献