© The Author(s), published by EDP Sciences, 2025

1 Introduction

Hyperspectral imagery constitutes three-dimensional data characterized by high spectral resolution and rich spectral information, facilitating precise pixel-level classification. Within such imagery, anomalous targets typically occupy a minimal number of pixels and exhibit spectra markedly distinct from their surrounding background [1–5]. Hyperspectral anomaly detection (HAD) identifies anomalous pixels without prior knowledge of target or background spectral signatures [6–8].

The research on hyperspectral anomaly detection can be divided into traditional and deep learning methods. The Reed-Xiaoli (RX) algorithm [9] is a classic statistical approach among traditional methods; it calculates the anomaly level of pixels by assuming that the background follows a multivariate normal distribution. The local variant of the RX algorithm [10] uses a dual-window to select the background instead of using the entire image, calculating the local anomaly level for each pixel. A weighted method is also used in the RX algorithm [11], which assigns low weights to abnormal samples and can better estimate the background distribution. The two-step generalized likelihood ratio test (2S-GLRT) [12] utilizes design criteria based on the generalized likelihood ratio test and its two-step modification to reduce computational complexity while improving detector performance. The development of compressive sensing theory has led to the use of reconstruction-based methods in HAD. The sparse representation (SR) model [13] reconstructs all pixels using an overcomplete dictionary, while the residuals represent the degree of pixel anomalies. Collaborative representation (CR) [14] is one of the commonly used reconstruction methods, which allows all atoms in the dictionary to participate in linear representation and is a pixel-by-pixel detection method. Low-rank representation (LRR) [15] divides data into sparse backgrounds and anomalies, focusing on describing the global structure of hyperspectral images. The attribute and edge-preserving filtering-based anomaly detection (AED) method [16] utilizes morphological filtering and support vector machines to achieve anomaly detection, integrating spatial and spectral information. With the deepening of research, deep learning-based HAD technology is gradually receiving attention, and its detection framework is divided into supervised and unsupervised approaches. Li et al. [17] proposed a supervised anomaly detection framework based on transferred deep CNN, which trains using differential pixel pairs formed by different categories of ground objects in reference images, with the difference between the central pixel and surrounding pixels as the anomaly level of the central pixel. Stacked autoencoder (SAE) [18] is a typical unsupervised deep learning network that reconstructs hyperspectral images through an encoder-decoder structure. Abnormal targets cannot be accurately identified by restoring their spectra. The Auto-AD [19] method uses full convolutional layers to construct autoencoders, which can suppress abnormal reconstruction. Fan et al. [20] proposed robust graph autoencoders (RGAE) that preserve the geometric structure between training samples and are robust to noise and anomalies during the training process.

Recent years have witnessed increasing adoption of transformer models in HAD for their potent global information extraction. Liu et al. [21] introduced BiGSeT, employing a binary mask-guided separation training strategy to address the identical mapping problem (IMP) via latent binary mask-based loss, effectively suppressing anomalies during background reconstruction. Ma et al. [22] developed a coarse-to-fine multiview anomaly coupling network integrating axial attention for global features and residual CNNs for fine-grained local characteristics, enhancing detection accuracy. Wu et al. [23] proposed a 3D-Transformer network leveraging spatial pixel correlations and spectral band similarities to reduce false alarms in background reconstruction. Xiao et al. [24] devised S2DWMTrans, aggregating global background perspectives while suppressing anomalies via dual-window neighboring pixels. Lian et al. [25] presented GT-HAD, a gated Transformer exploiting spatial-spectral similarity with dual-branch modeling and adaptive gating units. Wang et al. [26] established a self-supervised method using finite spatialwise attention (FSA) to mine background spectral clustering structures. Zhang et al. [27] created a differential network combining local punctured neighborhood extraction with detail and Transformer attention mechanisms. Li et al. [28] proposed an explicit background endmember learning framework (EBEL), which learned background endmember features using a cross-attention mechanism and represented hyperspectral image features through a cascaded cross-attention-based module, thereby improving anomaly detection performance. He et al. [29] proposed a convolutional Transformer-inspired autoencoder (CTA), which combined convolutional and Transformer architectures to capture both local and global features, thereby improving anomaly detection performance. Liu et al. [30] proposed SSHAD, an adaptive dual-domain learning method for HAD with state-space models. This method combined a spatial-wise selected state space module (SSSM) and a spectral-wise frequency division self-attention module (FDSM) to capture both spatial sparsity and inter-spectral similarity in hyperspectral images, thereby improving detection performance. Sun et al. [31] introduced CSSBR, a contrastive self-supervised learning-based background reconstruction method, which employed a pixel- and patch-level masking strategy and a dual-attention network (DAN) encoder to learn universal background representations without labeled samples. Guo et al. [32] proposed SSCMAE, a spatial-spectral cross-guided masked autoencoder, which used a guided mask based on spectral differences to suppress anomaly reconstruction and enhance background reconstruction accuracy. Additional contributions include: Spectrum Difference Enhanced Network (SDENet) [33] amplifying spectral discrepancies through variational mapping and Transformer; Transformer-Based Autoencoder Framework (TAEF) [34] modeling nonlinear mixing via autoencoder-Transformer integration; TGFA-AD [35] reconstructing abundance matrices with Transformer encoders and fractional convolution; adversarial refinement-based inpainting methods [36]; and the “train-once” anomaly enhancement network [37]. These studies collectively demonstrate the significant advancements in HAD enabled by deep learning and Transformer architectures. Each method emphasizes different aspects of feature extraction, background reconstruction, and anomaly suppression, contributing to the ongoing development of this field [38, 39].

With the rapid development of detector technology and UAV technology, handheld and UAV imaging spectrometers have been widely used, resulting in a large number of handheld/UAV spectrometer data [40]. This kind of hyperspectral image has the characteristics of “double high resolution”, that is, high spatial resolution and high spectral resolution, and the acquisition angle is variable. Data are often used immediately without various corrections, resulting in greater spectral variability and difficulties in extracting deep spectral features [41]. Due to the close data collection distance and high spatial resolution, the spatial distribution of the image is complex and large-scale continuous features may no longer exist (for example, leaves, branches, tree trunks, and grass may alternate in the space in the jungle). We hope to observe the performance of the proposed method on this type of data, so the data in this article are all private near ground data taken from near ground sources [42].

We propose a dual-window transformer framework with pyramid structure and constrained self-attention for anomaly detection in hyperspectral images. This framework only learns the differences in pixels between the center pixel and the dual-window. The selection of dual-window size is an unavoidable issue for all “window methods”. We construct an image pyramid by spatially sampling hyperspectral images, which are preprocessed and input layer by layer into the constrained self-attention transformer module. The output results are spatially upsampled as “intrinsic constraints” and fed back to the next layer’s constrained self-attention module along with “external constraints”, greatly reducing the impact of window size on detection results. External constraints are generated by SAE, which can enhance the robustness of the overall framework and provide information for the constrained self-attention transformer module at the first layer runtime.

The main contributions of the method proposed in this article can be summarized as follows:

1. We propose a dual-window transformer framework with pyramid structure and constrained self-attention. The proposed framework utilizes a pyramid structure to weaken the impact of window size on model performance.

2. The dual-window model can extract local information from hyperspectral images and fuse it with the global information feedback from the pyramid structure to the upper constrained self-attention transformer module, effectively detecting abnormal targets in hyperspectral images.

3. We conducted experimental verification using real hyperspectral images and compared them with state-of-the-art methods, ultimately proving the effectiveness of our proposed method.

2 Preliminary

2.1 Multi-head attention

The Transformer is a deep learning model proposed by Vaswani et al. [43] for natural language processing; it processes all elements in a sequence in parallel and can efficiently and quickly capture long-term dependencies in text. The Vision Transformer (ViT) uses the encoder module from the Transformer architecture for image classification tasks by segmenting images into patches and flattening them. This approach transforms the image recognition problem into a “sequence-to-sequence” problem [44]. The core of the Transformer encoder is the multi-head attention (MSA) mechanism, where each self-attention (SA) corresponds to a set of matrices W_q, W_k, and W_v, which are multiplied by the input X of SA to obtain the query Q, key K, and value V as follows:$$\begin{array}{ccc}\mathbf{Q}=\mathbf{X}{\mathbf{W}}_{\mathbf{q}},& \mathbf{K}=\mathbf{X}{\mathbf{W}}_{\mathbf{k}},& \mathbf{V}=\mathbf{X}{\mathbf{W}}_{\mathbf{V}}.\end{array}$$(1)

The corresponding output of SA is:$$\mathbf{SA}=\mathrm{attention} \left(\mathbf{Q}, \mathbf{K}, \mathbf{V}\right)=\mathrm{softmax}\left(\frac{\mathbf{Q}{\mathbf{K}}^{\mathrm{T}}}{\sqrt{d}}\right)\mathbf{V}.$$(2)

The product of Q and K ^T represents similarity, while V can be seen as a vector of a single input feature. Figure 1 illustrates the computation of multi-head attention as implemented in the actual code. It consists of multiple self-attention mechanisms. The input X is split into segments, and the corresponding self-attention is computed based on equations (1) and (2). These are then concatenated to obtain Z, which satisfies the following equation:$$\mathbf{Z}=\mathrm{MSA} \left(\mathbf{Q}, \mathbf{K}, \mathbf{V}\right)=\mathrm{concat}\left({\mathbf{SA}}_1, {\mathbf{SA}}_2, \mathrm{L}, {\mathbf{SA}}_h\right).$$(3)where h is the number of self-attention groups.

Fig. 1 The calculation process of multi-head attention.

2.2 Stacked autoencoder

The Autoencoder is an unsupervised neural network model designed to learn deep features of data and reconstruct the input. Essentially, it is a three-layer neural network comprising an input layer, a hidden layer, and a reconstruction layer [45]. Figure 2 shows the structure of a stacked Autoencoder, which is a symmetric multi-layer neural network formed by stacking multiple Autoencoders. The output of the lth layer in the stacked Autoencoder is:$${\mathbf{y}}^{\left(l\right)}=f\left[{\mathbf{W}}^{\left(l\right)}{\mathbf{y}}^{\left(l-1\right)}+{\mathbf{b}}^{\left(l\right)}\right].$$(4)

Fig. 2 Diagram of stacked autoencoder (SAE).

3 Proposed method

In this section, we specifically introduce the proposed dual-window transformer framework with pyramid structure and Constrained self-attention. Figure 3 shows the framework of the proposed method, a pyramid structure where the final result of each layer’s operation provides feedback for the next layer. The hyperspectral images of each scale in the pyramid are entered into Dual window Preprocessing and Constrained self-attention transformer module (CSATM) in order of scale. The framework’s core is the CSATM, whose weight of self-attention is Constrained by the upper layer of the pyramid. The framework consists of four parts:

1. Pyramid construction: Hyperspectral images are downsampled step by step spatially and input into subsequent modules. After completing all subsequent modules of a certain scale hyperspectral image, input the next scale hyperspectral image.

2. Dual-window preprocessing: The pixel-by-pixel moving dual-windows extract the hyperspectral image into a set of “center pixels – pixels within the dual-window”, and process the set into a form that can be received by CSATM. The center pixel and background pixel of the dual window at a certain position form a training sample.

3. CSATM: The core part of the framework is to learn spectral depth differences and provide pixel-level anomaly score maps.

4. Constraints generation module (CGM): Integrate intrinsic constraints and external constraints into self-attention constraints, and input into CSATM.

Fig. 3 Framework of the proposed method.

3.1 Image pyramid

Input the hyperspectral image ${\mathbf{Y}}^{\left(0\right)}\in {\mathbb{R}}^{\mathrm{H}\times \mathrm{W}\times \mathrm{B}}$ and perform progressive spatial downsampling at a ratio of 1/2 to obtain Y ⁽⁰⁾, Y ⁽²⁾, and Y ⁽³⁾. Y ⁽³⁾ will enter the subsequent modules as the input for the first layer, and after the intrinsic constraints and external constraints of the first layer are fed back to CSATM, Y ⁽²⁾ will be inputted. The same operation is used for Y ⁽¹⁾ and Y ⁽⁰⁾.

3.2 Dual-window preprocessing

Take the input Y ⁽⁰⁾ of the fourth layer as an example. Y ⁽⁰⁾ performs boundary padding to ensure normal processing of boundaries by the dual-window and is then extracted as a set of “center pixels – dual-window inner pixels” by the dual-window with outer window size win_out and inner window size win_in. The set is represented as:$$\mathrm{\Omega }=\left\{\left({\mathbf{P}}_j^{*},{\mathbf{P}}_j^i\right)\begin{array}{c}i=\mathrm{1,2}, \dots , N\\ i=\mathrm{1,2}, \dots , M\end{array}\right\}$$(5)where $N=\mathrm{wi}{\mathrm{n}}_{\mathrm{out}}^2-\mathrm{wi}{\mathrm{n}}_{\mathrm{in}}^2$, M = H × W.

The purpose of the overall framework is to learn the level of difference between the central pixel and the surrounding background, and the required residuals are obtained as follows:$${\mathbf{\epsilon }}_j^i={\mathbf{P}}_j^{*}-{\mathbf{P}}_j^i.$$(6)

In this section, the hyperspectral image Y ⁽⁰⁾ ∈ ℝ ^{H × W × B} is converted into ε  ∈ ℝ ^{M × N × B} , which can be processed by CSATM.

3.2 CSATM

The differential data ε ∈ ℝ ^{M × N × B} is encoded as deep features in this module, and the anomaly score is finally obtained. The fully connected layer in the Embedded module performs a linear transformation on ε and adds positional information. The encoded vector is passed through a constrained transformer encoder and finally processed by Mean and MLP to obtain the anomaly score S. The forward process is as follows:$$\begin{array}{cc}{\mathbf{z}}_0=\mathbf{\epsilon }\mathbf{E}+{\mathbf{E}}_{\mathbf{pos}}, & \mathbf{E}\in {\mathbb{R}}^{B\times D},{\mathbf{E}}_{\mathbf{pos}}\in {\mathbb{R}}^{M\times N\times D}\\ {\mathbf{Z}}_t^{\text{'}}=\mathrm{CMSA}\left(\mathrm{LN}\left({\mathbf{Z}}_{t-1}\right)\right)+{\mathbf{Z}}_{t-1}, & t=\mathrm{1,2},\dots ,L\\ {\mathbf{Z}}_t=\mathrm{MLP}\left(\mathrm{LN}\left({\mathbf{Z}}_t^{\text{'}}\right)\right)+{\mathbf{Z}}_t^{\text{'}},& t=\mathrm{1,2},\dots ,L\\ \mathbf{S}=\mathrm{MLP}\left(\mathrm{Mean}\left(\mathrm{LN}\left({\mathbf{Z}}_t\right)\right)\right)+{\mathbf{Z}}_t^{\text{'}}& \mathbf{S}\in {\mathbb{R}}^M.\end{array}$$(7)

Where CMSA(x) = concat(CSA₁, CSA₂, …, CSA _h ). The new weights obtained by adding constraints to self-attention weights are as follows:$$\mathbf{CSA}=\mathrm{Softmax}\left[\mathbf{Q}{\mathbf{K}}^T\mathrm{Norm}\left({\mathbf{\gamma }\mathbf{\gamma }}^T\right)\right]\mathbf{V},$$(8) $$\mathbf{\gamma }=\mathrm{Flattening}\left({\mathbf{\chi }}_{\mathrm{I}}+{\mathbf{\chi }}_{\mathrm{E}}\right),$$(9) $${\mathbf{\chi }}_{\mathrm{I}}=\left\{{\mathbf{x}}_{\mathrm{I}}^i{\mathbf{x}}_{\mathrm{I}}^i={\mathbf{x}}_{\mathrm{I}}^{*}-{\mathbf{x}}_{\mathrm{I}}^i\right\}, {\mathbf{\chi }}_{\mathrm{A}}=\left\{{\mathbf{x}}_{\mathbf{E}}^i\right|{\mathbf{x}}_{\mathrm{E}}^i={\mathbf{x}}_{\mathrm{E}}^{*}-{\mathbf{x}}_{\mathrm{E}}^i\}.$$(10)where ${\mathbf{x}}_{\mathrm{I}}^{*}$ is the value corresponding to the central pixel in the intrinsic constraints; ${\mathbf{x}}_{\mathrm{I}}^i(i=\mathrm{1,2},\dots , \mathrm{N})$ is the value corresponding to pixels within the dual-window in the intrinsic constraints; ${\mathbf{x}}_{\mathrm{E}}^{*}$ is the value corresponding to the central pixel in the external constraints; ${\mathbf{x}}_{\mathbf{E}}^i(i=\mathrm{1,2},\dots , N)$ is the value corresponding to the pixels within the double window in the external constraints. Figure 4 shows the positions of ${\mathbf{x}}_{\mathrm{I}}^{*}$ and ${\mathbf{x}}_{\mathrm{I}}^i$, with the same distribution of external constraints. ${\mathbf{x}}_{\mathrm{I}}^i$ is the central pixel to be tested, and ${\mathbf{x}}_{\mathrm{I}}^{*}$ is the surrounding background pixel.

Fig. 4 Diagram of intrinsic constraints.

3.4 GCM

The results of each layer will be input into CSATM as intrinsic constraints. The training of the first layer does not provide intrinsic constraints for the previous layer. We stagger the number of layers of intrinsic constraints and external constraints and provide the first layer of external constraints to the first layer of CSATM. External constraints obtained from SAE represent the addition of global information, which can make the model more robust. Algorithm 1 briefly describes the acquisition of additional constraints, taking Y⁽⁰⁾ as an example.

The hyperspectral image is input into SAE, but the anomaly cannot be well reconstructed by the network due to its unique characteristics. Therefore, the residual between the original image and the reconstructed image can reflect the location of the anomaly. Apply the RX algorithm on residuals to obtain external constraints. The result ${\mathbf{T}}^{\left(3\right)}\in {\mathbb{R}}^{\frac{H}{8}\times \frac{W}{8}}$ generated by the first layer is spatially upsampled twice to obtain the first intrinsic constraint, which is fed back to CSATM along with the second external constraint ${\mathbf{S}}^{\left(2\right)}\in {\mathbb{R}}^{\frac{H}{4}\times \frac{W}{4}}$. When running to the last layer, the weighted average of the intrinsic constraint ${\mathbf{T}}^{\left(0\right)}\in {\mathbb{R}}^{H\times W}$ and the last external constraint ${\mathbf{S}}^{\left(0\right)}\in {\mathbb{R}}^{H\times W}$ is used as the final result.

4 Experimental results and analysis

4.1 Datasets and experimental setup

This article uses five real hyperspectral data for experiments, and their pseudocolor images and the corresponding ground-truth maps are shown in Figure 5. The spectral component of the hyperspectral imager is Acousto-Optic Tunable Filter (AOTF), and we have set the wavelength range to 449–801 nm, which includes 89 bands with band intervals of 4 nm.

1. Meadow: The image resolution is 1000 × 400, with fake turf-A placed as the abnormal target, and the background includes building walls and lawns.

2. Holly: The image resolution is 600 × 496, with fake turf-B placed as the abnormal target, and the background includes holly flower beds and architectural shadows. Meadow and Holly were both filmed on May 26, 2022, with geographic coordinates (38°27′W, 114°30′E) and a ground spatial resolution of approximately 8.4 mm, both of which belong to typical urban near ground scenes.

3. Cement Street: The image resolution is 500 × 300. Filmed on April 6, 2024, with geographic coordinates (38°27′W, 114°30′E) and a ground spatial resolution of 1.5 mm, it belongs to a cement road background and targets two alloy aircraft models.

4. Jungle-I and Jungle-II: Filmed on April 6, 2024, with geographic coordinates (38°27′W, 114°30′E) and a ground spatial resolution of 3.3 mm, all of which belong to a close-range jungle background, and the image resolution is 1000 × 1000. Different types of fake turf-C and fake turf-D were placed in Jungle-I and Jungle-II. Meadow serves as the training set for the entire framework.

Fig. 5 Pseudocolor images (left) and the corresponding ground-truth maps (right) of five real hyperspectral datasets. (a) Meadow; (b) Holly; (c) Cement Street; (d) Jungle-I; (e) Jungle-II.

All experiments were implemented on the Ubuntu system using PyTorch 1.8 and Python 3.8, with two GeForce GRX 2080 GPUs included. Table 1 shows the model parameter settings for the Transformer module and SAE in the proposed framework. Meadow will serve as the training set for the framework.

4.2 Evaluation metrics

There are objective and subjective analyses of the results of HAD. The subjective analysis mainly involves decision-makers combining ground-truth maps to visually analyze the detection results. The most commonly used methods in objective analysis are receiver operating characteristics (ROC) and area under the curve (AUC). The ROC curve establishes a correlation between the probability of false alarm (PFA) and the probability of detection (PD), which is based on a common threshold τ. The traditional two-dimensional ROC curve is essentially composed of τ-PFA relationship curve and τ-PD relationship curve. The definitions of PD and PFA are as follows:$$\mathrm{PD}=\frac{N_{\mathrm{d}}}{N_{\mathrm{t}}},\mathrm{FPA}=\frac{N_{\mathrm{f}}}{N_{\mathrm{b}}}.$$(11)where N_d is the number of real target pixels detected, which is the number of pixels that belong to the target and are considered the target by the detector, N_t represents the total number of target pixels in the image. N_f represents the number of detected false alarm pixels, which belong to the background but are considered targets by the detector. N_b represents the total number of background pixels in the image.

AUC quantitatively describes the degree of deviation of the ROC curve to the upper left, which is the area formed by the ROC curve surrounded by the horizontal axis. The values are as follows:$$\mathrm{AUC}={\int }_0^1\mathrm{ROC}\left(x\right)\mathrm{d}x.$$(12)

The greater the offset of the ROC curve to the upper left, the larger the area below the curve line, and the larger the AUC, indicating better detection performance. The flatter the ROC curve, the smaller the area below the curve, and the smaller the AUC, the worse the detection effect.

4.3 Hyperparameter analysis

The framework proposed in this article has some pre-set hyperparameters. This section will analyze the impact of hyperparameters on the framework through ablation experiments and select appropriate hyperparameters to achieve optimal detection performance.

4.3.1 The impact of dual-window size on the model

Set the range of the outer window size win_out to [3, 5, 7] and the range of the inner window size win_in to [1, 3, 5]. Different win_in and win_out will be combined into six sets of dual-window parameters. In this section of the experiment, the heads (number of self-attention) of CASTM are set to 1, and the depth (number of encoder blocks) is set to 5. The experiment will be conducted on the remaining four datasets, with AUC values as the evaluation metric. Table 2 shows the detection results for different dual-window sizes. On each dataset, the difference between the highest and lowest AUC is less than 0.01, indicating that changes in the dual-window parameter group have a very small impact on the detection performance of the model. Based on this conclusion, when win_out is fixed, the influence of the thickness (th = win_out − win_in − 1) of the double window on the detection results can be ignored. Figure 6 shows the detection results for each dataset at th = 1 for different dual-window. Holly and Jungle-I showed mild performance, while Cement Street and Jungle-II changed when the dual-window reached (9, 11), but the overall change was not significant. We take the dual-window parameter group (9, 11) with the highest average AUC value from the four datasets as the optimal parameter.

Fig. 6 AUC under different dual-windows when heads = 1.

4.3.2 The impact of the number of heads and depths on the model

The number of heads affects the model’s ability to capture differences across the entire spectral band, and an appropriate number of heads should be considered. The depth of the model represents the number of encoder blocks, which affects the model’s ability to extract features based on the difference between pixels in the dual-window and the center pixel. In this section of the experiment, we set the dual-windows to (9, 11). Figure 7 shows the detection results of each dataset under different numbers of heads at depth = 5. The experiment shows that the model performs best when heads = 6. Figure 8 shows the detection results of each dataset at different depths when heads = 6, with the best performance achieved at depth = 5.

Fig. 7 AUC under different numbers of heads when depth = 5.

Fig. 8 AUC under different depths when heads = 6.

In the end, we chose heads = 6, depth = 5, and (win_in, win_out) = (9, 11) as the optimal hyperparameter group for the framework.

4.4 Ablation experiment

This section will discuss the role and effectiveness of the proposed constrained self attention transformer module in the framework. The experiment will test the anomaly detection performance of traditional self attention and constrained self attention on various datasets. Table 3 shows the results of the experiment. Although CSATM performs slightly worse than MSA on the Cement Street dataset, their detection results have AUC values close to 1, and the difference can be ignored. More attention should be paid to the performance on the other three datasets. The performance of CSATM on Holly, Jungle-I, and Jungle-II datasets has significantly improved compared to MSA, indicating that CSATM has stronger feature extraction ability and detection performance. The main reason for the excellent performance of CSATM is that it obtains additional information based on MSA, including local anomaly information at the downsampling scale and global anomaly information provided by SAE. These additional information enable CSATM to adaptively adjust its self-attention weights and improve the performance of the model.

4.5 Comparative experiment

In order to effectively analyze and evaluate our proposed detection framework, the comparison methods we use include:

1. RX [4]: Assumes background adherence to Gaussian distribution. Estimates the probability density function via background covariance and mean vectors, defining anomaly level as test pixel deviation from background statistical distribution.

2. 2S-GLRT [7]: Adaptive detector for Gaussian backgrounds with unknown covariance matrices, implementing generalized likelihood ratio testing.

3. CRD [9]: Sparse representation-based detection method constructing dictionaries to represent targets/backgrounds. Represents target signals as linear combinations of dictionary atoms, separating targets/backgrounds through sparse coefficient optimization.

4. RGAE [15]: Graph-regularized autoencoder framework maintaining geometric sample structures and demonstrating noise/anomaly robustness during training.

5. Auto-AD [14]: Skip-connected fully convolutional autoencoder employing adaptive weighted loss to suppress abnormal pixel contributions during reconstruction.

6. CTA [24]: Integrates K-means clustering (extracting pseudo-background/anomalous samples) with convolutional Transformer autoencoders. Combines convolutional layers and multi-head attention to capture local-global features, enhancing anomaly-background separability.

7. GT-HAD [20]: Leverages spatial-spectral similarity during reconstruction with du-al-branch modeling (background/anomaly features) under content similarity constraints. Incorporates adaptive gating units via Content Matching Method (CMM) for branch activation control.

8. TAEF [30]: Embeds Extended Multilinear Mixture Model (EMLM) in decoder while utilizing Transformer-based encoder. Effectively reconstructs HSI backgrounds and characterizes high-order nonlinear mixing phenomena.

The setting with hyperparameters in the comparison method is:

1. 2S-GLRT: The size of the dual-window (win_in, win_out) = (5, 7).

2. CRD: The size of the dual-window (win_in, win_out) = (3, 11), Lagrange Multiplier λ = 0.1.

3. RGAE: Tradeoff Parameter λ = 10-2, Number of Superpixels S = 150, Dimension of Hidden Layer n_hi d = 20.

Figures 9–12 shows the detection results of each method on the Holly, Cement Street, Jungle-I, and Jungle-II datasets. From the overall detection results, traditional detection methods (RX, 2S-GLRT, and CRD) have weak detection performance, and their main characteristics are: 1) low target anomaly scores; 2) a large number of target pixels are missed and scattered; 3) a large number of false alarms in the detection results of the Cement Street dataset. The reason for these phenomena is that hyperspectral images captured near the ground have strong spectral uncertainty, and the phenomenon of “same substance but different spectrum, same spectral foreign objects” is more obvious. Traditional detection methods cannot extract deep spectral features, which can easily lead to missed detections and false alarms. The detection performance of RGAE, Auto-AD, GT-HAD, and TAEF is basically the same, but the detection rate of Jungle-I and Jungle-II data sets is not high. These models are all based on reconstruction based deep learning methods. The background of the Holly and Cement Street datasets is relatively simple, and these methods can effectively reconstruct the background to distinguish anomalous targets. The backgrounds of the Jungle-I and Jungle-II datasets are relatively complex, and the performance of reconstruction based methods on both datasets has significantly decreased. The detection results of CTA are not ideal, except for detecting edges in the Cement Street dataset, no targets are detected in the other three datasets. The training sample quality and model performance of CTA depend on its clustering-based module. The hyperspectral dataset we provide has a large spatial size and contains a large number of spectral training samples. The clustering-based module cannot effectively partition the anomaly and background sets, resulting in rapid model degradation. The proposed method has good integrity in detecting targets in four datasets, and its anomaly score is high, but there are some false alarms in the detection results. Figure 13 shows the ROC curves and log scale ROC curve of the detection results of each method on the Holly, Cement Street, Jungle-I, and Jungle-II datasets. 2S-GLRT has little detection effect on near-ground datasets. CTA is very accurate in distinguishing the background, but there are also issues with undetectable or incomplete detection of targets. Our proposed method performs the best overall, but its performance on Cement Streets is not as good as RGAE, Auto-AD, GT-HAD, and TAEF. Table 4 shows the AUC values of the ROC curves for the detection results of each method on the Holly, Cement Street, Jungle-I, and Jungle-II datasets. Holly and Cement Street have relatively simple backgrounds and generally perform well compared to the methods used, while Jungle-I and Jungle-II are close-range jungle backgrounds with complex backgrounds, resulting in a decrease in their detection performance. Our proposed method has the highest average AUC, but the performance varies on different datasets. Jungle-I and Jungle-II have the same background, but the proposed method has slightly different detection performance. The SAE in the proposed method is based on a reconstructed model, which adjusts the self attention weights together with the global anomaly information provided by the upper layer of the pyramid, resulting in better model performance than the reconstruction method. The experiments in this section have effectively demonstrated the superiority of the proposed method.

Fig. 9 The detection results of different methods on the dataset (Holly). (a) RX; (b) 2S-GLRT; (c) CRD; (d) RGAE; (e) Auto-AD; (f) CTA; (g) GT-HAD; (h) TAEF; (i) Proposed method; (j) Ground-truth maps.

Fig. 10 The detection results of different methods on the dataset (Cement Street). (a) RX; (b) 2S-GLRT; (c) CRD; (d) RGAE; (e) Auto-AD; (f) CTA; (g) GT-HAD; (h) TAEF; (i) Proposed method; (j) Ground-truth maps.

Fig. 11 The detection results of different methods on the dataset (Jungle-I). (a) RX; (b) 2S-GLRT; (c) CRD; (d) RGAE; (e) Auto-AD; (f) CTA; (g) GT-HAD; (h) TAEF; (i) Proposed method; (j) Ground-truth maps.

Fig. 12 The detection results of different methods on the dataset (Jungle-II). (a)RX; (b) 2S-GLRT; (c) CRD; (d) RGAE; (e) Auto-AD; (f) CTA; (g) GT-HAD; (h) TAEF; (i) Proposed method; (j) Ground-truth maps.

Fig. 13 ROC curve(left) and log scale ROC curve (right) of detection results. (a) Holly. (b) Cement Street. (c) Jungle-I. (d) Jungle-II.

5 Conclusion

This paper proposes a novel dual-window transformer framework featuring a pyramid structure and constrained self-attention for hyperspectral image anomaly detection. The core innovation lies in its ability to extract deep spectral differences between central pixels and their surrounding background through the dual-window transformer. A specifically designed constrained self-attention mechanism enhances this process. Furthermore, the pyramid structure integrates global contextual information from its upper layers into the CASTM, achieving an effective fusion of local and global information. This hierarchical design also significantly mitigates the impact of dual-window size selection on the model’s performance, thereby improving the framework’s overall robustness. Comprehensive parameter analyses across four collected datasets validate the robustness of the proposed method. Comparative experiments demonstrate that our method is particularly well-suited for hyperspectral imagery characterized by complex scenes and significant spectral uncertainty, achieving a higher detection rate. These results confirm the method’s rationality and superiority. Essentially, our approach represents a hybrid of dual-window training-based and reconstruction-based paradigms, successfully integrating both global and local anomalous information. This integration offers a novel and effective strategy for enhancing HAD performance. The proposed method shows great promise for practical applications, such as large-area reconnaissance and preliminary screening prior to precise target identification. Future work will focus on two primary objectives: reducing the false alarm rate and further improving the computational efficiency of the framework.

Funding

This research did not receive any specific funding.

Conflicts of interest

The authors declare that they have no competing interests to report.

Data availability statement

Some hyperspectral images used in this paper are public data sets commonly used in the field, which can be obtained from the Internet according to the description of the data set in this paper. The remaining dataset was taken by the author themselves and can be obtained or inquired about through email.

Author contribution statement

All authors have reviewed, discussed, and agreed to their personal contributions. The specific situation is as follows: Conceptualization, Bing Zhou; Methodology, Algorithms, Analysis and Writing, Lei Deng; Data visualization and Review, Zhaorui Li.

References

All Tables

All Figures

	Fig. 1 The calculation process of multi-head attention.
In the text

	Fig. 2 Diagram of stacked autoencoder (SAE).
In the text

	Fig. 3 Framework of the proposed method.
In the text

	Fig. 4 Diagram of intrinsic constraints.
In the text

	Fig. 5 Pseudocolor images (left) and the corresponding ground-truth maps (right) of five real hyperspectral datasets. (a) Meadow; (b) Holly; (c) Cement Street; (d) Jungle-I; (e) Jungle-II.
In the text

	Fig. 6 AUC under different dual-windows when heads = 1.
In the text

	Fig. 7 AUC under different numbers of heads when depth = 5.
In the text

	Fig. 8 AUC under different depths when heads = 6.
In the text

	Fig. 9 The detection results of different methods on the dataset (Holly). (a) RX; (b) 2S-GLRT; (c) CRD; (d) RGAE; (e) Auto-AD; (f) CTA; (g) GT-HAD; (h) TAEF; (i) Proposed method; (j) Ground-truth maps.
In the text

	Fig. 10 The detection results of different methods on the dataset (Cement Street). (a) RX; (b) 2S-GLRT; (c) CRD; (d) RGAE; (e) Auto-AD; (f) CTA; (g) GT-HAD; (h) TAEF; (i) Proposed method; (j) Ground-truth maps.
In the text

	Fig. 11 The detection results of different methods on the dataset (Jungle-I). (a) RX; (b) 2S-GLRT; (c) CRD; (d) RGAE; (e) Auto-AD; (f) CTA; (g) GT-HAD; (h) TAEF; (i) Proposed method; (j) Ground-truth maps.
In the text

	Fig. 12 The detection results of different methods on the dataset (Jungle-II). (a)RX; (b) 2S-GLRT; (c) CRD; (d) RGAE; (e) Auto-AD; (f) CTA; (g) GT-HAD; (h) TAEF; (i) Proposed method; (j) Ground-truth maps.
In the text

	Fig. 13 ROC curve(left) and log scale ROC curve (right) of detection results. (a) Holly. (b) Cement Street. (c) Jungle-I. (d) Jungle-II.
In the text