# loftr

**Repository Path**: wangyanling10/loftr

## Basic Information

- **Project Name**: loftr
- **Description**: loftr仓库，做特征点检测的算法
- **Primary Language**: Unknown
- **License**: Apache-2.0
- **Default Branch**: quadtree_attn
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2024-12-19
- **Last Updated**: 2025-05-07

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

For environment and data setup, please refer to [LoFTR](https://github.com/zju3dv/LoFTR).

## Training
The results can be reproduced when training with 8 gpus. Please run the following commands.
```
sh scripts/reproduce_train/indoor_quadtree_ds.sh
```
The parameter top K can be reduced for speeding up. The performance won't drop too much. Please set this parameter in cfg.LOFTR.COARSE.TOPKS.

## Testing
```
sh scripts/reproduce_test/indoor_ds_quadtree.sh
```

## Sample to Test an Image Pair

- Download outdoor weights from this [Github Release](https://github.com/Tangshitao/QuadTreeAttention/releases/tag/QuadTreeAttention_feature_match)

- Run the following command:

```bash
python3 test_one_image_pair_sample.py --weight_path ./outdoor.ckpt \
        --config_path ./configs/loftr/outdoor/loftr_ds_quadtree.py \
        --query_path ./assets/phototourism_sample_images/london_bridge_19481797_2295892421.jpg \
        --ref_path ./assets/phototourism_sample_images/london_bridge_49190386_5209386933.jpg
```

![Feature Matching Sample](./docs/images/feature_matching_result.jpg)



## export onnx

cann中的einsum算子不支持三个输入的场景，在Linear_attention.py中存在此场景，需要做算子替换

```python
v_length = values.size(1)
values = values / v_length  # prevent fp16 overflow
KV = torch.einsum("nshd,nshv->nhdv", K, values)  # (S,D)' @ S,V
Z = 1 / (torch.einsum("nlhd,nhd->nlh", Q, K.sum(dim=1)) + self.eps)
queried_values = torch.einsum("nlhd,nhdv,nlh->nlhv", Q, KV, Z) * v_length
```

einsum是基于爱因斯坦求和约定做矩阵求和操作，核心思想是通过下标标签来描述参与计算的张量维度，并根据这些标签自动确定哪些维度需要相乘并求和。  根据爱因斯坦求和约定原理和torch.einsum算子计算逻辑的描述，计算过程可以拆分为：张量维度对齐->矩阵点乘->转置->根据输出下标做sum求和，一共4步，除了矩阵点乘外其他三个操作是非必须的，由算子的输出下标决定是否使用。

替换后如下

```python
q_shape = Q.shape
kv_shape = KV.shape
z_shape = Z.shape
Qr=Q.reshape(q_shape[0], q_shape[1], q_shape[2], q_shape[3], 1)
KVr=Q.reshape(kv_shape[0], 1, kv_shape[1], kv_shape[2], kv_shape[3])
Zr=Q.reshape(z_shape[0], z_shape[1], z_shape[2], 1)
queried_values = ((Qr * KVr).sum(3) * Zr) *v_length
```

执行下面命令：

```
python3 test_one_image_pair_sample.py --weight_path ./weights/outdoor.ckpt --export_onnx True --config_path ./configs/loftr/outdoor/loftr_ds_quadtree.py  --query_path ./assets/phototourism_sample_images/london_bridge_19481797_2295892421.jpg  --ref_path ./assets/phototourism_sample_images/london_bridge_49190386_5209386933.jpg
```



## onnx infer by onnxruntime

```
python3 test_onnx_infer.py --weight_path ./weights/loftr.onnx  --config_path ./configs/loftr/outdoor/loftr_ds_quadtree.py  --query_path ./assets/phototourism_sample_images/london_bridge_19481797_2295892421.jpg  --ref_path ./assets/phototourism_sample_images/london_bridge_49190386_5209386933.jpg
```