---
title: flash-attention-windows-wheel
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/flash-attention-windows-wheel"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/flash-attention-windows-wheel.md"
repository: AI-ModelScope/flash-attention-windows-wheel
last_updated: 2025-07-01
license: bsd-3-clause
library_name:
  - pytorch
frameworks:
  - pytorch
downloads: 214
stars: 1
---

# flash-attention-windows-wheel

> flash-attention-windows-wheel - AI-ModelScope 在 ModelScope 开源的模型。Windows wheels of flash-attention

AI-ModelScope/flash-attention-windows-wheel 是 ModelScope 魔搭社区上的机器学习模型，采用 bsd-3-clause 许可。

- **Repository**: AI-ModelScope/flash-attention-windows-wheel
- **License**: bsd-3-clause
- **Downloads**: 214
- **Stars**: 1
- **Last updated**: 2025-07-01

Source: https://www.modelscope.cn/models/AI-ModelScope/flash-attention-windows-wheel

---

Windows wheels of [flash-attention](https://github.com/Dao-AILab/flash-attention)

Build cuda wheel steps
- First clone code
```
git clone https://github.com/Dao-AILab/flash-attention
cd flash-attention
```
- Switch tag branch, such as `v2.7.0.post2` (you can get latest tag by `git describe --tags` or list all available tags by `git tag -l`)

```
git checkout -b v2.7.0.post2 v2.7.0.post2
```

- Download WindowsWhlBuilder_cuda.bat into `flash-attention`

- To build with MSVC, please open the "Native Tools Command Prompt for Visual Studio". The exact name may depend on your version of Windows, Visual Studio, and cpu architecture (in my case it was "x64 Native Tools Command Prompt for VS 2022".)

  My Visual Studio Installer version

  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/662db5cd34f94a2b82f317b0/nCCBd7_MYr1PrYZOrEnQw.png)

- Switch python env and make sure the corresponding torch cuda version is installed

- Start task
```
# Build with 1 parallel workers (I used 8 workers on i9-14900KF-3.20GHz-RAM64G, which took about 30 minutes.)
# If you want reset workers, you should edit `WindowsWhlBuilder_cuda.bat` and modify `set MAX_JOBS=1`.(I tried to modify it by parameters, but failed)
WindowsWhlBuilder_cuda.bat

# Build with sm80 and sm120
WindowsWhlBuilder_cuda.bat CUDA_ARCH="80;120" 

# Enable cxx11abi
WindowsWhlBuilder_cuda.bat CUDA_ARCH="80;120" FORCE_CXX11_ABI=TRUE
```
- Wheel file will be placed in the `dist` directory