---
title: keyfan-grok-1-hf
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/keyfan-grok-1-hf"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/keyfan-grok-1-hf.md"
repository: AI-ModelScope/keyfan-grok-1-hf
last_updated: 2024-03-25
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - grok
architectures:
  - GrokForCausalLM
library_name:
  - pytorch
frameworks:
  - pytorch
downloads: 4082
stars: 0
---

# keyfan-grok-1-hf

> keyfan-grok-1-hf - AI-ModelScope 在 ModelScope 开源的模型。Unofficial dequantized weight of grok-1 in HF Transformers format.

AI-ModelScope/keyfan-grok-1-hf 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可。

- **Repository**: AI-ModelScope/keyfan-grok-1-hf
- **License**: apache-2.0
- **Tasks**: text-generation
- **Downloads**: 4082
- **Stars**: 0
- **Last updated**: 2024-03-25

Source: https://www.modelscope.cn/models/AI-ModelScope/keyfan-grok-1-hf

---

Unofficial dequantized weight of [grok-1](https://huggingface.co/xai-org/grok-1) in HF Transformers format.

Note: If you haven't download the weight yet, please use the `fp32` revision instead which uses float32 precision for RMSNorm and Router layer for better consistency.

The (fp32) weights are converted using the [script here](https://gist.github.com/chu-tianxiang/ec310e15d56949fd0f351cb5f65ee7a1) ran inside the [grok-1 repo](https://github.com/xai-org/grok-1). Since downloading the dequantized weight needs twice as much time, it's recommended to download the original weight and convert on your own.

#### Benchmarks 
(I ran with `load_in_8bit` using [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) due to limited hardware, so the result will be slightly worse)
* MMLU 5-shot: 0.7166
* BBH 3-shot: 0.5204