---
title: deepthought-8b-llama-v0.01-alpha
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/deepthought-8b-llama-v0.01-alpha"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/deepthought-8b-llama-v0.01-alpha.md"
repository: AI-ModelScope/deepthought-8b-llama-v0.01-alpha
last_updated: 2024-12-07
license: llama3.1
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - llama
architectures:
  - LlamaForCausalLM
parameters: 8.0B
tensor_type:
  - BF16
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - pytorch
language:
  - en
inference_backends:
  - "deploy_task text/emb"
  - "lmdeploy 0.9.1"
  - "lmdeploy_turbomind 0.9.1"
  - "sglang 0.5.2"
  - "vllm 0.9.2"
downloads: 1234
stars: 0
---

# deepthought-8b-llama-v0.01-alpha

> deepthought-8b-llama-v0.01-alpha - AI-ModelScope 在 ModelScope 开源的模型。Deepthought-8B is a small and capable reasoning model built on LLaMA-3.1 8B, designed to make AI reasoning more transparent and controllable. Despite its relatively small size, it achieves…

AI-ModelScope/deepthought-8b-llama-v0.01-alpha 是 ModelScope 魔搭社区上的 8.0B 参数text-generation模型，采用 llama3.1 许可，可用 deploy_task text/emb、lmdeploy 0.9.1、lmdeploy_turbomind 0.9.1 部署。

- **Repository**: AI-ModelScope/deepthought-8b-llama-v0.01-alpha
- **License**: llama3.1
- **Tasks**: text-generation
- **Parameters**: 8.0B
- **Inference backends**: deploy_task text/emb, lmdeploy 0.9.1, lmdeploy_turbomind 0.9.1, sglang 0.5.2, vllm 0.9.2
- **Downloads**: 1234
- **Stars**: 0
- **Last updated**: 2024-12-07

Source: https://www.modelscope.cn/models/AI-ModelScope/deepthought-8b-llama-v0.01-alpha

---

# Deepthought-8B

Deepthought-8B is a small and capable reasoning model built on LLaMA-3.1 8B, designed to make AI reasoning more transparent and controllable. Despite its relatively small size, it achieves sophisticated reasoning capabilities that rival much larger models.

## Model Description

Deepthought-8B is designed with a unique approach to problem-solving, breaking down its thinking into clear, distinct, documented steps. The model outputs its reasoning process in a structured JSON format, making it easier to understand and validate its decision-making process.

### Key Features

- **Transparent Reasoning**: Step-by-step documentation of the thought process
- **Programmable Approach**: Customizable reasoning patterns without model retraining
- **Test-time Compute Scaling**: Flexible reasoning depth based on task complexity
- **Efficient Scale**: Runs on 16GB+ VRAM
- **Structured Output**: JSON-formatted reasoning chains for easy integration

Try out Deepthought-8B on our Ruliad interface: https://chat.ruliad.co

## Technical Requirements

- Python 3.6+
- PyTorch
- Transformers library
- 16GB+ VRAM
- Optional: Flash Attention 2 for improved performance

## Installation

```bash
pip install torch transformers
# Optional: Install Flash Attention 2 for better performance
pip install flash-attn
```

## Usage

1. First, set your HuggingFace token as an environment variable:
```bash
export HF_TOKEN=your_token_here
export HF_HUB_ENABLE_HF_TRANSFER=1
```

2. Use the model in your Python code:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Initialize the model
model_name = "ruliad/deepthought-8b-llama-v0.01-alpha"
tokenizer = AutoTokenizer.from_pretrained(
    model_name,
    add_bos_token=False,
    trust_remote_code=True,
    padding="left",
    torch_dtype=torch.bfloat16,
)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="flash_attention_2",  # Use "eager" (or omit) if flash_attn is not installed
    use_cache=True,
    trust_remote_code=True,
)
```

3. Run the provided example script:
```bash
python deepthought_inference.py
```

## Example Output

The model provides structured reasoning in JSON format:

```json
{
  "step": 1,
  "type": "problem_understanding",
  "thought": "Understanding the user's objective for the task."
}
```

Each reasoning chain includes multiple steps:
1. Problem understanding
2. Data gathering
3. Analysis
4. Calculation (when applicable)
5. Verification
6. Conclusion drawing
7. Implementation

## Performance

Deepthought-8B demonstrates strong performance across various benchmarks:
- Step-by-step problem-solving
- Coding and mathematical tasks
- Instruction following with transparent reasoning
- Scalable performance with test-time compute

## Limitations

Current known limitations include:
- Complex mathematical reasoning
- Long-context processing
- Edge case handling

## License

The model is available under a commercial license for enterprise use.

## Citation

If you use this model in your research, please cite:

```bibtex
@misc{Deepthought2024,
  author = {Ruliad},
  title = {Deepthought-8B: A Small and Capable Reasoning Model},
  year = {2024},
  publisher = {Ruliad}
}
```

## Support

For questions and feedback:
- Twitter: @ruliad_ai
- Email: team@ruliad.co