---
title: RWKV-4-world-one-state-ultrachat-COT-65k
canonical_url: "https://www.modelscope.cn/models/AI-ModelScope/RWKV-4-world-one-state-ultrachat-COT-65k"
md_url: "https://www.modelscope.cn/models/AI-ModelScope/RWKV-4-world-one-state-ultrachat-COT-65k.md"
repository: AI-ModelScope/RWKV-4-world-one-state-ultrachat-COT-65k
last_updated: 2023-10-23
license: apache-2.0
pipeline_tag: text-generation
tasks:
  - text-generation
library_name:
  - pytorch
frameworks:
  - pytorch
downloads: 17
stars: 2
---

# RWKV-4-world-one-state-ultrachat-COT-65k

> RWKV-4-world-one-state-ultrachat-COT-65k - AI-ModelScope 在 ModelScope 开源的模型。RWKV v4 7B world model finetuned with ultrachat , COT and some novel instructions data, commitpackft and so on

AI-ModelScope/RWKV-4-world-one-state-ultrachat-COT-65k 是 ModelScope 魔搭社区上的text-generation模型，采用 apache-2.0 许可。

- **Repository**: AI-ModelScope/RWKV-4-world-one-state-ultrachat-COT-65k
- **License**: apache-2.0
- **Tasks**: text-generation
- **Downloads**: 17
- **Stars**: 2
- **Last updated**: 2023-10-23

Source: https://www.modelscope.cn/models/AI-ModelScope/RWKV-4-world-one-state-ultrachat-COT-65k

---

# RWKV v4 7B world model 
finetuned with ultrachat , COT and some novel instructions data, commitpackft and so on

use full ultrachat and cot data, about 3B tokens

if you wanna do Role play, use [this model](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/RWKV-world-novel-one-state-ultrachat-cot-tuned-Role-play-65k.pth)


# Contributor
[@JL-er](https://huggingface.co/JL-er) 
[@Remixa](https://huggingface.co/Remixa)


# Design of experiment
this model lose multi-turn chat ability,cause from using whole ultrachat datasets.

so i continue tuned multi-turn datasets with 2 aspects

1.[role play](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/RWKV-world-novel-one-state-ultrachat-cot-tuned-Role-play-65k.pth)

2.[novel multiturn instruction](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/rwkv-world-one-novel-cot-ultrachat-novel-instructions.pth)

# Training details
[wandb.ai](https://wandb.ai/one-/one-rwkv-64k)

# CAses

![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/_1dJo549ldgX6q0JUwC6c.jpeg)

![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/7969wbHaJpBq2n6xvfC7C.jpeg)

# Usage
adjust tempp and topp on different scenario.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/cGDF6b4-x_9rcwMdl1KPp.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/_QduMpcGkCoC00DTlXQop.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/9CqrfEpcJcxtLoX5ffEFo.png)

# COT and lookback

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/hUxTVgjLBMcFqxQX9HoxL.png)
[this model](https://huggingface.co/xiaol/RWKV-4-world-one-state-novel-tuned-65k) can do above task with 100% acc.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/mXyfoD1_jlNqBElQfn_Fk.png)

## role play model

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/2ns794U56M1592w6Uy1dk.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/6CQp2kh56FFa-TmNUrld3.png)

## novel

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/wPXTYWFWfi-mPmoPbmfWg.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/ZgRK8N6jnxR3HawvFiUve.png)
# demo site(temporary)
[online showcase](https://rwkv.ai-creator.net/risu)