Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.9.8 NPU 驱动支持 release v1.0.1 吗 #154

Open
ZIFENG278 opened this issue Jan 10, 2025 · 4 comments
Open

0.9.8 NPU 驱动支持 release v1.0.1 吗 #154

ZIFENG278 opened this issue Jan 10, 2025 · 4 comments

Comments

@ZIFENG278
Copy link

在使用 0.9.8 NPU 驱动下使用 v1.0.1 的代码,在加载模型时会报错

rkllm init start
rkllm-runtime version: 1.0.1

Query platform error!
@waydong
Copy link
Collaborator

waydong commented Jan 13, 2025

Hi, 请尝试使用1.1.4最新版本。

@ZIFENG278
Copy link
Author

v1.0.1 下会自动统计 token/s, 但是 v1.1.4 把这个统计功能去掉了,请问如何才能计算这些值

@waydong
Copy link
Collaborator

waydong commented Jan 13, 2025

v1.0.1 下会自动统计 token/s, 但是 v1.1.4 把这个统计功能去掉了,请问如何才能计算这些值

export RKLLM_LOG_LEVEL=1,会打印性能日志

@ZIFENG278
Copy link
Author

我在 v1.1.4 example 下的 test.py 转换 qwen1,8b, data_quant.json 是用 chatgpt 生成的100 条,编译用的是 llm_demo.cpp 没改动,但是在 3588 上的效果特别差, 甚至不回答,我应该怎么修复呢,在 1.0.1 好像都没什么问题

rock@rock-5b-plus:~/ssd/rkllm$ ./llm_demo ./qwen.rkllm 1024 1024
rkllm init start
I rkllm: rkllm-runtime version: 1.1.4, rknpu driver version: 0.9.8, platform: RK3588

rkllm init success

**********************可输入以下问题对应序号获取回答/或自定义输入********************

[0] 把下面的现代文翻译成文言文: 到了春风和煦,阳光明媚的时候,湖面平静,没有惊涛骇浪,天色湖光相连,一片碧绿,广阔无际;沙洲上的鸥鸟,时而飞翔,时而停歇,美丽的鱼游来游去,岸上与小洲上的花草,青翠欲滴。
[1] 以咏梅为题目,帮我写一首古诗,要求包含梅花、白雪等元素。
[2] 上联: 江边惯看千帆过
[3] 把这句话翻译成中文: Knowledge can be acquired from many sources. These include books, teachers and practical experience, and each has its own advantages. The knowledge we gain from books and formal education enables us to learn about things that we have no opportunity to experience in daily life. We can also develop our analytical skills and learn how to view and interpret the world around us in different ways. Furthermore, we can learn from the past by reading books. In this way, we won't repeat the mistakes of others and can build on their achievements.
[4] 把这句话翻译成英文: RK3588是新一代高端处理器,具有高算力、低功耗、超强多媒体、丰富数据接口等特点

*************************************************************************


user: 你是谁
robot: 慢慢的是谁



user: 3
把这句话翻译成中文: Knowledge can be acquired from many sources. These include books, teachers and practical experience, and each has its own advantages. The knowledge we gain from books and formal education enables us to learn about things that we have no opportunity to experience in daily life. We can also develop our analytical skills and learn how to view and interpret the world around us in different ways. Furthermore, we can learn from the past by reading books. In this way, we won't repeat the mistakes of others and can build on their achievements.
robot: 

user: 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants