Toggle navigation
Toggle navigation
This project
Loading...
Sign in
万朱浩
/
Venue-Ops
Go to a project
Toggle navigation
Projects
Groups
Snippets
Help
Toggle navigation pinning
Project
Activity
Repository
Pipelines
Graphs
Issues
0
Merge Requests
0
Wiki
Network
Create a new issue
Builds
Commits
Authored by
戒酒的李白
2025-08-04 14:06:45 +0800
Browse Files
Options
Browse Files
Download
Email Patches
Plain Diff
Commit
645242a55269cafc06af2d08b276f9853a970de6
645242a5
1 parent
1331a5b9
Updated how the fine-tuned BERT model is stored.
Show whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
48 additions
and
3 deletions
.gitignore
WeiboSentiment_Finetuned/BertChinese-Lora/README.md
WeiboSentiment_Finetuned/BertChinese-Lora/predict.py
WeiboSentiment_Finetuned/BertChinese-Lora/predict_pipeline.py
.gitignore
View file @
645242a
...
...
@@ -181,6 +181,7 @@ WeiboSentiment_Finetuned/GPT2-Lora/models/
WeiboSentiment_Finetuned/GPT2-AdapterTuning/models/
WeiboSentiment_Finetuned/BertChinese-Lora/models/
WeiboSentiment_LLM/models/
WeiboSentiment_Finetuned/BertChinese-Lora/model/
# LoRA 和 Adapter 权重
*/adapter_model.safetensors
...
...
WeiboSentiment_Finetuned/BertChinese-Lora/README.md
View file @
645242a
...
...
@@ -64,8 +64,15 @@ print("正面情感" if prediction == 1 else "负面情感")
-
`predict_pipeline.py`
: 使用pipeline方式的预测程序
-
`README.md`
: 使用说明
## 模型存储
-
首次运行时会自动下载模型到当前目录的
`model`
文件夹
-
后续运行会直接从本地加载,无需重复下载
-
模型大小约400MB,首次下载需要网络连接
## 注意事项
-
首次运行时会自动下载模型,需要网络连接
-
模型
大小约400MB,下载可能需要一些时间
-
模型
会保存到当前目录,方便后续使用
-
支持GPU加速,会自动检测可用设备
-
如需清理模型文件,删除
`model`
文件夹即可
\ No newline at end of file
...
...
WeiboSentiment_Finetuned/BertChinese-Lora/predict.py
View file @
645242a
...
...
@@ -16,12 +16,26 @@ def main():
# 使用HuggingFace预训练模型
model_name
=
"wsqstar/GISchat-weibo-100k-fine-tuned-bert"
local_model_path
=
"./model"
try
:
# 加载模型和分词器
# 检查本地是否已有模型
import
os
if
os
.
path
.
exists
(
local_model_path
):
print
(
"从本地加载模型..."
)
tokenizer
=
AutoTokenizer
.
from_pretrained
(
local_model_path
)
model
=
AutoModelForSequenceClassification
.
from_pretrained
(
local_model_path
)
else
:
print
(
"首次使用,正在下载模型到本地..."
)
# 下载并保存到本地
tokenizer
=
AutoTokenizer
.
from_pretrained
(
model_name
)
model
=
AutoModelForSequenceClassification
.
from_pretrained
(
model_name
)
# 保存到本地
tokenizer
.
save_pretrained
(
local_model_path
)
model
.
save_pretrained
(
local_model_path
)
print
(
f
"模型已保存到: {local_model_path}"
)
# 设置设备
device
=
torch
.
device
(
'cuda'
if
torch
.
cuda
.
is_available
()
else
'cpu'
)
model
.
to
(
device
)
...
...
WeiboSentiment_Finetuned/BertChinese-Lora/predict_pipeline.py
View file @
645242a
...
...
@@ -15,11 +15,34 @@ def main():
# 使用pipeline方式 - 更简单
model_name
=
"wsqstar/GISchat-weibo-100k-fine-tuned-bert"
local_model_path
=
"./model"
try
:
# 检查本地是否已有模型
import
os
if
os
.
path
.
exists
(
local_model_path
):
print
(
"从本地加载模型..."
)
classifier
=
pipeline
(
"text-classification"
,
model
=
model_name
,
model
=
local_model_path
,
return_all_scores
=
True
)
else
:
print
(
"首次使用,正在下载模型到本地..."
)
# 先下载模型
from
transformers
import
AutoTokenizer
,
AutoModelForSequenceClassification
tokenizer
=
AutoTokenizer
.
from_pretrained
(
model_name
)
model
=
AutoModelForSequenceClassification
.
from_pretrained
(
model_name
)
# 保存到本地
tokenizer
.
save_pretrained
(
local_model_path
)
model
.
save_pretrained
(
local_model_path
)
print
(
f
"模型已保存到: {local_model_path}"
)
# 使用本地模型创建pipeline
classifier
=
pipeline
(
"text-classification"
,
model
=
local_model_path
,
return_all_scores
=
True
)
print
(
"模型加载成功!"
)
...
...
Please
register
or
login
to post a comment