Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MCTS Sampler #2967

Merged
merged 63 commits into from
Feb 8, 2025
Merged
Changes from 1 commit
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
ebbbfb0
mcts init
lxline Jan 17, 2025
8871d16
step continue & faster prm
lxline Jan 17, 2025
acb62b4
fix
lxline Jan 17, 2025
dee44d3
more parallel
lxline Jan 17, 2025
f1fa335
more parallel
lxline Jan 17, 2025
68767d5
fix
lxline Jan 17, 2025
6862f70
Merge commit '68767d54' into sampler
lxline Jan 17, 2025
890d2fd
catch client error
lxline Jan 18, 2025
eb923cc
ctime
lxline Jan 18, 2025
170721b
sample prompt
lxline Jan 21, 2025
88e6b40
unique log & expand pruning
lxline Jan 21, 2025
288c11a
logger time
lxline Jan 21, 2025
068ba81
rollout with multi-engines
lxline Jan 21, 2025
b2f5c60
add next-prompt
lxline Jan 21, 2025
6e76f6d
change args
lxline Jan 22, 2025
4331c17
prefer add ground_truth
lxline Jan 22, 2025
15aac92
collect_filter_threshold
lxline Jan 22, 2025
44546a4
base_url and api_key in args
lxline Jan 22, 2025
2410221
Merge branch 'main' into sampler
lxline Jan 22, 2025
f7a71f6
client prm
lxline Jan 22, 2025
78fde7f
update generated results
lxline Jan 22, 2025
d5808c8
check terminated in orm
lxline Jan 22, 2025
67b4400
Merge branch 'main' into sampler
lxline Jan 23, 2025
1f74f6c
rollout args change
lxline Jan 23, 2025
aacf1b6
stop_words \n
lxline Jan 23, 2025
48caf73
sys_prompt from file
lxline Jan 24, 2025
6327e6e
fix
lxline Jan 24, 2025
21c576d
terminate state back propagate
lxline Jan 24, 2025
6c0d293
Merge branch 'modelscope:main' into main
lxline Jan 24, 2025
7ec1d1a
add "enable_prefix_caching" args for vllm engine. (#2939)
Leoyzen Jan 23, 2025
c2cebd0
Fix vllm docs link & fix web-ui (#2970)
Jintao-Huang Jan 23, 2025
d363d84
Fix sample (#2971)
tastelikefeet Jan 23, 2025
edbf4b6
support merge-lora & quant (#2973)
Jintao-Huang Jan 23, 2025
1113182
support create_checkpoint_symlink (#2975)
Jintao-Huang Jan 23, 2025
2642978
Sampling and RFT (#2977)
tastelikefeet Jan 23, 2025
4eb8723
support auto dataset mapping (#2976)
Jintao-Huang Jan 23, 2025
d8a7ed8
sys_prompt from file
lxline Jan 24, 2025
54288d1
fix
lxline Jan 24, 2025
e02ae6b
support qwen2_5 long (#2982)
Jintao-Huang Jan 24, 2025
94135ed
merge main into sampler
lxline Jan 24, 2025
b61b7f7
fix node.correct & fix prefer_pairs
lxline Jan 26, 2025
322fe9c
fix file save & no system_prompt
lxline Jan 27, 2025
ccdf03d
Merge branch 'modelscope:main' into main
lxline Jan 27, 2025
e01ba81
perform_infer & collect tree
lxline Jan 30, 2025
ce2116f
fix
lxline Jan 30, 2025
1e5a86c
stop_reason & fix
lxline Jan 30, 2025
57b7990
result add query
lxline Jan 30, 2025
a944eba
fix
lxline Jan 30, 2025
cd5a83a
fix collect_from_mct
lxline Jan 31, 2025
fbbbdb2
fix vllm_engine
lxline Jan 31, 2025
9712cc6
fix perform_infer
lxline Jan 31, 2025
bb39b6c
fix perform_infer & pre-commit
lxline Jan 31, 2025
33f9741
Merge branch 'modelscope:main' into main
lxline Jan 31, 2025
5fc63fb
merge main
lxline Jan 31, 2025
94c3c7a
Merge branch 'modelscope:main' into main
lxline Feb 1, 2025
3db7309
Merge branch 'main' into sampler
lxline Feb 1, 2025
626b23f
fix
lxline Feb 1, 2025
55c061e
examples
lxline Feb 1, 2025
e354c32
pre commit
lxline Feb 1, 2025
958d3e3
less log & change example arg
lxline Feb 1, 2025
6d9b5de
Merge branch 'modelscope:main' into main
lxline Feb 8, 2025
cc5ed77
Merge branch 'main' into sampler
lxline Feb 8, 2025
5445239
fix
lxline Feb 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
prefer add ground_truth
lxline committed Jan 22, 2025
commit 4331c1754d679b091078b359ded7fc827ba081c2
1 change: 1 addition & 0 deletions swift/experimental/sampling/mcts.py
Original file line number Diff line number Diff line change
@@ -324,6 +324,7 @@ def _collect(curr_node: LanguageNode):
if curr_node.children[-1].outcome_reward - curr_node.children[0].outcome_reward > 0.6:
results.append(json.dumps({
"query": query,
"ground_truth": ground_truth,
"path": curr_node.path,
"good": curr_node.children[-1].path[-1],
"good_score": curr_node.children[-1].outcome_reward,