Skip to content

Tags: modelscope/evalscope

Tags

v0.5.5

Toggle v0.5.5's commit message
Release version 0.5.5

v0.5.2

Toggle v0.5.2's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Dev/refactor 0.5 (#91)

* update test_run_all

* update readme

* update readme

* update readme

* update logger for swift deploy in UTs

* update UTs

* update ut

* update ut

* update ut

* update ut

* update example

* add summarizer for eval_datasets

* update

* update

* update version to 0.5.1

* update ut

* test for summ

* update

* update version and req

* update version

v0.4.3

Toggle v0.4.3's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Refactor setup and add UTs (#85)

* add opencompass cmd parser

* fix dry-run

* update

* refactor for backend & add backend module

* add example for oc

* update

* update readme and example

* temp

* update opencompass parser

* add opencompas cli and chat_medium collection

* update cli and eval_api

* add OpenCompassArguments for cli

* update eval_api for models

* update version

* add check end for ms-opencompass

* add backend args merge and datasets filter

* add models and api meta template

* assert models

* add tmpfile for task config

* fix datasets in self.args.config

* update

* pop dataset_name key in datasets

* set meta_template to None for mmlu, ceval, ...

* fix models path

* update test task

* update eval task

* update test tasks

* add debug info

* add example for swift eval

* add download data

* update

* update import for eval_datasets

* add json config for toolbench eval

* add entry task in run.py; update example_eval_swift_openai_api

* add yaml and json config

* update example

* update gsm8k

* update example

* update summarizer

* support opencompass backend in Summaryzer

* update summarizer

* update exmaple

* update example; set dataset_dir in config.py

* update version

* update args name

* update example

* set mmlu to 0-shot for swift

* add api key for openai api

* update eval datasets

* add limit

* add limit example

* update datasets

* update readme for oc backend

* update example

* update readme for oc backend

* add readme for en

* fix eval_config assertion

* fix eval_backend assertion

* fix eval_backend assertion

* fix eval_backend assertion

* add ut for swift-eval

* add test run all

* update tests

* add logger for ut

* update

* update pypi source

* add debug

* update

* fix swift deploy subprocess

* update

* add check service

* update

* update

* fix check swift server

* update terminate process

* update

* update example

* update example

* update example

* fix eval_backend checking

* update version

* update UTs and examples

* refactor setup

* add eval_backend and eval_config in config.py

* fix pr issues

v0.2.8

Toggle v0.2.8's commit message
feat: add openai sdk (for service api) support.

v0.2.7

Toggle v0.2.7's commit message
feat: update data/model/general_qa adapter & add eval&infer engine.

v0.2.6

Toggle v0.2.6's commit message
update version

v0.2.5

Toggle v0.2.5's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fix/general qa load (#30)

* fix dataset_name_or_path in run when Local dataset_hub

* update

* udpate

* fix general qa load encode

* refine gsm8k