Tags: modelscope/evalscope
Tags
Dev/refactor 0.5 (#91) * update test_run_all * update readme * update readme * update readme * update logger for swift deploy in UTs * update UTs * update ut * update ut * update ut * update ut * update example * add summarizer for eval_datasets * update * update * update version to 0.5.1 * update ut * test for summ * update * update version and req * update version
Refactor setup and add UTs (#85) * add opencompass cmd parser * fix dry-run * update * refactor for backend & add backend module * add example for oc * update * update readme and example * temp * update opencompass parser * add opencompas cli and chat_medium collection * update cli and eval_api * add OpenCompassArguments for cli * update eval_api for models * update version * add check end for ms-opencompass * add backend args merge and datasets filter * add models and api meta template * assert models * add tmpfile for task config * fix datasets in self.args.config * update * pop dataset_name key in datasets * set meta_template to None for mmlu, ceval, ... * fix models path * update test task * update eval task * update test tasks * add debug info * add example for swift eval * add download data * update * update import for eval_datasets * add json config for toolbench eval * add entry task in run.py; update example_eval_swift_openai_api * add yaml and json config * update example * update gsm8k * update example * update summarizer * support opencompass backend in Summaryzer * update summarizer * update exmaple * update example; set dataset_dir in config.py * update version * update args name * update example * set mmlu to 0-shot for swift * add api key for openai api * update eval datasets * add limit * add limit example * update datasets * update readme for oc backend * update example * update readme for oc backend * add readme for en * fix eval_config assertion * fix eval_backend assertion * fix eval_backend assertion * fix eval_backend assertion * add ut for swift-eval * add test run all * update tests * add logger for ut * update * update pypi source * add debug * update * fix swift deploy subprocess * update * add check service * update * update * fix check swift server * update terminate process * update * update example * update example * update example * fix eval_backend checking * update version * update UTs and examples * refactor setup * add eval_backend and eval_config in config.py * fix pr issues