-
Notifications
You must be signed in to change notification settings - Fork 393
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Installation is a mess, instructions are a mess #41
Comments
We've tested the following installation instructions and pip package. What error message did you encounter, can you post it here?
|
pip 23.3.1 from C:\Python311\Lib\site-packages\pip (python 3.11) With your commands : First 6 ok,
and conda -v :
help pls |
I successfully install it on WSL 2 (Windows 11) with following step:Environment
Here is the Environment in WSL
Install OpenChat on Windows 11 (WSL)Using the instructions a bit different from aboveSame for create conda env
⭐ Alomst same, specify the python version
Same for install cudatoolkit in conda
⭐ Alomst same, but specify the torch version.
Same
⭐ Before install flash-attn, I need to Install g Compiler first. (Basically, GPT-4 teaches me everything)
Same
I try almost 6 hours on making it works on Windows but fail, So I try WSL 2
Use this just works for me
Use this to see more parameters settings
Test with request |
@imoneoi For exampleRun ServerFor single GPT (e.g. RTX 3090, 4090)
For cluster with multiple GPUs
For more parameters settings
Original README has run server instruction above, instead of right after the
|
Just made a PR #70 |
After a little bit of experimenting and getting errors related to cuda versions, here are up-to-date steps (for 3.6.1) that worked for me (conda 4.10.3): conda create -y --name openchat3.6
conda activate openchat3.6
conda install -y python==3.11
conda install nvidia/label/cuda-12.1.1::cuda-toolkit
pip3 install torch
pip3 install packaging ninja
pip3 install --no-build-isolation "flash-attn<2"
pip3 install ochat==3.6.1 Finally: python -m ochat.serving.openai_api_server --model openchat/openchat-3.6-8b-20240522 |
ChatGPT helped remove all the swear words.
I've encountered difficulties trying to set this up on Ubuntu, MacOS, and Windows. I've noticed some inconsistencies in the instructions, and it seems some tools and libraries might be either too new or outdated. It would be greatly appreciated if these issues could be addressed to make the project more user-friendly for everyone. Thank you.
The text was updated successfully, but these errors were encountered: