Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llama.cpp convert to vlang? :) just curious, or some port of it. #21277

Open
2 tasks
ouvaa opened this issue Apr 13, 2024 · 7 comments
Open
2 tasks

llama.cpp convert to vlang? :) just curious, or some port of it. #21277

ouvaa opened this issue Apr 13, 2024 · 7 comments
Labels
Feature Request This issue is made to request a feature.

Comments

@ouvaa
Copy link

ouvaa commented Apr 13, 2024

Describe the feature

anyone working on porting llama.cpp to vlang? that'll be something.

Use Case

llama.cpp being used by vlang

Proposed Solution

No response

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

Version used

anything

Environment details (OS name and version, etc.)

ubuntu 22.04

Note

You can use the 👍 reaction to increase the issue's priority for developers.

Please note that only the 👍 reaction to the issue itself counts as a vote.
Other reactions and those to comments will not be taken into account.

@ouvaa ouvaa added the Feature Request This issue is made to request a feature. label Apr 13, 2024
@JalonSolov
Copy link
Contributor

Have you tried doing v translate llama.cpp?

@ouvaa
Copy link
Author

ouvaa commented Apr 15, 2024

@JalonSolov

v translate llama.cpp

C2V is not installed. Cloning C2V to /root/.vmodules/c2v ...
Compiling c2v ...
C to V translator 0.4.0
"/usr/local/src/llama.cpp" is a directory, processing all C files in it recursively...

  translating ./tests/test-c.c ... c2v_output/test-c.v:334:6: error: cannot register fn `Ggml_opt_callback`, another type with this name exists
  332 | }
  333 | 
  334 | type Ggml_opt_callback = fn (voidptr, int, &f32, &bool)
      |      ~~~~~~~~~~~~~~~~~
  335 | type Ggml_log_callback = fn (Ggml_log_level, &i8, voidptr)
  336 | struct Ggml_opt_params {

Internal vfmt error while formatting file: /usr/local/src/llama.cpp/./c2v_output/test-c.v.
 took   618 ms ; output .v file: ./c2v_output/test-c.v
  translating ./ggml-alloc.c  ... c2v_output/ggml-alloc.v:97:2: error: struct embedding must be declared at the beginning of the struct body
   95 | struct Ggml_backend_i { 
   96 |     get_name fn (Ggml_backend_t) &i8
   97 |     c.free fn (Ggml_backend_t)
      |     ~~~~~~
   98 |     get_default_buffer_type fn (Ggml_backend_t) Ggml_backend_buffer_type_t
   99 |     set_tensor_async fn (Ggml_backend_t, &Ggml_tensor, voidptr, usize, usize)

Internal vfmt error while formatting file: /usr/local/src/llama.cpp/./c2v_output/ggml-alloc.v.
 took   364 ms ; output .v file: ./c2v_output/ggml-alloc.v
  translating ./ggml.c        ... c2v_output/ggml.v:23172:3: error: inline comment is deprecated, please use line comment
23170 | n_gradient_accumulation: 1, 
23171 | adam: (unnamed at ./ggml.h {
23172 |         /*FAILED TO FIND STRUCT "(unnamed at ./ggml.h"*/10000, 1, 0, 2, 0.00100000005, 0.899999976, 0.999500012, 9.99999993E-9, 9.99999974E-6, 0.00100000005, 0}
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
23173 |         , 
23174 | }

Internal vfmt error while formatting file: /usr/local/src/llama.cpp/./c2v_output/ggml.v.
 took  6821 ms ; output .v file: ./c2v_output/ggml.v
  translating ./ggml-backend.c ... c2v_output/ggml-backend.v:915:2: error: inline comment is deprecated, please use line comment
  913 | fn ggml_backend_cpu_buffer_type() Ggml_backend_buffer_type_t {
  914 |     ggml_backend_cpu_buffer_type := Ggml_backend_buffer_type {
  915 |     /*FAILED TO FIND STRUCT "Ggml_backend_buffer_type"*/Ggml_backend_buffer_type_i {
      |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  916 |     get_name: ggml_backend_cpu_buffer_type_get_name, 
  917 | alloc_buffer: ggml_backend_cpu_buffer_type_alloc_buffer,

Internal vfmt error while formatting file: /usr/local/src/llama.cpp/./c2v_output/ggml-backend.v.
 took   468 ms ; output .v file: ./c2v_output/ggml-backend.v
  translating ./ggml-quants.c ... c2v_output/ggml-quants.v:240:3: error: use `?` instead of `?void`
  238 | fn quantize_row_q4_0_reference(x &Float, y &Block_q4_0, k int)  {
  239 |     qk := 32
  240 |     (void(sizeof(if (k % qk == 0){ 1 } else {0})) , )
      |      ~~~~
  241 |     nb := k / qk
  242 |     for i := 0 ; i < nb ; i    {

Internal vfmt error while formatting file: /usr/local/src/llama.cpp/./c2v_output/ggml-quants.v.
 took  1905 ms ; output .v file: ./c2v_output/ggml-quants.v
  translating ./ggml-mpi.c    ... ./ggml-mpi.c:5:10: fatal error: 'mpi.h' file not found
#include <mpi.h>
         ^~~~~~~
1 error generated.

The file ./ggml-mpi.c could not be parsed as a C source file.
C2V command: '/root/.vmodules/c2v/c2v' 'llama.cpp'
C2V failed to translate the C files. Please report it via GitHub.

@JalonSolov
Copy link
Contributor

The last one is easy enough... you have to tell c2v (the program called by v translate) where the mpi.h header can be found. See https://github.com/vlang/c2v?tab=readme-ov-file#configuration for details on how to do that.

The rest of those messages, though, are bugs in c2v. Looks like it hasn't been updated to latest V standards. :-\

@medvednikov
Copy link
Member

с2v is tested via ci, DOOM is translated and run on each commit (and verified pixel by pixel)

so it's always updated to latest V standards

@JalonSolov
Copy link
Contributor

Not quite... for example, as shown above, c2v is generating inline comments, which V no longer supports.

@hholst80
Copy link
Contributor

hholst80 commented May 5, 2024

@ouvaa is there a version without MPI? MPI is a very special high performance message passing library. It would be nice to get llama2.c working first?

https://github.com/karpathy/llama2.c

@trufae
Copy link
Contributor

trufae commented May 5, 2024

Llama is c not c. So dont expect c2v to work. Also llama is moving really fast with lots of changes everyday and support for many hardware. I dont think it makes sense to port it to v. But maybe just wrap the public c api from v. Like its done in the llama-cpp-python api.

for fun purposes porting gpt2 to c which is already in c should be easy and fun. But wont compete with perf or features offered by llama, also supporting gguf is a key feature for any inference engine nowadays

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature Request This issue is made to request a feature.
Projects
None yet
Development

No branches or pull requests

5 participants