Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hot code swapping #68

Open
andrewrk opened this issue Jan 14, 2016 · 17 comments
Open

hot code swapping #68

andrewrk opened this issue Jan 14, 2016 · 17 comments
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. frontend Tokenization, parsing, AstGen, Sema, and Liveness.
Milestone

Comments

@andrewrk
Copy link
Member

  • Have the compiler run continuously, watching the file system for source
    changes and automatically perform multithreaded compilation to build projects
    quickly.
  • Hot code swapping. When integrated with the previous feature, you could
    press "save" in your editor and see the change immediately in your running
    software.

Of course this only works for some kinds of changes - obviously if you make changes to your initialization code, you won't see a difference. But it would probably work at the function level, so you could swap out any individual function for a new one.

@andrewrk andrewrk added enhancement Solving this issue will likely involve adding new logic or components to the codebase. lowpriority labels Jan 14, 2016
@PavelVozenilek
Copy link

This feature (automatic rebuild in background & hot code swapping) is available for C in Projucer IDE, built upon LLVM infrastructure.

Old demo 1, old demo 2, few implementation details.

Be aware that it took several years to implement and the implementor said that "it was the most complicated thing I ever did".

@0joshuaolson1
Copy link

0joshuaolson1 commented May 25, 2018

@PavelVozenilek The Projucer link has died, sadly.

@PavelVozenilek
Copy link

@ccll
Copy link

ccll commented Jun 14, 2018

I always wonder how should hot-swap deal with data (data structure layout and long-lived in-memory state), it seems rather difficult compared to hot-swap stateless code (like a pure function).
In some languages which support closure, it's even more complicated.
Are there any articles on this subject?

@isaachier
Copy link
Contributor

isaachier commented Jun 14, 2018

@ccll that is the biggest issue. Doing it in C/C is currently possible using a dynamically loaded library. See this blog post on the topic.

@themagnet
Copy link

here https://molecular-matters.com/products_livepp.html is a commercial product which realizes hot-reload for c/c projects

@jaccarmac
Copy link

I wonder if looking at the way Common Lisp deals with these things would be useful at all. Sure, CL has a fatter runtime than Zig at the binary level, but:

  1. If there is a watcher like @andrewrk hints we could possibly have it keep track of the old structure of the code it is watching.
  2. CL contains a full-blown object system and is able to manage updating all live instances of a CLOS class. Updating structs would be comparatively simpler, size changes might be the hardest part to deal with?

@ccll
Copy link

ccll commented Jul 11, 2018

@isaachier Thanks, that article is very helpful!
I can see it passes a DLL-specific 'game_state' around, but what if the layout of 'game_state' got changed between reloads? I can imagine there will be lots of crashes/segfaults/misbehavior.

Now i've integrated C live coding successfully with libtcc, which hot-swap pure code flawlessly.

About the state transition, I can think of a solution which serialize the state out (to a layout-agnostic format), allocate new state, then serialize in. It should work, but involves a lot of manual implementation of serializing code and is not very generic/automatic.

@andrewrk andrewrk modified the milestones: 0.4.0, 0.5.0 Sep 28, 2018
@andrewrk andrewrk modified the milestones: 0.5.0, 0.6.0 Apr 10, 2019
@ghost ghost mentioned this issue Aug 7, 2019
5 tasks
@andrewrk andrewrk modified the milestones: 0.6.0, 0.7.0 Oct 2, 2019
@andrewrk andrewrk added the frontend Tokenization, parsing, AstGen, Sema, and Liveness. label Nov 27, 2019
@pgruenbacher
Copy link

nim-lang/RFCs#511
Nim seems to have this implemented as a reference.

@pixelherodev
Copy link
Contributor

Regarding code swapping in C, see https://github.com/fungos/cr

@cshenton
Copy link
Contributor

cshenton commented Jan 1, 2020

This would be a killer feature for me in game dev. Commercial game engines (Unity/Unreal) tend to implement their own hotswapping layers for engine code or rely on scripting languages to get iteration times down.

To have this supported at the language level would definitely reinforce the "so productive you have to use it" narrative for zig in games.

@alkeryn
Copy link

alkeryn commented Apr 15, 2020

@cshenton
One of the possible thing you can already do in C and C and probably in zig as well is have most of your core logic into a dll / so file and having the program check for it and reload it on change in the debug builds.

It isn't "native" hot code reloading supported into the language, but it allow for a similar workflow
as an example : https://youtu.be/LNRfbZlppLo?t=60

altough it isn't the same as if it was builtin and it would be great if it was, you might find it useful !

Hoping this wasn't a necrobump.

@NightMachinery
Copy link

Lisps have a great interactive dev experience. You can eval the last expression, the current line, the selected region, the current 'cell' (a region delimited by special comments), the current file, etc. They also autocomplete based on the living program in the REPL, and you can inspect the variables.

It would be awesome to have this workflow supported. Julia does this (more or less; structs cannot yet be redefined) with its VSCode plugin, so it's not like emacs is needed to support this.

@andrewrk andrewrk modified the milestones: 0.8.0, 0.9.0 Jun 4, 2021
@andrewrk andrewrk modified the milestones: 0.9.0, 0.10.0 Nov 21, 2021
@dischoen
Copy link

I always wonder how should hot-swap deal with data (data structure layout and long-lived in-memory state), it seems rather difficult compared to hot-swap stateless code (like a pure function). In some languages which support closure, it's even more complicated. Are there any articles on this subject?

You could have a look at how it is done in erlang. However, that's a different world..

Another thing, which I haven't seen in the comments so far would be security.
Would an attacker be able to patch production code at runtime?

@andrewrk
Copy link
Member Author

andrewrk commented Jan 24, 2022

I did some live coding on this feature recently, and we got to a proof-of-concept status:

It's in the hcs branch. It depends on further progress on the x86_64 backend to be generally useful. It's also for Linux only currently. Each OS will need its own hot code swapping strategy.

@andrewrk andrewrk modified the milestones: 0.10.0, 0.11.0 Apr 17, 2022
@ziglang ziglang deleted a comment from chris2860 Oct 16, 2022
@andrewrk andrewrk modified the milestones: 0.11.0, 0.12.0 Apr 9, 2023
@qm3ster
Copy link

qm3ster commented Jul 29, 2023

There has also been https://www.jakubkonka.com/2022/03/16/hcs-zig.html (hcs-macos branch) by @kubkon

@acodervic
Copy link

I need HRC(Hot Reload Code) , If Zig has HRC It will be perfect language .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. frontend Tokenization, parsing, AstGen, Sema, and Liveness.
Projects
None yet
Development

No branches or pull requests