-
Notifications
You must be signed in to change notification settings - Fork 26.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[NEXT-1143] Dev mode slow compilation #48748
Comments
same here, and in docker env is even worse, seems like is processing same files over and over without caching them. |
Same for me also dev env ,navigating to different pages via link component is pretty slow |
1 its same here, hitting the page first time seems fine but routing via links gets stuck |
last canary version has a better cold build times improvements, still slow like 2-5 seconds (in docker env) waiting but much better the version im talking about is 13.3.2-canary.6 |
Hey, @jeengbe there have been some patch updates (13.3.1 -> 13.3.4) did it improve for you? |
Hi @denu5, unfortunately, I can't report any real performance changes since I opened this issue. You might want to check out the above linked issue in the TypeScript repo though - might be related. |
As @jeengbe mention there is no performance improvement, there is also a lot of I/O I don’t know why, one request gets pretty much like 1gb-2gb of io. And it is very slow. |
Unfortunately, I can't confirm this for my case |
That’s pretty good, in my case there is a lot of I/O, maybe is because I’m using material-ui but I think is too much even though. |
Possibly, it would align with what your trace shows: #48407 (comment) |
I see that slow route changes in dev mode are showing a '[Fast Refresh] rebuilding' message in the browser console. Sometimes it performs a full page reload when changing routes even if no files have been edited. |
entry next-app-loader
spans?)entry next-app-loader
spans?)
its slowing down the development..! |
Having the same issue here, in the Docker environment it's come to a point where it's almost unusable, and sometimes I even have to do a hard reload, after waiting too long for navigation. This is the case both with component from next/navigation, as with the router.push (useRouter hook imported from next/navigation). We're using Next.js 13.4.2. |
same here, it is almost not usable in dokcer enviorements, but also outside docker is very slow, something is not working nice. this is painfully slow. |
Yeah same for me. I used to remote develop inside our k8s cluster but dev --turbo is super slow inside a container and causes my health check endpoint to sigkill it regularly. The whole app router is super slow when containerized in Dev mode. It works perfectly fine when I run both on my local machine and connect it via reverse proxy. This way it's faster than the old setup (which was not significantly faster before) and takes advantage of preloading pages via next/link. I see inconsistencies in caching too where it's a mix of instant navigation or long builds (around 3.5k modules for some things) around 2-10 sec. Also there is this weird thing happening that a page compiles just fine and then later it grinds to a halt being stuck in waiting for compiling forever until the pod is crashed. |
I love next, but this is a complete show stopper. Sometimes it takes 10 seconds outside of docker for me on a Mac M2 to navigate one page. This is insane. |
Yep even more I get sometimes 50 seconds in a simple page, that’s because is also building other things related to that in pralllel I guess. not even outside docker, i just make a test to work outside docker and timing is exactly the same no difference…. Is getting slower and slower |
Btw webpack lazy building cold is faster than turbopack 🙂 by far |
Yes! I'm surprised this is not more prevalent as an issue atm; unless turbo will somehow fix all of this in 13.5 and they're waiting to address it. What configs do you have for the faster webpack builds? I've tried quite a bit and can't lower my built time by much. I need a temporary fix for this ASAP :( |
A month later no updates on this? Makes development on appDir absolutely impossible. @timneutkens ? Linked a bunch of related issues on this: |
I confirm that next app dir on dev mode and dynamic routing are very very slow on docker now |
Changes in the past weekI've been investigating this over the past week. Made a bunch of changes, some make a small impact, some make a large impact. Here's a list:
You can try them using Help InvestigateIn order to help me investigate this I'll ideally need an application that can be run, if you can't provide that (I understand if you can't) please provide the If possible follow these steps which would give me the best picture to investigate:
Known application-side slowdownsTo collect things I've seen before that cause slow compilation as this is often the root cause:
This and other slowdown reports are currently the top priority for our team. We'll continue optimizing Next.js with webpack where possible. |
entry next-app-loader
spans?)
Changed the initial post in this issue to reflect my reply above in order to ensure people see it as the first thing when opening the issue. I'm going to close the duplicate issues reporting similar slowdowns in favor of this one. I'll need help from you all to ensure this thread doesn't spiral in "It is slow" comments that are not actionable when e.g. without traces / reproduction / further information. Thank you 🙏 |
Hi @timneutkens, sorry to hijack this issue but I'm trying to debug some slow compile times that our time has been experiencing and I'd like to know how do you process and load the trace file into Jaeger or a GUI representation (as seen in your last comment). I've been unable to find anything documenting or mentioning this process. I've found the Also, I'm running a local jaeger deployment from https://github.com/vercel/opentelemetry-collector-dev-setup and if I run execute the $ cargo run ../../traces-sites/vanilla.json
Finished `dev` profile [unoptimized debuginfo] target(s) in 0.33s
Running `/<redacted>/next.js/target/debug/send-trace-to-jaeger ../../traces-sites/vanilla.json`
Jaeger trace will be available on http://127.0.0.1:16686/trace/a3fd2081f16b2ef3
body = Ok("No value at $[7].tags.layer reading List<Span> from json")
body = Ok("No value at $[0].tags.layer reading List<Span> from json")
body = Ok("No value at $[19].tags.layer reading List<Span> from json") but when I open http://127.0.0.1:16686/trace/a3fd2081f16b2ef3, all I see is a |
@lluiscab I'd recommend using |
I suspected my styles being responsible. And it is indeed much better then 2-3 weeks ago, when it really would take 20-30 seconds to recompile. About dynamic imports - I implemented a dynamic loading of translation json files using But on the other hand - when I use Example: // this works fine:
import dynamic from "next/dynamic";
export const DebugPanelDynamic = dynamic(() => import("@/components/layout/DebugPanel").then((m) => m.DebugPanel)); |
I can confirm this that it was because of anti-virus in my case. My code compile time were in xxx ms , but just when "Bitdefender" was installed it sky rocket to to 20 - 50 seconds!! . Found this post and https://stackoverflow.com/a/78236532/507203 which suggested me to exclude. Right when it get excluded it goes back to ms. |
@timneutkens Thanks for taking the time to help people here! I'm working on migrating our main web app from ejected CRA into Next and I'm running into some intense numbers for development builds. So the main issues are: way slower restarts (no cache at all with app router?) and 18 MB client JS which takes significantly longer to load into the browser. The amount of modules is also concerning. Switching to Turbo looks a lot better (even without cache) but sadly some quirks in my context are preventing me from jumping into Turbo now. In short:
I've sent you the trace in a DM on Twitter/X, I've looked into it myself but I'm having trouble coming to any useful conclusions (besides our app just being huge). Edit: forgot to mention one very obvious thing: there's a lot of duplication of modules in the client bundle. Thanks again for looking into these, I'd love to get us fully on board! |
@tiriana The reason that doesn't work is that you're combining path aliases |
@Harbinder-web can you elaborate on what exactly is slow? Seeing initial compile take ~5 seconds, that's mostly because of the amount of barrel imports / large libraries you're importing like Mantine and react-icons, even with barrel optimization these can still be quite large Recompiles when changing a file are taking 300-500ms. |
@GriffinSauce will follow up on twitter 👍 |
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
@Harbinder-web can you make sure those steps are followed when collecting the trace? Then I can correctly inspect the trace 👍 |
We are also experiencing this. For context: We are using Mac M1 & M2 in development and are on optimizePackageImports: [
"@mui/icons-material",
"@mui/material",
"mui-core",
"date-fns",
"lodash-es",
],
}, We cannot use turbopack yet because of #68101. On every page load I have absurd compile times:
I am not sure if it matters, but we are roughly following https://nextjs.org/docs/app/building-your-application/routing/internationalization - so this is why you see the [lang] everywhere. @timneutkens I know you want stack traces, but I got an error generating one: If you give me any kind of instructions for debugging, I will follow along. |
@kl-thamm please at least share the trace file per instructions in the initial post. |
@timneutkens Did you see what i posted?
Edit: Create a private repo, uploaded trace from yesterday and shared access with you. If I can do more let me know how. |
Hey @kl-thamm, I'm still missing the |
Hey, I am pretty sure that is related to the error I described above, because thats the only output. Let me know how to debug that and I will get back asap. I googled around a bit myself, but didnt find anything helpful. Edit: Will try to just try again in the meantime. I added our next config to the repo you are in |
As said the cpuprofile files don't matter, the |
As we dont use turbopack, I just have the trace file now and added that to the repo. Thanks for pointing out the location. That was the missing piece, as the profile was created in another directory. |
@kl-thamm Thanks, had a look, seems the majority of time spent is on reading files from disk. Additional pointers:
Any specific reason you're not able to use Turbopack? |
Thanks a lot. I will make some changes based on that and report back with a new trace (if issues persist). However: We ran the same code in plain react at some point and it was insanely much faster at that point (that was before supabase however). Is it expected that:
Yes, thats ok!
Yes:
Edit changes made:
Remarks:
Edit 2: |
@timneutkens Hey there, we've also been struggling a little with compilation times. I have a feeling that in our case it's just due to the size of the application and the amount of dependencies and files we have but I figured it would be worth double-checking with someone much more knowledgeable. Would you be able to take a look at the trace? Here is the gist: https://gist.github.com/wadamek65/735f9efa17cdc50d9dcc353b099d4ac8 For some reason no CPU profile was generated for me even though I followed the instructions and ran the app with |
@timneutkens here is performance gist file, only issue you can face maybe is, we are using https://gist.github.com/rzzapp/ef8815c4162f4e2ff6fd479550740572 Edit: Additional infos:
|
@wadamek65 looking at your trace there is a lot of time unaccounted for, which would be coming from the plugins you added: https://gist.github.com/wadamek65/735f9efa17cdc50d9dcc353b099d4ac8#file-next-config-js-L1-L6. Would recommend removing these first. Also seeing slow file reads, given you're on windows make sure anti-virus which blocks file reads is disabled on your code. |
@rzzapp this stands out:
|
@timneutkens thanks for checking! Yes, I'm on Mac also my team. Will check with our hardware supplier team in case if we are having something locked in our system which can cause file reading unbearably slow. But generally laptops are running quite fast on smaller projects of CRA or vite.
Tbh we don't have any custom postcss, we only have tailwindcss and everything runs based on that. also worth to mentioned we do have custom nextjs proxy/middleware which run on each route change to detect it should load our next app pages or fallback legacy app (just redirects to that url which is external project) maybe this can cause slowness but I'm not sure. I remember once I tried to disable it, it was still slow. Probably nothing about this one is serious but worth to try to toggle it again on my side I'll try few scenarios based on this then can provide few more details. |
@timneutkens Thanks for taking a look! I think you're right; I think the slowness comes from the Vanilla Extract plugin, probably in combination with Mantine that we're using. I've seen quite a few issues regarding this in their repository. I tried adding Thanks for taking the time to look into this. I really appreciate it! By the way, is there an instruction somewhere on how to load those traces into Jaeger and debug by ourselves? |
Hello @timneutkens, were you able to look into this? |
@timneutkens thanks really for good inside, we did actually able to drop our compile/load time from 100-200s to around 1-2s. Main issues we missed:
Also we upgrade from So far cold start takes around 10-20s but later gets much much faster. |
@timneutkens Can we get a bit more systematic here and maybe also collect more general instructions? I know the detailed debugs are helpful for individual users and also for next to understand what is going on, but it will not allow for a good dx out of the box. Many recommendations you need to find by browsing through very large GitHub issues and lots of it is outdated. In addition to that, there still seem to be a more fundamental issues, because I keep reading here about initial compile times of 10s and even reloads that exceed seconds after a very fair share of optimization. I can only compare that to other frameworks I worked with and given the very high hardware demands (e.g. 64GB RAM and M1 Max are not sufficient to be really fast), this is frustrating. So what I gathered so far:
Maybe there is a chance you add something to the next documentation about this? And then back to my problem:
|
Changes in the past week
I've been investigating this over the past week. Made a bunch of changes, some make a small impact, some make a large impact. Here's a list:
pages
andapp
and you're only working onapp
it will no longer compile the runtime forpages
. Note: this shifts the compilation of the runtime to when you first open a pageYou can try them using
npm install next@canary
.Help Investigate
In order to help me investigate this I'll ideally need an application that can be run, if you can't provide that (I understand if you can't) please provide the
.next/trace
file.If possible follow these steps which would give me the best picture to investigate:
npm install next@canary
(use the package manager you're using) -- We want to make sure you're using the very latest version of Next.js which includes the fixes mentioned earlier.rm -rf .next
NEXT_CPU_PROF=1
andNEXT_TURBOPACK_TRACING=1
(regardless of if you're using Turbopack, it only affects when you do use Turbopack) environment variable. E.g.:NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 npm run dev
NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 yarn dev
NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 pnpm dev
ctrl c
).next/trace
file to https://gist.github.com -- Please don't run trace-to-tree yourself, as I use some other tools (e.g. Jaeger) that require the actual trace file..next/trace.log
as well, if it's too large for GitHub gists you can upload it to Google Drive or Dropbox and share it through that.next.config.js
(if you have one) to https://gist.github.comKnown application-side slowdowns
To collect things I've seen before that cause slow compilation as this is often the root cause:
react-icons
, material icons, etc. Most of these libraries publish barrel files with a lot of re-exports. E.g. material-ui/icons ships 5500 module re-exports, which causes all of them to be compiled. You have to addmodularizeImports
to reduce it, here's an example: long compile times locally - along with "JavaScript heap out of memory" since upgrade to NextJS 13 #45529 (comment)content
setting that tries to read too many files (e.g. files not relevant for the application)This and other slowdown reports are currently the top priority for our team. We'll continue optimizing Next.js with webpack where possible.
The Turbopack team is currently working on getting all Next.js integration tests passing when using Turbopack as we continue working towards stability of Turbopack.
Original post
Verify canary release
Provide environment information
Which area(s) of Next.js are affected? (leave empty if unsure)
No response
Link to the code that reproduces this issue
https://github.com/DigitalerSchulhof/digitaler-schulhof
To Reproduce
Note that I have been unable to replicate this issue in a demo repository.
Describe the Bug
The issue is that Next.js is generally slow in dev mode. Navigating to new pages takes several seconds:
The only somewhat reasonable time would be 600ms for the API route
/api/schulhof/auth/login/route
, even though that is still quite a lot slower than what it should be given its size.It also doesn't look right to compile ~1500 modules for each page, as most of them should be cached. The pages are not very different.
Even an empty API route takes several hundreds of ms. The following example contains solely type exports:
I am not exactly sure how to read trace trees, but what stands out is that there are (over multiple runs) several
entry next-app-loader
that take 2 seconds to complete:Find both dev and build traces here: https://gist.github.com/jeengbe/46220a09846de6535c188e78fb6da03e
Note that I have modified
trace-to-tree.mjs
to include event times for all events.It also seems unusual that none of the modules have child traces.
Expected Behavior
Initial load and navigating should be substantially faster.
Which browser are you using? (if relevant)
No response
How are you deploying your application? (if relevant)
No response
From SyncLinear.com | NEXT-1143
The text was updated successfully, but these errors were encountered: