Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Module federation: internal async boundary #11811

Open
ScriptedAlchemy opened this issue Oct 24, 2020 · 16 comments
Open

Module federation: internal async boundary #11811

ScriptedAlchemy opened this issue Oct 24, 2020 · 16 comments
Labels
enhancement module-federation issues related to module federation webpack-5

Comments

@ScriptedAlchemy
Copy link
Member

ScriptedAlchemy commented Oct 24, 2020

Feature request

Posting here for visibility @sokra

Projects that work synchronously, like next.js - cause problems with sharing vendor code. Since next has no async boundary internally to pause execution while container negotiations take place between runtimes.

To solve this, we change the webpack runtime startup to prevent base app boot till sharing is resolved.

What is the expected behavior?
I want to use federated imports without requiring a async import above the modules I'm planning to share which are static imports.

What is motivation or use case for adding/changing the behavior?

It's inconvenient and unreliable waiting on third parties to get with the program and update projects to be async.

More users will be able to use MF regardless of framework limitations by delegating the responsibility to webpack runtime directly.

How should this be implemented in your opinion?
Quoting @sokra :

We modify StartupEntrypointRuntimeModule to work similar to ShareRuntimeModule.

Are you willing to work on this yourself?
Hell yes

Fork and relevant branch https://github.com/ScriptedAlchemy/webpack/pull/5
Playground for testing: https://github.com/ScriptedAlchemy/webpack-federation-playground
This is work in progress

@Ivan-Parushev
Copy link

Any progress on this?

I very much looking forward to building federated next.js application.

@ScriptedAlchemy
Copy link
Member Author

This was too complex to implement last we tried.

@mkotsollaris
Copy link

Do we have any updates on this? I am very much looking to build next federated apps too.

@ScriptedAlchemy
Copy link
Member Author

I created a plugin called nextjs-MF which solves the problem for nextjs

@mkotsollaris
Copy link

Thanks Zack. I believe nextjs-mf works great for CSR, but doesn't support SSR apps. For SSR, the only workaround I've seen is this one which uses telenkos' package

@ScriptedAlchemy
Copy link
Member Author

I have others which are not publicly available. You can use fetch() and post to have remotes SSR themselves inside a next/dynamic instance server side

@robertohein
Copy link

any updates on this problem?

@ScriptedAlchemy
Copy link
Member Author

any updates on this problem?

Too complex if I remember correctly

@micro-chipset
Copy link

Do you plan to continue working here?

@ScriptedAlchemy
Copy link
Member Author

No plans to continue working on this problem.

@vankop vankop added the module-federation issues related to module federation label Nov 18, 2021
@crodriguez-plitzi
Copy link

crodriguez-plitzi commented Jan 24, 2022

@ScriptedAlchemy @sokra hi, i would like to revive this topic, after few days debugging the whole sharing modules i think we can do some changes to make sync works

basically the main problem is
Screenshot from 2022-01-24 02-54-24

when we do the initial load and we try to load shared dependencies, like react in this example, this is trying to call the next code

"webpack/sharing/consume/default/react/react": () => (loadSingletonFallback("default", "react", () => (__webpack_require__.e("vendors-node_modules_react_index_js").then(() => (() => (__webpack_require__(/*! react */ "./node_modules/react/index.js"))))))),

logically this is wrong because, in the initial load we should not wait for the chunk, instead of that, dependencies should be loaded and stored in cache and when this be required by chunks we should be able to read the cache to get the references,

the second things is that this should be considered a cascade imports

Screenshot from 2022-01-24 03-04-56

i was trying to fix it but im not very involved in the core of react, but after debugging this im thinking that the fix should not be too complicated to make MF be able to work in sync mode, and can be optional with props to keep original functionality like is actually,

the reason that MF works async is because primitive dependencies (from node_modules) are already loaded, but this doesnt make sense if from the beginning we dont have access to them, basically the HOST must have it loaded but the logic should not require to have eager enabled, this can be refactored to works in a better way to support scenarios like

scenario1: HOST1 -----> HOST2 ------> PLUGINS
scenario2: HOST1 -----> PLUGINS

what we should check if in the initial load shared dependencies arent loaded, should be loaded

when we works based on the shared we should consider that the HOST should provide these shared dependencies to keep some armony between dependencies

the main file where the problem i think is located is this one

/webpack/lib/sharing/ConsumeSharedRuntimeModule.js

@ScriptedAlchemy
Copy link
Member Author

I believe the problem becomes that in order for webpack to attempt to negotiate what version of react to use, since it's a singleton, it would require loading the remotes to know which version of a dependency should be consumed, and these dependencies are usually only discovered by executing the application. If it's possible to do, I'm all ears.

But it does present a challenge to start an application if the dependencies are not hoisted and frontloaded asynchronously.

I believe there is a workaround that if you use shareKey and import, and give the object some key other than react, like fakeReact: it won't move out of the graph into another chunk. However you are then in host only mode, similar to eager:true

@crodriguez-plitzi
Copy link

crodriguez-plitzi commented Jan 24, 2022

was thinking that and its true, the main problem is webpack and how he negotiate the shared resources, and thats why im suggesting it, and about versions can be designed in the way that we can store the versions also, and if the shared that is expecting the child MF plugin or the chunk is not there, we can implement a way to continue loading it in the current way getting this from node_modules

at high level should be like this
User Story
APP (entry point, chunk1, chunk2) and will load as MF (plugin1)
config = react will be singleton

how webpack should manage this complex scenario in sync way

something like this
image (53)

@crodriguez-plitzi
Copy link

crodriguez-plitzi commented Jan 24, 2022

webpack load process and negotiations should be based on tree logic, the first one that loads a shared dependency can be considered the host when this is configured as singleton, and works based on cache, if a MF plugin or chunk or anything else is trying to load a shared library and the version dont match, should load it and store it in the cache and keep the same flow

in any case if u want to use same resource across the whole execution this should keep using requiredVersion: false and also, this can solve the problem about run the whole stuff in sync way and finally remove the bootstrap hack way that is being used to make it works

But it does present a challenge to start an application if the dependencies are not hoisted and frontloaded asynchronously.

I believe there is a workaround that if you use shareKey and import, and give the object some key other than react, like fakeReact: it won't move out of the graph into another chunk. However you are then in host only mode, similar to eager:true

if a shared resource is remote, we can detect it and also we could add new attributes in webpack config to make it explicit,
but to be honest shared singleton shouldnt be remote, for remote we have explicitly remotes

new ModuleFederationPlugin({
     name: "app1",
     filename: "remoteEntry.js",
     remotes: {
       app2: "app2@http://localhost:3005/remoteEntry.js",
     },
     exposes: {
       "./Button": "./src/Button",
     },
     // sharing code based on the installed version, to allow for multiple vendors with different versions
     shared: [
       {
         react: { singleton: true, requiredVersion: false },
         "react-dom": { singleton: true, requiredVersion: false },
         "react-redux": { singleton: true, requiredVersion: false },
       },
     ],
   })

hopefully this makes sense, sadly i still reading webpack to try be able to help, but the logic and negotiations i think is pretty clear

i updated this project just in case https://github.com/crodriguez-plitzi/webpack-federation-playground

@piterjov
Copy link

So it is not possible to federate modules from nextJS CSR without a plugin at all? What about sidecar (sub-app) that Jack showed in his video?

@ScriptedAlchemy
Copy link
Member Author

So it is not possible to federate modules from nextJS CSR without a plugin at all? What about sidecar (sub-app) that Jack showed in his video?

It's possible, I maintain both SSR and CSR implementations.

Sidecars work and are free - but I know it can sometimes be a little tricky to get shared modules on next.js via sidecars sometimes, that can be worked around tho

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement module-federation issues related to module federation webpack-5
Projects
None yet
Development

No branches or pull requests

9 participants