Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add module federation plugin #1764

Open
wants to merge 68 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
898f785
feat: support module federation
xusd320 Nov 21, 2024
45c519c
feat: mf exposes to remote entries
xusd320 Nov 23, 2024
aebf286
chore: code styles
xusd320 Nov 23, 2024
2ec6a9f
feat: mf container entry impl
xusd320 Nov 24, 2024
768259e
fix: mf container entry
xusd320 Nov 24, 2024
391a4fc
fix: mf runtime initOptions
xusd320 Nov 25, 2024
d98c303
feat: add containter references
xusd320 Nov 26, 2024
aac2c2d
feat: impl mf remote
xusd320 Nov 29, 2024
7a61e41
feat: improve mf exposes
xusd320 Dec 3, 2024
d15d7c6
fix: mf exposes runtime factory
xusd320 Dec 4, 2024
7d801e3
fix: mf plugin execution order
xusd320 Dec 4, 2024
b450d89
chore: update mf demo
xusd320 Dec 4, 2024
5f62944
feat: generate mf manifest in rust
xusd320 Dec 5, 2024
a842cbd
fix: remote stats.json
xusd320 Dec 5, 2024
5044465
refactor: code styles
xusd320 Dec 5, 2024
d67f792
chore: add some FIXME
xusd320 Dec 5, 2024
609e94f
refactor: mf plugin mods files
xusd320 Dec 6, 2024
528f8cc
refactor: mf plugin mods files
xusd320 Dec 6, 2024
60452a3
chore: remove dead code
xusd320 Dec 6, 2024
71ff5b8
--wip-- [skip ci]
xusd320 Dec 10, 2024
bbc7780
Merge remote-tracking branch 'origin' into feat/module-federation-plugin
xusd320 Dec 10, 2024
1f75e17
fix: remote stats.json
xusd320 Dec 10, 2024
6fc54c9
fix: typos
xusd320 Dec 11, 2024
17170fb
Merge remote-tracking branch 'origin' into feat/module-federation-plugin
xusd320 Dec 12, 2024
9efd0e6
Merge remote-tracking branch 'origin' into feat/module-federation-plugin
xusd320 Dec 13, 2024
367561b
chore: simpify mf runtime codes fmt
xusd320 Dec 13, 2024
b7c9136
refactor: mf containter plugin
xusd320 Dec 13, 2024
606ca18
feat: mf shared workaround
xusd320 Jan 8, 2025
465c32f
feat: mf shared workaround
xusd320 Jan 9, 2025
9f3ef78
fix: runtime template and remove some useless codes
xusd320 Jan 10, 2025
76ae884
fix: mf dev server
xusd320 Jan 13, 2025
d78087d
fix: mf shared config
xusd320 Jan 15, 2025
dbd6a38
feat: supports chunk group exclude
xusd320 Jan 16, 2025
4ba5bb3
feat: mf patch code splitting
xusd320 Jan 20, 2025
fb15a73
feat: mf shared manifest
xusd320 Jan 20, 2025
eeec6b5
feat: add config hash for mf shared module
xusd320 Jan 20, 2025
3687f8a
Merge remote-tracking branch 'origin' into feat/module-federation-plugin
xusd320 Jan 21, 2025
ca5c7d5
chore: update mako typings
xusd320 Jan 21, 2025
38dc51b
chore: code styles
xusd320 Jan 21, 2025
bccb33e
chore: fix typo
xusd320 Jan 21, 2025
5ff27c8
chore: code styles
xusd320 Jan 22, 2025
441bed8
perf: improve performance
xusd320 Jan 23, 2025
e4b1d1d
chore: code styles
xusd320 Jan 23, 2025
d7aefa8
chore: rename types
xusd320 Jan 23, 2025
7a38151
feat: add options to disable mf manifest
xusd320 Jan 23, 2025
3c88b49
feat: entry config should be defined as BTreeMap
xusd320 Jan 23, 2025
6462064
fix: mf shared consume and supports eager config
xusd320 Jan 23, 2025
2ac48bc
fix: mf shared eager
xusd320 Jan 23, 2025
1b8b9d9
fix: not generate chunk for mf remote module
xusd320 Jan 24, 2025
5378030
fix: typos
xusd320 Jan 24, 2025
c4e1ff9
feat: add entry filename supports
xusd320 Feb 6, 2025
4a0eabb
chore: remove meaning less changes
xusd320 Feb 6, 2025
755f5e0
fix: entry filename and mf config
xusd320 Feb 6, 2025
f66deea
Merge branch 'master' into feat/module-federation-plugin
xusd320 Feb 6, 2025
d3105d9
release: @umijs/[email protected]
xusd320 Feb 6, 2025
b35d7ca
fix: ignore shared dep when it is been external
xusd320 Feb 7, 2025
771e641
Revert "release: @umijs/[email protected]"
xusd320 Feb 7, 2025
6179982
release: @umijs/[email protected]
xusd320 Feb 7, 2025
f9e1174
fix: skip serialize mf manifest remoteEntry if none
xusd320 Feb 7, 2025
b121dbf
fix: mf manifest remoteEntry address
xusd320 Feb 7, 2025
8e2364a
Revert "release: @umijs/[email protected]"
xusd320 Feb 7, 2025
9dca297
fix: typo
xusd320 Feb 7, 2025
b6c6995
fix: mako mf manifest publicPath
xusd320 Feb 8, 2025
23dacc2
fix: mf manifest panic
xusd320 Feb 8, 2025
2a9df44
fix: mf typings
xusd320 Feb 8, 2025
aba1029
test: add e2e test for mf
xusd320 Feb 8, 2025
a59da24
fix: typo
xusd320 Feb 8, 2025
ebc21d6
chore: update README
xusd320 Feb 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<div align="center">

<img src="https://img.alicdn.com/imgextra/i2/O1CN01kdmA7X1FVqCPcRi3L_!!6000000000493-2-tps-584-584.png" alt="Mako logo" width="160" height="160" />

# Mako 🦈
Expand All @@ -10,7 +10,7 @@
[![](https://badgen.net/npm/license/umi)](https://www.npmjs.com/package/@umijs/mako)
[![codecov](https://codecov.io/gh/umijs/mako/graph/badge.svg?token=ptCnNedFGf)](https://codecov.io/gh/umijs/mako)

Mako `['mɑːkoʊ]` is an **extremely fast**, **production-grade** web bundler based on **Rust**.
Mako `['mɑːkoʊ]` is an **extremely fast**, **production-grade** web bundler based on **Rust**.

✨ See more at [makojs.dev](https://makojs.dev).
</div>
Expand Down Expand Up @@ -56,6 +56,7 @@ This project is inspired by:
- [oxc-resolver](https://github.com/oxc-project/oxc-resolver) by [@Boshen](https://github.com/Boshen) which powered the resolver of Mako.
- [Oxc](https://github.com/oxc-project/oxc/) by [@Boshen](https://github.com/Boshen) from which we learned a lot about how to develop efficiently in Rust.
- [biome](https://github.com/biomejs/biome) by [@ematipico](https://github.com/ematipico) from which we learned a lot about how to develop efficiently in Rust.
- [module-federation](https://github.com/module-federation/core) by [@ScriptedAlchemy](https://github.com/ScriptedAlchemy),which inspired a lot and powered the module federation feature of Mako.

## LICENSE

Expand Down
1 change: 1 addition & 0 deletions crates/binding/src/js_plugin.rs
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,7 @@ impl Plugin for JsPlugin {
&self,
content: &mut Content,
path: &str,
_is_entry: bool,
context: &Arc<Context>,
) -> Result<Option<Content>> {
if let Some(hook) = &self.hooks.transform_include {
Expand Down
21 changes: 21 additions & 0 deletions crates/binding/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -156,9 +156,30 @@ pub struct BuildParams {
rscClient?: false | {
"logServerComponent": "error" | "ignore";
};
moduleFederation?: {
name: string;
// filename?: string;
exposes?: Record<string, string>;
shared: Record<string,
{ singleton?: bool; strictVersion?: bool; requiredVersion?: string;
/* eager?: bool; */ /* shareScope?: string; */
}
>;
remotes?: Record<string, string>;
runtimePlugins?: string[];
shareScope?: string;
shareStrategy?: "version-first" | "loaded-first";
implementation: string;
};
experimental?: {
webpackSyntaxValidate?: string[];
requireContext?: bool;
ignoreNonLiteralRequire?: bool;
magicComment?: bool;
detectCircularDependence?: { ignore?: string[] };
rustPlugins?: Array<[string, any]>;
centralEnsure?: bool,
importsChecker?: bool,
};
watch?: {
ignoredPaths?: string[];
Expand Down
57 changes: 55 additions & 2 deletions crates/mako/src/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ use crate::ast::file::{Content, File, JsContent};
use crate::ast::utils::get_module_system;
use crate::compiler::{Compiler, Context};
use crate::generate::chunk_pot::util::hash_hashmap;
use crate::module::{Module, ModuleAst, ModuleId, ModuleInfo};
use crate::module::{FedereationModuleType, Module, ModuleAst, ModuleId, ModuleInfo, ModuleSystem};
use crate::plugin::NextBuildParam;
use crate::resolve::ResolverResource;
use crate::resolve::{ConsumeSharedInfo, RemoteInfo, ResolverResource};
use crate::utils::thread_pool;

#[derive(Debug, Error)]
Expand Down Expand Up @@ -46,6 +46,15 @@ impl Compiler {
rs.send(result).unwrap();
});
};

let build_consume_share_with_pool = |consume_share_info: ConsumeSharedInfo| {
let rs = rs.clone();
let context = self.context.clone();
thread_pool::spawn(move || {
let result = Self::build_consume_shared_module(consume_share_info, context.clone());
rs.send(result).unwrap();
});
};
let mut count = 0;
for file in files {
count += 1;
Expand Down Expand Up @@ -127,13 +136,22 @@ impl Compiler {
ResolverResource::Ignored(_) => {
Self::create_ignored_module(&path, self.context.clone())
}
ResolverResource::Remote(remote_into) => {
Self::create_remote_module(remote_into)
}
ResolverResource::Shared(consume_share_info) => {
count += 1;
build_consume_share_with_pool(consume_share_info.clone());
Self::create_empty_module(&dep_module_id)
}
};

// 拿到依赖之后需要直接添加 module 到 module_graph 里,不能等依赖 build 完再添加
// 是因为由于是异步处理各个模块,后者会导致大量重复任务的 build_module 任务(3 倍左右)
module_ids.insert(module.id.clone());
module_graph.add_module(module);
}

module_graph.add_dependency(&module_id, &dep_module_id, dep.dependency);
}
if count == 0 {
Expand Down Expand Up @@ -267,6 +285,12 @@ __mako_require__.loadScript('{}', (e) => e.type === 'load' ? resolve() : reject(
result
}
}
pub fn build_consume_shared_module(
consume_share_info: ConsumeSharedInfo,
_context: Arc<Context>,
) -> Result<Module> {
Ok(Self::create_consume_share_module(consume_share_info))
}

pub fn build_module(
file: &File,
Expand All @@ -279,6 +303,7 @@ __mako_require__.loadScript('{}', (e) => e.type === 'load' ? resolve() : reject(
let content = context.plugin_driver.load_transform(
&mut content,
&file.path.to_string_lossy(),
file.is_entry,
&context,
)?;
file.set_content(content);
Expand Down Expand Up @@ -329,4 +354,32 @@ __mako_require__.loadScript('{}', (e) => e.type === 'load' ? resolve() : reject(
let module = Module::new(module_id, is_entry, Some(info));
Ok(module)
}

pub(crate) fn create_remote_module(remote_info: RemoteInfo) -> Module {
Module {
is_entry: false,
id: remote_info.module_id.as_str().into(),
info: Some(ModuleInfo {
resolved_resource: Some(ResolverResource::Remote(remote_info.clone())),
federation: Some(FedereationModuleType::Remote),
..Default::default()
}),
side_effects: true,
}
}

pub(crate) fn create_consume_share_module(consume_share_info: ConsumeSharedInfo) -> Module {
Module {
is_entry: false,
id: consume_share_info.module_id.as_str().into(),
info: Some(ModuleInfo {
deps: consume_share_info.deps.clone(),
resolved_resource: Some(ResolverResource::Shared(consume_share_info.clone())),
federation: Some(FedereationModuleType::ConsumeShare),
module_system: ModuleSystem::Custom,
..Default::default()
}),
side_effects: true,
}
}
}
8 changes: 6 additions & 2 deletions crates/mako/src/build/analyze_deps.rs
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,14 @@ impl AnalyzeDeps {
);
match result {
Ok(resolver_resource) => {
resolved_deps.push(ResolvedDep {
let resolved_dep = ResolvedDep {
resolver_resource,
dependency: dep,
});
};
context
.plugin_driver
.after_resolve(&resolved_dep, &context)?;
resolved_deps.push(resolved_dep);
}
Err(_err) => {
missing_deps.insert(dep.source.clone(), dep);
Expand Down
9 changes: 7 additions & 2 deletions crates/mako/src/compiler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ use crate::generate::optimize_chunk::OptimizeChunksInfo;
use crate::module_graph::ModuleGraph;
use crate::plugin::{Plugin, PluginDriver, PluginGenerateEndParams};
use crate::plugins;
use crate::plugins::module_federation::ModuleFederationPlugin;
use crate::resolve::{get_resolvers, Resolvers};
use crate::share::helpers::SWC_HELPERS;
use crate::stats::StatsInfo;
Expand Down Expand Up @@ -314,7 +315,6 @@ impl Compiler {
Arc::new(plugins::bundless_compiler::BundlessCompilerPlugin {}),
);
}

if std::env::var("DEBUG_GRAPH").is_ok_and(|v| v == "true") {
plugins.push(Arc::new(plugins::graphviz::Graphviz {}));
}
Expand All @@ -327,6 +327,10 @@ impl Compiler {
plugins.push(Arc::new(plugins::central_ensure::CentralChunkEnsure {}));
}

if let Some(mf_cfg) = config.module_federation.as_ref() {
plugins.push(Arc::new(ModuleFederationPlugin::new(mf_cfg.clone())));
}

if let Some(minifish_config) = &config._minifish {
let inject = if let Some(inject) = &minifish_config.inject {
let mut map = HashMap::new();
Expand Down Expand Up @@ -423,7 +427,7 @@ impl Compiler {
.entry
.values()
.map(|entry| {
let mut entry = entry.to_string_lossy().to_string();
let mut entry = entry.import.to_string_lossy().to_string();
let is_browser = matches!(
self.context.config.platform,
crate::config::Platform::Browser
Expand Down Expand Up @@ -492,6 +496,7 @@ impl Compiler {
self.context
.plugin_driver
.generate_end(&params, &self.context)?;

self.context.plugin_driver.write_bundle(&self.context)?;
Ok(())
}
Expand Down
32 changes: 22 additions & 10 deletions crates/mako/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ mod code_splitting;
mod dev_server;
mod devtool;
mod duplicate_package_checker;
pub mod entry;
mod experimental;
mod external;
mod generic_usize;
Expand All @@ -12,6 +13,7 @@ mod macros;
mod manifest;
mod minifish;
mod mode;
pub mod module_federation;
mod module_id_strategy;
mod optimization;
mod output;
Expand All @@ -28,9 +30,9 @@ mod tree_shaking;
mod umd;
mod watch;

use std::collections::{BTreeMap, HashMap};
use std::collections::HashMap;
use std::fmt;
use std::path::{Path, PathBuf};
use std::path::Path;

pub use analyze::AnalyzeConfig;
use anyhow::{anyhow, Result};
Expand All @@ -42,6 +44,7 @@ pub use devtool::{deserialize_devtool, DevtoolConfig};
pub use duplicate_package_checker::{
deserialize_check_duplicate_package, DuplicatePackageCheckerConfig,
};
use entry::{Entry, EntryItem};
use experimental::ExperimentalConfig;
pub use external::{
ExternalAdvanced, ExternalAdvancedSubpath, ExternalAdvancedSubpathConverter,
Expand All @@ -54,6 +57,7 @@ pub use manifest::{deserialize_manifest, ManifestConfig};
use miette::{miette, ByteOffset, Diagnostic, NamedSource, SourceOffset, SourceSpan};
pub use minifish::{deserialize_minifish, MinifishConfig};
pub use mode::Mode;
use module_federation::ModuleFederationConfig;
pub use module_id_strategy::ModuleIdStrategy;
pub use optimization::{deserialize_optimization, OptimizationConfig};
use output::get_default_chunk_loading_global;
Expand Down Expand Up @@ -135,7 +139,7 @@ pub enum CopyConfig {
#[derive(Deserialize, Serialize, Debug)]
#[serde(rename_all = "camelCase")]
pub struct Config {
pub entry: BTreeMap<String, PathBuf>,
pub entry: Entry,
pub output: OutputConfig,
pub resolve: ResolveConfig,
#[serde(deserialize_with = "deserialize_manifest", default)]
Expand Down Expand Up @@ -228,6 +232,7 @@ pub struct Config {
default
)]
pub check_duplicate_package: Option<DuplicatePackageCheckerConfig>,
pub module_federation: Option<ModuleFederationConfig>,
// 是否开启 case sensitive 检查,只有mac平台才需要开启
#[serde(rename = "caseSensitiveCheck")]
pub case_sensitive_check: bool,
Expand Down Expand Up @@ -369,7 +374,13 @@ impl Config {
for ext in JS_EXTENSIONS {
let file_path = root.join(file_path).with_extension(ext);
if file_path.exists() {
config.entry.insert("index".to_string(), file_path);
config.entry.insert(
"index".to_string(),
EntryItem {
filename: None,
import: file_path,
},
);
break 'outer;
}
}
Expand All @@ -382,28 +393,29 @@ impl Config {
// normalize entry
config.entry.iter_mut().try_for_each(|(k, v)| {
#[allow(clippy::needless_borrows_for_generic_args)]
if let Ok(entry_path) = root.join(&v).canonicalize()
if let Ok(entry_path) = root.join(&v.import).canonicalize()
&& entry_path.is_file()
{
*v = entry_path;
v.import = entry_path;
} else {
for ext in JS_EXTENSIONS {
#[allow(clippy::needless_borrows_for_generic_args)]
if let Ok(entry_path) = root.join(&v).with_extension(ext).canonicalize()
if let Ok(entry_path) =
root.join(&v.import).with_extension(ext).canonicalize()
&& entry_path.is_file()
{
*v = entry_path;
v.import = entry_path;
return Ok(());
}

if let Ok(entry_path) = root
.join(&v)
.join(&v.import)
.join("index")
.with_extension(ext)
.canonicalize()
&& entry_path.is_file()
{
*v = entry_path;
v.import = entry_path;
return Ok(());
}
}
Expand Down
37 changes: 37 additions & 0 deletions crates/mako/src/config/entry.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
use std::collections::BTreeMap;
use std::path::PathBuf;

use serde::{Deserialize, Serialize};
use serde_json::Value;

#[derive(Serialize, Debug)]
pub struct EntryItem {
#[serde(default)]
pub filename: Option<String>,
pub import: PathBuf,
}

pub type Entry = BTreeMap<String, EntryItem>;

impl<'de> Deserialize<'de> for EntryItem {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
let value: serde_json::Value = serde_json::Value::deserialize(deserializer)?;
match &value {
Value::String(s) => Ok(EntryItem {
filename: None,
import: s.into(),
}),
Value::Object(_) => {
Ok(serde_json::from_value::<EntryItem>(value).map_err(serde::de::Error::custom)?)
}
_ => Err(serde::de::Error::custom(format!(
"invalid `{}` value: {}",
stringify!(deserialize_umd).replace("deserialize_", ""),
value
))),
}
}
}
Loading
Loading