Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: npm workspace and better Deno workspace support #24334

Merged
merged 101 commits into from
Jul 4, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
101 commits
Select commit Hold shift click to select a range
3660270
Start on workspace refactor.
dsherret Jun 17, 2024
2c591ef
Some work on tasks
dsherret Jun 18, 2024
81dc4a8
Switching more to workspace
dsherret Jun 18, 2024
271df70
More.
dsherret Jun 18, 2024
b155d9a
Remove cli/args/package_json.rs
dsherret Jun 18, 2024
c1d52f8
some work on deno compile
dsherret Jun 18, 2024
c8dbbb1
tasks
dsherret Jun 19, 2024
04fd3c6
update publishing
dsherret Jun 19, 2024
03b30f4
Adding fmting
dsherret Jun 19, 2024
1e8b196
Merge branch 'main' into workspace_start
dsherret Jun 19, 2024
cb6b545
some work on linting
dsherret Jun 19, 2024
5e9fd3f
finish lint
dsherret Jun 20, 2024
4fe5441
Some work on lsp and installing from all the packages
dsherret Jun 20, 2024
19ad5c4
separate out test options from workspace test options
dsherret Jun 20, 2024
6eed364
finish up test
dsherret Jun 20, 2024
c04afc7
Add bench
dsherret Jun 20, 2024
49b1a23
compiling
dsherret Jun 20, 2024
76dcc18
handle base properly
dsherret Jun 20, 2024
4205667
fix slow types linting
dsherret Jun 20, 2024
2033d89
Use SharedLocal and fix some vendor tests.
dsherret Jun 21, 2024
297d191
Start making import map optional
dsherret Jun 21, 2024
8583651
Start investigating @types issue
dsherret Jun 21, 2024
df4e20e
Fix. Wasn't thinking.
dsherret Jun 21, 2024
a7b5857
fix info
dsherret Jun 21, 2024
c5df354
fix watcher tests
dsherret Jun 21, 2024
77c0977
working npm tests
dsherret Jun 21, 2024
d38c216
some import specifier fixes
dsherret Jun 22, 2024
ffa9bb1
Fix some lsp issues.
dsherret Jun 24, 2024
e887d93
Fix test
dsherret Jun 24, 2024
64c20ac
We understand the npm workspace now
dsherret Jun 24, 2024
171bed6
Make deno compile work with relative specifiers
dsherret Jun 24, 2024
5c330c1
Fix in import_map/deno_emit
dsherret Jun 25, 2024
2b4247c
Compile fixes
dsherret Jun 25, 2024
3c9fec7
Fix some byonm stuff
dsherret Jun 25, 2024
e691125
Merge branch 'main' into workspace_start
dsherret Jun 25, 2024
4d3ac9e
Update to actual deps
dsherret Jun 26, 2024
e844ee1
refactor: move `PackageJson` to deno_config
dsherret Jun 26, 2024
5770c6b
Update
dsherret Jun 26, 2024
1ca7b82
fix failing tests
dsherret Jun 26, 2024
6b65431
Update deno_config
dsherret Jun 26, 2024
25439b0
fix failing test by clearing package json cache
dsherret Jun 26, 2024
c72410e
fix formatting
dsherret Jun 26, 2024
f41a348
Merge branch 'main' into workspace_start
dsherret Jun 26, 2024
b1e4645
Merge branch 'package_json_deno_config' into workspace_start
dsherret Jun 26, 2024
11eb6ff
Fixes for latest changes
dsherret Jun 26, 2024
1e28b4e
clippy
dsherret Jun 26, 2024
f626f10
Resolve todo
dsherret Jun 26, 2024
3fe1853
update from what will be main
dsherret Jun 26, 2024
c4b55bf
Resolve some todos
dsherret Jun 26, 2024
698d9aa
Merge branch 'main' into workspace_start
dsherret Jun 26, 2024
1281f76
Merge branch 'main' into workspace_start
dsherret Jun 26, 2024
217a217
fix failing tests
dsherret Jun 26, 2024
163bef4
npm workspace resolution
dsherret Jun 27, 2024
05ce93e
handle going to npm workspace deps for npm specifiers too
dsherret Jun 27, 2024
de1b298
Add test for workspace
dsherret Jun 27, 2024
2fd7ffa
add an npm specifier referencing a workspace package
dsherret Jun 27, 2024
0a9e1c3
move this down to deno_config
dsherret Jun 27, 2024
33ab587
updates for latest
dsherret Jun 27, 2024
2c35049
update to new Workspace::discover signature
dsherret Jun 28, 2024
6e9c118
Support `@workspace` tag in an npm specifier
dsherret Jun 28, 2024
a063f9a
Support pkg json "workspace:" specifiers
dsherret Jun 28, 2024
91c3c1b
Move test
dsherret Jun 28, 2024
8851529
Initialize node_modules dir with used workspace members
dsherret Jun 28, 2024
8dd40a9
Fix some tests and update var names
dsherret Jun 29, 2024
66c863b
Update workspace basic test
dsherret Jun 29, 2024
5df7730
Merge branch 'main' into workspace_start
dsherret Jun 29, 2024
44c189e
Merge branch 'main' into workspace_start
dsherret Jun 29, 2024
07a57b4
Update for failing compile
dsherret Jun 29, 2024
ebfe959
Merge branch 'main' of https://github.com/denoland/deno into workspac…
dsherret Jul 1, 2024
9a1e921
improve resolving root dir for deno compile
dsherret Jul 1, 2024
497c22d
Resolve more todos about errors in sub path resolution
dsherret Jul 1, 2024
b0fc65d
Add tests for and fix formatting
dsherret Jul 1, 2024
914be64
Fix some issues and add tests for linting
dsherret Jul 1, 2024
bb1988e
Add no slow types workspace lint test
dsherret Jul 1, 2024
36c1ca0
Merge branch 'main' into workspace_start
dsherret Jul 1, 2024
7fee5f6
Format and ignore some files from formatting
dsherret Jul 1, 2024
ea41334
Add tests for exports not found in npm workspace
dsherret Jul 1, 2024
59fb3db
Merge branch 'main' into workspace_start
dsherret Jul 1, 2024
f8cf42a
diagnostics
dsherret Jul 1, 2024
57fdbde
Add tests for lockfile going to the workspace root
dsherret Jul 1, 2024
8a97d8c
Add test for check
dsherret Jul 2, 2024
9ce4caa
Update deno_config
dsherret Jul 2, 2024
803b097
Add bench tests
dsherret Jul 2, 2024
754b308
add test
dsherret Jul 2, 2024
9d38ecf
Merge branch 'main' into workspace_start
dsherret Jul 2, 2024
6aaac91
fix specs
dsherret Jul 2, 2024
1448564
was using a backslash in a path
dsherret Jul 2, 2024
3f7569f
Bump ci cache
dsherret Jul 2, 2024
7e6644f
Switch back to resolving based on multiple args.
dsherret Jul 3, 2024
db95f58
update some tests
dsherret Jul 3, 2024
43f1f5f
Merge branch 'main' into workspace_start
dsherret Jul 3, 2024
372ceed
update tests
dsherret Jul 3, 2024
0c98628
fix copyright
dsherret Jul 3, 2024
51bed91
add tests for vendor
dsherret Jul 3, 2024
8ac81df
update test for pkg with no deps in lockfile in workspace
dsherret Jul 3, 2024
2a957e0
Merge branch 'main' into workspace_start
dsherret Jul 3, 2024
650a43d
- Fix private registries not working with auto-install in deno compile
dsherret Jul 3, 2024
ae7ebec
maybe fix windows tests
dsherret Jul 3, 2024
4d0d459
Updates from review
dsherret Jul 3, 2024
21673a3
Merge branch 'main' into workspace_start
dsherret Jul 3, 2024
b282f75
Upgrade deno_unsync. Remove fatal diagnostic test because it's in den…
dsherret Jul 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Switch back to resolving based on multiple args.
  • Loading branch information
dsherret committed Jul 3, 2024
commit 7e6644fbf99263d8f79e11caa0c37b3f86b4dd36
2 changes: 0 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -376,3 376,6 @@ opt-level = 3
opt-level = 3
[profile.release.package.zstd-sys]
opt-level = 3

[patch.crates-io]
deno_config = { path = "../deno_config" }
96 changes: 62 additions & 34 deletions cli/args/flags.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 36,7 @@ use std::path::PathBuf;
use std::str::FromStr;

use crate::args::resolve_no_prompt;
use crate::util::collections::CheckedSet;
use crate::util::fs::canonicalize_path;

use super::flags_net;
Expand Down Expand Up @@ -862,28 863,41 @@ impl Flags {
args
}

/// Extract the directory path the config file should be discovered from.
/// Extract the directory paths the config file should be discovered from.
///
/// Returns `None` if the config file should not be auto-discovered.
pub fn config_path_arg(&self, current_dir: &Path) -> Option<PathBuf> {
pub fn config_path_args(&self, current_dir: &Path) -> Option<Vec<PathBuf>> {
fn resolve_multiple_files(
files: &[String],
current_dir: &Path,
) -> Vec<PathBuf> {
let mut seen = CheckedSet::with_capacity(files.len());
let result = files
.iter()
.filter_map(|p| {
let path = normalize_path(current_dir.join(p).parent()?);
if seen.insert(&path) {
Some(path)
} else {
None
}
})
.collect::<Vec<_>>();
if result.is_empty() {
vec![current_dir.to_path_buf()]
} else {
result
}
}

use DenoSubcommand::*;
match &self.subcommand {
Fmt(FmtFlags { files, .. }) => Some(
files
.include
.iter()
.filter_map(|p| Some(normalize_path(current_dir.join(p).parent()?)))
.next()
.unwrap_or_else(|| current_dir.to_path_buf()),
),
Lint(LintFlags { files, .. }) => Some(
files
.include
.iter()
.filter_map(|p| Some(normalize_path(current_dir.join(p).parent()?)))
.next()
.unwrap_or_else(|| current_dir.to_path_buf()),
),
Fmt(FmtFlags { files, .. }) => {
Some(resolve_multiple_files(&files.include, current_dir))
}
Lint(LintFlags { files, .. }) => {
Some(resolve_multiple_files(&files.include, current_dir))
}
Run(RunFlags { script, .. })
| Compile(CompileFlags {
source_file: script,
Expand All @@ -894,17 908,17 @@ impl Flags {
|| module_specifier.scheme() == "npm"
{
if let Ok(p) = module_specifier.to_file_path() {
Some(p.parent().unwrap().to_path_buf())
Some(vec![p.parent().unwrap().to_path_buf()])
} else {
Some(current_dir.to_path_buf())
Some(vec![current_dir.to_path_buf()])
}
} else {
// When the entrypoint doesn't have file: scheme (it's the remote
// script), then we don't auto discover config file.
None
}
} else {
Some(current_dir.to_path_buf())
Some(vec![current_dir.to_path_buf()])
}
}
Task(TaskFlags {
Expand All @@ -914,11 928,11 @@ impl Flags {
// attempt to resolve the config file from the task subcommand's
// `--cwd` when specified
match canonicalize_path(&PathBuf::from(path)) {
Ok(path) => Some(path),
Err(_) => Some(current_dir.to_path_buf()),
Ok(path) => Some(vec![path]),
Err(_) => Some(vec![current_dir.to_path_buf()]),
}
}
_ => Some(current_dir.to_path_buf()),
_ => Some(vec![current_dir.to_path_buf()]),
}
}

Expand Down Expand Up @@ -9236,30 9250,44 @@ mod tests {
fn test_config_path_args() {
let flags = flags_from_vec(svec!["deno", "run", "foo.js"]).unwrap();
let cwd = std::env::current_dir().unwrap();
assert_eq!(flags.config_path_arg(&cwd), Some(cwd.clone()));

assert_eq!(flags.config_path_args(&cwd), Some(vec![cwd.clone()]));

let flags = flags_from_vec(svec!["deno", "run", "sub_dir/foo.js"]).unwrap();
let cwd = std::env::current_dir().unwrap();
assert_eq!(
flags.config_path_arg(&cwd),
Some(cwd.join("sub_dir").clone())
flags.config_path_args(&cwd),
Some(vec![cwd.join("sub_dir").clone()])
);

let flags =
flags_from_vec(svec!["deno", "run", "https://example.com/foo.js"])
.unwrap();
assert_eq!(flags.config_path_arg(&cwd), None);
assert_eq!(flags.config_path_args(&cwd), None);

let flags =
flags_from_vec(svec!["deno", "lint", "dir/a.js", "dir/b.js"]).unwrap();
assert_eq!(flags.config_path_arg(&cwd), Some(cwd.join("dir")));
flags_from_vec(svec!["deno", "lint", "dir/a/a.js", "dir/b/b.js"])
.unwrap();
assert_eq!(
flags.config_path_args(&cwd),
Some(vec![cwd.join("dir/a/"), cwd.join("dir/b/")])
);

let flags = flags_from_vec(svec!["deno", "lint"]).unwrap();
assert_eq!(flags.config_path_arg(&cwd), Some(cwd.clone()));
assert_eq!(flags.config_path_args(&cwd), Some(vec![cwd.clone()]));

let flags =
flags_from_vec(svec!["deno", "fmt", "dir/a.js", "dir/b.js"]).unwrap();
assert_eq!(flags.config_path_arg(&cwd), Some(cwd.join("dir")));
let flags = flags_from_vec(svec![
"deno",
"fmt",
"dir/a/a.js",
"dir/a/a2.js",
"dir/b.js"
])
.unwrap();
assert_eq!(
flags.config_path_args(&cwd),
Some(vec![cwd.join("dir/a/"), cwd.join("dir/b/")])
);
}

#[test]
Expand Down
4 changes: 2 additions & 2 deletions cli/args/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -892,9 892,9 @@ impl CliOptions {

let workspace = match &flags.config_flag {
deno_config::ConfigFlag::Discover => {
if let Some(start_dir) = flags.config_path_arg(&initial_cwd) {
if let Some(start_dirs) = flags.config_path_args(&initial_cwd) {
Workspace::discover(
WorkspaceDiscoverStart::Dir(&start_dir),
WorkspaceDiscoverStart::Dirs(&start_dirs),
&WorkspaceDiscoverOptions {
fs: &DenoConfigFsAdapter::new(&RealFs),
pkg_json_cache: Some(
Expand Down
4 changes: 3 additions & 1 deletion cli/lsp/language_server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3557,7 3557,9 @@ impl Inner {
.unwrap_or_else(|| self.initial_cwd.clone());
// todo: we need a way to convert config data to a Workspace
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @nayeemrmn what's done here is probably good enough for now.

let workspace = Arc::new(Workspace::discover(
deno_config::workspace::WorkspaceDiscoverStart::Dir(&initial_cwd),
deno_config::workspace::WorkspaceDiscoverStart::Dirs(&[
initial_cwd.clone()
]),
&WorkspaceDiscoverOptions {
fs: &DenoConfigFsAdapter::new(&deno_runtime::deno_fs::RealFs),
pkg_json_cache: None,
Expand Down
36 changes: 36 additions & 0 deletions cli/util/collections.rs
Original file line number Diff line number Diff line change
@@ -0,0 1,36 @@
use std::marker::PhantomData;

pub struct CheckedSet<T: std::hash::Hash ?Sized> {
_kind: PhantomData<T>,
checked: std::collections::HashSet<u64>,
}

impl<T: std::hash::Hash ?Sized> Default for CheckedSet<T> {
fn default() -> Self {
Self {
_kind: Default::default(),
checked: Default::default(),
}
}
}

impl<T: std::hash::Hash ?Sized> CheckedSet<T> {
pub fn with_capacity(capacity: usize) -> Self {
Self {
_kind: PhantomData,
checked: std::collections::HashSet::with_capacity(capacity),
}
}

pub fn insert(&mut self, value: &T) -> bool {
self.checked.insert(self.get_hash(value))
}

fn get_hash(&self, value: &T) -> u64 {
use std::collections::hash_map::DefaultHasher;
use std::hash::Hasher;
let mut hasher = DefaultHasher::new();
value.hash(&mut hasher);
hasher.finish()
}
}
1 change: 1 addition & 0 deletions cli/util/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 2,7 @@

// Note: Only add code in this folder that has no application specific logic
pub mod checksum;
pub mod collections;
pub mod console;
pub mod diff;
pub mod display;
Expand Down