Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sync.waitgroup behaves differently from Golang #6870

Open
axetroy opened this issue Nov 18, 2020 · 7 comments
Open

sync.waitgroup behaves differently from Golang #6870

axetroy opened this issue Nov 18, 2020 · 7 comments
Labels
Bug This tag is applied to issues which reports bugs.

Comments

@axetroy
Copy link

axetroy commented Nov 18, 2020 •
edited
Loading

V version:
V 0.1.29 ed874ff
OS:
macos 11.0.1

What did you do?

I wrote a function to traverse directories using goroutine

// main.v
// run: v run main.v
module main

import os
import sync

fn walk(dir string, mut wg sync.WaitGroup, ch chan int) {
	defer {
		wg.done()
	}

	ch <- 1

	println('before: $ch.len')

	files := os.ls(dir) or {
		panic(err)
	}

	s := <-ch or {
		panic(err)
	}

	println('after: $ch.len')

	if s <0 {
		// do nothing
	}

	for file in files {
		target := os.join_path(dir, file)
		if os.is_dir(target) {
			wg.add(1)
			go walk(target, mut wg, ch)
		} else {
			// println(target)
			// ch <- 1
		}
	}
}

fn main() {
	mut wg := sync.new_waitgroup()

	wg.add(1)

	mut ch := chan int{cap: 255}

	go walk("/path/to/dir", mut wg, ch)

	wg.wait()
	ch.close()

	println('done')
}

What did you expect to see?

The proceeding should exit, wg.Wait() should execute like expect.

What did you see instead?

At least I didn't see anything wrong from the above code.

Normally, the process will exit normally.

But in reality, the process will hang. wg.wait() will never execute

For comparison, I wrote an equivalent Golang code

// main.go
// run: go run main.go
package main

import (
	"io/ioutil"
	"path"
	"sync"
)

func walk(dir string, wg *sync.WaitGroup, ch chan int) {
	defer wg.Done()

	ch <- 1

	println("before", len(ch))

	files, err := ioutil.ReadDir(dir)

	if err != nil {
		panic(err)
	}

	s := <-ch

	println("after", len(ch))

	if s < 0 {
		// do nothing
	}

	for _, file := range files {
		target := path.Join(dir, file.Name())
		if file.IsDir() {
			wg.Add(1)
			go walk(target, wg, ch)
		} else {
			// println(target)
			// ch <- 1
		}
	}
}

func main() {
	wg := sync.WaitGroup{}

	wg.Add(1)

	var ch = make(chan int, 255)

	go walk("/path/to/dir", &wg, ch)

	wg.Wait()
	close(ch)

	println("done")
}

and Golang's code works fine for me.
@axetroy axetroy added the Bug This tag is applied to issues which reports bugs. label Nov 18, 2020
@zakuro9715
Copy link
Member

I tried it. walking './vlib' is works. But walking '/home/zakuro/src' is hungup. In golang, Works both.

I think, This problem occurs when target files too many.

@axetroy
Copy link
Author

axetroy commented Feb 15, 2021

update:

V 0.2.2 c057b45, timestamp: 2021-02-15 18:56:26 0200

still got this problme

@danieldaeschle
Copy link
Member

@zakuro9715 for me it worked

@axetroy
Copy link
Author

axetroy commented Sep 12, 2021 •
edited
Loading

V 0.2.4 ccf6285

still got this with another error

V panic: `go main__walk()`: Resource temporarily unavailable
0   test                                0x000000010166a41d panic_error_number   77
1   test                                0x00000001016574cf main__walk   895
2   test                                0x000000010165712c main__walk_thread_wrapper   44
3   libsystem_pthread.dylib             0x00007fff203d58fc _pthread_start   224
4   libsystem_pthread.dylib             0x00007fff203d1443 thread_start   15

@JalonSolov
Copy link
Contributor

Interesting error.

I just tried your code on Manjaro Linux, with latest V 0.2.4 cd7d482 and it ran.

There were a few odd messages, but those are mostly because there are still 'panic's in vlib where there should be optionals.

I don't think your latest error is a V problem.

@medvednikov
Copy link
Member

Works ok now.

@axetroy
Copy link
Author

axetroy commented Sep 4, 2022

Works ok now.

@medvednikov Hi. I have retry it and it seems doesn't works for now

Current V version: V 0.3.1 436b19c, timestamp: 2022-09-04 14:15:26 0300

$ v run test.v
...
...
...
after: 227
after: 224
before: 244
after: 231
before: 228
after: 227
before: 232
before: 255
after: 254
after: 253
before: 254
before: 255
after: 254
after: 253
V panic: `go main__walk()`: Resource temporarily unavailable
v hash: 436b19c
0   test                                0x000000010ddf83ed panic_error_number   77
1   test                                0x000000010de2ea68 main__walk   1192
2   test                                0x000000010dde32b1 main__walk_thread_wrapper   49
3   test                                0x000000010de3d8ac GC_start_routine   100
4   libsystem_pthread.dylib             0x00007ff805d9e4e1 _pthread_start   125
5   libsystem_pthread.dylib             0x00007ff805d99f6b thread_start   15
before: 254
after: 253
before: 254
after: 253
V panic: `go main__walk()`: Resource temporarily unavailable
v hash: 436b19c
0   test                                0x000000010ddf83ed panic_error_number   77
1   test                                0x000000010de2ea68 main__walk   1192
2   test                                0x000000010dde32b1 main__walk_thread_wrapper   49

@danieldaeschle danieldaeschle reopened this Sep 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug This tag is applied to issues which reports bugs.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@medvednikov @zakuro9715 @axetroy @danieldaeschle @JalonSolov