Thank you for working on this! It definitely looked like there was significant overlap in the tests and many could be combined to reduce the overhead.

It may be possible to use xdist just for spark tests, I suspect it will cause problems for other tests but it might be worth experimenting. Maybe we could set OMP_NUM_THREADS before running the tests. The main goal here is to reduce the time of the CI cpu python tests step, which currently stands at around 25mins.

Yeah, Let's fix them one by one. Currently, this PR can reduce the most time-consuming test.

@trivialfis please help to trigger the CI for the build. Thx

I started the CI

Without this PR, It takes 70s to finish test_classifier_with_cross_validator 70.03s call python/test_spark/test_spark_local.py::XgboostLocalTest::test_classifier_with_cross_validator With this PR, it only takes 17s to be finished 17.43s call python/test_spark/test_spark_local.py::XgboostLocalTest::test_classifier_with_cross_validator

I also tried enabling pytest with parallelly running by pytest-xdist and it worked well with pyspark tests, and the time can be reduced from 4m49.335s to 2m19.574s.

I don't know if we need to enable pytest-xdist for python tests?

BTW, I also found some tests may have overlaps, like test_classifier_distributed_weight_eval and test_regressor_distributed_weight_eval, seems we don't need to test all kinds of estimators @WeichenXu123, please correct me.

Hi, could you please tell me how to count running time of the functions in the c codebase? Thanks!