[Will Charles (drwillcharles)]

Why are these changes needed?

When using ray>=2, there is an import error for tune.report.

AttributeError: module 'ray.tune' has no attribute 'report'

I used the Ray migration guide to update this and use the new report function if the ray version is not 1.x .

I see that in the FLAML codebase there are already a few workarounds like this so I think I can do something similar without needing a full migration to Ray 2.x

Related issue number

Related to #874 - although I still think a full migration to Ray>=2 is needed in the future.

Checks

• [ x] I've used pre-commit to lint the changes in this PR (note the same in integrated in our CI checks).
• [x ] I've included any doc changes needed for https://microsoft.github.io/FLAML/. See https://microsoft.github.io/FLAML/docs/Contribute#documentation to build and test documentation locally.
• [ x] I've added tests (if relevant) corresponding to the changes introduced in this PR.
• [x ] I've made sure all auto checks have passed.

[Will Charles (drwillcharles)]

@microsoft-github-policy-service agree company="Evotec"

[Li Jiang (thinkall)]

Thank you, Will Charles (@drwillcharles) , for the PR!

[Li Jiang (thinkall)]

Hi Will Charles (@drwillcharles) , could you check the test errors? Looks like the change will lead to test timeout issue. Thanks.

[Will Charles (drwillcharles)]

It appears the test _test_hf_data() on test_gpu.py gets stuck in a loop because of an error in Ray. Strangely - this test is not even run because of the naming convention. (_test_hf_data() instead of test_hf_data()).

I have loosened the pydantic dependency and hope this solves the issue.

[Will Charles (drwillcharles)]

OpenML is currently down so a lot of tests are getting connection errors. Will wait until this is back online before I resume development.

From the tests that have run successfully, it appears there are still some environment issues to fix

[Will Charles (drwillcharles)]

I've figured out that the jobs are timing out because Ray cannot find the appropriate resource

There is a function _test_hf_data in test/test_gpu.py which I don't think is even picked up in the tests (because of the underscore).
However, when running this file, it gets stuck in an infinite loop. I think before this just failed silently.

Can I delete this?

[Li Jiang (thinkall)]

I've figured out that the jobs are timing out because Ray cannot find the appropriate resource

There is a function _test_hf_data in test/test_gpu.py which I don't think is even picked up in the tests (because of the underscore). However, when running this file, it gets stuck in an infinite loop. I think before this just failed silently.

Can I delete this?

Sorry for the late reply, Will Charles (@drwillcharles). It's interesting that this underscored test is picked up. For bypassing the test, I think we can detect whether there is a GPU resource and skip the test if None. WDYT? Thanks.

[Will Charles (drwillcharles)]

No problem Li Jiang (@thinkall). I did add a check for the GPU resource and even deleted the test to check. Turns out I was wrong and the problem appears to be test_lrl2 on test_model.py.

Looking at the line lr.fit(X, y, budget=1e-5) it appears the test should just stop when the budget is exceeded (almost instantaneous). However, that appears not to be happening. I have not yet found a solution to this and it appears #1402 is having the same problem in the tests.

[Li Jiang (thinkall)]

No problem Li Jiang (@thinkall). I did add a check for the GPU resource and even deleted the test to check. Turns out I was wrong and the problem appears to be test_lrl2 on test_model.py.

Looking at the line lr.fit(X, y, budget=1e-5) it appears the test should just stop when the budget is exceeded (almost instantaneous). However, that appears not to be happening. I have not yet found a solution to this and it appears #1402 is having the same problem in the tests.

Thank you for pointing out the root cause test, Will Charles (@drwillcharles) . I fixed the tests in #1406 .

[Will Charles (drwillcharles)]

No problem Li Jiang (@thinkall). I did add a check for the GPU resource and even deleted the test to check. Turns out I was wrong and the problem appears to be test_lrl2 on test_model.py.
Looking at the line lr.fit(X, y, budget=1e-5) it appears the test should just stop when the budget is exceeded (almost instantaneous). However, that appears not to be happening. I have not yet found a solution to this and it appears #1402 is having the same problem in the tests.

Thank you for pointing out the root cause test, Will Charles (@drwillcharles) . I fixed the tests in #1406 .

Thanks Li Jiang (@thinkall) ! I have implemented your fix so hopefully these tests pass now 🤞