fix(eva): keep the lowest scores in the factor short group

[Divyansh Barodiya (Divyansh151005)]

Description

The model-performance cumulative-return chart builds equal-count score groups with

x[len(x) // N * i : len(x) // N * (i + 1)]

after sorting scores from high to low. When the number of names is not divisible by N, that slice stops before the end of the ranking. The leftover names are the lowest scores, so they never enter the short group.

calc_group_returns now keeps that remainder in the last group and takes the slice with iloc. Group1 is still the highest score. long-short is Group1 minus the last group, and long-average still subtracts the mean label of the whole cross-section, including names with a missing score. Days that already divide evenly are unchanged.

Motivation and Context

Fixes #2379

A 7-name cross-section with 5 groups used to report a short-group return of the 5th-best name. The two worst names were dropped, so the long-short spread was 7 - 3 instead of 7 - mean(3, 2, 1). That chart is the factor-evaluation view of a model's ranked book.

How Has This Been Tested?

• Pass the test by running: pytest qlib/tests/test_all_pipeline.py under upper directory of qlib.
• If you are adding a new feature, test on your own test scripts.

test_all_pipeline.py trains a CSI300 LightGBM workflow and needs the market dataset. These tests build a synthetic score panel.

$ python -m pytest tests/misc/test_factor_group_return.py -q --tb=short
....                                                                     [100%]
4 passed in 2.69s

flake8 on the changed Python files, with the ignore list from the developer guide, reported no issues. black -l 120 reformatted qlib/contrib/eva/alpha.py and left the other Python files unchanged.

Screenshots of Test Results (if appropriate):

1. Pipeline test: not run. It needs the CSI300 dataset and a model fit.
2. Your own tests: tests/misc/test_factor_group_return.py, 4 passed.

Types of changes

• Fix bugs
• Add new feature
• Update documentation

[Arhan Canli (arhancanli)]

Read the diff and traced the 7-name case by hand: sorted high to low the scores are 7,6,5,4,3,2,1, size = 1, so Group1..Group4 hold one name each and Group5 takes iloc[4:7] = 3,2,1. That matches your test, and the old len(x)//N*(i+1) slice did drop the bottom len % N names, so the fix is real.

One thing the new docs and docstring should say differently: the buckets are no longer equal-count. With 299 names a day, size = 59, so Groups 1-4 have 59 names and Group5 has 63; with 7 names it is 1,1,1,1,3. The docs now say "Equal-count buckets", which is only true when n % n_groups == 0. It also matters for how people read long-short: it now compares a 59-name mean against a 63-name mean, and the Group5 mean is a little less noisy than Group1's. I would reword to "equal-size buckets of len // n_groups names, with the remainder added to the last".

Two smaller points:

• This changes published numbers for anyone whose daily universe is not divisible by N (a CSI300 day with a few suspensions, for example), so it deserves a line in the changelog saying long-short and Group5 move slightly on those dates.
• A test for a date with fewer names than groups (size == 0 returns all NaN, as before) would pin that behaviour, since the docstring now promises the short leg is "not truncated" and on those dates there is no short leg at all.

[Divyansh Barodiya (Divyansh151005)]

Thanks for tracing the 7-name case. I pushed a follow-up.

The docstring and the report page now say each bucket has len(day) // n_groups names, and the last bucket keeps the remainder. long-short is a mean of that smaller long bucket against a larger short bucket when the day does not divide evenly.

CHANGELOG.md notes that Group5 and long-short change on dates where the number of names is not divisible by the group count.

test_fewer_names_than_groups_is_all_nan covers a day with 3 names and 5 groups. Every group, long-short, and long-average is NaN, which is the previous size == 0 behaviour.

$ python -m pytest tests/misc/test_factor_group_return.py -q --tb=short
.....                                                                    [100%]
5 passed in 1.22s

[Divyansh Barodiya (Divyansh151005)]

Thanks for tracing the 7-name case. I pushed a follow-up.

The docstring and the report page now say each bucket has len(day) // n_groups names, and the last bucket keeps the remainder. long-short is a mean of that smaller long bucket against a larger short bucket when the day does not divide evenly.

CHANGELOG.md notes that Group5 and long-short change on dates where the number of names is not divisible by the group count.

test_fewer_names_than_groups_is_all_nan covers a day with 3 names and 5 groups. Every group, long-short, and long-average is NaN, which is the previous size == 0 behaviour.

$ python -m pytest tests/misc/test_factor_group_return.py -q --tb=short
.....                                                                    [100%]
5 passed in 1.22s