镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

CI runs in normal (non-debug) mode fail to find bin/unit/splinter_test, causing CI jobs to abort. #287

Closed

Description

@gapisback

This issue is a continuation of the attempts to get stable CI runs that was logged under issue #282. Downgrading Ubuntu to 20.04 has not resolved that instability. The symptom of the failure is something like this:

TEST 28/28 util:test_debug_hex_encode [OK]
RESULTS: 28 tests (28 ok, 0 failed, 0 skipped) ran in 114311 ms
+ set +x
test.sh: Fast unit tests: 115 s [ 0h 1m 55s ]

+ bin/unit/splinter_test test_inserts
./test.sh: line 49: bin/unit/splinter_test: No such file or directory

... whereas the same changes to test.sh are able to run this new test just fine in debug builds.

Additional info from @rosenhouse below gives pointers to chase this down.

  1. Looking at the latest CI results from your PR, I notice that the 3 jobs which are failing are the non-debug jobs, namely ci/clang, ci/gcc and ci/rust.
  2. Looking at the top-level job definitions here I see that those are the only three jobs which call the steps_pr_build_test function in the ytt template. Other jobs call other functions like steps_pr_debug_build_test
  3. Comparing the steps_pr_build_test versus steps_pr_debug_build_test in the ytt functions file, I see that the steps_pr_build_test job first builds a docker image, and then runs tests inside the built image.
  4. By contrast, the steps_pr_debug_build_test job does a build-and-test in a single step, within the build-env image.
    Digging in further, the build-image-then-test-it workflow is building this top-level Dockerfile. I see that it is doing a non-recursive copy of the bin subdirectory. When I follow the instructions at the top of that Dockerfile to build the image, I can see that there’s a bin directory but no unit subdirectory. Note this is the current behavior on main and is what you see if you just
docker run --rm -it projects.registry.vmware.com/splinterdb/splinterdb find .

In short, I recommend you change the Dockerfile to do a recursive copy of the entire bin directory. Reference for the COPY directive is here. You can test it locally using the instructions at the top of our Dockerfile.

Activity

  1. self-assigned this
    on Feb 24, 2022
  2. rosenhouse commented on Feb 24, 2022

    @rosenhouse
    Contributor

    Just to clarify, the issue isn't general "instability" -- I expect it's consistently reproducible and entirely confined to #277, because that PR modifies the test.sh script to call binaries that are nested below bin/unit/:

    https://github.com/vmware/splinterdb/pull/277/files#diff-3722d9ba8feb2d3feac8ce71a209a638d4b404e1c53f937188761181594023e2R117-R118

    run_with_timing "Splinter inserts tests" bin/unit/splinter_test test_inserts
    run_with_timing "Splinter lookups tests" bin/unit/splinter_test --memtable-capacity-mib 4 test_lookups
  3. added a commit that references this issue on Feb 26, 2022
  4. added a commit that references this issue on Feb 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions