镜像站点 · 本页由第三方 GitHub 只读镜像提供，非 GitHub 官方站点，不接受任何登录或凭据输入。前往 github.com
Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

[Nvidia][CI] Jetson toolchain files public acess #18637

Closed
#18677
Closed
[Nvidia][CI] Jetson toolchain files public acess#18637
#18677
Labels
Feature request

Description

@ChaiBapchya
ChaiBapchya
opened on Jun 29, 2020

Description

Currently, CI for Jetson builds uses aarch64 cross depedencies package based on Jetpack 4.3

However, it needs to be manually downloaded using SDK Manager tool and placed in a private S3 bucket because we don't have license to redistribute them.

Fix would be to make it publicly available under nvidia machine learning apt repository http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/

For reference
https://github.com/apache/incubator-mxnet/blob/master/ci/docker/Dockerfile.build.jetson#L61-L65

@leezu @ptrendx @TristonC

Activity

  1. ChaiBapchya
    added
    Feature request
    on Jun 29, 2020

  2. ChaiBapchya commented on Jun 29, 2020

    @ChaiBapchya
    ChaiBapchya
    on Jun 29, 2020
    ContributorAuthor

    Currently, MXNet devs won't be able to reproduce CI issue for Jetson via

    ci/build.py --docker-registry mxnetci --platform jetson --docker-build-retries 0 --shm-size 500m /work/runtime_functions.sh build_jetson

    As it fails for

    Step 11/17 : RUN aws s3 cp s3://mxnet-ci-prod-private-slave-data/nvidia/sdkm_downloads/cuda-repo-ubuntu1804-10-0-local-10.0.326-410.108_1.0-1_amd64.deb . &&     dpkg -i cuda-repo-ubuntu1804-10-0-local-10.0.326-410.108_1.0-1_amd64.deb &&     rm cuda-repo-ubuntu1804-10-0-local-10.0.326-410.108_1.0-1_amd64.deb &&     apt-key add /var/cuda-repo-10-0-local-10.0.326-410.108/7fa2af80.pub &&     aws s3 cp s3://mxnet-ci-prod-private-slave-data/nvidia/sdkm_downloads/cuda-repo-cross-aarch64-10-0-local-10.0.326_1.0-1_all.deb . &&     dpkg -i cuda-repo-cross-aarch64-10-0-local-10.0.326_1.0-1_all.deb &&     rm cuda-repo-cross-aarch64-10-0-local-10.0.326_1.0-1_all.deb &&     apt-get update &&     apt-get install -y -f &&     apt-get install -y cuda-cross-aarch64 cuda-cross-aarch64-10-0 &&     rm -rf /var/lib/apt/lists/*
 ---> Running in 0ee2a3406e2c
fatal error: Unable to locate credentials

    This needs access to the private mxnet CI prod account bucket which isn't feasible.

  3. leezu commented on Jun 29, 2020

    @leezu
    leezu
    on Jun 29, 2020
    Contributor

    Thanks @ChaiBapchya for opening the tracking issue! Previously I was given an early-June 2020 ETA for public availability of the files. I just followed up with the owner. Ideally the files are available already / will be soon and we just need to update the Dockerfile.

  4. leezu commented on Jul 8, 2020

    @leezu
    leezu
    on Jul 8, 2020
    Contributor

    Nvidia helped to make the files available publicly:

    https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cross-aarch64_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cudart-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cufft-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cupti-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-curand-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cusolver-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cusparse-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-driver-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-misc-headers-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-npp-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-nsight-compute-addon-l4t-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-nvgraph-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-nvml-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-nvrtc-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cublas/libcublas-cross-aarch64_10.2.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/n/nsight-compute/nsight-compute-addon-l4t-2019.5.0_2019.5.0.14-1_all.deb

    Now the Dockerfile can be updated to make use of the public files and the private S3 bucket can be deleted.

  5. ChaiBapchya commented on Jul 8, 2020

    @ChaiBapchya
    ChaiBapchya
    on Jul 8, 2020
    ContributorAuthor

    Yes working on that fix.

  6. leezu commented on Jul 8, 2020

    @leezu
    leezu
    on Jul 8, 2020
    Contributor

    Thank you @ChaiBapchya!

  7. leezu
    closed this as completedin #18677on Jul 15, 2020

  8. josefhernandez commented on Sep 11, 2021

    @josefhernandez
    josefhernandez
    on Sep 11, 2021

    I'm getting back this error on versions master, 1.8.0 and 1.7.0
    Last week everything compiled ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Feature request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions