DCI Downloader

dci-downloader is a useful tool for downloading the latest versions of Red Hat products

TLDR

$ sudo yum -y install https://packages.distributed-ci.io/dci-release.el7.noarch.rpm
$ sudo yum -y install dci-downloader
$ source ~/dcirc.sh
$ dci-downloader RHEL-8.5 /tmp/repo

Table of Contents

Requirements

Red Hat SSO

DCI is connected to the Red Hat SSO. You will need a Red Hat account.

Remoteci

A remoteci is a Virtual Machine or a baremetal server running RHEL. You should check that your remoteci:

  • Is running the latest RHEL 7 release.
  • Has enough free space in the destination folder.
  • Should be able to reach:
    • https://api.distributed-ci.io (443).
    • https://repo.distributed-ci.io (443).

Installation

The dci-downloader is packaged and available as a RPM files.

$ sudo yum -y install https://packages.distributed-ci.io/dci-release.el7.noarch.rpm
$ sudo yum -y install dci-downloader

Limitation

At the moment it is not possible to perform two parallel downloads in two different processes. If you use a configuration file with multiple topics, the download is done synchronously topic after topic.

Configuration

Remoteci creation

DCI is connected to the Red Hat SSO. You need to log in https://www.distributed-ci.io with your redhat.com SSO account. Your user account will be created in our database the first time you connect.

After the first connection you can create a remoteci. Go to https://www.distributed-ci.io/remotecis and click Create a new remoteci button. Once your remoteci is created, you can retrieve the connection information in the Authentication column. Save those information in ~/dcirc.sh file.

At this point, you can validate your credentials with the following commands:

$ source ~/dcirc.sh
$ dcictl remoteci-list

If you see your remoteci in the list, everything is working great so far.

Topic access

Before using dci-downloader, we need to check the list of topic (version of the product) you have access to:

$ source ~/dcirc.sh
$ dcictl topic-list

If you don't see any topic then you should contact your EPM at Red Hat which will give you access to the topic you need.

Usage

You can now download the latest version of a product using dci-downloader.

RHEL examples

Example command to download the latest RHEL-8.5 components into /tmp/repo folder.

$ dci-downloader RHEL-8.5 /tmp/repo

RHOSP examples

/!\ Starting with OSP15, OSP topics are now supported by dci-downloader. Older topics require the --all flag.

From OSP15 onwards:

$ dci-downloader OSP16.1 /tmp/repo

For older OSP topics

$ dci-downloader --all OSP13 /tmp/repo

dci-downloader also allows to mirror the container images associated with the component to a local anonymous registry.

$ dci-downloader OSP16.1 /tmp/repo --registry local_registry_host:5000

ℹ NOTE: except when using dci-openstack-agent, it's the user's responsibility to provide a working anonymous registry. If needed a simple way to setup one is by using Docker Distribution registry in a container

$ podman run --rm -p 5000:5000 registry:2

ℹ NOTE: dci-downloader does not currently clean/purge/untag any image on the registry, leaving this responsibility to the the user until a suitable solution is provided.

Options

By default dci-downloader will download all variants for x86_64 architecture without debug RPMs.

Download other architectures

To download other architectures you can specify those using --arch option

Download x86_64 and ppc64le architectures for RHEL-8.5 topic:

$ dci-downloader RHEL-8.5 /tmp/repo --arch x86_64 --arch ppc64le

Specific variants

To download only specific variants you can specify those using --variant

Download only AppStream and BaseOS variants:

$ dci-downloader RHEL-8.5 /tmp/repo --variant AppStream --variant BaseOS

Filters

By default dci-downloader download the latest components attached to a topic. But if you want to filter those component, you can use the --filter option.

Download only the latest RHEL-8.5 nigthly compose

$ dci-downloader RHEL-8.5 /tmp/repo --filter=compose:nigthly

Download the whole compose

To download everything you can add the --all flag

$ dci-downloader RHEL-8.5 /tmp/repo --all

Debug RPMs

To download debug RPMs you can add the --debug flag

Download RHEL-8.5 compose with debug rpms

$ dci-downloader RHEL-8.5 /tmp --debug

Settings file

You can use a settings file to send parameters to parameterize dci-downloader.

Use --settings parameter:

$ dci-downloader --settings "/etc/dci-rhel-agent/settings.yml"

All settings from settings.yml file will overwrite cli parameters.

Examples of a settings file:

version: "2"
download_folder: /var/www/html
topics:
  - name: RHEL-9.0
    filters:
      - type: compose
        tag: nightly
  - name: RHEL-8.4
    filters:
      - type: compose
        tag: milestone
    archs:
      - ppc64le
    variants:
      - AppStream
      - name: BaseOS
        with_debug: true
  - name: RHEL-8.2

SSL certificates

dci-downloader downloads SSL certificates prior the download phase. To customize the CRT and KEY path you can change DCI_KEY_FILE and DCI_CERT_FILE env variables.

If you are using settings.yml file change

dci_key_file: /etc/dci-rhel-agent/dci.key
dci_cert_file: /etc/dci-rhel-agent/dci.crt

Proxy

If you are using a proxy, just add HTTP_PROXY or HTTPS_PROXY env variable pointing to your proxy.

License

Apache License, Version 2.0 (see LICENSE file)

Help

If something is not working correctly, ensure you have the latest version installed

$ dci-downloader --version
$ dci-downloader --help

To contact DCI see Questions and help

results matching ""

    No results matching ""