Private Composer Repositories with GitLab

GitLab provides free package repositories for package managers of various languages such as NPM, Maven, and Go. The recent GitLab version 13.2 adds support for Composer package registries.

With GitLab, you can now have your own private Composer repository for Git repositories hosted on GitLab. All for free, for those who use the hosted GitLab.com and those who self-host GitLab.

How Composer uses repositories

packagist.org is the default repository for Composer. Unless otherwise disabled, Composer will look up the packages in packagist.org. Anyone with a Packagist account and add a Git repository to Packagist, and with a webhook, Packagist keeps an up to date list of versions available for a given package.

Composer supports adding custom repositories. Composer will lookup packages in the configured repositories as well.

Composer 2 supports fine-tuning custom repositories with filters, canonical repositories, and priorities.

VCS and Path repositories

It is possible to add a local path on disk, or a URL to a Version Control System (such as Git) to the base composer.json file, and Composer will lookup available versions in said paths and URLs.

{
    "repositories": [
        {"type": "vcs", "url": "https://github.com/example/lib-example"},
        {"type": "path", "url": "../../packages/my-package"}
    ]
}

When a package is requested, Composer now looks up package versions in the configured VCS and path locations.

This is often quite slow, because Composer needs to load a lot of data about the repositories to determine the available versions and check the composer.json file of each version.

Artifacts and individual packages

Composer also provides features to lookup Zip files within a remote location (useful with CI build systems that emit Zip artifacts). It is even possible to declare a package within the composer.json file itself, and point it to a specific URL that Composer will fetch. These ad-hoc packages do not need to contain a composer.json file of its own either.

Artifacts and individual packages are not as used as widely as Composer repositories or VCS/path repositories, and are only mentioned for the sake of completion of this article.

Composer repositories

Composer can work with custom Composer repositories as well. packagist.org is the default repository, but if as long as the custom repository implements the REST API Composer communicates with, Composer can search and download packages from custom repositories.

Packagist.com, from the creators of Composer project, is probably the most popular commercial service.

Open source alternatives include Satis from Composer project, and various project-specific repositories.

{
    "repositories": [
        {"type": "composer", "url": "https://example.com"}
    ]
}

With a custom composer repo, Composer can query the repo URL to find packages, and call the API once a package is downloaded if the repository is interested in keeping count of package downloads.

Using a composer repository is by far, the fastest and most secure way to host and use private packages because it provides a cleaner API to lookup package information, and granular access to packages and package information.

---

Private composer repositories from GitLab

GitLab offers free unlimited private repositories and a CI/CD platform in its free plan. The recent addition of package repositories in the free edition now enables Composer users to easily run private Composer repositories.

With a private Composer repository, you can create a collection of PHP Composer packages, and have Composer download them from GitLab package repository. This can provide ease of use with a UI to view packages, and delete them from the UI.

A few things to keep in mind:

• Packages and package versions must be published via the API. There is no UI to create packages and versions. Packages and their versions can be viewed and deleted from the browser UI.
• A personal access token is necessary to publish a package.
• We use GitLab CI to automatically publish new Git tags to the Package registry.
• A personal access token is required to access the private Composer repository.

1. Create a Project Group

If you do not have one already, create a new project group. All projects published under a project group will be available in the Composer repository.

Once created, you will notice the group is a given a numeric group ID. This ID will be used in the repository URL, making all packages within the group share the same URL.

2. Create a project under the group

With a Project Group created, you can now create a project under the newly created group. If you already have a project group and a project ID, you can skip this step.

3. Enable Package Registry

In project Settings → General page, and under Visibility, project features, permissions, enable "Pipelines" and "Packages" features.

4. Create a Personal Access Token to publish

In order to publish packages, a user must be authenticated to the Package Registry. Under a user account that has access to the group, create a Personal Access Token.

Enter an appropriate name, optionally an expiration date. Under Scopes, grant api access.

5. Add Personal Access Token as a masked CI variable

In project Settings → CI / CD page, and under Variables, add a new variable. This variable will be used in GitLab CI to publish the package automatically.

Make sure to enable "Mask variable". This will hide the token in CI logs. Unless you are using a strict protected branch for releases, or tag patterns, disable the "Protect variable" option.

In this example, we use the key DEPLOY_TOKEN, which we call in the next step.

6. GitLab CI task to automatically publish versions

GitLab Packages features does not automatically publish each Git tag to a version. There is no user-interface to create Composer compatible versions either.

GitLab provides an API endpoint that can be called to publish a package with a specific version or branch. GitLab also provides a free CI/CD platform that can be used to publish new versions of the Composer package to GitLab Package Registry.

If you do not have already, create a new file in Git repository root with name .gitlab-ci.yml (mind the dot at the beginning of the file name) with contents below:

deploy_composer:
  only:
    - tags
  stage: test
  script:
    - curl -sS --show-error --fail --data tag=${CI_COMMIT_TAG} "https://__token__:${DEPLOY_TOKEN}@gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/composer"

Note that this is an overly simplified example, but a slightly more complete GitLab CI .gitlab-ci.yml file would be:

stages:
  - test
  - deploy

test:
  stage: test
  image: phpwatch/docker-php:latest
  before_script:
    - php -v
    - composer install --prefer-dist -q --no-progress
  script:
    - ./vendor/bin/phpunit -v --coverage-text --colors=never --stderr

deploy_composer:
  only:
    - tags
  stage: deploy
  script:
    - curl -sS --show-error --fail --data tag=${CI_COMMIT_TAG} "https://__token__:${DEPLOY_TOKEN}@gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/composer"

In deloy_composer step, that runs at deploy stage (which means the previous steps such test are successful), we issue a curl call to POST to GitLab packages API to publish the newly created tag. only: [tags] clause ensures that we do not make this API call on other events such as standard commit pushes, merge requests, etc.

${DEPLOY_TOKEN} value is the CI environment variable set at the previous step.

If a valid composer.json file is present, GitLab will create a new package version and return a 201 response. If you use the snippet above, the CI job will only be successful if GitLab accepted the new tag.

7. List published packages

You can view and delete published packages at Packages & Registries → Packages Registry page.

---

How to use GitLab private repositories

Consuming packages hosted in GitLab private composer repositories is straight forward.

Add private repository as a composer repository

You can add your new private Composer repository to the root composer.json file.

{
    "repositories": [
        {
            "type": "composer",
            "url": "https://gitlab.com/api/v4/group/<GROUP_ID>/-/packages/composer/packages.json"
        }
    ]
}

The <GROUP_ID> value is the Group ID the project belongs to. See the screenshot at step 1: Create a Project Group.

Alternately, you can navigate to any package already published, and instructions to use the package will be shown at the end of the page:

Private repository authentication

Before Composer can consume the new private package repository, it needs to be authenticated.

This is can be done with Basic Authentication supported in Composer.

A Personal Access Token is used for this. See Step 4: Create a Personal Access Token to publish. Ideally, use an access token with read_repository access, but not api access. An api token is read/write and can be used to publish packages.

A Personal Access Token should never be committed to the Git repository. Use the global auth.json, a git-ignored local auth.json file, or the COMPOSER_AUTH environment` variable to inject access credentials.

{
    "http-basic":  {
        "gitlab.com":  {
            "username":  "___token___",
            "password":  "<ACCESS_TOKEN>"
        }
    }
}

• __TOKEN__ is a special username GitLab expects to use in API calls. Do not change this.
• Replace <ACCESS_TOKEN> with a valid personal access token.

You can create this configuration from command line too:

Per-project configuration (Make sure to add auth.json to .gitignore file):

composer config http-basic.gitlab.com ___token___ <ACCESS_TOKEN>

Global configuration:

composer g config http-basic.gitlab.com ___token___ <ACCESS_TOKEN>

Although not recommended, it is possible to embed the credentials to the composer.json file itself too:

{
    "repositories": [
        {
            "type": "composer",
            "url": "https://___token___:<ACCESS_TOKEN>@gitlab.com/api/v4/group/<GROUP_ID>/-/packages/composer/packages.json"
        }
    ]
}

• Replace <GROUP_ID> with the group ID (from step #1).
• Replace <ACCESS_TOKEN> with the "personal" access token.

It's highly recommended to not embed the personal access tokens in the composer.json file. It appears that there is a problem with Composer authenticating to GitLab with auth.json credentials. This post will be updated when it is fixed.

---

GitLab package repositories provide free Composer repositories for organizations, as an alternative to Packagist.com (which offers a lot more features, and also proxies as a packagist.org repository). It is easy to setup and run compared to a self-hosted option.

Note that under the personal free accounts, there is a limit of 500MB of storage for package registries. This limit is across the whole account. Free for open source packages. Package files themselves are stored in the registry (npmjs.org-akin). Consider using a .gitattributes file with export-ignore rules to exclude unnecessary files from being packaged to the stored zip files.

GitHub also provides package registry features, but they do not support Composer (yet?).