GitLab 10.5 released: integration with Let's Encrypt, Gemnasium dependency checks and external CI / CD files

Original author: GitLab
  • Transfer

Картинка для привлечения внимания

In GitLab 10.5, we added the ability to easily encrypt GitLab traffic and scale pipeline management, improved application security, and much more.

Reduced GitLab Secure Deployment Time

It is impossible to talk about Internet security without mentioning the HTTPS protocol. Its use is mandatory in cases where the GitLab instance is open to the public. HTTPS provides two key benefits. Firstly, it is the encryption of incoming and outgoing traffic during interactions with the server, so that user credentials and other important information are protected from interception. Secondly, it is an opportunity to confirm site identification. Without such a check, there is the possibility of an accidental login to the wrong site.

The benefits of HTTPS are particularly important for remote use or for users of mobile applications, as they often use an insecure Wi-Fi connection, which increases the risk of data leakage or connecting to fraudulent sites.

However, despite all the above advantages, setting up HTTPS and requesting certificates can require significant time and effort, in particular with credit cards and key management. In GitLab 10.5, we added integration with Let's Encrypt , an automated, free, and public certification authority. SSL certificates can now be obtained instantly with one option. Connecting Let's Encrypt for your GitLab instance guarantees traffic encryption and confirmation of the identity of your site. Integration with Let's Encrypt is available in both the paid and free versions of GitLab.

Pipeline Management Scaling

In this release, we add in general simple functionality with great potential.

The use of DevOps in a corporate environment is associated with certain difficulties. For many of our largest customers, the DevOps team is responsible for supplying CI / CD pipelines to a large number of development teams throughout the organization. Managing such processes is quite time-consuming, since previously there was no scalable way to deliver the configuration of pipelines. Because of this, you had to manually copy the code between the files of .gitlab-ci.yml various projects. In addition to excessive efforts, this approach created the likelihood of errors. In addition, it was impossible to track that the testing and deployment processes are conducted stably for each repository.

Starting with GitLab 10.5, you have the opportunity to include external files in the definition of the CI / CD pipeline . These files can be either local (contained in the same repository) or remote (accessible via HTTP / HTTPS). The inclusion of local files allows you to split a bulky file .gitlab-ci.yml into modules that are easier to work with. Using remote files allows you to distribute these modules across thousands (potentially, and millions) of repositories. You now have a simple and stable way to supply pipelines configuration.

Improving Security Testing Using Gemnasium

Less than a month ago, GitLab acquired Gemnasium . As we promised, we immediately provide our users with advanced Gemnasium functionality for dependency checking. It often happens that after such acquisitions, companies deliver innovations as separate add-ons. However, the philosophy of GitLab is the architecture of a single application , which is designed in such a way that development, testing, security and operations teams can work simultaneously with the same data in the same interface. Therefore, we fully integrate Gemnasium technology into the GitLab CI / CD, greatly improving the testing functionality.

Thanks to advanced algorithms and an extended vulnerability database, improved results in JavaScript, Ruby and Python. Also added support for PHP and Java.

Do not miss new opportunities

This article is about 26 GitLab enhancements (18 of which are available in the open source version). A complete list of improvements can be found here . Further in the article we will talk about key innovations in version 10.5

We invite you to our meetings!

GitLab MVP badge

( MVP ) This Month - Hiroyuki Sato

Hiroyuki has been contributing to the development of GitLab from the earliest days of the project, and this month it becomes MVP for the third time (for the first time this happened already in GitLab 5.1!). In version 10.5, he added the ability to view a merge request for a commit , which makes it easy to track changes and speeds up the development process.

Thanks again, Hiroyuki! Although we know that joining the GitLab Hall of Fame is the best award, we also send Hiroyuki handmade tanuki, socks and a GitLab t-shirt.

Instant SSL with Let's Encrypt for GitLab

GitLab often stores the source code of a project - intellectual property, the privacy of which must be protected. The fundamental step for this is to use HTTPS to encrypt the identity and to verify the identity of the sites.

Previously, to connect to HTTPS in GitLab, it was necessary to perform a number of steps: create a certificate request, pay a certification authority, copy files to a GitLab server, and finally configure GitLab to use them.

In 10.5, we greatly simplified this process by integrating with Let's Encrypt . If your instance is accessible over the Internet via the HTTP protocol, then all you need to do now to connect to HTTPS is assign it letsencrypt['enable'] = true in a file gitlab.rb and reconfigure it. Done, HTTPS is connected to the main GitLab domain!

Future releases of Let's Encrypt will be enabled by default , and support for other GitLab elements such as Registry , Pages, and Mattermost will be added .

Instant SSL with Let's Encrypt for GitLab

More information about integration with Let's Encrypt here

Include external files in CI / CD pipeline definitions

CI / CD pipelines are defined in the YML file located in the project repository. It often happens that the same job definitions are used for many different projects. It is also not uncommon for existing snippets to simply be copied from documentation and examples.

In GitLab 10.5, it is now possible to import external files into the main configuration using a new keyword include . These files can be either local (from the same repository) or remote (publicly accessible via HTTP / HTTPS). Good examples of work that can be reused this way are security checks and deployment configurations.

Documentation for including external files

Include external files in CI/CD pipeline definition

Gemnasium dependency checks

GitLab recently acquired Gemnasium , and we immediately started working together to integrate this great technology into our testing functionality.

Thanks to databases of existing vulnerabilities and improved detection of new ones, in GitLab 10.5 it became possible to obtain extremely accurate reports on the security of dependencies of your application in the following languages:

  • Ruby
  • Java (Maven)
  • Javascript (NPM)
  • Python
  • Php

If you already use Auto DevOps , then you do not need to change anything. If you copied the job description into your pipeline, update it to access the new functionality. More information on the example page .

Supported Languages ​​and Security Audit Documentation

Gemnasium dependency checks

Track additional browser performance metrics

In GitLab 10.3, we added performance testing in the browser , with which you can quickly assess the impact of the merge request on performance. In this release, we analyze three additional metrics: speed index , transfer size, and number of requests.

We also added the ability to save the entire report as an artifact, which provides easy access to a large amount of performance and debugging data. If you use Auto DevOps , such reports will be saved automatically.

More information about browser performance testing here

Track additional browser performance metrics

Git LFS 2 Lock Support

In Git LFS 2.0.0 has been added to lock the support. Now this functionality is also supported in GitLab, so you can block LFS files using the Git LFS client. Locked files can be easily identified by the corresponding icon.

Support for locking any file or directory has been added in GitLab Premium 8.9. With its help, you can block files through the GitLab interface. In GitLab Premium 10.5, we integrated Git LFS lock with GitLab lock.

More information on using Git LFS here

Git LFS 2 locking support

Other improvements in GitLab 10.5

View Epics in Roadmap Mode

In this release, we introduce the first iteration of roadmaps (roadmap). Roadmaps allow you to simultaneously view several epics of one group (or even a subgroup) at once with ordering by time. Now you can easily see when the epics begin and end relative to each other.

This innovation simplifies the planning and tracking of progress over time, as well as demonstrating overlapping workflows. For example, you want to launch a new feature of your product in the second quarter of 2018 and plan to conduct an appropriate marketing campaign. In this case, you will create one epic for working on functionality and another for marketing. In roadmap mode, both of these epics will be displayed at the same time, which will simplify control over the start and end times.

View Epics in roadmap

Roadmap documentation

Dynamic management of secret variables

Secret variables are used to configure CI / CD pipelines so that sensitive information is hidden from external access. Users with a Master access level can define them in the CI / CD> Settings menu ; however, with this approach, variables can be defined only one at a time.

In GitLab 10.5, we have added dynamic management of secret variables with support for adding multiple variables at the same time, which simplifies working with them.

Dynamic management of secret variables

Secret Variable Documentation

Instance-level Auto DevOps Domain Definition

Auto DevOps can automatically deploy your application to the Kubernetes cluster, however, to access it you must provide the domain name associated with the public IP address of this cluster.

Starting with GitLab 10.5, you can determine the domain name at the instance level and use it for the entire organization: it will be used automatically for any project in which a domain name is not specified.

Auto DevOps Base Domain Documentation

Migrating GitLab Groups

Now you can transfer entire GitLab groups from one to another, which simplifies the process of structuring groups and subgroups.

Transfer groups

GitLab Group Documentation

Ingress objects configured for Prometheus metrics

Deployed Ingress objects are now configured for Prometheus metrics, making it easy to track the following system metrics: latency, throughput, and error rate.

More information about deploying Ingress on Kubernetes here

Additional Merge Requests

We have simplified the logic of approval of merge requests: now it’s even easier to configure and use permissions.

In particular, it has now become possible to approve the merge request, even if the required number of statements has already been collected. Reviewers are no longer dependent on each other and can approve merge requests when they are comfortable. This approach encourages more users to participate in the code review process.

However, if the workflow requires it, you can manually set the number of claims you need according to your reviewers.

As before, previously submitted claims can be deleted.

Approve additionally in Merge Requests

Merge Request Approval Documentation

Display tasks of all subgroups on the group tasks page

Now it is possible to display the tasks of all subgroups on the tasks page of their common group. This is very useful in cases where the hierarchy of subgroups forms a tree of several levels, and you need to view all its tasks in one place. Examples of such situations can be teams with microservices distributed across many projects and groups, or organizations with a complex hierarchy of teams.

Exactly the same changes were made for the merge request page.

View all issues of all subgroups on group issues page

Task documentation

Colored Icons in GitLab Flavored Markdown

GitLab Flavored Markdown (GFM) now supports color icons. Just enter the appropriate color code in any place where GFM is supported (for example, descriptions and comments of tasks and merge requests), and GitLab will create a color icon in this place. This innovation will be especially useful for designers and front-end developers, as it will simplify the joint work on designs containing various colors, without having to leave GitLab.

Thanks to @thetonyrom and @raviolicode for their contributions!

Color chips in GitLab Flavored Markdown

GitLab Flavored Markdown Documentation

Push to create a project

Now, when you push the repository into a project with a nonexistent name (in your personal namespace), a private project with that name will be automatically created. Thanks to this, creating projects will be even faster!

More information on creating projects here.

Disaster recovery with a single secondary base is now publicly available

Disaster Recovery functionality uses Geo replication to quickly move to another site with minimal effort in the event of a disaster. Failover in single secondary configurations is now publicly available.

Disaster Recovery Documentation

Epic Search and Filter Menus

In this release, we added search and filtering menus to the epic page. This is the same search that is used throughout GitLab for tasks and merge requests. In this release, it became possible to filter epics by author, as well as search for titles and descriptions of epics. Additionally, sorting epics by date of creation or last update is available.

This makes it possible to take advantage of the lists that you are probably used to when working with tasks and merge requests. With the help of search and filtering, it will be easier for you to manage exactly those epics that you need. In the future we plan to add additional filters - and we will start with filtering by tags .

Epics search and filter bar

Epic Documentation

Sustainable deployment of public projects

Auto DevOps (currently available in beta) can automatically deploy your application to a Kubernetes cluster. However, this process may stop if the cluster needs to restart pods - for example, if someone moved them and the original image cannot be found in the cache.

Starting with GitLab 10.5, public projects automatically configure the cluster so that it has access to the GitLab Container Registry even after the deployment pipeline finishes, which will make your application environment more stable.

Read more about the beta version of Auto DevOps

Opportunity for developers to create projects in groups

With each release, we are improving the flexibility of the GitLab access model: group or server administrators now have access to a setting that allows users with the Developer role to create projects.

Allow developers to create projects in groups

Read more in the documentation on access levels for creating a project.

Easy Prometheus integration in Kubernetes

In GitLab 10.4, we added the ability to deploy Prometheus in a Kubernetes cluster connected to it with one click . In this release, we have moved further: we have made the integration of the project with Prometheus automatic.

GitLab uses the Kubernetes API to query a deployed Prometheus server, and this server does not have to be open for access from outside the cluster.

More information about application control with Prometheus

Global Search API

We have added global search support to the GitLab API. In fact, this is the same global search from the GitLab web interface, only wrapped in an API - to allow external systems to take advantage of this functionality.

This will allow teams to create custom workflows - for example, search for something in files and send statistics on the results.

The API works regardless of whether you have Elasticsearch (available only for GitLab Starter and above).

See the Global Search API documentation for more information.

Label List Page Redesign

We have simplified the design of the list of labels page, and now it will become easier for you to understand the labels and manage each of them. We also brought the icons in line with our new design and regrouped links to tasks and merge requests for each specific label - they now take up less space in width, which simplifies working with them.

Labels list page redesign

See the tag documentation for details.

Transition to related merge requests from the commit page

Links to merge requests related to the commit are now displayed directly on the commit page. This is useful when you are viewing the repository commit history and want to know the general purpose and technical context of the commit. Now you can easily move to the merge request, which generated the commit directly from the page of this commit. From the merge request, in turn, you will see all the previous discussions; and if a task was associated with it (for example, using the task closing mechanism , you can even go back to it to see its purpose.

Thanks for this feature @hiroponz .

Navigate to related merge requests from commit page

Documentation of the transition to related merge requests from the commit page

Localization improvements

We continue to work on the localization of GitLab: in this version, we externalized the lines in the task and merge requests menu, on the repository graphs and repository diagrams page. The opportunity is available to any member of our translation community. If you want to join the GitLab translation, we will be glad to see you .

More on GitLab Localization

Hashed Storage Beta

Hashed storage is a new type of storage behavior introduced in version 10.0. Instead of associating the project URL with the folder structure in which the repository will be stored on disk, we associate a hash based on the project identifier. This makes the folder structure unchanged, which helps to get rid of additional requirements for state synchronization between URLs and disk structure. This means that you can rename a group, user or project with one transaction in the database - instantly.

Find out more in the hashed storage documentation.

Omnibus Enhancements

  • In GitLab Mattermost 4.6 , the download speed of the channel decreased and the compliance of section 508 was improved (Section 508)
  • Chef has been updated to 12.21.31
  • Chef Omnibus has been updated to 5.6.3.
  • SELinux rules have been added to quickly search for SSH keys .

Read more in the Omnibus GitLab documentation

GitLab Runner 10.5

Also in this release we release the next version of GitLab Runner - 10.5. GitLab Runner is an open source project used to run CI / CD jobs and send results back to GitLab.

The most important changes ::

A complete list of changes is in the CHANGELOG GitLab Runner .

See the GitLab Runner documentation for details.

Performance enhancements

A few performance improvements in GitLab 10.5 that you just can't help but mention:

You can see the full list of performance improvements for this version in the documentation.

Detailed release notes and update / installation instructions can be found in the original English post: GitLab 10.5 released with Let's Encrypt integration, Gemnasium dependency checks, and CI / CD external files .

We worked on the translation from English rishavant and sgnl_05 .