Omnibus

Serve your dish on all the tables

Balasankar C

https://balasankarc.in/talks/fossasia

Me

  • Balasankar "Balu" C
  • Jr. Distribution Engineer @ GitLab
  • Debian Developer
  • GNOME Foundation Member & Mozillian

Agenda

  • Why?
  • Introduction
  • DSLs
  • Process
  • Output
  • Caching
  • omnibus-gitlab walkthrough

Why?

  • Automate everything!
  • Safeguard from dependency version mismatch
    • Eg: libssl1.0 v/s libssl1.1 ;)
  • OS Package Managers
    • Familiar install/upgrade/removal process
  • Reliability in what you provide to users
    • Guaranteed to work

Introduction

Project DSL

name            "gitlab-ce"
maintainer      "GitLab Inc."
homepage        "https://gitlab.com"
replaces        "gitlab"
install_path    "/opt/gitlab"
build_version   Omnibus::BuildVersion.new.semver
build_iteration 1

dependency "preparation"
dependency "postgresql"
dependency "gitlab-cookbooks"
dependency "gitaly"

exclude "\.git*"

Software DSL

name 'bzip2'
default_version '1.0.6'

license 'BSD-2-Clause'
license_file 'LICENSE'

dependency 'zlib'
dependency 'openssl'

version '1.0.6' do
  source md5: '00b516f4704d4a7cb50a1d97e6e8e15b'
end
source url: "http://www.bzip.org/#{version}/#{name}-#{version}.tar.gz"

relative_path "#{name}-#{version}"

build do
...
end

Software DSL (Build Process)

build do
  env = with_standard_compiler_flags(with_embedded_path)

  # Avoid warning where .rodata cannot be used when making a shared object
  env['CFLAGS'] << ' -fPIC'

  # The list of arguments to pass to make
  args = "PREFIX='#{install_dir}/embedded' VERSION='#{version}'"

  patch source: 'makefile_take_env_vars.patch', env: env
  patch source: 'soname_install_dir.patch', env: env if mac_os_x?

  make args.to_s, env: env
  make "#{args} -f Makefile-libbz2_so", env: env
  make "#{args} install", env: env
end

Process

  • Read sources definitions
  • Get sources
  • Build software
  • Perform checks
  • Generate output files

Output

  • Platform specific packages
    • Deb on Debian/Ubuntu/Mint etc.
    • RPM on RedHat/CentOS/openSUSE
  • Package metadata file
    • Complete information of package and included softwares
  • Version Manifest file
    • JSON format
    • Proper versions of all software - git commit SHAs

Caching

  • Software source cache
    • S3 bucket backed
  • Build cache
    • Software definitions, for faster subsequent builds
    • Based on lightweight git tags
    • Cache invalidation - a change in
      • Order of build
      • Software definition
      • Version

Advanced Features

  • GPG Signing of packages
  • Changelog generation
  • Multi-level debugging
  • Publishing to different backends like Artifactory, S3, etc.

Summary

  • Full stack installers are a great idea.
  • Product developers, take a look at Omnibus
  • Ruby based, easy-to-understand and rich DSL
  • Single, general process to build your product for multiple operating systems
  • Platform specific build process with caching

Thanks & Credits

Thank You

This presentation is licensed under Creative Commons Share Alike license. Logos of software presented are trademark of respective owners.

balasankar@gitlab.com

@cbalasankar