platform/upstream/gstreamer.git
2 years agocerbero: Always fetch sources with four parallel jobs
Nirbheek Chauhan [Tue, 17 Aug 2021 10:34:42 +0000 (16:04 +0530)]
cerbero: Always fetch sources with four parallel jobs

The default number of parallel jobs is two, which is too few. We can
easily use four or more. Should speed up image builds and also
downloading of (new) sources that aren't already cached in the image.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/411>

2 years agogitlab: fix incorrect build-tools path being used on macOS
Matthew Waters [Thu, 29 Jul 2021 12:41:06 +0000 (22:41 +1000)]
gitlab: fix incorrect build-tools path being used on macOS

The relevant CI log

dyld: Library not loaded: /Users/gst-ci/builds/gstreamer/cerbero/cerbero-build/dist/darwin_universal/x86_64/lib/liborc-0.4.0.dylib
  Referenced from: /Users/gst-ci/builds/xhaakon/gstreamer-sharp/cerbero-build/dist/darwin_universal/x86_64/bin/orcc
  Reason: image not found

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/410>

2 years agoci: build universal macos packages instead of x86_64 only
Matthew Waters [Wed, 17 Feb 2021 05:32:38 +0000 (16:32 +1100)]
ci: build universal macos packages instead of x86_64 only

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/408>

3 years agoci_template: rebuild some images
Tim-Philipp Müller [Wed, 30 Jun 2021 15:45:52 +0000 (16:45 +0100)]
ci_template: rebuild some images

We need a newer cache for fontconfig since it renamed master branch
to main now and the wrap file has depth=1

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/407>

3 years agodocker: fedora: Add intel-mediasdk package.
Víctor Manuel Jáquez Leal [Mon, 24 May 2021 15:10:11 +0000 (17:10 +0200)]
docker: fedora: Add intel-mediasdk package.

Add intel-mediasdk package to build gst-msdk plugin in gst-plugins-bad.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/403>

3 years agoci_template: rebuild some images
Jordan Petridis [Mon, 7 Jun 2021 13:02:03 +0000 (16:02 +0300)]
ci_template: rebuild some images

We need a newer cache for harfbuzz since it renamed master branch
to main now and the wrap file has depth=1

https://gitlab.freedesktop.org/gstreamer/gst-build/-/merge_requests/246#note_948717

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/404>

3 years agoci_template: mark the manifest job as not allowed to fail
Jordan Petridis [Mon, 24 May 2021 20:11:10 +0000 (23:11 +0300)]
ci_template: mark the manifest job as not allowed to fail

With gitlab 13.12 the behavior of jobs that depend of manual
jobs changes, and they are considered skipped by default.

But in our pipeline we want to enforce that the whole pipeline
runs and succeds and the manual status is only there to reduce
CI load.

For more details see:

https://about.gitlab.com/blog/2021/05/20/dag-manual-fix/

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/404>

3 years agodocker/fedora: Bump image tag
Jordan Petridis [Fri, 4 Jun 2021 09:21:21 +0000 (12:21 +0300)]
docker/fedora: Bump image tag

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/406>

3 years agodocker/fedora: Fix find cli arg in prepare.sh script
Jordan Petridis [Fri, 4 Jun 2021 09:38:29 +0000 (12:38 +0300)]
docker/fedora: Fix find cli arg in prepare.sh script

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/406>

3 years agodocker/fedora: Bump to latest rustup
Philippe Normand [Sat, 22 May 2021 15:00:43 +0000 (16:00 +0100)]
docker/fedora: Bump to latest rustup

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/406>

3 years agodocker/fedora: Reverse meson installation
Philippe Normand [Sat, 22 May 2021 15:00:28 +0000 (16:00 +0100)]
docker/fedora: Reverse meson installation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/406>

3 years agodocker/fedora: Fix rpmfusion urls
Philippe Normand [Sat, 22 May 2021 14:59:54 +0000 (15:59 +0100)]
docker/fedora: Fix rpmfusion urls

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/406>

3 years agoci_template: gstreamer-full: enable auto features
Stéphane Cerveau [Thu, 29 Apr 2021 12:19:57 +0000 (14:19 +0200)]
ci_template: gstreamer-full: enable auto features

Enable auto features for gstreamer-full but
disable "introspection" for now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/402>

3 years agodocker: fedora: update string version
Stéphane Cerveau [Tue, 30 Mar 2021 13:38:03 +0000 (15:38 +0200)]
docker: fedora: update string version

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/400>

3 years agodocker: fedora: add qt5 devel package.
Stéphane Cerveau [Tue, 30 Mar 2021 08:27:33 +0000 (10:27 +0200)]
docker: fedora: add qt5 devel package.

add qt5 devel package to build qt plugin

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/400>

3 years agoci_template: use older windows image for cerbero
Jordan Petridis [Fri, 5 Mar 2021 06:28:47 +0000 (08:28 +0200)]
ci_template: use older windows image for cerbero

Current windows image doesn't seem to be able to spawn procceses for reason,
use a different image until this is debuged

```
Running command 'c:/builds/alatiera/gst-ci/cerbero-build/build-tools/bin/meson --prefix=c:/builds/alatiera/gst-ci/cerbero-build/dist/msvc_x86_64 --libdir=lib -Ddebug=true --default-library=both -Doptimization=2 --backend=ninja --wrap-mode=nodownload -Db_vscrt=md --native-file c:/builds/alatiera/gst-ci/cerbero-build/sources/msvc_x86_64/gstreamer-1.0-1.19.0.1/_builddir/meson-native-file.txt -Dlibunwind=disabled -Ddbghelp=enabled -Dintrospection=disabled -Dexamples=disabled'
failed to create process
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/399>

3 years agoci_template: fix WINDOWS_CONTAINER_SCRIPT_URL location
Jordan Petridis [Fri, 5 Mar 2021 05:12:35 +0000 (07:12 +0200)]
ci_template: fix WINDOWS_CONTAINER_SCRIPT_URL location

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/399>

3 years agoci_template: build windows image in the ci
Jordan Petridis [Wed, 27 Jan 2021 07:54:59 +0000 (09:54 +0200)]
ci_template: build windows image in the ci

Build the images in the ci runners like we do,
for the linux containers.

Kudos to mesa for the powershell script that
reimplements the ci-templates behavior.

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4304

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/392>

3 years agodocker/windows: make the powershell scripts more robust
Jordan Petridis [Wed, 27 Jan 2021 05:16:23 +0000 (07:16 +0200)]
docker/windows: make the powershell scripts more robust

Check the exit codes of various commands and fail
appropriatly upon error.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/392>

3 years agoci_template: use meson subproject --rebase for the windows builds as well
Jordan Petridis [Wed, 16 Dec 2020 15:23:15 +0000 (17:23 +0200)]
ci_template: use meson subproject --rebase for the windows builds as well

Similar to !375

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/382>

3 years agogitlab/cerbero: Don't fail when there's no macOS cache
Nirbheek Chauhan [Thu, 18 Feb 2021 09:19:41 +0000 (14:49 +0530)]
gitlab/cerbero: Don't fail when there's no macOS cache

`test -d "${CERBERO_HOME}/dist/${ARCH}"` will cause the build to error
out if the dist dir doesn't exist, which will happen if there's no
cache, for example:
https://gitlab.freedesktop.org/seungha.yang/gst-plugins-bad/-/jobs/7405565

What we actually want to do, is to not perform the rsync if there's no
cache, since it means we want to rebuild from scratch; for instance,
if the current MR uses a custom cerbero branch of the same name which
won't have a prebuilt cache.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/398>

3 years agodocker/windows: Update msys2 installer to 2021-02-15 version.
Biswapriyo Nath [Sat, 13 Feb 2021 15:21:35 +0000 (20:51 +0530)]
docker/windows: Update msys2 installer to 2021-02-15 version.

This fixes the PGP key installation issue.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/396>

3 years agogitlab: Allow du to fail in cerbero scripts
Nirbheek Chauhan [Wed, 3 Feb 2021 08:35:19 +0000 (14:05 +0530)]
gitlab: Allow du to fail in cerbero scripts

It's purely for informative reasons. `du` will fail on the sources dir
if a branch name has unicode in it due to an MSYS/MinGW bug. The long
term fix is to from MSYS/MinGW to MSYS/MinGW-W64 or MSYS2/MinGW-W64.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/395>

3 years agohotdoc: use latest pip version
Mathieu Duponchelle [Mon, 1 Feb 2021 13:59:04 +0000 (14:59 +0100)]
hotdoc: use latest pip version

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/394>

3 years agodocker/fedora: run git gc after cloning the subprojects
Jordan Petridis [Wed, 16 Dec 2020 15:16:39 +0000 (17:16 +0200)]
docker/fedora: run git gc after cloning the subprojects

Most of the subprojects specify depth=1 in the wrap file,
so this doesn't have a big effect atm.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/375>

3 years agocitemplate: use meson 0.56 and suproject update --reset
Jordan Petridis [Wed, 16 Dec 2020 15:22:40 +0000 (17:22 +0200)]
citemplate: use meson 0.56 and suproject update --reset

The windows build will be updated in a later
commit.

Close #73 #74 #75

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/375>

3 years agocitemplate: only spawn a single pipeline in Merge Requests
Jordan Petridis [Tue, 26 Jan 2021 06:39:42 +0000 (08:39 +0200)]
citemplate: only spawn a single pipeline in Merge Requests

This is the 3rd or 4th time we are trying this yes, (!331),
but hear me out. This time is for Real!

Jokes aside, its possible to avoid spawning >1 pipeline per event
with gitlab 13.8.

https://gitlab.com/gitlab-org/gitlab/-/issues/201845

https://gitlab.com/gitlab-org/gitlab/-/issues/299409

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/391>

3 years agogitlab: update macos/ios tags for newer vm image
Matthew Waters [Fri, 15 Jan 2021 12:29:33 +0000 (23:29 +1100)]
gitlab: update macos/ios tags for newer vm image

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/389>

3 years agoci_template: remove artifacts workaround for cerbero msvc
Jordan Petridis [Wed, 6 Jan 2021 18:48:41 +0000 (20:48 +0200)]
ci_template: remove artifacts workaround for cerbero msvc

This was fixed 3 weeks ago and should no longer be needed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/387>

3 years agoci_template: re-enable the extraction of logs for windows jobs
Jordan Petridis [Wed, 23 Dec 2020 02:46:19 +0000 (04:46 +0200)]
ci_template: re-enable the extraction of logs for windows jobs

Most of the networking issues with the runner should be fixed now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/385>

3 years agogstreamer-full: enable features unit test
Stéphane Cerveau [Fri, 9 Oct 2020 12:11:31 +0000 (14:11 +0200)]
gstreamer-full: enable features unit test

As gst-build can select features to be exposed by
gstreamer-full, the test will check that feature
are present or not present according to the current
configuration.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/381>

3 years agoRevert "ci: check commit using ci-fairy"
Jordan Petridis [Wed, 2 Dec 2020 10:38:57 +0000 (12:38 +0200)]
Revert "ci: check commit using ci-fairy"

This reverts commit 12c1689aaaf307feb50f465565f6d752f16a1b0e.

Revert ci-fairy check-commits job for now as it seems a bit
buggy. It picks up commits that are not part of the branch
and breaks the CI.

Will investigate further and reintroduce it later.

3 years agoci_template: job to complain about allow-edit
Jordan Petridis [Fri, 27 Nov 2020 01:05:11 +0000 (03:05 +0200)]
ci_template: job to complain about allow-edit

Make sure merge requests have enabled the "allow-edit"
option in the MR so we can rebase them.

Close gstreamer/gstreamer-project#34

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/304>

3 years agoci: check commit using ci-fairy
Guillaume Desmottes [Thu, 11 Jun 2020 12:45:24 +0000 (14:45 +0200)]
ci: check commit using ci-fairy

See https://freedesktop.pages.freedesktop.org/ci-templates/ci-fairy.html#checking-commits

In this setup it will check for any leftover fixup!/squash! commits and
that messages don't have the 'Signed-off-by:' tag.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/304>

3 years agocitemplate: build devtools for windows builds
Jordan Petridis [Mon, 16 Nov 2020 13:10:28 +0000 (15:10 +0200)]
citemplate: build devtools for windows builds

explicitly enable the cairo plugin build as well

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/377>

3 years agoRevert "cerbero: Never use cached cerbero git repo"
Jordan Petridis [Mon, 16 Nov 2020 13:40:16 +0000 (15:40 +0200)]
Revert "cerbero: Never use cached cerbero git repo"

This reverts commit c3b5693736aea7798df87b10de48239ef7e43de7.

This broke the build and needs further investigation

https://gitlab.freedesktop.org/gstreamer/gst-ci/-/jobs/5630095

3 years agocerbero: Never use cached cerbero git repo
Nirbheek Chauhan [Thu, 12 Nov 2020 08:33:26 +0000 (14:03 +0530)]
cerbero: Never use cached cerbero git repo

Sometimes the git repo cached on the docker image gets corrupted on
Windows, in which case there is no way to clean things and resolve
that error. For example: https://gitlab.freedesktop.org/bilboed/gst-plugins-bad/-/jobs/5517037

```
++ cp -a C:/cerbero/.git .
cp: cannot create regular file `./.git/objects/pack/pack-cbdde9dc4dd07bff6761e6715e20d7c0abe62b9e.idx': Permission denied
cp: cannot create regular file `./.git/objects/pack/pack-cbdde9dc4dd07bff6761e6715e20d7c0abe62b9e.pack': Permission denied
```

Always delete the cached git repo and copy a pristine copy from the
image. This will also avoid accumulation of garbage in the git repo.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/376>

3 years agocitemplate: remove fdo.expires-after label
Jordan Petridis [Thu, 15 Oct 2020 14:37:53 +0000 (17:37 +0300)]
citemplate: remove fdo.expires-after label

This is meant to be used for containers that are going
to be periodically rebuilt. For our usecase specifying
`fdo.upstream-repo` is enough to garbage collect all
the tags from the forks.

We also need to rebuild the images, so they don't have
include the label anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/368>

3 years agodocker/cerbero: remove bootstrap invocation
Jordan Petridis [Thu, 22 Oct 2020 13:04:01 +0000 (16:04 +0300)]
docker/cerbero: remove bootstrap invocation

`bootstrap --build-tools=no --toolchains=no` now do nothing, and
just repeat what the bootstrap line above does which is to only
run the system bootstraper

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/368>

3 years agocitemplate: tweak merge-request rules
Jordan Petridis [Wed, 14 Oct 2020 18:26:35 +0000 (21:26 +0300)]
citemplate: tweak merge-request rules

With the MergeRequest rules template, its was impossible
to manually create or trigger a pipeline for a branch/commit
that was not part of a merge request.

This tweaks the workflow:rules such that there will always be
a pipeline for each commit (which will be set to a manual job trigger).

There is the downside that we will now be triggering 2 pipelines in
case of merge requests, but they will require manual action to be
started.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/369>

3 years agogstreamer-full: enable unit test
Stéphane Cerveau [Fri, 9 Oct 2020 12:11:31 +0000 (14:11 +0200)]
gstreamer-full: enable unit test

Run test-gst-full in gstreamer-full job

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/365>

3 years agofedora: rebuild image so it includes a newer subproject cache
Jordan Petridis [Fri, 9 Oct 2020 09:14:32 +0000 (12:14 +0300)]
fedora: rebuild image so it includes a newer subproject cache

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/364>

3 years agoci_template: try to enable merge-request pipelines again
Jordan Petridis [Thu, 8 Oct 2020 10:57:36 +0000 (13:57 +0300)]
ci_template: try to enable merge-request pipelines again

Gitlab was slightly bugged in previous version, and we
had to use branch-only pipelines. Let's try this again.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/363>

3 years agoEnable pango plugin on Windows
Xavier Claessens [Thu, 1 Oct 2020 18:44:14 +0000 (14:44 -0400)]
Enable pango plugin on Windows

This tests that all fallbacks works to get pango plugin built with no
system dependencies available.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/360>

3 years agoRevert "citemplate: Actually use the new cerbero_setup.sh script"
Jordan Petridis [Sun, 4 Oct 2020 13:30:46 +0000 (13:30 +0000)]
Revert "citemplate: Actually use the new cerbero_setup.sh script"

This reverts commit 19d7ff53424c5bd104b73755f9f640bff8169392

3 years agocitemplate: Actually use the new cerbero_setup.sh script
Nirbheek Chauhan [Fri, 2 Oct 2020 16:49:57 +0000 (22:19 +0530)]
citemplate: Actually use the new cerbero_setup.sh script

Revert me after merge.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/361>

3 years agocitemplate: Consistent naming for cerbero_setup.sh script
Nirbheek Chauhan [Fri, 2 Oct 2020 17:10:30 +0000 (22:40 +0530)]
citemplate: Consistent naming for cerbero_setup.sh script

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/361>

3 years agocerbero ci: Make before_script wiping more thorough
Nirbheek Chauhan [Fri, 2 Oct 2020 16:47:53 +0000 (22:17 +0530)]
cerbero ci: Make before_script wiping more thorough

Will avoid packaging errors due to the old package file still being
around from a previous job, such as in:

https://gitlab.freedesktop.org/nirbheek/cerbero/-/jobs/4804147

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/361>

3 years agowindows: rebuild docker image
Jordan Petridis [Mon, 28 Sep 2020 11:00:24 +0000 (14:00 +0300)]
windows: rebuild docker image

also:

Revert "citemplate: set the cerbero msvc build to allow_failure again"

This reverts commit 3dddfacfbf710fa71777488145594eb0abe11ee4.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/359>

3 years agocerbero ci: Wipe in before_script because gitlab doesn't
Nirbheek Chauhan [Thu, 1 Oct 2020 04:30:49 +0000 (10:00 +0530)]
cerbero ci: Wipe in before_script because gitlab doesn't

Noticed in this job, which failed because we rebuilt on top of an
existing builddir:

https://gitlab.freedesktop.org/nirbheek/cerbero/-/jobs/4772728

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/358>

3 years agocitemplate: set the cerbero msvc build to allow_failure again
Jordan Petridis [Tue, 29 Sep 2020 15:07:06 +0000 (18:07 +0300)]
citemplate: set the cerbero msvc build to allow_failure again

Very weird things going on..

```
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 8608: character maps to <undefined>
```

https://gitlab.freedesktop.org/Hosang/gst-plugins-bad/-/jobs/4731636

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/356>

3 years agocitempalte: fix string comparitson in rules
Jordan Petridis [Mon, 28 Sep 2020 04:24:27 +0000 (07:24 +0300)]
citempalte: fix string comparitson in rules

When quoting in rules it takes the string as literal and doesn't
expand the variable

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/353>

3 years agoRevert "Switch MSVC job to manual till we can upload deps cache"
Nirbheek Chauhan [Thu, 24 Sep 2020 00:06:46 +0000 (05:36 +0530)]
Revert "Switch MSVC job to manual till we can upload deps cache"

This reverts commit bdbec90a4939d5c31f800cc9f07646c820a7bd76.

Everything seems to be working fine.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/352>

3 years agoRevert "cerbero ci: Actually use the new cerbero_setup.sh script"
Nirbheek Chauhan [Wed, 23 Sep 2020 23:48:24 +0000 (05:18 +0530)]
Revert "cerbero ci: Actually use the new cerbero_setup.sh script"

This reverts commit ef981bafc9540ca795e3ed0a6eb37950859d5da5.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/352>

3 years agodocker/cerbero: Do not remove toolchain downloads
Nirbheek Chauhan [Wed, 23 Sep 2020 08:48:57 +0000 (14:18 +0530)]
docker/cerbero: Do not remove toolchain downloads

We will use these when running bootstrap in the actual jobs.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/350>

3 years agocerbero ci: Add hacks for build tools
Nirbheek Chauhan [Tue, 22 Sep 2020 14:06:28 +0000 (19:36 +0530)]
cerbero ci: Add hacks for build tools

Instead of putting a hammer-like symlink to ../../gstreamer/cerbero,
do recipe-specific relocation hacks. Fixes random errors like:

```
++ ln -sf /builds/nirbheek/cerbero ../../gstreamer/cerbero
ln: ../../gstreamer/cerbero/cerbero: cannot overwrite directory
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/350>

3 years agocerbero ci: Refactor cerbero_before_script
Nirbheek Chauhan [Mon, 21 Sep 2020 18:34:08 +0000 (00:04 +0530)]
cerbero ci: Refactor cerbero_before_script

We were doing a few things suboptimally:

1. Making a symlink from the current directory to
   ../../gstreamer/cerbero so that build-tools kept working, which
   should not be necessary since cerbero relocates all binaries inside
   the cerbero homedir after extracting the cache.

2. Hard-coding the cerbero builddir when building on Windows because
   `pwd` was outputting an MSYS-style path which Python can't read

3. rsync-ing all the contents of $CERBERO_HOST_DIR, which is
   bad because we should be building on top of the artifact cache.
   The only thing we want from $CERBERO_HOST_DIR is cerbero-sources
   (the network cache), and we can setup the toolchain during
   bootstrap with the new --system and --toolchains arguments which
   were split out of --system-only.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/350>

3 years agocerbero ci: Actually use the new cerbero_setup.sh script
Nirbheek Chauhan [Mon, 21 Sep 2020 21:09:25 +0000 (02:39 +0530)]
cerbero ci: Actually use the new cerbero_setup.sh script

Revert after merging.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/350>

3 years agomanifest: Check all deps logs when picking cerbero commit
Nirbheek Chauhan [Mon, 21 Sep 2020 16:56:20 +0000 (22:26 +0530)]
manifest: Check all deps logs when picking cerbero commit

Instead of picking the 'slowest build', look at all the deps logs and
pick the commit that is mentioned in the maximum number of deps logs.

In practice, this will either be the newest commit or the previous
commit. If it's not the newest commit, we will warn and use an older
one.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/349>

3 years agocitemplate: set PYTHONFAULTHANDLER for tests
Jordan Petridis [Tue, 15 Sep 2020 13:38:39 +0000 (16:38 +0300)]
citemplate: set PYTHONFAULTHANDLER for tests

In case python segfaults, which might happen for things with
a C-FFI like pygobject, make it so we get a proper backtrace.

https://docs.python.org/3/using/cmdline.html#envvar-PYTHONFAULTHANDLER

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/341>

3 years agomanifest: Fix URL for cerbero deps log
Nicolas Dufresne [Mon, 21 Sep 2020 15:57:59 +0000 (11:57 -0400)]
manifest: Fix URL for cerbero deps log

There was a missing 'f' prefix, so the {branch} was not expanded.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/348>

3 years agomanifest: Build useful error message
Nicolas Dufresne [Mon, 21 Sep 2020 15:57:26 +0000 (11:57 -0400)]
manifest: Build useful error message

This helps detecting issue with optional steps like cerbero deps
lookup.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/348>

3 years agoRevert "cerbero ci: Also disable the deps build"
Nirbheek Chauhan [Mon, 21 Sep 2020 14:03:17 +0000 (19:33 +0530)]
Revert "cerbero ci: Also disable the deps build"

This reverts commit 555c49f9d17aa7a8b9e650941002b52e4e6dd325.

The job is working fine now and takes slightly more time than the iOS
build, so it can be re-enabled.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/347>

3 years agocerbero ci: Sprinkle some 'time' commands in the script
Nirbheek Chauhan [Sat, 19 Sep 2020 06:28:37 +0000 (11:58 +0530)]
cerbero ci: Sprinkle some 'time' commands in the script

Sprinkle some 'time' commands so we can measure how long things take
and we can know where to spend time optimizing first.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/346>

3 years agoci: Switch cerbero script URL back to gst-ci master
Nirbheek Chauhan [Thu, 17 Sep 2020 19:06:55 +0000 (00:36 +0530)]
ci: Switch cerbero script URL back to gst-ci master

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/345>

3 years agocerbero ci: Also disable the deps build
Nirbheek Chauhan [Thu, 17 Sep 2020 18:20:56 +0000 (23:50 +0530)]
cerbero ci: Also disable the deps build

There seems to be some issues uploading the cache from the Windows
runner.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/344>

3 years agocerbero ci: Try to fix deps build on the gstreamer namespace
Nirbheek Chauhan [Thu, 17 Sep 2020 11:06:59 +0000 (16:36 +0530)]
cerbero ci: Try to fix deps build on the gstreamer namespace

Can't ln twice: it does a cp on MSYS.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/344>

3 years agocerbero ci: Don't expand the ssh private key
Nirbheek Chauhan [Thu, 17 Sep 2020 16:21:49 +0000 (21:51 +0530)]
cerbero ci: Don't expand the ssh private key

Because of set -x we were printing the key into the CI logs:
https://gitlab.freedesktop.org/gstreamer/cerbero/-/jobs/4572836#L797

Don't ever expand the variable in shell.

3 years agoSwitch MSVC job to manual till we can upload deps cache
Nirbheek Chauhan [Thu, 17 Sep 2020 10:52:06 +0000 (16:22 +0530)]
Switch MSVC job to manual till we can upload deps cache

The deps cache job is failing:

https://gitlab.freedesktop.org/gstreamer/cerbero/-/jobs/4568410

Because of which MRs in other repos are unable to download the cache
and are building everything from scratch, which is inflating CI times
to 2 hours:

https://gitlab.freedesktop.org/marian/gst-plugins-bad/-/jobs/4567025

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/343>

3 years agocitemplate: post-merge fix for url path
Jordan Petridis [Wed, 16 Sep 2020 23:14:55 +0000 (02:14 +0300)]
citemplate: post-merge fix for url path

switch it so it looks at the stable branch for the cerbero_setup
script.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/342>

3 years agocitemplate: add initial cerbero msvc builds
Jordan Petridis [Fri, 7 Aug 2020 12:29:39 +0000 (15:29 +0300)]
citemplate: add initial cerbero msvc builds

This commit adds the last bits needed to have a working
CI job of cerbero using mingw and msvc17 toolchain.

It adds a new stage to the docker image that mirrors
the setup of the cerbero fedora docker build, and
tweaks the cerbero_setup script run by jobs with
a couple windows specific fixes.

Close #39

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/334>

3 years agogitlab/cerbero_setup.sh: refactor some tests to if/else statments
Jordan Petridis [Thu, 6 Aug 2020 17:48:01 +0000 (20:48 +0300)]
gitlab/cerbero_setup.sh: refactor some tests to if/else statments

and replace the ccache check

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/329>

3 years agoci_template: move the cerbero scripts from yaml to a new file
Jordan Petridis [Thu, 6 Aug 2020 17:21:44 +0000 (20:21 +0300)]
ci_template: move the cerbero scripts from yaml to a new file

Its hard to manage ever growing bash scripts in yaml, and
its even harder to run them locally to reproduce the environment.

This is essentially a copy-paste of the scripts, to make review
easier. We can refactor later.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/329>

3 years agocitemplate: workaround stale subproject cache
Jordan Petridis [Tue, 15 Sep 2020 13:58:40 +0000 (16:58 +0300)]
citemplate: workaround stale subproject cache

The revision of the wrap for pygobject was switched recently [1]

[1]: https://gitlab.freedesktop.org/gstreamer/gst-build/-/merge_requests/189

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/339>

3 years agodocker/fedora: install more debug symbols
Jordan Petridis [Thu, 10 Sep 2020 18:10:37 +0000 (21:10 +0300)]
docker/fedora: install more debug symbols

These are needed for the webrtcbin tests

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/339>

3 years agodocker/windows: move the git user config outside the script
Jordan Petridis [Tue, 1 Sep 2020 18:30:29 +0000 (21:30 +0300)]
docker/windows: move the git user config outside the script

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/333>

3 years agodocker/windows: tweak installation of git
Jordan Petridis [Fri, 28 Aug 2020 16:30:41 +0000 (19:30 +0300)]
docker/windows: tweak installation of git

Make sure we set autocrlf to false, which is needed by cerbero.
Additionally remove useless for the container integrations with
the shell.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/333>

3 years agocerbero/fedora: install which into the image
Jordan Petridis [Wed, 9 Sep 2020 14:31:08 +0000 (17:31 +0300)]
cerbero/fedora: install which into the image

We used to get this as transient dependency it seems,
which broke rebuilds of the img once w/e pkg was
depending on it, stopped.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/338>

3 years agocitemplate: update the buildah bootstrap image
Jordan Petridis [Wed, 9 Sep 2020 14:34:39 +0000 (17:34 +0300)]
citemplate: update the buildah bootstrap image

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/338>

3 years agoci_template: correctly spell the global var
Jordan Petridis [Wed, 9 Sep 2020 13:00:54 +0000 (16:00 +0300)]
ci_template: correctly spell the global var

Close #83

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/337>

3 years agocitemplate: rework gitlab rules
Jordan Petridis [Wed, 26 Aug 2020 11:55:58 +0000 (14:55 +0300)]
citemplate: rework gitlab rules

Rework rules of jobs to prever duplicate pipelines.

https://docs.gitlab.com/ce/ci/yaml/README.html#prevent-duplicate-pipelines

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/331>

3 years agoci_template: remove rules from the docker build
Jordan Petridis [Thu, 27 Aug 2020 12:35:12 +0000 (15:35 +0300)]
ci_template: remove rules from the docker build

We can run the jobs always automatically since, most of the
time they will be copying/reffing the existance image in the
registry.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/331>

3 years agocitemplate: Allow spaces in filenames when indenting
Nirbheek Chauhan [Wed, 5 Aug 2020 20:04:18 +0000 (01:34 +0530)]
citemplate: Allow spaces in filenames when indenting

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/328>

3 years agocitemplate: add support for .indentignore file to indent job
Tim-Philipp Müller [Tue, 4 Aug 2020 23:32:17 +0000 (00:32 +0100)]
citemplate: add support for .indentignore file to indent job

See https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1465

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/328>

4 years agocitemplate: skip integration testsuites for gstreamer-sharp
Jordan Petridis [Sun, 26 Jul 2020 10:01:06 +0000 (13:01 +0300)]
citemplate: skip integration testsuites for gstreamer-sharp

Close #80

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/326>

4 years agocitemplate: make it so docker builds only run for detached pipelines
Jordan Petridis [Tue, 7 Jul 2020 16:25:19 +0000 (19:25 +0300)]
citemplate: make it so docker builds only run for detached pipelines

Currently, if you have a MR open, there 2 pipelines being
triggered. One normal, and one detached.

Previously, if you were to rebuild an image, the jobs of the
docker build stage would be executed concurrently, race
and end up both rebuilding the image.

Make them manual for normal pipelines to avoid such occurrence
and waste of resources.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/320>

4 years agocitemplate: properly pin the image of the docs job
Jordan Petridis [Fri, 24 Jul 2020 15:53:39 +0000 (18:53 +0300)]
citemplate: properly pin the image of the docs job

we don't need the image to be copied or rebuilt, as
its currently pinned shortly and won't change anytime soon.

ammends dc486667426d0d31284e25001cfece45879595a8 !324

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/325>

4 years agocitemplate: revert the docs job to the previous image build
Jordan Petridis [Thu, 23 Jul 2020 12:16:55 +0000 (15:16 +0300)]
citemplate: revert the docs job to the previous image build

The update to f31 from !271 triggered an error in hotdoc it seems,
so rollback for now till we figure it out.

https://gitlab.freedesktop.org/gdesmott/gst-plugins-base/-/jobs/3764215

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/324>

4 years agodocker/fedora: update base to fedora 31
Jordan Petridis [Wed, 10 Jun 2020 14:47:08 +0000 (17:47 +0300)]
docker/fedora: update base to fedora 31

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/271>

4 years agogitlab: Support android tarballs compressed with xz
Nirbheek Chauhan [Fri, 10 Jul 2020 16:59:37 +0000 (22:29 +0530)]
gitlab: Support android tarballs compressed with xz

Needed for
https://gitlab.freedesktop.org/gstreamer/cerbero/-/merge_requests/540.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/322>

4 years agogitlab: update hotdoc
Mathieu Duponchelle [Thu, 9 Jul 2020 23:09:26 +0000 (01:09 +0200)]
gitlab: update hotdoc

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/321>

4 years agodocker/fedora: add gtest to get openh264 to build again
Jordan Petridis [Mon, 29 Jun 2020 13:05:03 +0000 (16:05 +0300)]
docker/fedora: add gtest to get openh264 to build again

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/315>

4 years agogst-build: only enable werror for gst recipes
Matthew Waters [Mon, 30 Sep 2019 04:44:10 +0000 (14:44 +1000)]
gst-build: only enable werror for gst recipes

It may throws various warning that are treated as errors for other
repositories

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/171>

4 years agocitemplate: use the default keyword for our global policy
Jordan Petridis [Thu, 2 Jul 2020 14:21:58 +0000 (17:21 +0300)]
citemplate: use the default keyword for our global policy

It was introduced in a recentish release along with pipelines
for merge requests, but it required the use of rules.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/318>

4 years agocitemplate: avoid the need to upgrade buildah in docker build
Jordan Petridis [Thu, 2 Jul 2020 12:43:13 +0000 (15:43 +0300)]
citemplate: avoid the need to upgrade buildah in docker build

We only need the newer version from the image, to support
the --arch arg, but we don't really use it yet so we
could live without it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/318>

4 years agocitemplate: run check in gst-examples but not integration tests
Jordan Petridis [Wed, 1 Jul 2020 15:34:07 +0000 (18:34 +0300)]
citemplate: run check in gst-examples but not integration tests

gst-example contains tests for check but not integrations suite

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/318>

4 years agocitemplate: run the docs job in more modules
Jordan Petridis [Wed, 1 Jul 2020 15:32:26 +0000 (18:32 +0300)]
citemplate: run the docs job in more modules

Needed to catch regressions in the docs cache files.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/318>

4 years agocitemplate: slightly improve logging of docker build jobs
Jordan Petridis [Wed, 1 Jul 2020 14:22:33 +0000 (17:22 +0300)]
citemplate: slightly improve logging of docker build jobs

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/318>

4 years agoci: use gst-indent from docker image
Guillaume Desmottes [Mon, 15 Jun 2020 12:06:59 +0000 (14:06 +0200)]
ci: use gst-indent from docker image

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/318>

4 years agogitlab: Update hotdoc
Thibault Saunier [Wed, 1 Jul 2020 23:56:16 +0000 (19:56 -0400)]
gitlab: Update hotdoc