Lin Yang [Tue, 10 Dec 2013 12:00:33 +0000 (20:00 +0800)]
Add profile info into accept tag name
Change accept tag name format to accept/{OBS_PRJ_NAME}/{DATE.TIME} to insert
profile info.
Fixes: #1511
Change-Id: I1d1e5087e3966e4ec52b804c631037c7cfef8093
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Ed Bartosh [Sun, 8 Dec 2013 11:37:35 +0000 (13:37 +0200)]
policycheck: Check if specs found in packaging/ directory
If there is no spec files in packaging/ directory job should fail and
send message back to gerrit informing submitter about the issue.
Change-Id: Ib6a129fe94fc9e011fd9b33be31f4083b6d73bc2
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sun, 8 Dec 2013 11:41:46 +0000 (13:41 +0200)]
policycheck: fail the job if gbs export fails
If gbs export fails job should fail as this is definitely policy issue.
Ignoring this doesn't make sense.
Change-Id: Ibf377af948e229d6c01f5cb3bda7bdeba83ea07e
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sun, 8 Dec 2013 11:30:23 +0000 (13:30 +0200)]
policycheck: Check if packaging/ directory exist
if there is no packaging/ directory in the project policycheck should
fail and send message back to Gerrit.
Currently if project doesn't have packaging directory job produces
non-informative traceback:
Traceback (most recent call last):
File "jenkins-scripts/job_policycheck.py", line 196, in <module>
sys.exit(main())
File "jenkins-scripts/job_policycheck.py", line 172, in main
if not export(prjdir, packagingdir, mygit, events, mygerrit):
File "jenkins-scripts/job_policycheck.py", line 62, in export
tempd = tempfile.mkdtemp(dir=os.path.join(prjdir, packagingdir))
File "/usr/lib64/python2.7/tempfile.py", line 318, in mkdtemp
_os.mkdir(file, 0700)
and doesn't report anything back to submitter.
Fixes: #1526
Change-Id: Ic98c26e04c45ff22eeb9136acfe006159d9eff5c
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sun, 8 Dec 2013 13:06:11 +0000 (15:06 +0200)]
Catch backenddb EntityError exception
When there is no map for some OBS project backenddb API raises
EntityError exception. It should be catched.
Fixes: #1528
Change-Id: I4c7fa8c5562c4d01a5d9b67e9a4a5d1d38e4b082
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sun, 8 Dec 2013 10:15:03 +0000 (12:15 +0200)]
Check if key is in mapping to avoid KeyError
This change should fix KeyError traceback in parsing description. In
current code everything with ':' is considered as key:value, which is
incorrect assumption as ':' is a very common symbol.
Fixes: #1524
Change-Id: I29deb4e410ba820535e09d8c4808894296c40cda
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 9 Dec 2013 18:17:08 +0000 (20:17 +0200)]
job_pre_release_gerrit: Check if prerelease enabled for OBS project
To avoid triggering workflow for all projects prerelease_enabled
function is implemented and used in job_pre_release_gerrit to filter out
unrelated projects.
Fixes: #1531
Change-Id: I65f3cd98bedacf61e7dae769b94f4c7091c52d4a
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 9 Dec 2013 09:32:41 +0000 (11:32 +0200)]
Fixed linking and unlinking repositories
Linking and unlinking was changing only first repository in the project.
The rest repos were left unchanged.
Fixes: #1530
Change-Id: Ia40fc6249f244644375a1b71f2c9a8ddfccc2067
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Wed, 4 Dec 2013 12:26:29 +0000 (14:26 +0200)]
Check return code of sync function
When sync filed job_imager should also fail.
Fixes: #1513
Change-Id: I261b6c1844f1c2aa2e19e8b968f08a39d4b93e96
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Hasan Wan [Wed, 4 Dec 2013 06:44:22 +0000 (08:44 +0200)]
fix the releaesid key name mistake when update value
Change-Id: I08168e89393b7d49e4b20bd49fe4931257d61119
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Hasan Wan [Wed, 4 Dec 2013 06:42:22 +0000 (08:42 +0200)]
Fix a typo when get exception
Change-Id: I9761ec87c71684b0c060f75d4e906bff8388bdca
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Hasan Wan [Wed, 4 Dec 2013 06:15:40 +0000 (08:15 +0200)]
enable createrepo_c for repomaker
Change-Id: I53166ed7cace598b7b7e569f9c9fa425d148560f
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Ed Bartosh [Mon, 2 Dec 2013 16:39:22 +0000 (18:39 +0200)]
Code cleanup of job_pre_release_obs
Pylinted + code cleanup.
Change-Id: Ifaa2adb1cd57d5dcaeee5f0abf7e4871266960d2
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sun, 1 Dec 2013 21:17:07 +0000 (23:17 +0200)]
Cleanup prerelese repos from all snapshots
When cleaning up prerelease repos we should consider all snapshots, not
just the latest one.
Change-Id: I85b2d1dfedeb979eaf135ca5f4416ef54c3fbb93
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sun, 1 Dec 2013 20:16:56 +0000 (22:16 +0200)]
Reimplemented logic of finding published repos
Current logic assumes that all repos of one OBS projects have the same
snapshot id, which is not always true. First of all due to the bug in
boss-repomaker they're always different. Second, for one OBS project
some repos can be published, but some can't because of package build
failures, so snapshot ids can be different for different repos in the
same project and they has to be found and cleaned up.
As it's impossible to find repo only by OBS project name more
sophisticated logic should be used:
- Find all keys, starting from OBS prject name
- Part after colon may be repo name
- Repo name should not contain colon as some projects may have the same
prefix, e.g. Tizen:IVI and Tizen:IVI:Release
Change-Id: I018105dc52244d8530b26daa5a4b47bf6cc6ea1a
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 2 Dec 2013 16:11:11 +0000 (18:11 +0200)]
Used repo name as a part of release key
PartOf and Release keys in repos.yaml are not uniq. Using them as a
uniq identifier leads to breakage of workflow as they're continuously
rewritten by repomaker for every snapshot for the repos with the
same PartOf and Release keys in repos.yaml
This change uses repo name instead of PartOf and Release. Name is
uniq in repos.yaml, so this should hopefully fix the issue.
Note, that this change requires repomaker to store release ids using
repo name. This is done in this change:
https://otctools.jf.intel.com/review/#/c/8189/
Change-Id: I63e033bff640ec2ae93e1d522dce55758aff2acd
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 2 Dec 2013 16:06:38 +0000 (18:06 +0200)]
Pylinted job_pre_release_obs.py
Change-Id: I65f7ff443ecfbb941f91cd8bf2e92ada2d6f82fc
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sat, 30 Nov 2013 15:34:37 +0000 (17:34 +0200)]
Refined warning message
This is quite important message. Let's make it a bit more clear.
Change-Id: I3653a702511d06e1e93e7697cab5ea3e7cfd85f4
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 2 Dec 2013 15:57:39 +0000 (17:57 +0200)]
Used OBS repo to find repo name from repos.yaml
This is an adaptation of job_pre_release_obs and job_create_snapshot to
the change in the key format for obs -> repo mappings. Now OBS repository
is also used to find repo name. This change passes OBS repository to
Snapshot class to find repo name properly.
Change-Id: I30adf0c8f3efceaabeec77a468613cacf2ec1e23
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sat, 30 Nov 2013 14:59:25 +0000 (16:59 +0200)]
Added obs repository as a part of obs->repo mapping
OBS project can be mapped to multiple repo names in repos.yaml depending
on obs repository, for example:
Tizen:Mobile OBS project currently maps to 3 repos:
Tizen-mobile for OBS repo atom,
Tizen-mobile-rd-pq for OBS repo arm,
Tizen-emulator for OBS repo emulator.
so, it's not possible to find repo name by using OBS project.
OBS repository should be taken into consideration as well.
This is a serious design fault in the current code.
This change is trying to fix this by using <OBS project>:<OBS repository>
key in obs->repo mapping API.
Change-Id: Ibfb7b77b3efbbbde65d8deeff1b90e08e0695977
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Lin Yang [Thu, 28 Nov 2013 03:39:53 +0000 (11:39 +0800)]
Clean up use obsolete function end()
Because end() function has been already removed, should not use it any more.
Change-Id: Id02d2e7aaec145236557b8fa228098e818e2aeb1
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lin Yang [Thu, 28 Nov 2013 03:24:11 +0000 (11:24 +0800)]
Change return code when init TempPackage exception
One buildcheck project will send out multiply OBS_REPO_PUBLISH_STATE events, the
first build will delete this project, the following builds will rasie this
exception. In this case, it should return 0.
Change-Id: I6816b9fe61cb9b4e9a6baaae9b415587795fccfc
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lingchaox Xin [Wed, 27 Nov 2013 02:58:18 +0000 (10:58 +0800)]
Fix is_new_package method not found
It is not fixed completely in 9340906 commit
Change-Id: I862e941d0b78f4cb8b68818da59d924335f3cb71
Lin Yang [Wed, 27 Nov 2013 06:34:39 +0000 (14:34 +0800)]
Return directly if post comment to gerrit failed
Change-Id: I3f4c7c5da7089645caaecfb49970ce60d5da656b
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Hasan Wan [Fri, 22 Nov 2013 03:25:24 +0000 (03:25 +0000)]
Trigger the next job with project name info
Change-Id: I5b8e1534a445fb340f9ffdad38bc02cb32f4e109
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Lin Yang [Thu, 21 Nov 2013 07:06:50 +0000 (15:06 +0800)]
Catch GerritError when post comment back to gerrit
Change-Id: Ie00925f18f10cca8560e65e45f934758f2395cb7
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lin Yang [Tue, 26 Nov 2013 07:03:28 +0000 (15:03 +0800)]
Enable more flexible pattern to sync code
When sync code from outside to inside gerrit, need more flexible pattern to
control which refname need to be synced, like only sync one branch.
Change-Id: I65bafca9574c7d7a0a89d728fefc4b08e4132a0a
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lin Yang [Wed, 20 Nov 2013 06:15:51 +0000 (14:15 +0800)]
Split the output of _git_inout to lines
Change-Id: I93f4ff3bd892137806e151015d841e29ae21cb81
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Hasan Wan [Fri, 15 Nov 2013 07:24:26 +0000 (09:24 +0200)]
Fix the image creation job not trigger issues
Fixes: #1462
Change-Id: Ia07fdf3132a7a61b65df622ef9b289931d5691a0
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Change-Id: I3b35bf56963106674e46c49c5c485a9d9b76d4fb
Hasan Wan [Fri, 15 Nov 2013 04:21:02 +0000 (06:21 +0200)]
job_submit.py are required in jenkins-scripts package
Change-Id: I7e40d106cd74fb3199d4e49c7f423401eef0aea3
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Lin Yang [Wed, 13 Nov 2013 07:15:43 +0000 (15:15 +0800)]
Do not need to specify specfile when gbs rb
Because gbs can automatically handle multiply specfile issue, don't need to
specify specfile in gbs command any more.
Change-Id: I09707dc3f66302a4362dea5057743314d59aa1b9
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Ed Bartosh [Tue, 12 Nov 2013 15:30:45 +0000 (17:30 +0200)]
Used atexit to remove temporary directories
Change-Id: I171819fa87dea5933bbbe1592a6ffd759683b410
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Tue, 12 Nov 2013 12:39:24 +0000 (14:39 +0200)]
job_submitobs: Pylinted
Change-Id: Ic250816f3d8a091f37d8d30bc448180fb518a927
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Tue, 12 Nov 2013 11:53:44 +0000 (13:53 +0200)]
Moved ref sync from job_sync_git to job_submitobs
This is done to avoid submitting to obs on sync instance if sync fails.
It's impossible to do when jobs are running asynchronously.
This change also reduces amount of code as it removes duplication
code used in both jobs.
Fixes: #1437
Change-Id: Iba10ec0126963b65296ba516cc7215a86212db36
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 11 Nov 2013 19:24:08 +0000 (21:24 +0200)]
Catch git errors, print error message.
Fixes: #1407
Change-Id: Ifaa3dc82da2d094582c075b49905138ec6813997
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Lin Yang [Wed, 13 Nov 2013 06:51:36 +0000 (14:51 +0800)]
No need to checkout upstream pristine-tar branch
Because gbs new version can automatically checkout upstream and pristine-tar
branch before gbs export, don't need to checkout these branches manually.
Change-Id: I477070020fa38e1651a46a6727b9d366045b1ad0
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Ed Bartosh [Mon, 11 Nov 2013 19:06:24 +0000 (21:06 +0200)]
Catched git errors
Used _git_inout instead of deprecated _git_getoutput.
Catched errors and raised local exception.
Change-Id: Ib7e844a0740e1215c18e8862dabedd0a5add5307
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Tue, 12 Nov 2013 16:16:56 +0000 (18:16 +0200)]
Pylinted common/buildservice.py
Fixed one typo. Got rid of unused variables.
Change-Id: Icf597b912100dd0f6e60b88dca990d87b55cc678
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Tue, 12 Nov 2013 16:02:56 +0000 (18:02 +0200)]
Catch urllib2.URLError
Caught urllib2.URLError exception. Included more info into error
message.
Fixes: #1408
Change-Id: Ib86635c50e305e8228cf62840c9cee527bc42bc8
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Lingchaox Xin [Thu, 14 Nov 2013 05:42:45 +0000 (13:42 +0800)]
Fix rpm packaging bug
Remove `/var/lib/jenkins/userContent/jenkins-scripts' link only when
uninstall the package.
Change-Id: I8fffb85cb60b6b5bd0f12796281404b4d50fdb6b
Lingchaox Xin [Wed, 13 Nov 2013 07:00:02 +0000 (15:00 +0800)]
Fix creating temp workspace since packaging dir is a relative path
Change-Id: I362547d80ea7537fd5cd0c2b8a2dbf7fb3c86de0
Lingchaox Xin [Tue, 29 Oct 2013 08:54:04 +0000 (16:54 +0800)]
Check whether `gbs export` succeeds before remote building
Feature #1396
Change-Id: Iedfadd42d40e759dafdcb16d9d221890270b24f1
Lin Yang [Mon, 11 Nov 2013 14:08:24 +0000 (22:08 +0800)]
Check whether need to sync project to inside obs
For sync instance, need to check whether this project match the regular
expression in jobs/configuration to sync this submission to inside obs.
Fixes: #1426
Change-Id: Ide8addde38a40dab1487d7d97f1a477dc6dbdbff
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Hasan Wan [Mon, 11 Nov 2013 06:26:17 +0000 (06:26 +0000)]
Generate repo manifest file
Fixes: #1392
Change-Id: I1f6b055ec59ad8bc5caaef681ff09bc08bd5c9d3
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Hasan Wan [Wed, 6 Nov 2013 03:17:34 +0000 (03:17 +0000)]
Keep another copy of buildconf in repodata
Change-Id: I530a4287b8bd692bbf065244cb4367211b729ed1
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Hasan Wan [Wed, 6 Nov 2013 03:16:59 +0000 (03:16 +0000)]
Fix a mistake of previous patch
Change-Id: I861a910652999013830d2eba20d3eebc66fa42fd
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Lin Yang [Fri, 8 Nov 2013 06:33:02 +0000 (14:33 +0800)]
Fix delete obs package failure in linked project
Before delete obs package, we should make sure this package still exist in obs.
Origin function bs.exists() will return true if this package don't exist in this
project, but in another linked project. So move to get_package_real_project_name
to check it.
Fixes: #1428
Change-Id: I969aac13df6df532694ef7ed3a30adfc8e7ec5c1
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lin Yang [Fri, 8 Nov 2013 04:40:25 +0000 (12:40 +0800)]
Fix create symlinks under userContent failure
Safer way to rewrite the symlink when install and upgrade package.
Fixes: #1433
Change-Id: I4f62f47255faae35010316082e665b52466200d6
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Ed Bartosh [Mon, 4 Nov 2013 15:39:57 +0000 (17:39 +0200)]
Catch GerritError exception
Wrapped gerrit.Gerrit.* calls into try/except to catch GerritError
exception and produce human-readable error messages.
Fixes: #1436
Change-Id: Id7aec6ced7a8803987eb765e51bc667f0b4de557
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 4 Nov 2013 14:48:05 +0000 (16:48 +0200)]
Refactored common/gerrit
Now Gerrit class raises GerritException when it gets non-zero return
code from ssh gerrit interface. Previously errors were silently ignored.
Change-Id: I68d48a010c3427e00daa1306127de5e3b9e17bbd
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Hasan Wan [Mon, 4 Nov 2013 07:48:18 +0000 (09:48 +0200)]
retry the obs operation when get ObsError
Fixes: #1103
Change-Id: I68d5935133b94167456a2d81bc747bdaa5e14342
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Hasan Wan [Mon, 4 Nov 2013 07:17:53 +0000 (09:17 +0200)]
make retry function as decorator
Change-Id: Ibbe7114af871b3cafc2dde6e4085b52e6737f20e
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Hasan Wan [Thu, 31 Oct 2013 04:14:57 +0000 (04:14 +0000)]
move xml2obj to common/utils
Change-Id: Ie4dac18217667e0095cad8c2ca958055c949d656
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Hasan Wan [Thu, 31 Oct 2013 03:53:19 +0000 (03:53 +0000)]
Trigger post snapshot creation jobs after snapshot created
Change-Id: I702320a5eff7af494c1725a0358bbd78a6b574f8
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Hasan Wan [Thu, 31 Oct 2013 03:03:41 +0000 (03:03 +0000)]
Fix an error with the project init
Change-Id: Ic979ab0574df02b23b65f3f9374a2b7bf30fdfc0
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Lin Yang [Thu, 31 Oct 2013 07:55:47 +0000 (15:55 +0800)]
Fix no logfile exception in buildcheck job
If obs package is in some states, like unresolvable, it will throw exception
when try to fetch build log for this package. And we need to send out build log
only for build failed package.
Fixes: #1429
Change-Id: I3f3754bc7200b15d720d8f23de86015110cd426f
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lingchaox Xin [Tue, 29 Oct 2013 08:51:57 +0000 (16:51 +0800)]
Handle gbs commands errors
Before this, gbs errors will be ignored
Fix #1406
Change-Id: I4f0db8902e23567c5ef0615f50279b952a4e654a
Lingchaox Xin [Tue, 29 Oct 2013 02:00:25 +0000 (10:00 +0800)]
Use oscapi directly rather than invoke core.meta_exists
Change-Id: I9b7945c56d385b032cb8b965d2b47c3e7c25e080
Lin Yang [Fri, 25 Oct 2013 04:12:55 +0000 (12:12 +0800)]
Fix import gbp.rpm.parse_spec failure
In gbp new version, it change parse_spec() function name to SpecFile().
Fixes: #1419
Change-Id: I74bb5971080038e89a573dd5b86cc2567600de9f
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lin Yang [Mon, 28 Oct 2013 06:18:30 +0000 (14:18 +0800)]
Fix submitobs job cannot send out email
Use ${BUILD_TAG}.env file as file parameter, instead of ${random}.mail.env, to
trigger downstream job mail-sender to match jenkins job conf change.
Fixes: #1422
Change-Id: Ie791588eaadd26036b0e909427023cb07ff5fb91
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lin Yang [Tue, 29 Oct 2013 02:43:38 +0000 (10:43 +0800)]
Handle manually created requests in obs
The description section format of requests manually created by admin is
different as default format, so catch this kind of exception when trying parse
the request description and ignore this request.
Fixes: #1397
Change-Id: Ibba0406fa1161cf2f0fc519d545e610e339fecf5
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lin Yang [Mon, 28 Oct 2013 11:53:07 +0000 (19:53 +0800)]
Change tools repo to otctools obs repo
To get the latest released tools, change tools dependency repo from tizen.org
repo to otctools obs repo.
Change-Id: I1c5320a89535636e6492d1b45f7128f309dde829
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Ed Bartosh [Wed, 23 Oct 2013 11:31:44 +0000 (14:31 +0300)]
Fixed KeyError exception in job_load_repos
This happened in production:
Traceback (most recent call last):
File "jenkins-scripts/job_load_repos.yaml.py", line 75, in <module>
sys.exit(main())
File "jenkins-scripts/job_load_repos.yaml.py", line 70, in main
update_repos_conf_on_filesystem(os.path.join(prjdir, 'repos.yaml'))
File "jenkins-scripts/job_load_repos.yaml.py", line 47, in
update_repos_conf_on_filesystem
repo['SnapshotDir'])
KeyError: 'SnapshotDir'
Change-Id: I8fe89fa7f509f001824215791cee4f24be6e1a9d
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Lin Yang [Tue, 22 Oct 2013 13:09:35 +0000 (21:09 +0800)]
Fix push to heads/accepted/* hierarchy failure
When parse one tag info, it should return the commit id the tag attached, not
the hash number the tag itself.
Fixes: #1395
Change-Id: I7f82c84c35924a42d67dc7b8e17e7c9c48e13b3a
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lingchaox Xin [Tue, 24 Sep 2013 02:59:55 +0000 (10:59 +0800)]
Catch git push tag exception
Fixes: #1344
Change-Id: I2b174f5ce142a54353361b6e03c8febfc01472e4
Signed-off-by: Lingchaox Xin <lingchaox.xin@intel.com>
Hasan Wan [Thu, 17 Oct 2013 07:23:00 +0000 (07:23 +0000)]
Add a new scripts to generate the buildlogs dir for snapshots
Change-Id: I91934f21fd9c15f7a834312d2ccf4a1dc075bbc6
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Ed Bartosh [Thu, 17 Oct 2013 14:21:30 +0000 (17:21 +0300)]
Moved userContent/jenkins-scripts symlink to -common
userContent/jenkins-scripts symlink should be in -common for two
reasons:
- it's used by also by -sync instances.
- jenkins-scripts directory exists in -common package
Change-Id: Ia82d4e44c511bd3765da0dc0dee5222624a4d3b0
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Lin Yang [Tue, 15 Oct 2013 12:58:35 +0000 (20:58 +0800)]
Auto create symlinks to jenkins-scripts
In %post section, automatically create symlinks /var/lib/jenkins/userContent/
jenkins-scripts to /var/lib/jenkins/jenkins-scripts.
Fixes: #1356
Change-Id: Ib8972db7bb1db616db377a0c3a4fca5f33e33ee5
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Ed Bartosh [Tue, 15 Oct 2013 10:28:18 +0000 (13:28 +0300)]
Deleted job_deletetag
The reason is that we don't need to delete any tags. We may need another
job to delete old tags, but we don't need to delete them just because
the change was rejected like job_deletetag does.
Moreover job_deletetag has bugs. It may end up in infinite loop if it
can't force push to git repo. Removing the job automatically fixes this
issue.
Fixes: #1379
Change-Id: I182f8abdaa6f656edc8bb686698fffebbfa41092
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 14 Oct 2013 11:57:57 +0000 (14:57 +0300)]
Improved error handling in job_mail_sender
Now jobs performs sanity check for configuration parameters and returns
1 if it fails.
Fixes: #1225
Change-Id: I4c7223589ec601f22b1675f75dacaa1388e05c33
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 14 Oct 2013 11:46:31 +0000 (14:46 +0300)]
Created Snapshot and Prerelease classes
Snapshot and Prerelease classes provide simple OO interface to Snapshot
and Prerelease metadata. Previously get_repo_data function was used, but
it was hard to read and snapshot and prerelease data was mixed there.
That complexity was the first reason to replace it with Snapshot and
Prerelease classes.
Another reason was snapshot increment. It would be quite hard to
implement it using functions. Object oriented approach makes it simpler
to implement and more readable.
Change-Id: I1f77e3de047cd6666df30a3f40040115a9554b1f
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 14 Oct 2013 11:44:22 +0000 (14:44 +0300)]
Fixed typo in method name
mailer.py: Using get method: req.req -> req.get
Change-Id: Ia53384e1af4bb2b346ffac051495b69b52b8f172
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Lin Yang [Mon, 14 Oct 2013 07:34:29 +0000 (15:34 +0800)]
Support multiply levels branch name in submit tag
Branch name maybe contain '/', like devel/efl. Originly, submission with tag
submit/devel/efl/
20131009.205635 is treated as
submit branch: devel date: efl/
20131009.205635 instead of
submit branch: devel/efl date:
20131009.205635.
Fixes: #1378
Change-Id: I57abe3cf1022535ff7e77bf7cefd88a198ce262f
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Hasan Wan [Fri, 11 Oct 2013 03:32:53 +0000 (06:32 +0300)]
Ignore the project delete event for non-prerelease project
Fixes: #1338
Change-Id: I10bb70260a4f04ce0cd38e69bd8759024e67d965
Signed-off-by: Hasan Wan <hasan.wan@intel.com>
Ed Bartosh [Sun, 6 Oct 2013 17:21:09 +0000 (20:21 +0300)]
Isolated job_submitobs to separate package
This change resolves conflicting runtime dependencies:
jenkins-scripts-tzs -> jenkins-scripts-common(with job_submitobs) -> gbs
-> depanneur -> modified build and obs-server -> build
The reason of the conflict is that jenkins-scripts-tzs package has to be
installed on the same host as obs-server because tzs appliance is one
computer with all services(Jenkins, OBS, Gerrit) installed on it.
So, job_submitobs should be removed from -common to avoid conflicts
between our modified build and non-modified build, required by
obs-server.
This complicated packaging issue can be resolved by moving job_submitobs
to separate package jenkins-scripts-submitobs.
This is funny in a sense that jenkins-scripts doesn't require gbs build
functionality, which depends on depanneur and build.
Note that this is temporary solution. There are at least 2 solutions to
this issue:
1. Upgrade our build package to the version, required by obs-server
(also temporary until the time when OBS depends on newer build)
2. - Move gbs remotebuild API to separate package
- Add dependency to gbs-remotebuid to jenkins-scripts
- Use gbs export and gbs remotebuild API in jenkinkins scripts
- Move job_sumbitobs back to jenkins-scripts-common
Even better would be to start using gbs service instead of gbs export in
job_submitobs. gbs remotebuild is still required, but gbs export is not
required in this case.
Change-Id: I0395e69000bc5695546c719a020cacf2bb77c3bc
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sun, 6 Oct 2013 16:47:17 +0000 (19:47 +0300)]
Added tzs appliance package
Added jenkins-scripts-tzs binary package. Moved scripts, used in
jenkins-scripts and jenkins-scripts-tzs to -common.
Change-Id: I526884d4ff39147c536e14deeb6ecaa4a918c2ce
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Fri, 4 Oct 2013 09:11:29 +0000 (12:11 +0300)]
Implemented job_create_snapshot
Job converts live OBS repo to snapshot repo, extracts .ks files from it
and triggers images creation.
This job is a next step towards replacing repomaker. It does the same as
boss-participant-repomaker.
Change-Id: Ibfc684123acc159821e8485451f9b7b546d2b232
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Wed, 2 Oct 2013 10:31:35 +0000 (13:31 +0300)]
Fixed mistake in constructing snapshot_path
Fixed snapshot path generation. Repomaker now makes repo in the right
location.
Change-Id: I59ff093d02fab32d5ee98346bb1ebb4808314b13
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Tue, 1 Oct 2013 13:38:50 +0000 (16:38 +0300)]
job_submit: Check if target project exists
In snapshot mode there is no need for all target projects to exist. If
some of the projects don't exist job_submit should still be able to
submit to those projects, which exist.
In applience mode it's even more important as applience usually has only
one target project in OBS.
Change-Id: I1aee31688434a441dfa11fb9e2a0c6b55c8c04e5
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 30 Sep 2013 10:40:25 +0000 (13:40 +0300)]
Added snapshot_dir to get_repo_data's return value
This field will be used by snapshot creation functionality. As
get_repo_data used in 2 places: snapshot creation and prerelease, it
should return all values used in both workflows.
Change-Id: I967ce538b5a7811bd3acf356c8249bd73bc5587a
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Fri, 27 Sep 2013 14:05:53 +0000 (17:05 +0300)]
Moved get_prerelease_data function
Moved get_prerelease_data from job_pre_release_obs to common/repoutils
and renamed it to get_repo_data. The reason of this move is that this
API will be used in snapshot creation functionality, i.e. will be called
from 2 job_ scripts.
Change-Id: Ia2976f6ccf799eea1fa626b5a48c32125dd536e6
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Tue, 24 Sep 2013 11:40:30 +0000 (14:40 +0300)]
Created job_submit out of job_pre_release_gerrit
job_sumbit can be run in two modes: prerelease and snapshot. In
prerelease mode it does the same as job_pre_release_gerrit was doing. In
snapshot mode it uploads package straight to target project.
Change-Id: I6714070654cd3f1196b112e3970e39e7aa77371e
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Fri, 4 Oct 2013 09:43:07 +0000 (12:43 +0300)]
Fixed wrong field name in requests/mailer code
Event, which comes from OBS doesn't have 'type' field. 'event_type'
should be used.
Looks like this code has never been tested and as a result caused
serious production issue: all email notifications have been lost.
Fixes: #1360
Change-Id: Ie73949d4d270219ee2261597b2418ec1a3a8b806
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Fri, 4 Oct 2013 09:38:01 +0000 (12:38 +0300)]
Fixed KeyError crash in mailer.py
Looks like event, which comes from obs plugin doesn't have
'origintype' field.
====Notify the tag owner====================================
Traceback (most recent call last):
File "/srv/jenkins/jenkins-scripts/job_request.py", line 315, in <module>
sys.exit(main())
File "/srv/jenkins/jenkins-scripts/job_request.py", line 282, in main
notify_submiter(event_fields, data)
File "/srv/jenkins/jenkins-scripts/job_request.py", line 261, in notify_submiter
os.getenv('EMAIL_TEMPLATES_DIR'))
File "/srv/jenkins/jenkins-scripts/requests/mailer.py", line 238, in mailer
if not tri['cond'] or tri['cond'](_bs, request):
File "/srv/jenkins/jenkins-scripts/requests/mailer.py", line 58, in <lambda>
'cond': lambda bs, req: req['origintype'] == 'change_devel',
KeyError: 'origintype'
Change-Id: I28f3866c3b37b7782576c9e613c8904050053526
Ed Bartosh [Wed, 2 Oct 2013 15:34:17 +0000 (18:34 +0300)]
Moved delete_package call after pushing accepted tag
request_accepted funcion deletes package from OBS before making and
pushing accepted tag. As OBS operations can cause unexpected errors
it's better to delete project after making a tag. Another point is
that making a tag is more important and anything, which can prevent
this operation should be done after it.
Change-Id: I1199738a88adca26f293909645cd736b71637cf3
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Wed, 2 Oct 2013 15:26:19 +0000 (18:26 +0300)]
Check if package exists before deleting it
job_request tries to delete package from source project when request
is accepted, rejected or revoked. However, in OBS 2.4 this is done by
OBS and package doesn't exist. It caused this traceback in production:
Traceback (most recent call last):
File "/srv/jenkins/jenkins-scripts/job_request.py", line 313, in <module>
sys.exit(main())
File "/srv/jenkins/jenkins-scripts/job_request.py", line 306, in main
request_accepted(data, gerrit, gitprj)
File "/srv/jenkins/jenkins-scripts/job_request.py", line 161, in request_accepted
delete_package(data['OBS_REQ_PRJ_SRC'], data['OBS_REQ_PKG_SRC'])
File "/srv/jenkins/jenkins-scripts/job_request.py", line 136, in delete_package
_bs.delete_package(prj, pkg)
File "/srv/jenkins/jenkins-scripts/common/buildservice.py", line 383, in delete_package
core.delete_package(self.apiurl, project, package)
File "/usr/lib/python2.7/site-packages/osc/core.py", line 4697, in delete_package
http_DELETE(u)
File "/usr/lib/python2.7/site-packages/osc/core.py", line 2924, in http_DELETE
def http_DELETE(*args, **kwargs): return http_request('DELETE', *args, **kwargs)
File "/usr/lib/python2.7/site-packages/osc/core.py", line 2909, in http_request
fd = urllib2.urlopen(req, data=data)
File "/usr/lib64/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib64/python2.7/urllib2.py", line 400, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 513, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 432, in error
result = self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 372, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 884, in http_error_401
url, req, headers)
File "/usr/lib64/python2.7/urllib2.py", line 859, in http_error_auth_reqed
response = self.retry_http_basic_auth(host, req, realm)
File "/usr/lib64/python2.7/urllib2.py", line 872, in retry_http_basic_auth
return self.parent.open(req, timeout=req.timeout)
File "/usr/lib64/python2.7/urllib2.py", line 400, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 513, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 438, in error
return self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 372, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 521, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: Not Found
Change-Id: I2a6b56ba40feecf83eeabb59169ae19b835231d7
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Wed, 2 Oct 2013 15:13:44 +0000 (18:13 +0300)]
Converted VM_CPUS and VM_MEMORY to int
When VM_CPUS and VM_MEMORY are taken from configuration file they have
to be converted to int to avoid this traceback:
Traceback (most recent call last):
File "/var/lib/jenkins/jobs/image-creator/workspace/jenkins-scripts/job_imager.py", line 142, in <module>
sys.exit(main())
File "/var/lib/jenkins/jobs/image-creator/workspace/jenkins-scripts/job_imager.py", line 94, in main
os.getenv("VM_CPUS", 8), basedir)
File "/var/lib/jenkins/jobs/image-creator/workspace/jenkins-scripts/job_imager.py", line 33, in run_inside_vm
(vm_memory, vm_cpus, vm_image, basedir)
TypeError: %d format: a number is required, not str
Change-Id: Ib18cbd320f013e8b3d9075dadf22840131e07bc1
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 30 Sep 2013 15:09:29 +0000 (18:09 +0300)]
Fixed wrong config parameter
Job is failing because it uses REDIS_HOSTNAME config parameter instead
of REDIS_HOST. As REDIS_HOSTNAME does not exist job crashes this way:
Traceback (most recent call last):
File "jenkins-scripts/job_load_repos.yaml.py", line 74, in <module>
sys.exit(main())
File "jenkins-scripts/job_load_repos.yaml.py", line 71, in main
return load_repos(os.path.join(prjdir, 'repos.yaml'))
File "jenkins-scripts/job_load_repos.yaml.py", line 26, in load_repos
int(os.getenv('REDIS_PORT')))
File "/srv/jenkins/jobs/load-repo-conf/workspace/jenkins-scripts/common/backenddb.py", line 141, in __init__
self._redis.exists('anything')
File "/usr/lib/python2.7/site-packages/redis/client.py", line 465, in exists
return self.execute_command('EXISTS', name)
File "/usr/lib/python2.7/site-packages/redis/client.py", line 334, in execute_command
connection.send_command(*args)
File "/usr/lib/python2.7/site-packages/redis/connection.py", line 287, in send_command
self.send_packed_command(self.pack_command(*args))
File "/usr/lib/python2.7/site-packages/redis/connection.py", line 269, in send_packed_command
self.connect()
File "/usr/lib/python2.7/site-packages/redis/connection.py", line 214, in connect
sock = self._connect()
File "/usr/lib/python2.7/site-packages/redis/connection.py", line 226, in _connect
sock.connect((self.host, self.port))
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
TypeError: coercing to Unicode: need string or buffer, NoneType found
Fixes: #1355
Change-Id: I37a60fc7b960301fe06b1cc1efc45189c471c532
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Lin Yang [Tue, 24 Sep 2013 08:08:48 +0000 (16:08 +0800)]
Fix the issue of parse commit a tag point to
When use 'git rev-parse' to parse the commit a tag point to, it should add
^{commit} after tag name, otherwise git will return the tag SHA1.
Change-Id: I0791f96350b01b458417fcbc27f14b58b79c5742
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Lin Yang [Tue, 24 Sep 2013 07:49:01 +0000 (15:49 +0800)]
Push accepted commit to refs/heads/accepted/*
To use "accepted" branches as default ref in generic manifests, push accepted
commit to specific hierarchy refs/heads/accepted/* when sr accepted to specific
project.
E.g. package accepted to Tizen:Mobile should push to branch
refs/heads/accepted/tizen/mobile.
Fixes: #1345
Change-Id: I204ffb137210f4f1b6753ee152f9f0ab3fa1c88b
Signed-off-by: Lin Yang <lin.a.yang@intel.com>
Ed Bartosh [Mon, 23 Sep 2013 12:40:53 +0000 (15:40 +0300)]
Repomaker: Check out wrong output archs
Check if architectures, provided to repomaker are valid ones, i.e. they
don't need to be mapped. For example, in current code i586 and i686
architectures are mapped to ia32, so both of them are not valid output
directories and repomaker throws exception if they're found in the list
of output architectures.
Change-Id: I4bee1f172f73550784e8153eaadad2b5a4626550
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 23 Sep 2013 12:00:03 +0000 (15:00 +0300)]
Fixed Tester builds of Ref Updated events
Added OBS and Jenkins repos to packaging/.extra-repos to fix
installation issues. This package doesn't depend on OBS, but it needs
this fix because Tester installs all packages from target repo on
RefUpdate events.
This is a first commit of this kind. The only way to test it is to
accept it and see if Tester succeeds. If it is then similar fixes should
be committed to the rest of packages.
Change-Id: Ifb3b18936406b893dce553ee5741ce51079b7272
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 23 Sep 2013 11:01:41 +0000 (14:01 +0300)]
Started work for 0.15
Updated version and .changes
Change-Id: I0eaef25f07c14161405cf37f99fe13210212cec6
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 23 Sep 2013 10:53:11 +0000 (13:53 +0300)]
Release 0.14
Changed versioning. Updated changelog.
Change-Id: Id26c2fe43cf246746a0970e3f401ffae8d52088b
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sun, 22 Sep 2013 12:23:46 +0000 (15:23 +0300)]
Skip packages for not specified architectures
Output architectures are passed to repomaker API from caller. Packages
with not specified architectures should not go to output repository.
Change-Id: I370e1c67542ebda45a5185b50569821d37f0ac61
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Sun, 22 Sep 2013 11:57:01 +0000 (14:57 +0300)]
Added i686 packages to the repomaker test
Thanks Qiang for this suggestion. Adding i686 packages improves the test
in a way that 2 mapped architectures(i586 and i686) are tested.
Change-Id: Ib32255520fdca733082f8779c3c5eb60ebe54bc5
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Thu, 19 Sep 2013 08:14:19 +0000 (11:14 +0300)]
Fixed processing of architectures in repomaker
Repomaker was getting list of architectures analyzinng packages in the
incoming repository. If repository doesn't contain some architectures
those were skipped. This was a mistake. Now repomaker gets list of
architectures as a parameter and creates output repos for all
architectures even if no packages found in incoming repo for some of
them.
Fixes: #1324
Change-Id: I2e4fa4df61ae1fcaa64d0792648391493c962001
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Lingchaox Xin [Tue, 17 Sep 2013 05:33:49 +0000 (13:33 +0800)]
Optimize updating git repo logic
Change-Id: I3ed80bba2536a008bd225ffabb0b90094209a40a
Ed Bartosh [Mon, 16 Sep 2013 08:47:46 +0000 (11:47 +0300)]
Renamed mic-seed to mic-appliance
Mic-seed is less informative and straightforward name from my point of
view. It explains better what this image is for.
Change-Id: I1fae68d0fb2271c7ecd619865cb19bfa963b2064
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Ed Bartosh [Mon, 16 Sep 2013 21:09:45 +0000 (00:09 +0300)]
Repomaker: consider noarch when collect packages
When repomaker collects packages from live repository it should also
consider noarch packages.
Currently it skips noarch/ directories. If submission also contains
architecture-dependent packages noarch packages just silently skipped.
However, when submission contains only noarch packages then nothing is
copied and this causes image creation failure. We were 'lucky' that
submissions we've tested didn't contain noarch packages. That's why we
didn't notice this bug.
This is a serious bug and requires to be fixed also in pre_release_obs
code, where only binary architectures are processed.
This changes fixes only repomaker. With this fix repomaker is able to
collect noarch packages.
This change partly fixes issue 1324.
Change-Id: Iba6f978ef2207aa9562f4e46c1bc596142f73ef4
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>