tools/weles.git
6 years agoRefactor job lister tests 35/189435/3
Alexander Mazuruk [Mon, 17 Sep 2018 16:45:29 +0000 (18:45 +0200)]
Refactor job lister tests

Previous tests of job lister had little to no value.

Change-Id: Iae66d511b9610b03102a1a4e84aca9d3e8a47a89
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRefactor artifact lister tests 44/186444/18
Alexander Mazuruk [Thu, 2 Aug 2018 09:39:13 +0000 (11:39 +0200)]
Refactor artifact lister tests

Previous tests of artifact lister handler had little to no value.

Change-Id: I95dc0e3b1a74e009b1de031ebb315f251d1d8601
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoUnify JobFilter returned by server 34/189434/2
Alexander Mazuruk [Mon, 17 Sep 2018 15:20:38 +0000 (17:20 +0200)]
Unify JobFilter returned by server

Further description in job_lister_handler.go file and Unify
ArtifactFilter returned by server commit.

Change-Id: Ieda839e30ee6be96187cb9551f9b4cb768298db9
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoUnify ArtifactFilter returned by server 14/188914/6
Alexander Mazuruk [Tue, 11 Sep 2018 16:19:53 +0000 (18:19 +0200)]
Unify ArtifactFilter returned by server

Due to how json array marshalling works in go:

Having following struct:

type A struct{
    Key1 []string
    Key2 []string
}

When following empty JSON was sent:
{ Key1: [], Key2: [] }

We would receive a struct:
Key1, Key2 - one element slice with empty string

If we would send:
{ Key2: [] }

We would receive a struct:
Key1 - empty slice
Key2 - one element slice with empty string

This commit unifies empty values returned by server to always be empty
slice.

Change-Id: I24ff156dc98f5a1e160858cc3ce02ecd44387129
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoSet defaults on JobSorter 09/188909/9
Alexander Mazuruk [Tue, 11 Sep 2018 14:22:44 +0000 (16:22 +0200)]
Set defaults on JobSorter

Additionally, this commit unifies how server treats query params
(pagination). They are always ignored if pagination is turned off
(either by user or admin)

Test have been hotfixed to pass and will be fixed in future commits on
this branch.

Change-Id: I12eb875972ac818b0630f10ae95289ff7aef7f88
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAllow user to sort with JobSortByID 07/188907/9
Alexander Mazuruk [Tue, 11 Sep 2018 14:21:18 +0000 (16:21 +0200)]
Allow user to sort with JobSortByID

Previously, it was only possible to sort by ID by sending JSON which did
not contain SortBy field.

Change-Id: Ia91bacc1ba8b4df9809faaefc0b54c3196cdcf19
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoMove default ArtifactSortOrder to server 43/186443/15
Alexander Mazuruk [Tue, 31 Jul 2018 17:16:45 +0000 (19:16 +0200)]
Move default ArtifactSortOrder to server

Previously database package was responsible for setting default
value.

Test were fixed to pass but have little to no value. Next commit will
change tests to be more readable and have less "magic".

Additionally this commit unifies how server treats query params
(pagination). They are always ignored if pagination is turned off
(either by user or admin)

Change-Id: I3d9eaaf517596394537eb8bf9333ebc74c43812c
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoUnify paths to Dryad tools 42/185942/11
Pawel Wieczorek [Fri, 3 Aug 2018 16:48:49 +0000 (18:48 +0200)]
Unify paths to Dryad tools

Although binaries for Dryad tools (STM and FOTA) are placed in
"/usr/bin", their wrappers which should be primarily used are available
in "/usr/local/bin". This patch unifies those paths until tools are
provided as Dryad API.

Change-Id: Ic34658a7466fd3887996c0248fcdd10fb51f43b2
Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
6 years agoPass Artifact DB path to Dryad jobs 43/185943/12
Pawel Wieczorek [Fri, 3 Aug 2018 17:41:51 +0000 (19:41 +0200)]
Pass Artifact DB path to Dryad jobs

After reverse SSHFS introduction for transferring files between Weles
and Dryad assigned for a given job, Artifact DB had to be available
under proper mountpoint on Dryad. This patch propagates path to Artifact
DB from command line flag through Dryad Job Manager to newly created
Dryad Jobs.

Change-Id: I36fdec4ec6f04b2e544a38218b0faeeec1dddaa2
Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
6 years agoFix artifact test which failed occasionally in CI 38/189138/4
Alexander Mazuruk [Thu, 13 Sep 2018 22:27:50 +0000 (00:27 +0200)]
Fix artifact test which failed occasionally in CI

Channels in artifacts should be synchronized, for the artifacts to set
state of the artifact only after it is written to the database.
Currently, there might be requested information about artifact which is
not yet in the db. This should be done in separate commit

Change-Id: Ic6da4ea9fe63ea8a42995ef837ad692557979afc
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoMake filtering one db transaction 18/189018/5
Alexander Mazuruk [Wed, 12 Sep 2018 16:40:16 +0000 (18:40 +0200)]
Make filtering one db transaction

Change-Id: I4a7d1a64addf08bd4f469b47705c9ff7c0a6b7bd
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdjust fixtures to produce more than one artifact per job id 29/186929/13
Alexander Mazuruk [Tue, 14 Aug 2018 12:39:13 +0000 (14:39 +0200)]
Adjust fixtures to produce more than one artifact per job id

Change-Id: I6c0354c974c054f6b57e9b5b9479278e8c1c979a
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoMove ArtifactNotFound error to database pkg 45/186445/17
Alexander Mazuruk [Fri, 3 Aug 2018 12:03:29 +0000 (14:03 +0200)]
Move ArtifactNotFound error to database pkg

Previously server was checking for this error.

Change-Id: I17467a4569c4ec6c65954c0a3a97b12351ec8848
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRemove JobFilterAndSort from Weles pkg 75/187375/10
Alexander Mazuruk [Wed, 22 Aug 2018 10:14:55 +0000 (12:14 +0200)]
Remove JobFilterAndSort from Weles pkg

This struct is now in server/operations/jobs pkg, renamed
(automatically by go-swagger) to JobListerBody. This struct
is only used by server thus should not clutter main weles pkg.

Change-Id: I542c37c44951fa897432bf3ba7badd2867e4ebf3
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRemove ArtifactFilterAndSort from Weles pkg 55/186455/13
Alexander Mazuruk [Tue, 31 Jul 2018 17:09:49 +0000 (19:09 +0200)]
Remove ArtifactFilterAndSort from Weles pkg

This struct is now in server/operation/artifacts pkg, renamed
(automatically by go-swagger) to ArtifactListerBody. This struct
is only used by server thus should not clutter main weles pkg.

Change-Id: Ic7cb7ab68cb0feb506f318a5ae4b769065b8c96f
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoUpdate go-swagger, regenerate server 05/186705/10
Alexander Mazuruk [Mon, 13 Aug 2018 15:30:00 +0000 (17:30 +0200)]
Update go-swagger, regenerate server

This resolves bug where out of ArtifactInfo struct only
ArtifactDescription was marshalled to JSON:
https://github.com/go-swagger/go-swagger/issues/1617
Fixed in 0.16.0 release.

1. Updated Gopkg.toml
2. Re-generated Gopkg.lock
$ make Gopkg.lock
3. Build new swagger
$ make tools
4. Re-generate swagger server
$ make swagger-server-generate

Change-Id: I622e4a2500203f4236d8938031abfee136c69b81
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRemove unused Select function 54/186454/11
Alexander Mazuruk [Thu, 9 Aug 2018 15:58:44 +0000 (17:58 +0200)]
Remove unused Select function

Discussed with artifacts pkg author, it was left from previous
implementation.

Change-Id: I99f1035bd36e1e91ace82c64b046db053abcf5c2
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoMerge branch 'ci' 12/188812/1
Pawel Wieczorek [Mon, 10 Sep 2018 12:48:45 +0000 (14:48 +0200)]
Merge branch 'ci'

Change-Id: I9805c84f042771a22476dd93b4ee7b8ea58999a4

6 years agoAdd config for gometalinter 52/185552/23
Alexander Mazuruk [Tue, 31 Jul 2018 12:47:52 +0000 (14:47 +0200)]
Add config for gometalinter

As there are differences between Boruta and Weles (e.g. usage of
vendoring) we need different sets of linter policies.

Tool:
https://github.com/alecthomas/gometalinter

Install:
$ go get -u gopkg.in/alecthomas/gometalinter.v2

Using gometalinter in Weles repository root will use this config by
default.

Config file documentation:
https://github.com/alecthomas/gometalinter#configuration-file

Format of the config file is determined by Config struct:
https://github.com/alecthomas/gometalinter/blob/master/config.go

There is an issue with gas installation currently:
https://github.com/alecthomas/gometalinter/issues/508

Currently there is a pull request to add support for ignoring generated
files:
https://github.com/walle/lll/pull/4

In current patchset generated files are excluded manually (by pkg or
filename)

Change-Id: Ib750066370392d44555aa5b5d4f342acc06c9b36
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoSplit Controller's job listing and filtering 03/186603/14
Lukasz Wojciechowski [Fri, 10 Aug 2018 20:46:01 +0000 (22:46 +0200)]
Split Controller's job listing and filtering

The splitting improves readability of the code
and lowers cyclomatic complexity of the functions.

Method JobsControllerImpl.List has been split into:
* filter - returning JobInfos matching filter;
* sort - sorting them;
* paginate - calculating returned page indexes and sizes.

The List function by itself stays responsible for setting up critical
section for accessing jobs collection, runs above functions and returns
proper records and info.

One mechanism requires clarification. If JobID used as index in pagination
does not match filter, it must be temporary added to Jobs collection
for calculation of the index position after sorting.
The "extra" variable is used for containing information, if this job
is added to the collection.

Method Job.passesFilter has been split into several methods.
Each of them verifies if one of the Job fields passes through the filter:
* Job.passesCreatedAfterFilter,
* Job.passesCreatedBeforeFilter,
* Job.passesUpdatedAfterFilter,
* Job.passesUpdatedBeforeFilter,
* Job.passesInfoFilter,
* Job.passesJobIDFilter,
* Job.passesNameFilter,
* Job.passesStatusFilter,
exactly as the name of the function suggests.

Change-Id: I03bab42db51c168afc06d5e954dfcbd137637f12
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoFix linter errors in manager pkg 89/186289/18
Alexander Mazuruk [Wed, 8 Aug 2018 10:50:58 +0000 (12:50 +0200)]
Fix linter errors in manager pkg

Fixed possible context leak in reverse_sshfs.go

Verification:
$ go vet ./manager/...

Added directive for gometalinter/{gas,gosec} to ignore intentional
launch of subprocess with a variable.

Verification:
$ gometalinter.v2 --disable-all --vendor --enable=gas ./manager/...

Fixed variable shadowing.

Verification:
$ go vet -shadow ./manager/...

Fix unhandled errors:
- ignore unhandled errors
- use log library to log errors.

Verification:
$ gometalinter.v2 --disable-all --vendor --enable=errcheck

Change-Id: I777ca6ecfa8da73255efbb4898388b170641276f
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoHandle server shutdown error 11/186111/16
Alexander Mazuruk [Tue, 7 Aug 2018 04:59:50 +0000 (06:59 +0200)]
Handle server shutdown error

Use standard log library to log error.

Change-Id: I6c6f8ef627d21b929787fabef9794e59d69bf674
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoFix linter errors in controller pkg 10/186110/16
Alexander Mazuruk [Tue, 7 Aug 2018 03:24:38 +0000 (05:24 +0200)]
Fix linter errors in controller pkg

Remove shadowed variables:
in: downloaderimpl.go, jobscontrollerimpl_test.go

Verification:
$ go vet -shadow=true ./controller/...

Fix unhandled errors:
- ignore errors explicitly (using _)
in: boruterimpl.go, controller.go, dryaderimpl.go, jobscontrollerimpl.go

Use standard log library to log some errors.

Verification:
$ gometalinter.v2 --disable-all --vendor --enable=gas --enable=errcheck  ./controller/...

Remove unused parameter:
in: boruterimpl.go

Verification:
$ gometalinter.v2 --disable-all --vendor --enable=unparam   ./controller/...

Remove unneccessary cast:
in: controller_test.go

Verification:
$ gometalinter.v2 --disable-all --vendor --enable=unconvert  ./controller/...

Add directive for gometalinter/gocyclo to ignore passessFilter() func. This
funciton is readable as-is and breaking it down into smaller functions
would increase allocations while decreasing readability.

Added TODO comment on List() function for the author to break it down.

Thanks to gagarin for help with error handling.

Change-Id: I486a29aa5a83e91775d4d3f370f9d6a19a81b8e9
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoHandle fixture errors 08/186108/16
Alexander Mazuruk [Mon, 6 Aug 2018 15:45:15 +0000 (17:45 +0200)]
Handle fixture errors

Those errors are handled by panic. They will not occur due to fixed input.

gometalinter.v2 --disable-all --vendor --enable=gas --enable=errcheck  ./fixtures/...

Change-Id: I88a6aa7276bb3a48878620094ca6ea9345c29ded
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoFix error handling in artifacts{,/*} pkg 41/185941/17
Alexander Mazuruk [Fri, 3 Aug 2018 16:27:45 +0000 (18:27 +0200)]
Fix error handling in artifacts{,/*} pkg

- Handled a few unhandled errors (reported by gas and errcheck).

Verification:
$ gometalinter.v2 --disable-all --vendor --enable=gas --enable=errcheck  ./artifacts/...

Should return no errors.

Some errors should just be logged and are logged via standard log
library.

Additionally call to getID function and the function itself were
removed. They were not neccessary after moving ID field to ArtifactInfo
struct.

Change-Id: If0d249aa08e296c7b46bc06532fe51d835e1b5c6
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRemove shadowing variables from artifacts pkg 40/185940/14
Alexander Mazuruk [Fri, 3 Aug 2018 14:01:40 +0000 (16:01 +0200)]
Remove shadowing variables from artifacts pkg

Verification:

$ gometalinter.v2 --disable-all --enable=vetshadow --vendor ./artifacts/...

Change-Id: I25cb9d583075ff61fc210e4c1bd09111f7ff73cc

6 years agoFix line length to max 100 36/185936/14
Alexander Mazuruk [Fri, 3 Aug 2018 13:24:29 +0000 (15:24 +0200)]
Fix line length to max 100

Formatting is not super consistent. I'd love some input on a correct way
of formatting or should I leave it as is.

Verification:
G=`grep -RHI "Code generated" | grep -ve "^vendor" | sed -e 's|:.*||' | sort -u`; \
gometalinter.v2 --disable-all --linter='lll:lll -l 100 -w 4:PATH:LINE:MESSAGE' \
--vendor --skip="mock" --enable=lll ./... | grep -vF "$G"

Should return only 2 occurences of line going over the limit. Those
occurences have "// nolint: lll" comment added on the end of the line to
supress gometalinter from complaining.

Change-Id: I1ea1838233329de63d947de8488b9df682614249
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoSimplify loop in server_suite_test 09/186109/7
Alexander Mazuruk [Mon, 6 Aug 2018 13:56:12 +0000 (15:56 +0200)]
Simplify loop in server_suite_test

Change-Id: I66391fd2d26cfe0d2a237c4861eaade20d0985f7
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoMove building sort query to separate func 35/185935/9
Alexander Mazuruk [Mon, 30 Jul 2018 16:08:36 +0000 (18:08 +0200)]
Move building sort query to separate func

This is to reduce cyclomatic complexity of the function and increase
its readability.

Change-Id: I1db6505c90d2ca622ee4d86892dde2efcb9b19ca
igned-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>

6 years agoMove database filter query preparation to separate func 34/185934/9
Alexander Mazuruk [Mon, 30 Jul 2018 16:05:46 +0000 (18:05 +0200)]
Move database filter query preparation to separate func

This is to reduce cyclomatic complexity of the function and increase
readability.

Change-Id: I9292cb37c605e8cd6f6ba7e35af2d5e82556cc5c
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRemove redundant go:generate 02/186702/5
Alexander Mazuruk [Mon, 13 Aug 2018 12:00:17 +0000 (14:00 +0200)]
Remove redundant go:generate

This mock is generated via mock/mocks_generate.go.
go:generate in dryadjobmanager.go was redundant.

Change-Id: I03ca83bd49c82589c42ea451c625e9e1eb2ab1b6
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoMove manager mocks to mock pkgs 47/186547/4
Alexander Mazuruk [Thu, 9 Aug 2018 19:45:00 +0000 (21:45 +0200)]
Move manager mocks to mock pkgs

This removes dependency on mockgen binary in bin/dev-tools to build
weles. (go build triggered go generates in manager pkg which required
mockgen)

Change-Id: I1eb89dce25e2ac0cefcac5dd2ef0bc9bc71760c8
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRemove . imports from managers pkg 33/185933/8
Alexander Mazuruk [Mon, 30 Jul 2018 12:43:04 +0000 (14:43 +0200)]
Remove . imports from managers pkg

golint complains about using . imports.

managers package was the last one containing . imports.

Verification:

$ golint ./... | grep -v -e "^vendor*"

Change-Id: I603201cc627691881692b5990b00ad9b606f9e4f
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoChange ALL_CAPS to CamelCase in DryadJobStatus 32/185932/6
Alexander Mazuruk [Mon, 30 Jul 2018 12:37:04 +0000 (14:37 +0200)]
Change ALL_CAPS to CamelCase in DryadJobStatus

golint complains about using ALL_CAPS.

DJ_<STATUS> were renamed to DryadJobStatus<STATUS>

All occurrences were renamed.

Verification:

$ golint ./... |grep -v -e "^vendor*"

Should not report any warnings regarding ALL_CAPS.

Change-Id: I74afd3b2e24d368aaa63d999ef256df8f2af8566
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd keys to composite literals declarations 31/185931/5
Alexander Mazuruk [Mon, 30 Jul 2018 12:23:27 +0000 (14:23 +0200)]
Add keys to composite literals declarations

go vet complains about unkeyed composite literals.
This commit adds keys to all composite literal definitions.
This allows adding fields to structs without rewriting code.

Verification:
go vet -composites ./...

Should return no errors.

Change-Id: I59338e47afdddfa1efae2998c9cbdf6049d3a459
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoSimplify bool comparisons 30/185930/4
Alexander Mazuruk [Mon, 30 Jul 2018 11:19:53 +0000 (13:19 +0200)]
Simplify bool comparisons

Weles has been analyzed with gosimple:
https://github.com/dominikh/go-tools/tree/master/cmd/gosimple

Simplifications here are similiar to formatting code with gofmt as
follows:
$ gofmt -s -w

Additionally getTotal and getRemaining in artifacts/database/database.go
have been renamed to totalRecords and remainingRecords respectively.

Renaming reason:
https://golang.org/doc/effective_go.html#Getters

Verification:
$ gometalinter.v2 --disable-all --vendor --enable=gosimple ./...

Should report no errors

Change-Id: Id1c639e5398d7f07fff4c6c6815ff36c5b499751
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRemove unused assignments 29/185929/3
Alexander Mazuruk [Mon, 30 Jul 2018 09:26:07 +0000 (11:26 +0200)]
Remove unused assignments

Weles has been analyzed with ineffassign:
https://github.com/gordonklaus/ineffassign

https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices

Change-Id: Ib05a41f0f35c8b2831b9c5f5ecc85c1774c82555
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRemove unnecessary type conversions 28/185928/3
Alexander Mazuruk [Fri, 27 Jul 2018 16:42:11 +0000 (18:42 +0200)]
Remove unnecessary type conversions

Weles has been analyzed with unconvert:
https://github.com/mdempsky/unconvert

This should make code more readable.

Verification:
$ gometalinter.v2 --disable-all --vendor --enable=unconvert ./...

There should be only errors in server/server.go which is generated code.

Change-Id: I8288cb65b9faecd1b16854bf6f4bd3427dd3a01f
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoMerge branch 'api' 58/185558/1 cors
Pawel Wieczorek [Mon, 30 Jul 2018 16:09:57 +0000 (18:09 +0200)]
Merge branch 'api'

Change-Id: I73745abe146f5e9d6e3a9ba886f727da53a2b32e

6 years agoAdd protocol and IP to default Boruta address.
Alexander Mazuruk [Fri, 27 Jul 2018 15:53:40 +0000 (17:53 +0200)]
Add protocol and IP to default Boruta address.

Weles will not connect to Boruta if protocol and address are not
specified.

Change-Id: I42ad88c5b2b1c4de42347def1d505bee5581f652
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAllow disabling pagination in artifacts
Alexander Mazuruk [Wed, 25 Jul 2018 15:57:04 +0000 (17:57 +0200)]
Allow disabling pagination in artifacts

When ArtifactPagination.Limit is 0, .ID and .Forward are ignored.

Additionally, when empty ArtifactSorter is passed, correct sorting is
applied (ascending).

Change-Id: Ibc2cc7ca688c854b33762ee8639ceb9b0529a937
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAllow disabling pagination by setting limit to 0 56/185056/8
Lukasz Wojciechowski [Wed, 25 Jul 2018 14:24:04 +0000 (16:24 +0200)]
Allow disabling pagination by setting limit to 0

If JobPagination.Limit is set to 0, then:
* JobID and Forward fields are ignored;
* pagination in listing jobs is disabled;
* all records matching filter are returned.

This patch adjusts JobsController implementation to above new rule.
It modifies also tests to cover new feature.

Change-Id: I929b89aa100d8f491886d0861592de2004758424
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdd list artifacts path to API 82/181382/29
Alexander Mazuruk [Tue, 5 Jun 2018 13:25:31 +0000 (15:25 +0200)]
Add list artifacts path to API

There is an issue with generated marshalling and unmarshalling:
https://github.com/go-swagger/go-swagger/issues/1617
Currently generated code with explicit marshalling in artifact_info.go
is commented out. Implicit marshalling works fine.

Change-Id: I211ea81636fe0b7d25d4fb1e7d76b55dacae7121
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd info about default artifact sorting 39/185439/2
Alexander Mazuruk [Mon, 30 Jul 2018 14:38:20 +0000 (16:38 +0200)]
Add info about default artifact sorting

Change-Id: Idced62086616b1252a6a19575e64bea0ed6c14d4
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd pagination to artifacts/database 81/181381/28
Alexander Mazuruk [Tue, 5 Jun 2018 13:12:56 +0000 (15:12 +0200)]
Add pagination to artifacts/database

There was a problem with pagination key for ArtifactInfo objects.
Only ArtifactInfo.Path was unique and artifacts database ID was not
accessible. This patch moves ID field to ArtifactInfo from previous struct,
artifactInfoRecord which extended ArtifactInfo and was local to the
database package.

Added identifiers to structs in test as after adding field to
ArtifactInfo there were build errors.

Added some more verbose error messages in database Filtering function.

Added pagination and sorting.

Changed artifacts/database/database_suite_test.go package from
database_test to database as all tests are white-box.

And of course the main purpose of this commit- pagination and sorting of
artifacts is now possible.

Tests were added in separate package, server_suite_test.go contains
beforeSuite initialization which is used only by pagination_test.go
before/after suite has no effect on previous tests of database.

Change-Id: Ic74dfa403ea486faca94aa39e5cb063abdc31ac4
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd fixture generator module for testing 58/183758/17
Alexander Mazuruk [Mon, 9 Jul 2018 15:25:12 +0000 (17:25 +0200)]
Add fixture generator module for testing

It will be used in tests of artifacts/database (pagination)
as well as server (artifacts pagination).

Change-Id: I3eee5e6891461c46442bcb369e1d9ec0850f3a17
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd new types to enable artifact pagination 80/181380/26
Alexander Mazuruk [Tue, 5 Jun 2018 13:09:07 +0000 (15:09 +0200)]
Add new types to enable artifact pagination

Change-Id: I1d6cd83883288676703bb18fd3026e6603aacef4
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd list jobs path to API 25/171225/48
Alexander Mazuruk [Thu, 24 May 2018 04:43:50 +0000 (06:43 +0200)]
Add list jobs path to API

The following files should be reviewed (as they were the only files not
generated by the swagger automatically but had to be filled manually):
- errors.go
- swagger.yml
- cmd/weles-server/main.go
- server/configure_weles.go
- server/job_lister_handler.go
- server/job_lister_handler_test.go
- server/handlers_helpers.go
- server/server_suite_test.go
- server/page_limit.go

Change-Id: Iab9589ca6738831f8269ba3eeb20581bba35645b
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRemove dot imports usage from notifier package 13/181413/25
Lukasz Wojciechowski [Wed, 13 Jun 2018 12:12:30 +0000 (14:12 +0200)]
Remove dot imports usage from notifier package

Change-Id: I58ae24b3c86d06f2f487abe18543c50e27fef3d9
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoImplement pagination in listing jobs 12/181412/25
Lukasz Wojciechowski [Mon, 11 Jun 2018 16:18:07 +0000 (18:18 +0200)]
Implement pagination in listing jobs

Change-Id: I5a34576315118d0a946ee1f86cf2443e41ac744f
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoImplement sorting in ListJobs 11/181411/25
Lukasz Wojciechowski [Mon, 11 Jun 2018 13:27:59 +0000 (15:27 +0200)]
Implement sorting in ListJobs

Patch includes Aleksander's version of sorting function selector.

Change-Id: I624bba4a07e39b0129c08d5a2513ad1e874c32cc
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoImplement filtering in ListJobs 10/181410/25
Lukasz Wojciechowski [Mon, 11 Jun 2018 10:17:57 +0000 (12:17 +0200)]
Implement filtering in ListJobs

Change-Id: Iae458c029cfd416d39d4999bb745cf78e3720604
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdjust Controller to allow addition of pagination 67/180367/27
Lukasz Wojciechowski [Fri, 6 Jul 2018 15:08:48 +0000 (17:08 +0200)]
Adjust Controller to allow addition of pagination

The ListJobs signature has been changed to provide: better filtering,
sorting and pagination of listed jobs.

This patch provides a simple implementation ignoring filtering, sorting
and pagination. It returns all Jobs.
Following patches will add proper implementation.

The patch changes JobsController interface and requires rebuild of mock:
$ make mocks

Change-Id: Id0a806414d9090909274b438b18261193f88a9f7
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdd new types to weles to enable pagination 24/171224/35
Alexander Mazuruk [Tue, 27 Feb 2018 13:39:00 +0000 (14:39 +0100)]
Add new types to weles to enable pagination

Types generated by swagger:
- JobFilter containing filters
- JobSorter containing information about sorting the data
- JobSortBy containing enum of possible sorting keys for jobs
- SortOrder containing enum of possible sorting orders

Types added manually:
- JobPagination containing info how much data should be listed and where
should it start. Limit is int32 as Weles is intended to work on both 64
and 32 bit architectures. Due to varying integer size (which is slice
limit) on those archs, max slice len is constrained to be 32bit.
- ListInfo which holds number of total records fulfilling filter and
remaining records that have not yet been sent to the client.

Change-Id: I59e441da86856fd2bc57817ce8903ef96f669e97
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd cancel job path to API 13/170713/41
Alexander Mazuruk [Tue, 27 Feb 2018 14:17:28 +0000 (15:17 +0100)]
Add cancel job path to API

The following files should be reviewed (as they were the only files not
generated by the swagger automatically):
- swagger.yml
- server/configure_weles.go
- server/job_canceler_handler.go
- server/job_canceler_handler_test.go

Change-Id: I323b78f82c83a28f671f59105f1f5fc06a44be62
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd flags to set ArtifactManager init values 66/183566/13
Alexander Mazuruk [Fri, 6 Jul 2018 12:22:07 +0000 (14:22 +0200)]
Add flags to set ArtifactManager init values

This commit makes it possible to set ArtifactManager when executing
weles binary, rather than recompile it.

Change-Id: I14eb4c1c11c2bfc6af20bc25208874ce0e7901fd
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd create job path to API 12/170712/38
Alexander Mazuruk [Thu, 24 May 2018 03:06:13 +0000 (05:06 +0200)]
Add create job path to API

The following files were edited manually:
- swagger.yml
- job_id_string.go
- server/configure_weles.go
- server/job_creator_handler.go
- server/job_creator_handler_test.go
- server/managers.go
- server/server_suite_test.go
- server/test_sample.yml
- cmd/weles-server/main.go

Please note that documentation served with weles server is now preceded
by the basePath specified in the swagger.yml file. To check documentation
served alongside API you need to go to:
http://{host:port}/api/v1/docs

Change-Id: I6ae33208eea35834127e5c6ff0dda2d1e11f79cf
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoChange basic weles types to generated by swagger 08/170708/27
Alexander Mazuruk [Wed, 21 Feb 2018 12:24:31 +0000 (13:24 +0100)]
Change basic weles types to generated by swagger

Definitions from artifactmanager.go and jobmanager.go are removed.

Definitions which are used by swagger were added to swagger.yml.

Swagger server was regenerated.

ArtifactStatusChange was moved to separate file for consistency.

In ArtifactType const renamed:
YAMLFILE to YAML
TESTFILE to TEST
FILE suffix was redundant as each ArtifactType is a file.

In ArtifactType and ArtifactStatus consts AM_ prefix was changed to
ArtifactType and ArtifactStatus respectively. JobStatus consts JOB_
prefix was changed to JobStatus. This naming is more go-compliant.

Swagger generated models use strfmt.DateTime rather than time.Time.
strfmt.DateTime is a wrapper for time.Time extending it with swagger
validation.

controller and artifacts packages were adjusted to be compatible with
the above.

Currently artifact_info.go has difference between committed and
generated code. This is caused by the following issue:
https://github.com/go-swagger/go-swagger/issues/1617

Change-Id: If7687ac25113c93c6b130ae6555fd338c700fef8
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoRemove . imports from artifacts pkg 67/183567/9
Alexander Mazuruk [Wed, 4 Jul 2018 14:47:32 +0000 (16:47 +0200)]
Remove . imports from artifacts pkg

Go's vet tool complains about using dot imports.

Change-Id: Ifd9bedabc49b06196c67bf68a3d27923cfe557d9
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd initial swagger server 05/170705/20
Alexander Mazuruk [Wed, 21 Feb 2018 10:08:06 +0000 (11:08 +0100)]
Add initial swagger server

Swagger was chosen due to:
- Easy generation of server and client boilerplate code in many languages.
(in Weles project only golang is used but the swagger.yml file can be used
to generate Weles server/client boilerplate in any language)
- Clear, automatically generated documentation
- Promising standard that this tool is based on:
https://github.com/OAI/OpenAPI-Specification

This commit should be treated as the Weles server "scaffolding"

The initial swagger.yml file is added so there is something to generate
the server from. "tbd" will be removed in future commits.

Server was generated using following command:
./dev-tools/swagger generate server \
-A weles \
-f ./swagger.yml \
-m ../weles \
-s ./server \
-r ./COPYING \
--flag-strategy pflag \
--compatibility-mode=modern

Due to agreed name of the server package ("server") there was a conflict in
names in generated code causng build error. Generated code used
server as a name for an object instance which has been renamed to srv.

The Makefile recipe has additional --exclude-main flag which prevents the
swagger generator from overwriting cmd/weles-server/main.go

This flag must be set because:
- obviously as stated above regenerating main will cause build error
- all Weles objects will have to be created there
- additional Weles flags will have to be added there

Thus overwriting this file is a no-no and server regeneration should
only be performed using provided Makefile recipe.

The Makefile has also a recipe to generate html documentation which is
intended to be integrated with Sphinx documentation. This recipe
requires docker:
https://docs.docker.com/get-started/

---
Verification

Update your vendor directory
$ make vendor

To build server:
$ make server

You can check default swagger generated server flags:
$ bin/weles-server --help

When the server is running you can also see how the automatically
served swagger documentation looks like:
http://{host:port}/docs

host and port are flags that should be supplied when running Weles.
If not provided, localhost and random port will be used.

To regenerate swagger server code:
$ make swagger-server-generate

To generate html documentation:
$ make swagger-docs-html

Change-Id: I00cda82219944cc6ab1937f6d14f05d52c7fd4e5
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd swagger to development tools 06/170706/24
Alexander Mazuruk [Mon, 14 May 2018 16:56:26 +0000 (18:56 +0200)]
Add swagger to development tools

There was a problem with different go-swagger versions which made re-generating server
using different go-swagger versions yield different results than commited.

Version of go-swagger was constrained in Gopkg.toml file to resolve
above issue.

go-swagger and all of its dependencies were added to required rule in
Gopkg.toml to instruct dep to download them.

---
Verification:
$ make tools

In the bin/dev-tools/ directory there should be additional swagger
executable.

To check if the swagger tool has built correctly:

- serve specification UI:
$ bin/dev-tools/swagger serve https://raw.githubusercontent.com/swagger-api/swagger-spec/master/examples/v2.0/json/petstore-expanded.json

- validate specification:
$ bin/dev-tools/swagger validate https://raw.githubusercontent.com/swagger-api/swagger-spec/master/examples/v2.0/json/petstore-expanded.json

Change-Id: I4b96bf7181d9fb14d13fe1e2721ec564f66832bb
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoLock mockgen version and automate mock generation 67/180467/12
Alexander Mazuruk [Tue, 29 May 2018 10:45:25 +0000 (12:45 +0200)]
Lock mockgen version and automate mock generation

Version of mockgen tool has been constrained in dep's Gopkg.toml file.
It will ensure reproducible results of mocks re-generation.

Makefile 'tools' recipe has been added which will install mockgen, and
in the future- other tools supporting development.

Additionally 'mocks' recipe has been added which will generate mocks for
all Weles interfaces.

Go's generate tool has been tasked with mock generation.

"//go:generate" directives have been placed in separate mocks_generate.go
files in repository mock/ controller/mock/ and manager/ directories.

Mocks from manager have been re-generated using mockgen's reflect mode
as source mode does not work using binary not from GOPATH.
This resulted in changes in the following file:
manager/mock_dryad_job_runner_test.go

They work the same but due to different creation method look a bit
different.

ArtifactManager mock was also updated due to interface change:
mocks/artifactmanager.go

---
Verification:

Create mockgen binary:
$ make tools

Regenerate mocks:
$ make mocks

Check if the files have changed:
$ git status

(They should not change)

Additionally:
$ go test ./...
Should result in all tests passing.

Change-Id: Ie7d8ab89ac9b00960fe5473271287d7832d31e9f
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoAdd dep for dependency management 61/180061/7
Alexander Mazuruk [Wed, 23 May 2018 20:14:14 +0000 (22:14 +0200)]
Add dep for dependency management

Dep is introduced into the Weles project to lock dependencies and ensure
common and reproducible results among users of this package and its
developers. It will be used to lock both imported packages (when needed)
and to lock version of tools supporting development of Weles.

Gopkg.toml contains hand-written rules that govern dep's behaviour.

Gopkg.lock contains information that will enable anyone to download
dependencies in the same version as were used during development.

It has been decided that vendor directory shall not be committed thus was
added to .gitignore. This was decided due to projects contributors
preference not to keep all dependencies in the repository. It might
cause problems in case of dependency migration, name changes etc but
those are rare and should be maintained manually.

While make tool was designed to automate building projects written in
C-family of languages, we have decided to add Makefile to ease usage of
supporting tools such as dep, gomock, ginkgo, go-swagger.

I'd like to thank for all the input regarding Makefile from:
Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Aleksander Mistewicz <a.mistewicz@samsung.com>

---
Verification:

To build Weles you will need to have dep installed:
$ go get -u github.com/golang/dep/cmd/dep

To download dependencies needed to build weles:
$ make vendor

To verify you can try building parser as archive and check if the
gopkg.in/yaml.v2 is used from vendor:
$ go build -buildmode=archive -v -a ./parser

One of the output lines should be:
git.tizen.org/tools/weles/vendor/gopkg.in/yaml.v2

---
More about dep:
https://golang.github.io/dep/docs/introduction.html

If you do not have make installed:
https://www.gnu.org/software/make/

Change-Id: Ie787513b26c57117de06b1f6e0da25769d7a4bfe
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
6 years agoMerge branch 'artifacts' 35/183235/1 srun
Pawel Wieczorek [Tue, 3 Jul 2018 13:47:50 +0000 (15:47 +0200)]
Merge branch 'artifacts'

Change-Id: I7d422c12880842ce45ef7cb83c20eaf3b6b13192

6 years agoAdd downloader to Storage, implement PushArtifact 59/163859/7 artifacts
Katarzyna Gorska [Wed, 13 Dec 2017 14:10:46 +0000 (15:10 +0100)]
Add downloader to Storage, implement PushArtifact

Change-Id: If757c4745089eb308fca53c732790d3615c5dcb9
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
6 years agoMerge branch 'runner' 13/183013/1 ui
Pawel Wieczorek [Fri, 29 Jun 2018 14:16:56 +0000 (16:16 +0200)]
Merge branch 'runner'

Change-Id: Ic7344743f615ac74fbbf8c0e8c6c8fe46dd069fc

6 years agoAdd Boot to DeviceCommunicationProvider interface 25/182925/2
Aleksander Mistewicz [Fri, 22 Jun 2018 13:31:06 +0000 (15:31 +0200)]
Add Boot to DeviceCommunicationProvider interface

Boot starts DUT and prepares communication so that Login could work.
It can be called more than once to reset DUT's state.

Change-Id: I2c28028b557fa3ea862f5f176090b81767a240d2
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoImplement Test with test 01/177401/2
Aleksander Mistewicz [Wed, 29 Nov 2017 17:53:58 +0000 (18:53 +0100)]
Implement Test with test

Change-Id: Ic44cd543be463ef3434a23de679547c5e52e6fe9
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoImplement Boot with test 00/177400/2
Aleksander Mistewicz [Wed, 29 Nov 2017 17:29:02 +0000 (18:29 +0100)]
Implement Boot with test

Change-Id: I059792863037d06331fd624ba71543aedd7101ad
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoImplement Deploy with test 99/177399/2
Aleksander Mistewicz [Wed, 29 Nov 2017 17:22:56 +0000 (18:22 +0100)]
Implement Deploy with test

Change-Id: Ibc55e994878f639af1f0190c1debd332e35fd1c9
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd cmd provider to FOTA CLI 98/177398/2
Aleksander Mistewicz [Wed, 22 Nov 2017 10:31:14 +0000 (11:31 +0100)]
Add cmd provider to FOTA CLI

FOTA is a tool available on MuxPi providing ability to flash sdcard with
archives containing images published via http server.

This patch adds an easy way to construct command line call to the said
tool.

Change-Id: I8da6e5c49a4419e3f3343129456472363e55eb6d
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd example job definition to manager tests 97/177397/2
Aleksander Mistewicz [Wed, 29 Nov 2017 15:34:02 +0000 (16:34 +0100)]
Add example job definition to manager tests

Change-Id: Ib8f5dd6f989a0fb5a2edc4cddaab6c186b5d80d6
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd Config argument to Create of DryadJobManager 24/182924/1
Aleksander Mistewicz [Wed, 29 Nov 2017 15:32:38 +0000 (16:32 +0100)]
Add Config argument to Create of DryadJobManager

Change-Id: I85354ad5e1580d69a1d6ecb31e6e3cbfc567c00c
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoMerge branch 'dryad' 23/182923/1
Pawel Wieczorek [Thu, 28 Jun 2018 14:37:42 +0000 (16:37 +0200)]
Merge branch 'dryad'

Change-Id: I53430cc296457b031e5315b4c7f05af1fdd9dc33

6 years agoRename dryad_session_provider to session_provider 50/182250/2
Aleksander Mistewicz [Thu, 21 Jun 2018 13:49:29 +0000 (15:49 +0200)]
Rename dryad_session_provider to session_provider

Change-Id: I4e6dbbf5b06606e32cf3aef9e9dd9cb8d4d74134

6 years agoUse reverse SSHFS instead of scp 49/182249/2
Aleksander Mistewicz [Thu, 21 Jun 2018 13:33:24 +0000 (15:33 +0200)]
Use reverse SSHFS instead of scp

It solves the following problems:
 * some files did not fit into tmpfs or root filesystem,
 * some files could not be copied at all due to their contents,
 * paths needed to be translated.

Change-Id: I581ad3ddf22bf2e3e8aec0d2952f34f76ab90a41
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoMake Exec methods variadic 48/182248/2
Aleksander Mistewicz [Thu, 21 Jun 2018 13:30:51 +0000 (15:30 +0200)]
Make Exec methods variadic

Use mock instead of real implementation of SessionProvider in tests of
DeviceCommunicationProvider.

Change-Id: I6d62145b7982c28f93a2f44d259f0759dfd3dc6a
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd reverseSSHFS 37/176237/9
Aleksander Mistewicz [Fri, 13 Apr 2018 17:09:02 +0000 (19:09 +0200)]
Add reverseSSHFS

It mounts a local directory on a remote consuming one session of ssh
connection. It makes accessing files (even big) easy in the limited
environment provided by Dryad.

Change-Id: I2e881f8e1291bac28331dea584b81dcef92e331a
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd DeviceCommunicationProvider implementation 47/162647/16
Adam Malinowski [Mon, 27 Nov 2017 05:01:15 +0000 (06:01 +0100)]
Add DeviceCommunicationProvider implementation

Change-Id: I0b98831d3ee53de05831caf7564363fdf289fcf8
Signed-off-by: Adam Malinowski <a.malinowsk2@partner.samsung.com>
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd file transfer functions to SessionProvider 36/162036/18
Adam Malinowski [Tue, 28 Nov 2017 14:31:28 +0000 (15:31 +0100)]
Add file transfer functions to SessionProvider

Change-Id: Ief14d76d93bfd041bc9809d0599719859fde69dc
Signed-off-by: Adam Malinowski <a.malinowsk2@partner.samsung.com>
6 years agoAdd SessionManager implementation 35/162035/15
Adam Malinowski [Thu, 23 Nov 2017 09:18:23 +0000 (10:18 +0100)]
Add SessionManager implementation

MuxPi is needed to perform tests. SSH key needs to be generated and
its private part has to be saved to a local file. To run tests:

ginkgo manager/dryad -- \
    -keyFile=file \
    -address=address \
    -port=port \
    -userName=login

Where:
  file    - path to a file containing private part of SSH key generated
            on Dryad
  address - IP address of Dryad
  port    - SSH port to use for connection with Dryad (default: 22)
  login   - user name for connection with Dryad

Change-Id: I1b4cf55f8082b5b0d9ec1d0311f37f87f35fb753
Signed-off-by: Adam Malinowski <a.malinowsk2@partner.samsung.com>
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoMerge branch 'controller' 50/177650/1
Pawel Wieczorek [Wed, 2 May 2018 10:59:48 +0000 (12:59 +0200)]
Merge branch 'controller'

Change-Id: I7b318961f834834e5ea1ae22c615e74f8159737e

6 years agoAdd NewJobManager function 68/162368/12
Lukasz Wojciechowski [Thu, 30 Nov 2017 12:29:14 +0000 (13:29 +0100)]
Add NewJobManager function

NewJobManager function creates new instance of Controller
with all internal submodules. The created structure is returned
as JobManager interface.

This is the only valid method for creation whole Controller module
in Weles and should be used to create Weles service.

Change-Id: I8ff2051a23f714a99fc2642c94e34d247ec164d7
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoImplement ListJobs in Controller with tests 67/162367/11
Lukasz Wojciechowski [Mon, 16 Apr 2018 14:10:42 +0000 (16:10 +0200)]
Implement ListJobs in Controller with tests

Implementation enhances JobsController interface and implementation.
The mockup of JobsController is regenerated using:

mockgen -package mock -destination=controller/mock/jobscontroller.go \
        git.tizen.org/tools/weles/controller JobsController

Tests for both JobsControllerImpl and Controller are updated.

Change-Id: Ie30234ac754d19616ce2811e3f30a6834d28e296
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdd Controller with tests 22/162022/12
Lukasz Wojciechowski [Mon, 16 Apr 2018 13:34:14 +0000 (15:34 +0200)]
Add Controller with tests

Change-Id: I4681c0e085983094f9ce2699617bb9b52d2e086a
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoMerge branch 'manager' 80/177380/1
Pawel Wieczorek [Fri, 27 Apr 2018 12:14:49 +0000 (14:14 +0200)]
Merge branch 'manager'

Change-Id: I950cbd5ee07bb735ca5a8cfd46b1239039a70b31
Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
6 years agoImplement List with table driven tests 74/161474/4
Aleksander Mistewicz [Fri, 17 Nov 2017 15:50:40 +0000 (16:50 +0100)]
Implement List with table driven tests

Change-Id: Ifccf613b89cb6a21d4229b15affb533c0bb95de4
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd tests for panic recovery of dryadJob 73/161473/4
Aleksander Mistewicz [Fri, 17 Nov 2017 14:01:51 +0000 (15:01 +0100)]
Add tests for panic recovery of dryadJob

Change-Id: I7ac9ccb609a77ff5d0e4635c5d91e23cc88fd4ae
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd more complicated tests of dryadJob 72/161472/5
Aleksander Mistewicz [Fri, 17 Nov 2017 12:30:51 +0000 (13:30 +0100)]
Add more complicated tests of dryadJob

Change-Id: Ie015609731c8f25ecfe6bff31414f18390494e3b
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoUse mockups of generated interfaces in tests 71/161471/5
Aleksander Mistewicz [Thu, 16 Nov 2017 16:57:47 +0000 (17:57 +0100)]
Use mockups of generated interfaces in tests

Change-Id: I77760fa2028e3c096929e82683c8024e01ee3c06
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd generated mock of DryadJobRunner 70/161470/5
Aleksander Mistewicz [Wed, 29 Nov 2017 15:59:22 +0000 (16:59 +0100)]
Add generated mock of DryadJobRunner

Command used:
    mockgen -source=manager/manager.go \
            -destination=manager/mock_dryad_job_runner_test.go \
            -package manager

Change-Id: I015e79a3db12db02e2423bc6ce05cf4e9dfb6555
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd generated mock of interfaces in manager/dryad 69/161469/5
Aleksander Mistewicz [Thu, 16 Nov 2017 16:26:20 +0000 (17:26 +0100)]
Add generated mock of interfaces in manager/dryad

Command used:
    mockgen -source=manager/dryad/dryad.go \
            -destination=manager/mock_dryad_test.go \
            -package manager

Change-Id: I15860362b8fc144ddd9efe05e03efef2f5ebf03f
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
6 years agoAdd generated mocks for controller's internals 21/162021/11
Lukasz Wojciechowski [Mon, 27 Nov 2017 15:54:57 +0000 (16:54 +0100)]
Add generated mocks for controller's internals

mockgen -package mock -destination=controller/mock/boruter.go \
git.tizen.org/tools/weles/controller Boruter
mockgen -package mock -destination=controller/mock/downloader.go \
git.tizen.org/tools/weles/controller Downloader
mockgen -package mock -destination=controller/mock/dryader.go \
git.tizen.org/tools/weles/controller Dryader
mockgen -package mock -destination=controller/mock/parser.go \
git.tizen.org/tools/weles/controller Parser

Change-Id: Ie5fd939e0082c0f7975d2dbc416d730f28c311eb
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdd Dryader implementation with tests 20/162020/11
Lukasz Wojciechowski [Wed, 22 Nov 2017 08:53:17 +0000 (09:53 +0100)]
Add Dryader implementation with tests

DryaderImpl implements delegation and control of Job execution
in DryadJobManager.

Change-Id: I7d30f21bf0f838829f6c2d0b65362a6d7b66d0a3
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdd Dryader interface 19/162019/11
Lukasz Wojciechowski [Mon, 27 Nov 2017 08:15:12 +0000 (09:15 +0100)]
Add Dryader interface

Dryder interface allows delegation of Job execution and cancelation
to DryadJobManager.

Change-Id: I866f8290dc51d24ab65e2671edf306fc33f2e3c9
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdd Boruter implementation with tests 18/162018/11
Lukasz Wojciechowski [Tue, 21 Nov 2017 20:34:31 +0000 (21:34 +0100)]
Add Boruter implementation with tests

BoruterImpl implements communication with Boruta. It is used
for acquiring, monitoring and releasing Dryads for Weles' Jobs.

Change-Id: I91667cad4ccac176e4e01c697e2dbfbe7b54cfd3
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdd Boruter interface 17/162017/11
Lukasz Wojciechowski [Thu, 23 Nov 2017 17:02:23 +0000 (18:02 +0100)]
Add Boruter interface

Boruter defines interface for delegation of communication with Boruta.
Communication is used for acquiring, monitoring and releasing Dryads.

Change-Id: Id6c310220839c1d196f86b8c86123cd1a720f526
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdd generated mockup for boruta.Requests 16/162016/11
Lukasz Wojciechowski [Thu, 23 Nov 2017 17:01:10 +0000 (18:01 +0100)]
Add generated mockup for boruta.Requests

mockgen -package mock -destination=controller/mock/requests.go \
git.tizen.org/tools/boruta Requests

Change-Id: I3cd7673db7ff0eb5e7fb3ead8ea3d31a83430bee
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdd Downloader implementation with tests 15/162015/10
Lukasz Wojciechowski [Mon, 20 Nov 2017 05:04:31 +0000 (06:04 +0100)]
Add Downloader implementation with tests

DownloaderImpl is responsible for parsing Job's config and searching
for information about files (artifacts) related to the test.
It delegates to ArtifactsManager downloading images and testfiles
required to run Job (Push). It creates also paths for output
artifacts (Pull). It updates Job's config with proper paths
for these artifacts.

It does also monitor process of downloading files in ArtifactManager
and finally it notifies Controller when all files are downloaded
or in case of failure.

Change-Id: I578108dd1a924aaf36435e4dbc5d8dcce50214ee
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
6 years agoAdd Downloader interface 14/162014/8
Lukasz Wojciechowski [Thu, 23 Nov 2017 10:06:07 +0000 (11:06 +0100)]
Add Downloader interface

Downloader defines action run for a Job to download artifacts required
to run the Job.

Change-Id: I932b1630a97dc9573aa0cbf0c4726a91fb042339
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>