Alexander Mazuruk [Fri, 3 Aug 2018 13:24:29 +0000 (15:24 +0200)]
Fix line length to max 100
Formatting is not super consistent. I'd love some input on a correct way
of formatting or should I leave it as is.
Verification:
G=`grep -RHI "Code generated" | grep -ve "^vendor" | sed -e 's|:.*||' | sort -u`; \
gometalinter.v2 --disable-all --linter='lll:lll -l 100 -w 4:PATH:LINE:MESSAGE' \
--vendor --skip="mock" --enable=lll ./... | grep -vF "$G"
Should return only 2 occurences of line going over the limit. Those
occurences have "// nolint: lll" comment added on the end of the line to
supress gometalinter from complaining.
Change-Id: I1ea1838233329de63d947de8488b9df682614249
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 6 Aug 2018 13:56:12 +0000 (15:56 +0200)]
Simplify loop in server_suite_test
Change-Id: I66391fd2d26cfe0d2a237c4861eaade20d0985f7
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 30 Jul 2018 16:08:36 +0000 (18:08 +0200)]
Move building sort query to separate func
This is to reduce cyclomatic complexity of the function and increase
its readability.
Change-Id: I1db6505c90d2ca622ee4d86892dde2efcb9b19ca
igned-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 30 Jul 2018 16:05:46 +0000 (18:05 +0200)]
Move database filter query preparation to separate func
This is to reduce cyclomatic complexity of the function and increase
readability.
Change-Id: I9292cb37c605e8cd6f6ba7e35af2d5e82556cc5c
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 13 Aug 2018 12:00:17 +0000 (14:00 +0200)]
Remove redundant go:generate
This mock is generated via mock/mocks_generate.go.
go:generate in dryadjobmanager.go was redundant.
Change-Id: I03ca83bd49c82589c42ea451c625e9e1eb2ab1b6
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Thu, 9 Aug 2018 19:45:00 +0000 (21:45 +0200)]
Move manager mocks to mock pkgs
This removes dependency on mockgen binary in bin/dev-tools to build
weles. (go build triggered go generates in manager pkg which required
mockgen)
Change-Id: I1eb89dce25e2ac0cefcac5dd2ef0bc9bc71760c8
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 30 Jul 2018 12:43:04 +0000 (14:43 +0200)]
Remove . imports from managers pkg
golint complains about using . imports.
managers package was the last one containing . imports.
Verification:
$ golint ./... | grep -v -e "^vendor*"
Change-Id: I603201cc627691881692b5990b00ad9b606f9e4f
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 30 Jul 2018 12:37:04 +0000 (14:37 +0200)]
Change ALL_CAPS to CamelCase in DryadJobStatus
golint complains about using ALL_CAPS.
DJ_<STATUS> were renamed to DryadJobStatus<STATUS>
All occurrences were renamed.
Verification:
$ golint ./... |grep -v -e "^vendor*"
Should not report any warnings regarding ALL_CAPS.
Change-Id: I74afd3b2e24d368aaa63d999ef256df8f2af8566
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 30 Jul 2018 12:23:27 +0000 (14:23 +0200)]
Add keys to composite literals declarations
go vet complains about unkeyed composite literals.
This commit adds keys to all composite literal definitions.
This allows adding fields to structs without rewriting code.
Verification:
go vet -composites ./...
Should return no errors.
Change-Id: I59338e47afdddfa1efae2998c9cbdf6049d3a459
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 30 Jul 2018 11:19:53 +0000 (13:19 +0200)]
Simplify bool comparisons
Weles has been analyzed with gosimple:
https://github.com/dominikh/go-tools/tree/master/cmd/gosimple
Simplifications here are similiar to formatting code with gofmt as
follows:
$ gofmt -s -w
Additionally getTotal and getRemaining in artifacts/database/database.go
have been renamed to totalRecords and remainingRecords respectively.
Renaming reason:
https://golang.org/doc/effective_go.html#Getters
Verification:
$ gometalinter.v2 --disable-all --vendor --enable=gosimple ./...
Should report no errors
Change-Id: Id1c639e5398d7f07fff4c6c6815ff36c5b499751
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 30 Jul 2018 09:26:07 +0000 (11:26 +0200)]
Remove unused assignments
Weles has been analyzed with ineffassign:
https://github.com/gordonklaus/ineffassign
https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices
Change-Id: Ib05a41f0f35c8b2831b9c5f5ecc85c1774c82555
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Fri, 27 Jul 2018 16:42:11 +0000 (18:42 +0200)]
Remove unnecessary type conversions
Weles has been analyzed with unconvert:
https://github.com/mdempsky/unconvert
This should make code more readable.
Verification:
$ gometalinter.v2 --disable-all --vendor --enable=unconvert ./...
There should be only errors in server/server.go which is generated code.
Change-Id: I8288cb65b9faecd1b16854bf6f4bd3427dd3a01f
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Pawel Wieczorek [Mon, 30 Jul 2018 16:09:57 +0000 (18:09 +0200)]
Merge branch 'api'
Change-Id: I73745abe146f5e9d6e3a9ba886f727da53a2b32e
Alexander Mazuruk [Fri, 27 Jul 2018 15:53:40 +0000 (17:53 +0200)]
Add protocol and IP to default Boruta address.
Weles will not connect to Boruta if protocol and address are not
specified.
Change-Id: I42ad88c5b2b1c4de42347def1d505bee5581f652
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Wed, 25 Jul 2018 15:57:04 +0000 (17:57 +0200)]
Allow disabling pagination in artifacts
When ArtifactPagination.Limit is 0, .ID and .Forward are ignored.
Additionally, when empty ArtifactSorter is passed, correct sorting is
applied (ascending).
Change-Id: Ibc2cc7ca688c854b33762ee8639ceb9b0529a937
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Lukasz Wojciechowski [Wed, 25 Jul 2018 14:24:04 +0000 (16:24 +0200)]
Allow disabling pagination by setting limit to 0
If JobPagination.Limit is set to 0, then:
* JobID and Forward fields are ignored;
* pagination in listing jobs is disabled;
* all records matching filter are returned.
This patch adjusts JobsController implementation to above new rule.
It modifies also tests to cover new feature.
Change-Id: I929b89aa100d8f491886d0861592de2004758424
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Alexander Mazuruk [Tue, 5 Jun 2018 13:25:31 +0000 (15:25 +0200)]
Add list artifacts path to API
There is an issue with generated marshalling and unmarshalling:
https://github.com/go-swagger/go-swagger/issues/1617
Currently generated code with explicit marshalling in artifact_info.go
is commented out. Implicit marshalling works fine.
Change-Id: I211ea81636fe0b7d25d4fb1e7d76b55dacae7121
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 30 Jul 2018 14:38:20 +0000 (16:38 +0200)]
Add info about default artifact sorting
Change-Id: Idced62086616b1252a6a19575e64bea0ed6c14d4
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Tue, 5 Jun 2018 13:12:56 +0000 (15:12 +0200)]
Add pagination to artifacts/database
There was a problem with pagination key for ArtifactInfo objects.
Only ArtifactInfo.Path was unique and artifacts database ID was not
accessible. This patch moves ID field to ArtifactInfo from previous struct,
artifactInfoRecord which extended ArtifactInfo and was local to the
database package.
Added identifiers to structs in test as after adding field to
ArtifactInfo there were build errors.
Added some more verbose error messages in database Filtering function.
Added pagination and sorting.
Changed artifacts/database/database_suite_test.go package from
database_test to database as all tests are white-box.
And of course the main purpose of this commit- pagination and sorting of
artifacts is now possible.
Tests were added in separate package, server_suite_test.go contains
beforeSuite initialization which is used only by pagination_test.go
before/after suite has no effect on previous tests of database.
Change-Id: Ic74dfa403ea486faca94aa39e5cb063abdc31ac4
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 9 Jul 2018 15:25:12 +0000 (17:25 +0200)]
Add fixture generator module for testing
It will be used in tests of artifacts/database (pagination)
as well as server (artifacts pagination).
Change-Id: I3eee5e6891461c46442bcb369e1d9ec0850f3a17
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Tue, 5 Jun 2018 13:09:07 +0000 (15:09 +0200)]
Add new types to enable artifact pagination
Change-Id: I1d6cd83883288676703bb18fd3026e6603aacef4
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Thu, 24 May 2018 04:43:50 +0000 (06:43 +0200)]
Add list jobs path to API
The following files should be reviewed (as they were the only files not
generated by the swagger automatically but had to be filled manually):
- errors.go
- swagger.yml
- cmd/weles-server/main.go
- server/configure_weles.go
- server/job_lister_handler.go
- server/job_lister_handler_test.go
- server/handlers_helpers.go
- server/server_suite_test.go
- server/page_limit.go
Change-Id: Iab9589ca6738831f8269ba3eeb20581bba35645b
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Lukasz Wojciechowski [Wed, 13 Jun 2018 12:12:30 +0000 (14:12 +0200)]
Remove dot imports usage from notifier package
Change-Id: I58ae24b3c86d06f2f487abe18543c50e27fef3d9
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Mon, 11 Jun 2018 16:18:07 +0000 (18:18 +0200)]
Implement pagination in listing jobs
Change-Id: I5a34576315118d0a946ee1f86cf2443e41ac744f
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Mon, 11 Jun 2018 13:27:59 +0000 (15:27 +0200)]
Implement sorting in ListJobs
Patch includes Aleksander's version of sorting function selector.
Change-Id: I624bba4a07e39b0129c08d5a2513ad1e874c32cc
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Lukasz Wojciechowski [Mon, 11 Jun 2018 10:17:57 +0000 (12:17 +0200)]
Implement filtering in ListJobs
Change-Id: Iae458c029cfd416d39d4999bb745cf78e3720604
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Fri, 6 Jul 2018 15:08:48 +0000 (17:08 +0200)]
Adjust Controller to allow addition of pagination
The ListJobs signature has been changed to provide: better filtering,
sorting and pagination of listed jobs.
This patch provides a simple implementation ignoring filtering, sorting
and pagination. It returns all Jobs.
Following patches will add proper implementation.
The patch changes JobsController interface and requires rebuild of mock:
$ make mocks
Change-Id: Id0a806414d9090909274b438b18261193f88a9f7
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Alexander Mazuruk [Tue, 27 Feb 2018 13:39:00 +0000 (14:39 +0100)]
Add new types to weles to enable pagination
Types generated by swagger:
- JobFilter containing filters
- JobSorter containing information about sorting the data
- JobSortBy containing enum of possible sorting keys for jobs
- SortOrder containing enum of possible sorting orders
Types added manually:
- JobPagination containing info how much data should be listed and where
should it start. Limit is int32 as Weles is intended to work on both 64
and 32 bit architectures. Due to varying integer size (which is slice
limit) on those archs, max slice len is constrained to be 32bit.
- ListInfo which holds number of total records fulfilling filter and
remaining records that have not yet been sent to the client.
Change-Id: I59e441da86856fd2bc57817ce8903ef96f669e97
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Tue, 27 Feb 2018 14:17:28 +0000 (15:17 +0100)]
Add cancel job path to API
The following files should be reviewed (as they were the only files not
generated by the swagger automatically):
- swagger.yml
- server/configure_weles.go
- server/job_canceler_handler.go
- server/job_canceler_handler_test.go
Change-Id: I323b78f82c83a28f671f59105f1f5fc06a44be62
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Fri, 6 Jul 2018 12:22:07 +0000 (14:22 +0200)]
Add flags to set ArtifactManager init values
This commit makes it possible to set ArtifactManager when executing
weles binary, rather than recompile it.
Change-Id: I14eb4c1c11c2bfc6af20bc25208874ce0e7901fd
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Thu, 24 May 2018 03:06:13 +0000 (05:06 +0200)]
Add create job path to API
The following files were edited manually:
- swagger.yml
- job_id_string.go
- server/configure_weles.go
- server/job_creator_handler.go
- server/job_creator_handler_test.go
- server/managers.go
- server/server_suite_test.go
- server/test_sample.yml
- cmd/weles-server/main.go
Please note that documentation served with weles server is now preceded
by the basePath specified in the swagger.yml file. To check documentation
served alongside API you need to go to:
http://{host:port}/api/v1/docs
Change-Id: I6ae33208eea35834127e5c6ff0dda2d1e11f79cf
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Wed, 21 Feb 2018 12:24:31 +0000 (13:24 +0100)]
Change basic weles types to generated by swagger
Definitions from artifactmanager.go and jobmanager.go are removed.
Definitions which are used by swagger were added to swagger.yml.
Swagger server was regenerated.
ArtifactStatusChange was moved to separate file for consistency.
In ArtifactType const renamed:
YAMLFILE to YAML
TESTFILE to TEST
FILE suffix was redundant as each ArtifactType is a file.
In ArtifactType and ArtifactStatus consts AM_ prefix was changed to
ArtifactType and ArtifactStatus respectively. JobStatus consts JOB_
prefix was changed to JobStatus. This naming is more go-compliant.
Swagger generated models use strfmt.DateTime rather than time.Time.
strfmt.DateTime is a wrapper for time.Time extending it with swagger
validation.
controller and artifacts packages were adjusted to be compatible with
the above.
Currently artifact_info.go has difference between committed and
generated code. This is caused by the following issue:
https://github.com/go-swagger/go-swagger/issues/1617
Change-Id: If7687ac25113c93c6b130ae6555fd338c700fef8
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Wed, 4 Jul 2018 14:47:32 +0000 (16:47 +0200)]
Remove . imports from artifacts pkg
Go's vet tool complains about using dot imports.
Change-Id: Ifd9bedabc49b06196c67bf68a3d27923cfe557d9
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Wed, 21 Feb 2018 10:08:06 +0000 (11:08 +0100)]
Add initial swagger server
Swagger was chosen due to:
- Easy generation of server and client boilerplate code in many languages.
(in Weles project only golang is used but the swagger.yml file can be used
to generate Weles server/client boilerplate in any language)
- Clear, automatically generated documentation
- Promising standard that this tool is based on:
https://github.com/OAI/OpenAPI-Specification
This commit should be treated as the Weles server "scaffolding"
The initial swagger.yml file is added so there is something to generate
the server from. "tbd" will be removed in future commits.
Server was generated using following command:
./dev-tools/swagger generate server \
-A weles \
-f ./swagger.yml \
-m ../weles \
-s ./server \
-r ./COPYING \
--flag-strategy pflag \
--compatibility-mode=modern
Due to agreed name of the server package ("server") there was a conflict in
names in generated code causng build error. Generated code used
server as a name for an object instance which has been renamed to srv.
The Makefile recipe has additional --exclude-main flag which prevents the
swagger generator from overwriting cmd/weles-server/main.go
This flag must be set because:
- obviously as stated above regenerating main will cause build error
- all Weles objects will have to be created there
- additional Weles flags will have to be added there
Thus overwriting this file is a no-no and server regeneration should
only be performed using provided Makefile recipe.
The Makefile has also a recipe to generate html documentation which is
intended to be integrated with Sphinx documentation. This recipe
requires docker:
https://docs.docker.com/get-started/
---
Verification
Update your vendor directory
$ make vendor
To build server:
$ make server
You can check default swagger generated server flags:
$ bin/weles-server --help
When the server is running you can also see how the automatically
served swagger documentation looks like:
http://{host:port}/docs
host and port are flags that should be supplied when running Weles.
If not provided, localhost and random port will be used.
To regenerate swagger server code:
$ make swagger-server-generate
To generate html documentation:
$ make swagger-docs-html
Change-Id: I00cda82219944cc6ab1937f6d14f05d52c7fd4e5
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Mon, 14 May 2018 16:56:26 +0000 (18:56 +0200)]
Add swagger to development tools
There was a problem with different go-swagger versions which made re-generating server
using different go-swagger versions yield different results than commited.
Version of go-swagger was constrained in Gopkg.toml file to resolve
above issue.
go-swagger and all of its dependencies were added to required rule in
Gopkg.toml to instruct dep to download them.
---
Verification:
$ make tools
In the bin/dev-tools/ directory there should be additional swagger
executable.
To check if the swagger tool has built correctly:
- serve specification UI:
$ bin/dev-tools/swagger serve https://raw.githubusercontent.com/swagger-api/swagger-spec/master/examples/v2.0/json/petstore-expanded.json
- validate specification:
$ bin/dev-tools/swagger validate https://raw.githubusercontent.com/swagger-api/swagger-spec/master/examples/v2.0/json/petstore-expanded.json
Change-Id: I4b96bf7181d9fb14d13fe1e2721ec564f66832bb
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Tue, 29 May 2018 10:45:25 +0000 (12:45 +0200)]
Lock mockgen version and automate mock generation
Version of mockgen tool has been constrained in dep's Gopkg.toml file.
It will ensure reproducible results of mocks re-generation.
Makefile 'tools' recipe has been added which will install mockgen, and
in the future- other tools supporting development.
Additionally 'mocks' recipe has been added which will generate mocks for
all Weles interfaces.
Go's generate tool has been tasked with mock generation.
"//go:generate" directives have been placed in separate mocks_generate.go
files in repository mock/ controller/mock/ and manager/ directories.
Mocks from manager have been re-generated using mockgen's reflect mode
as source mode does not work using binary not from GOPATH.
This resulted in changes in the following file:
manager/mock_dryad_job_runner_test.go
They work the same but due to different creation method look a bit
different.
ArtifactManager mock was also updated due to interface change:
mocks/artifactmanager.go
---
Verification:
Create mockgen binary:
$ make tools
Regenerate mocks:
$ make mocks
Check if the files have changed:
$ git status
(They should not change)
Additionally:
$ go test ./...
Should result in all tests passing.
Change-Id: Ie7d8ab89ac9b00960fe5473271287d7832d31e9f
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Alexander Mazuruk [Wed, 23 May 2018 20:14:14 +0000 (22:14 +0200)]
Add dep for dependency management
Dep is introduced into the Weles project to lock dependencies and ensure
common and reproducible results among users of this package and its
developers. It will be used to lock both imported packages (when needed)
and to lock version of tools supporting development of Weles.
Gopkg.toml contains hand-written rules that govern dep's behaviour.
Gopkg.lock contains information that will enable anyone to download
dependencies in the same version as were used during development.
It has been decided that vendor directory shall not be committed thus was
added to .gitignore. This was decided due to projects contributors
preference not to keep all dependencies in the repository. It might
cause problems in case of dependency migration, name changes etc but
those are rare and should be maintained manually.
While make tool was designed to automate building projects written in
C-family of languages, we have decided to add Makefile to ease usage of
supporting tools such as dep, gomock, ginkgo, go-swagger.
I'd like to thank for all the input regarding Makefile from:
Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Aleksander Mistewicz <a.mistewicz@samsung.com>
---
Verification:
To build Weles you will need to have dep installed:
$ go get -u github.com/golang/dep/cmd/dep
To download dependencies needed to build weles:
$ make vendor
To verify you can try building parser as archive and check if the
gopkg.in/yaml.v2 is used from vendor:
$ go build -buildmode=archive -v -a ./parser
One of the output lines should be:
git.tizen.org/tools/weles/vendor/gopkg.in/yaml.v2
---
More about dep:
https://golang.github.io/dep/docs/introduction.html
If you do not have make installed:
https://www.gnu.org/software/make/
Change-Id: Ie787513b26c57117de06b1f6e0da25769d7a4bfe
Signed-off-by: Alexander Mazuruk <a.mazuruk@samsung.com>
Pawel Wieczorek [Tue, 3 Jul 2018 13:47:50 +0000 (15:47 +0200)]
Merge branch 'artifacts'
Change-Id: I7d422c12880842ce45ef7cb83c20eaf3b6b13192
Katarzyna Gorska [Wed, 13 Dec 2017 14:10:46 +0000 (15:10 +0100)]
Add downloader to Storage, implement PushArtifact
Change-Id: If757c4745089eb308fca53c732790d3615c5dcb9
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
Pawel Wieczorek [Fri, 29 Jun 2018 14:16:56 +0000 (16:16 +0200)]
Merge branch 'runner'
Change-Id: Ic7344743f615ac74fbbf8c0e8c6c8fe46dd069fc
Aleksander Mistewicz [Fri, 22 Jun 2018 13:31:06 +0000 (15:31 +0200)]
Add Boot to DeviceCommunicationProvider interface
Boot starts DUT and prepares communication so that Login could work.
It can be called more than once to reset DUT's state.
Change-Id: I2c28028b557fa3ea862f5f176090b81767a240d2
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Wed, 29 Nov 2017 17:53:58 +0000 (18:53 +0100)]
Implement Test with test
Change-Id: Ic44cd543be463ef3434a23de679547c5e52e6fe9
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Wed, 29 Nov 2017 17:29:02 +0000 (18:29 +0100)]
Implement Boot with test
Change-Id: I059792863037d06331fd624ba71543aedd7101ad
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Wed, 29 Nov 2017 17:22:56 +0000 (18:22 +0100)]
Implement Deploy with test
Change-Id: Ibc55e994878f639af1f0190c1debd332e35fd1c9
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Wed, 22 Nov 2017 10:31:14 +0000 (11:31 +0100)]
Add cmd provider to FOTA CLI
FOTA is a tool available on MuxPi providing ability to flash sdcard with
archives containing images published via http server.
This patch adds an easy way to construct command line call to the said
tool.
Change-Id: I8da6e5c49a4419e3f3343129456472363e55eb6d
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Wed, 29 Nov 2017 15:34:02 +0000 (16:34 +0100)]
Add example job definition to manager tests
Change-Id: Ib8f5dd6f989a0fb5a2edc4cddaab6c186b5d80d6
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Wed, 29 Nov 2017 15:32:38 +0000 (16:32 +0100)]
Add Config argument to Create of DryadJobManager
Change-Id: I85354ad5e1580d69a1d6ecb31e6e3cbfc567c00c
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Pawel Wieczorek [Thu, 28 Jun 2018 14:37:42 +0000 (16:37 +0200)]
Merge branch 'dryad'
Change-Id: I53430cc296457b031e5315b4c7f05af1fdd9dc33
Aleksander Mistewicz [Thu, 21 Jun 2018 13:49:29 +0000 (15:49 +0200)]
Rename dryad_session_provider to session_provider
Change-Id: I4e6dbbf5b06606e32cf3aef9e9dd9cb8d4d74134
Aleksander Mistewicz [Thu, 21 Jun 2018 13:33:24 +0000 (15:33 +0200)]
Use reverse SSHFS instead of scp
It solves the following problems:
* some files did not fit into tmpfs or root filesystem,
* some files could not be copied at all due to their contents,
* paths needed to be translated.
Change-Id: I581ad3ddf22bf2e3e8aec0d2952f34f76ab90a41
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Thu, 21 Jun 2018 13:30:51 +0000 (15:30 +0200)]
Make Exec methods variadic
Use mock instead of real implementation of SessionProvider in tests of
DeviceCommunicationProvider.
Change-Id: I6d62145b7982c28f93a2f44d259f0759dfd3dc6a
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Fri, 13 Apr 2018 17:09:02 +0000 (19:09 +0200)]
Add reverseSSHFS
It mounts a local directory on a remote consuming one session of ssh
connection. It makes accessing files (even big) easy in the limited
environment provided by Dryad.
Change-Id: I2e881f8e1291bac28331dea584b81dcef92e331a
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Adam Malinowski [Mon, 27 Nov 2017 05:01:15 +0000 (06:01 +0100)]
Add DeviceCommunicationProvider implementation
Change-Id: I0b98831d3ee53de05831caf7564363fdf289fcf8
Signed-off-by: Adam Malinowski <a.malinowsk2@partner.samsung.com>
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Adam Malinowski [Tue, 28 Nov 2017 14:31:28 +0000 (15:31 +0100)]
Add file transfer functions to SessionProvider
Change-Id: Ief14d76d93bfd041bc9809d0599719859fde69dc
Signed-off-by: Adam Malinowski <a.malinowsk2@partner.samsung.com>
Adam Malinowski [Thu, 23 Nov 2017 09:18:23 +0000 (10:18 +0100)]
Add SessionManager implementation
MuxPi is needed to perform tests. SSH key needs to be generated and
its private part has to be saved to a local file. To run tests:
ginkgo manager/dryad -- \
-keyFile=file \
-address=address \
-port=port \
-userName=login
Where:
file - path to a file containing private part of SSH key generated
on Dryad
address - IP address of Dryad
port - SSH port to use for connection with Dryad (default: 22)
login - user name for connection with Dryad
Change-Id: I1b4cf55f8082b5b0d9ec1d0311f37f87f35fb753
Signed-off-by: Adam Malinowski <a.malinowsk2@partner.samsung.com>
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Pawel Wieczorek [Wed, 2 May 2018 10:59:48 +0000 (12:59 +0200)]
Merge branch 'controller'
Change-Id: I7b318961f834834e5ea1ae22c615e74f8159737e
Lukasz Wojciechowski [Thu, 30 Nov 2017 12:29:14 +0000 (13:29 +0100)]
Add NewJobManager function
NewJobManager function creates new instance of Controller
with all internal submodules. The created structure is returned
as JobManager interface.
This is the only valid method for creation whole Controller module
in Weles and should be used to create Weles service.
Change-Id: I8ff2051a23f714a99fc2642c94e34d247ec164d7
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Mon, 16 Apr 2018 14:10:42 +0000 (16:10 +0200)]
Implement ListJobs in Controller with tests
Implementation enhances JobsController interface and implementation.
The mockup of JobsController is regenerated using:
mockgen -package mock -destination=controller/mock/jobscontroller.go \
git.tizen.org/tools/weles/controller JobsController
Tests for both JobsControllerImpl and Controller are updated.
Change-Id: Ie30234ac754d19616ce2811e3f30a6834d28e296
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Mon, 16 Apr 2018 13:34:14 +0000 (15:34 +0200)]
Add Controller with tests
Change-Id: I4681c0e085983094f9ce2699617bb9b52d2e086a
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Pawel Wieczorek [Fri, 27 Apr 2018 12:14:49 +0000 (14:14 +0200)]
Merge branch 'manager'
Change-Id: I950cbd5ee07bb735ca5a8cfd46b1239039a70b31
Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
Aleksander Mistewicz [Fri, 17 Nov 2017 15:50:40 +0000 (16:50 +0100)]
Implement List with table driven tests
Change-Id: Ifccf613b89cb6a21d4229b15affb533c0bb95de4
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Fri, 17 Nov 2017 14:01:51 +0000 (15:01 +0100)]
Add tests for panic recovery of dryadJob
Change-Id: I7ac9ccb609a77ff5d0e4635c5d91e23cc88fd4ae
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Fri, 17 Nov 2017 12:30:51 +0000 (13:30 +0100)]
Add more complicated tests of dryadJob
Change-Id: Ie015609731c8f25ecfe6bff31414f18390494e3b
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Thu, 16 Nov 2017 16:57:47 +0000 (17:57 +0100)]
Use mockups of generated interfaces in tests
Change-Id: I77760fa2028e3c096929e82683c8024e01ee3c06
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Wed, 29 Nov 2017 15:59:22 +0000 (16:59 +0100)]
Add generated mock of DryadJobRunner
Command used:
mockgen -source=manager/manager.go \
-destination=manager/mock_dryad_job_runner_test.go \
-package manager
Change-Id: I015e79a3db12db02e2423bc6ce05cf4e9dfb6555
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Aleksander Mistewicz [Thu, 16 Nov 2017 16:26:20 +0000 (17:26 +0100)]
Add generated mock of interfaces in manager/dryad
Command used:
mockgen -source=manager/dryad/dryad.go \
-destination=manager/mock_dryad_test.go \
-package manager
Change-Id: I15860362b8fc144ddd9efe05e03efef2f5ebf03f
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Lukasz Wojciechowski [Mon, 27 Nov 2017 15:54:57 +0000 (16:54 +0100)]
Add generated mocks for controller's internals
mockgen -package mock -destination=controller/mock/boruter.go \
git.tizen.org/tools/weles/controller Boruter
mockgen -package mock -destination=controller/mock/downloader.go \
git.tizen.org/tools/weles/controller Downloader
mockgen -package mock -destination=controller/mock/dryader.go \
git.tizen.org/tools/weles/controller Dryader
mockgen -package mock -destination=controller/mock/parser.go \
git.tizen.org/tools/weles/controller Parser
Change-Id: Ie5fd939e0082c0f7975d2dbc416d730f28c311eb
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Wed, 22 Nov 2017 08:53:17 +0000 (09:53 +0100)]
Add Dryader implementation with tests
DryaderImpl implements delegation and control of Job execution
in DryadJobManager.
Change-Id: I7d30f21bf0f838829f6c2d0b65362a6d7b66d0a3
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Mon, 27 Nov 2017 08:15:12 +0000 (09:15 +0100)]
Add Dryader interface
Dryder interface allows delegation of Job execution and cancelation
to DryadJobManager.
Change-Id: I866f8290dc51d24ab65e2671edf306fc33f2e3c9
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Tue, 21 Nov 2017 20:34:31 +0000 (21:34 +0100)]
Add Boruter implementation with tests
BoruterImpl implements communication with Boruta. It is used
for acquiring, monitoring and releasing Dryads for Weles' Jobs.
Change-Id: I91667cad4ccac176e4e01c697e2dbfbe7b54cfd3
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Thu, 23 Nov 2017 17:02:23 +0000 (18:02 +0100)]
Add Boruter interface
Boruter defines interface for delegation of communication with Boruta.
Communication is used for acquiring, monitoring and releasing Dryads.
Change-Id: Id6c310220839c1d196f86b8c86123cd1a720f526
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Thu, 23 Nov 2017 17:01:10 +0000 (18:01 +0100)]
Add generated mockup for boruta.Requests
mockgen -package mock -destination=controller/mock/requests.go \
git.tizen.org/tools/boruta Requests
Change-Id: I3cd7673db7ff0eb5e7fb3ead8ea3d31a83430bee
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Mon, 20 Nov 2017 05:04:31 +0000 (06:04 +0100)]
Add Downloader implementation with tests
DownloaderImpl is responsible for parsing Job's config and searching
for information about files (artifacts) related to the test.
It delegates to ArtifactsManager downloading images and testfiles
required to run Job (Push). It creates also paths for output
artifacts (Pull). It updates Job's config with proper paths
for these artifacts.
It does also monitor process of downloading files in ArtifactManager
and finally it notifies Controller when all files are downloaded
or in case of failure.
Change-Id: I578108dd1a924aaf36435e4dbc5d8dcce50214ee
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Thu, 23 Nov 2017 10:06:07 +0000 (11:06 +0100)]
Add Downloader interface
Downloader defines action run for a Job to download artifacts required
to run the Job.
Change-Id: I932b1630a97dc9573aa0cbf0c4726a91fb042339
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Tue, 14 Nov 2017 10:37:11 +0000 (11:37 +0100)]
Add Parser implementation with tests
ParserImpl implements parsing phase of Job handling.
It saves yaml file in ArtifactDB, parses it and sets up config.
Change-Id: Icb52b7c1943c29744f82532877aade2bb085c345
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Wed, 22 Nov 2017 16:39:51 +0000 (17:39 +0100)]
Add Parser interface
Parser defines action run for a Job to prepare config in parsing phase.
Change-Id: Iea53d08c89d5fbedf690036688ded53c40349741
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Wed, 29 Nov 2017 15:58:33 +0000 (16:58 +0100)]
Add Notifier implementation with tests
Change-Id: Ib37b4dccf043bb8d5b914b14436929b523446afe
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Wed, 29 Nov 2017 15:58:07 +0000 (16:58 +0100)]
Add Notifier interface
Change-Id: If77ebbe1df6cf2dd49bbee91efee2d8f0b9317d9
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Wed, 22 Nov 2017 16:37:31 +0000 (17:37 +0100)]
Add Notification struct
The structure is used to notify Controller about events occured
during processing a Job.
Change-Id: I38077eda6fc0cb156d8224b5af1280d5b8db88af
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Thu, 9 Nov 2017 17:00:25 +0000 (18:00 +0100)]
Add JobsController implementation with tests
JobsControllerImpl structure stores actual information about all Jobs
such as status, update time, config and Dryad access. It controls
collision free JobID creation. It stores state of Jobs execution
and saves data to DB. It implements JobsController interface.
Patches enhancing tests, written by Aleksander Mistewicz squashed into
this patch.
Change-Id: Ie3e3c46660c650c48cd80a93038b8cdd05f3fc21
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Signed-off-by: Aleksander Mistewicz <a.mistewicz@samsung.com>
Lukasz Wojciechowski [Tue, 21 Nov 2017 20:37:54 +0000 (21:37 +0100)]
Add Dryad structure content
Change-Id: Ifb754266fe16ef148fcc5c1dd7d7683b1a2638f2
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Wed, 22 Nov 2017 11:13:44 +0000 (12:13 +0100)]
Add generated mockup for JobsController
mockgen -package mock -destination=controller/mock/jobscontroller.go \
git.tizen.org/tools/weles/controller JobsController
Change-Id: I8b9fff217a683568dcedfd2971b94c23d75a0e4c
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Thu, 9 Nov 2017 16:59:32 +0000 (17:59 +0100)]
Define JobsController interface
JobsController interface is designed for managing Jobs inside Controller.
It is defined to provide additional layer for strict managing Job structures
only. This allows mocking up the interface in tests.
Change-Id: I3fa5054b017f6cd2b6c95f2e5a27acd72b226ee5
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Tue, 28 Nov 2017 11:45:56 +0000 (12:45 +0100)]
Add generated mockups for modules interfaces
mockgen -package mock -destination=mock/artifactmanager.go \
git.tizen.org/tools/weles ArtifactManager
mockgen -package mock -destination=mock/dryadjobmanager.go \
git.tizen.org/tools/weles DryadJobManager
mockgen -package mock -destination=mock/jobmanager.go \
git.tizen.org/tools/weles JobManager
mockgen -package mock -destination=mock/parser.go \
git.tizen.org/tools/weles Parser
Change-Id: I4401a3a64babe1165016baee749bdea296a55a59
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Lukasz Wojciechowski [Tue, 7 Nov 2017 11:12:49 +0000 (12:12 +0100)]
Fix typos in JobManager
Change-Id: Ic16bbbb50bd82b0d6b747f05f2fd0c409d2fcf1d
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Pawel Wieczorek [Wed, 18 Apr 2018 16:27:17 +0000 (18:27 +0200)]
Merge branch 'parser'
Change-Id: Iee519eef6c7f9f61f66d4bdeaa1a28f871ba2ec5
Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
Katarzyna Gorska [Tue, 28 Nov 2017 13:51:24 +0000 (14:51 +0100)]
Add downloadJob queue
Change-Id: I90afb2433977be3ba5b72935d88b289e28e97b2e
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
Katarzyna Gorska [Tue, 28 Nov 2017 13:25:32 +0000 (14:25 +0100)]
Add downloader base functions
Downloader implements ArtifactDownloader interface.
It schedules download jobs in queue and notifies about
artifacts' status changes.
Change-Id: I8ff33c46406dd305c62d3874bd6e013fc6606393
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
Katarzyna Gorska [Wed, 29 Nov 2017 19:44:59 +0000 (20:44 +0100)]
Add SetStatus with test
SetStatus changes artifact's status in ArtifactDB.
Change-Id: I3d00f1a4dce861220afee367433edb981286c40c
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
Katarzyna Gorska [Wed, 29 Nov 2017 19:11:20 +0000 (20:11 +0100)]
Implement List with tests
List is part of implementation of ArtifactManager.
It filters ArtifactDB with given filter
and returns list of all matching elements.
Change-Id: I746ea564e07d512ddf55e36d92f515e5f804b120
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
Katarzyna Gorska [Thu, 21 Dec 2017 13:20:39 +0000 (14:20 +0100)]
Implement Select in database package
Storage's Select allows to use one method for fetching
elements from ArtifactDB. Requested artifacts can be selected by
JobID, ArtifactType, ArtifactStatus or ArtifactAlias.
Change-Id: Idb5fe308cf7cac1dd638c31834b1499ef678822c
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
Katarzyna Gorska [Wed, 15 Nov 2017 16:52:46 +0000 (17:52 +0100)]
Add ArtifactDB to storage
ArtifactDB is designed to store information about downloaded artifacts.
As for now inserting ArtifactRecord is supported.
More operations (updates, deletes and selects) will be implemented soon.
Close was added to ArtifactManager interface.
Change-Id: I17ac260691d11518bcec5385eaf2689e389feb7f
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
Katarzyna Gorska [Tue, 5 Dec 2017 10:49:11 +0000 (11:49 +0100)]
Add artifacts/database package
Package database is responsible for Weles' artifact storage.
This patch introduces ArtifactDB struct responsible for database
connection and queries.
Change-Id: Ie74be2d5bb62a54c222351ae556fabe8e2be5f4c
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
Katarzyna Gorska [Thu, 9 Nov 2017 09:37:06 +0000 (10:37 +0100)]
Add Storage and stubs for interface implementation
Storage implements ArtifactManager interface.
Also ArtifactDownloader method Download parameters changed.
To properly download an artifact, we should pass not only ArtifactURI,
but also ArtifactPath.
Close method was added to ArtifactDownloader.
Change-Id: Ic707ee270cfd54d2f0bb9792d1b5f65ceae75e80
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
Leszek Zygo [Tue, 24 Oct 2017 10:16:44 +0000 (12:16 +0200)]
Add yaml parser functionality and tests
Change-Id: Iab4513293fbee901798c43f178ded8ee73585c9a
Signed-off-by: Leszek Zygo <l.zygo@samsung.com>
Leszek Zygo [Tue, 7 Nov 2017 16:09:20 +0000 (17:09 +0100)]
Add Weles parser interface
Change-Id: If5cb8bbdf033233510f847fd3279a32b7d4c3406
Signed-off-by: Leszek Zygo <l.zygo@samsung.com>
Leszek Zygo [Tue, 24 Oct 2017 09:57:57 +0000 (11:57 +0200)]
Add sample_yaml
Change-Id: Ic20bc4706c4461dc184d373443d85b5378bd3ca3
Signed-off-by: Leszek Zygo <l.zygo@samsung.com>
Lukasz Wojciechowski [Tue, 7 Nov 2017 11:12:49 +0000 (12:12 +0100)]
Define JobManager interface and basic Job types
Change-Id: Ic16bbbb50bd82b0d6b747f05f2fd0c409d2fcf1d
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Katarzyna Gorska [Fri, 3 Nov 2017 11:54:31 +0000 (12:54 +0100)]
Add ArtifactManager interface
Change-Id: Iaae3b462c525d10b25104d3b29256ac326f3c5a8
Signed-off-by: Katarzyna Gorska <k.gorska@samsung.com>
Lukasz Wojciechowski [Tue, 7 Nov 2017 20:46:10 +0000 (21:46 +0100)]
Define common Weles errors
Change-Id: I392e39b6454128d6000585fba6e8db976dff32c9
Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>