README.md

   1 ## Contents
   2 1. [Introduce Aurum](#introduce-aurum)
   3 2. [Command List](#command-list)
   4 3. [Running on TV](#running-on-tv)
   5 4. [Reference and Tip](#reference-and-tip)
   6
   7 ## Introduce Aurum
   8 - Aurum is a UI automation framework without UI Toolkit dependency.
   9
  10   Provides Commands to interact with the device UI by simulation user actions and introspection of the screen content.
  11
  12   It relies on the platform accessibility APIs to introspect the screen.
  13
  14 - User can use the IDL defined in Proto to create an automation app or script in a variety of languages without a language dependency.
  15
  16 ![image](https://media.github.sec.samsung.net/user/39889/files/ff684061-8bee-4a3f-855e-48ac38a5089a)
  17
  18 Please refer [grpc](https://grpc.io/) and [proto buffers](https://developers.google.com/protocol-buffers) for more information
  19
  20
  21 ## Command List
  22 <table>
  23
  24   <tr>
  25     <td> Name </td> <td> Description </td> <td> Proto definition </td> <td> Python usage </td>
  26   </tr>
  27
  28
  29   <tr>
  30     <td valign="top"> killServer </td>
  31     <td valign="top"> Terminate the org.tizen.aurum-bootstrap process </td>
  32     <td valign="top"> rpc killServer(ReqEmpty) returns (RspEmpty) {} </td>
  33     <td valign="top">
  34
  35   ```python
  36 stub.killServer(ReqEmpty())
  37   ```
  38 </td>
  39   </tr>
  40
  41
  42   <tr>
  43     <td valign="top"> findElements </td>
  44     <td valign="top"> If an object that satisfies a specific condition is searched on the screen and found, the object list is returned </td>
  45     <td valign="top"> rpc findElements(ReqFindElements) returns (RspFindElements) {}<br><br>ReqFindElements :<br>elementId<br>automationid<br>textfield<br>widgetType<br>widgetstyle<br> ischecked<br>ischeckable<br>iscliackable<br>isenabled<br>isfocused<br>isfocusable<br>isscrollable<br>isselected<br>isshowing<br>isactive<br>mindepth<br>maxdepth<br>packagename<br>textpartialmatch<br>xpath<br>geometry</td>
  46     <td valign="top">
  47
  48   ```python
  49 #Find Clickable object.
  50 def findClickableObject(stub):
  51     response = stub.findElements(ReqFindElements(isClickable=True))
  52
  53     for el in response.elements:
  54         return True
  55
  56     return False
  57
  58 #Find Object which has Button as text
  59 def findButtonTextObject(stub):
  60     response = stub.findElements(ReqFindElements(textField='Button'))
  61
  62     for el in response.elements:
  63         response = stub.getValue(ReqGetValue(elementId=el.elementId))
  64         return response.stringValue == 'Button'
  65
  66     return False
  67
  68 #Find entry type object
  69 def findEntryObject(stub):
  70     response = stub.findElements(ReqFindElements(widgetType='Elm_Entry'))
  71
  72     if len(response.elements) <= 0: return False
  73
  74     return True
  75
  76 #Find object with variouse condition
  77 def findTextWithShowingObject(stub):
  78     response = stub.findElements(ReqFindElements(textField='Bg', isShowing=True))
  79
  80     if len(response.elements) <= 0: return False
  81
  82     return True
  83   ```
  84 </td>
  85   </tr>
  86
  87   <tr>
  88     <td valign="top"> findElement </td>
  89     <td valign="top"> If an object that satisfies a specific condition is searched on the screen and found, the object is returned </td>
  90     <td valign="top"> rpc findElement(ReqFindElement) returns (RspFindElement) {}<br><br>ReqFindElement :<br>elementId<br>automationid<br>textfield<br>widgetType<br>widgetstyle<br> ischecked<br>ischeckable<br>iscliackable<br>isenabled<br>isfocused<br>isfocusable<br>isscrollable<br>isselected<br>isshowing<br>isactive<br>mindepth<br>maxdepth<br>packagename<br>textpartialmatch<br>xpath<br>geometry</td>
  91     <td valign="top">
  92
  93   ```python
  94 def findTextWithShowingObject(stub):
  95     response = stub.findElements(ReqFindElements(textField='Bg', isShowing=True))
  96
  97     if response.element is None:
  98         return False
  99
 100     return True
 101   ```
 102 </td>
 103   </tr>
 104
 105
 106   <tr>
 107     <td valign="top"> getValue </td>
 108     <td valign="top"> Get the value of the object </td>
 109     <td valign="top"> rpc getValue(ReqGetValue) returns (RspGetValue) {}<br><br>RspGetValue :<br>stringValue<br>intValue<br>doubleValue<br>boolValue </td>
 110     <td valign="top">
 111
 112   ```python
 113  def findButtonTextObject(stub):
 114     response = stub.findElements(ReqFindElements(textField='Button'))
 115
 116     for el in response.elements:
 117         response = stub.getValue(ReqGetValue(elementId=el.elementId))
 118         return response.stringValue == 'Button'
 119
 120     return False
 121   ```
 122 </td>
 123   </tr>
 124
 125
 126   <tr>
 127     <td valign="top"> setValue </td>
 128     <td valign="top"> Set the value of the object </td>
 129     <td valign="top"> rpc setValue(ReqSetValue) returns (RspSetValue) {}<br><br>RspGetValue :<br>stringValue<br>intValue<br>doubleValue<br>boolValue </td>
 130     <td valign="top">
 131
 132   ```python
 133  def setValueTest(stub):
 134     response = stub.findElements(ReqFindElements(widgetType='Elm_Entry'))
 135
 136     if len(response.elements) <= 0: return False
 137
 138     targetObj = response.elements[0].elementId
 139     testString = 'set test string by calling SetValue Method'
 140
 141     stub.setValue(ReqSetValue(elementId=targetObj, stringValue=testString))
 142     response = stub.getValue(ReqGetValue(elementId=targetObj))
 143
 144     if response.stringValue != testString:
 145         return False
 146
 147     return True
 148   ```
 149 </td>
 150   </tr>
 151
 152   <tr>
 153     <td valign="top"> getSize </td>
 154     <td valign="top"> Get geometry of the object </td>
 155     <td valign="top"> rpc getSize(ReqGetSize) returns (RspGetSize) {} </td>
 156     <td valign="top">
 157
 158   ```python
 159  def getSizeTest(stub):
 160     response = stub.findElements(ReqFindElements(textField='Button'))
 161     print("els", response)
 162
 163     for el in response.elements:
 164         response = stub.getSize(ReqGetSize(elementId=el.elementId))
 165         print(response)
 166
 167         return response.size.width + response.size.height > 0
 168
 169     return False
 170   ```
 171 </td>
 172   </tr>
 173
 174
 175   <tr>
 176     <td valign="top"> clear </td>
 177     <td valign="top"> Clear value of the object </td>
 178     <td valign="top"> rpc clear(ReqClear) returns (RspClear) {} </td>
 179     <td valign="top">
 180
 181   ```python
 182  def setValueAndClearTest(stub):
 183     response = stub.findElements(ReqFindElements(widgetType='Elm_Entry'))
 184
 185     if len(response.elements) <= 0: return False
 186
 187     targetObj = response.elements[0].elementId
 188     testString = 'set test string by calling SetValue Method'
 189
 190     stub.setValue(ReqSetValue(elementId=targetObj, stringValue=testString))
 191     response = stub.getValue(ReqGetValue(elementId=targetObj))
 192
 193     if response.stringValue != testString:
 194         return False
 195
 196     stub.clear(ReqClear(elementId=targetObj))
 197     response = stub.getValue(ReqGetValue(elementId=targetObj))
 198     if response.stringValue != '':
 199         return False
 200
 201     return True
 202   ```
 203 </td>
 204   </tr>
 205
 206
 207   <tr>
 208     <td valign="top"> getAttribute </td>
 209     <td valign="top"> Get attribute of the object </td>
 210     <td valign="top"> rpc getAttribute(ReqGetAttribute) returns (RspGetAttribute) {}<br><br>ReqGetAttribute :<br>VISIBLE<br>FOCUSABLE<br>FOCUSED<br>ENABLED<br>CLICKABLE<br>SCROLLABLE<br> CHECKABLE<br>CHECKED<br>SELECTED<br>SELECTABLE<br>SHOWING<br>ACTIVE<br><br>RspGetAttribute :<br>stringValue<br>intValue<br>doubleValue<br>boolValue </td>
 211     <td valign="top">
 212
 213   ```python
 214  def getAttributeTest(stub):
 215     response = stub.findElements(ReqFindElements(textField='Button'))
 216     if len(response.elements) <= 0: return False
 217
 218     checkList = [
 219             ['VISIBLE', True],
 220             ['FOCUSABLE', True],
 221             ['FOCUSED', False],
 222             ['ENABLED', True],
 223             ['CLICKABLE', True],
 224             ['SCROLLABLE', False],
 225             ['CHECKABLE', False],
 226             ['CHECKED', False],
 227             ['SELECTED', False],
 228             ['SELECTABLE',True],
 229             ['SHOWING', True],
 230     ]
 231     isFailed = False
 232
 233     for el in response.elements:
 234         for attr in checkList:
 235             if stub.getAttribute(ReqGetAttribute(elementId=el.elementId, attribute=attr[0])).boolValue != attr[1]:
 236                isFailed = True
 237
 238     if isFailed == True: return False
 239
 240     retrurn True
 241   ```
 242 </td>
 243   </tr>
 244
 245
 246   <tr>
 247     <td valign="top"> click </td>
 248     <td valign="top"> Generate click event </td>
 249     <td valign="top"> rpc click(ReqClick) returns (RspClick) {}<br><br>ReqClick :<br>elementId<br>coordination<br>Type :<br>ELEMENTID<br>COORD<br>ATSPI </td>
 250     <td valign="top">
 251
 252   ```python
 253  def clickTest(stub):
 254     response = stub.findElements(ReqFindElements(textField='Accessibility'))
 255
 256     if len(response.elements) <= 0: return False
 257
 258     for el in response.elements:
 259         stub.click(ReqClick(elementId=el.elementId, type='ELEMENTID'))
 260
 261     response = stub.findElements(ReqFindElements(textField='Screen Reader'))
 262     if len(response.elements) <= 0: return False
 263
 264     for el in response.elements:
 265         stub.click(ReqClick(coordination=Point(x=320, y=130), type='COORD'))
 266
 267     return True
 268   ```
 269 </td>
 270   </tr>
 271
 272
 273   <tr>
 274     <td valign="top"> longClick </td>
 275     <td valign="top"> Generate longclick (50ms) event </td>
 276     <td valign="top"> rpc longClick(ReqClick) returns (RspClick) {} </td>
 277     <td valign="top">
 278
 279   ```python
 280
 281  def longClickTest(stub):
 282     stub.longClick(ReqClick(coordination=Point(x=160, y=160), type='COORD'))
 283
 284     return False
 285   ```
 286 </td>
 287   </tr>
 288
 289
 290   <tr>
 291     <td valign="top"> flick </td>
 292     <td valign="top"> Generate flick event </td>
 293     <td valign="top"> rpc flick(ReqFlick) returns (RspFlick) {} </td>
 294     <td valign="top">
 295
 296   ```python
 297
 298  def flickTest(stub):
 299     for tryCnt in range(10):
 300         print('Flick to bottom to find "Spinner" item @ tries:', tryCnt)
 301         stub.flick(ReqFlick(startPoint=Point(x=160, y=359), endPoint=Point(x=160, y=1), durationMs=110))
 302         response = stub.findElements(ReqFindElements(textField='Spinner', isShowing=True)
 303
 304         for el in response.elements:
 305             return True
 306
 307     return False
 308   ```
 309 </td>
 310   </tr>
 311
 312
 313   <tr>
 314     <td valign="top"> touchDown </td>
 315     <td valign="top"> Generate touchDown event </td>
 316     <td valign="top"> rpc touchDown(ReqTouchDown) returns (RspTouchDown) {} </td>
 317     <td valign="top">
 318
 319   ```python
 320  def touchTest(stub):
 321     res = stub.touchDown(ReqTouchDown(coordination=Point(x=160,y=640)))
 322     print(res)
 323
 324     seq = res.seqId
 325     if seq < 0: return False
 326
 327     for yy in range(640, 30, -10):
 328         stub.touchMove(ReqTouchMove(coordination=Point(x=160,y=yy), seqId=seq))
 329
 330     stub.touchUp(ReqTouchUp(coordination=Point(x=160,y=30), seqId=seq))
 331
 332     return True
 333   ```
 334 </td>
 335   </tr>
 336
 337
 338   <tr>
 339     <td valign="top"> touchMove </td>
 340     <td valign="top"> Generate touchMove event </td>
 341     <td valign="top"> rpc touchMove(ReqTouchMove) returns (RspTouchMove) {} </td>
 342     <td valign="top">
 343
 344   ```python
 345 #See touchDown
 346   ```
 347 </td>
 348   </tr>
 349
 350
 351   <tr>
 352     <td valign="top"> touchUp </td>
 353     <td valign="top"> Generate touchUp event </td>
 354     <td valign="top"> rpc touchUp(ReqTouchUp) returns (RspTouchUp) {} </td>
 355     <td valign="top">
 356
 357   ```python
 358 #See touchDown
 359   ```
 360 </td>
 361   </tr>
 362
 363
 364   <tr>
 365     <td valign="top"> installApp </td>
 366     <td valign="top"> Install application </td>
 367     <td valign="top"> rpc installApp(stream ReqInstallApp) returns (RspInstallApp) {} </td>
 368     <td valign="top">
 369
 370   ```python
 371  def installAppTest(stub):
 372     response = stub.getAppInfo(ReqGetAppInfo(packageName='org.example.uicomponents'))
 373     if (response.isInstalled): return True
 374
 375     tpkFile = './org.tizen.uicomponents.arm.tpk'
 376     binaryChunk = get_file_chunks(tpkFile)
 377     response = stub.installApp(binaryChunk)
 378
 379     for waitCnt in range(10):
 380         response = stub.getAppInfo(ReqGetAppInfo(packageName='org.example.uicomponents'))
 381         print('tries:', waitCnt, 'isInstalled:', response.isInstalled)
 382         time.sleep(1)
 383
 384         if response.isInstalled: return True
 385
 386     return False
 387   ```
 388 </td>
 389   </tr>
 390
 391
 392   <tr>
 393     <td valign="top"> removeApp </td>
 394     <td valign="top"> Remove application </td>
 395     <td valign="top"> rpc removeApp(ReqRemoveApp) returns (RspRemoveApp) {} </td>
 396     <td valign="top">
 397
 398   ```python
 399
 400  def removeAppTest(stub):
 401     response = stub.getAppInfo(ReqGetAppInfo(packageName='org.example.uicomponents'))
 402     if (response.isInstalled): response = stub.removeApp(ReqRemoveApp(packageName='org.example.uicomponents'))
 403
 404     for waitCnt in range(10):
 405         response = stub.getAppInfo(ReqGetAppInfo(packageName='org.example.uicomponents'))
 406         print('tries:', waitCnt, 'isInstalled:', response.isInstalled)
 407         time.sleep(1)
 408
 409         if response.isInstalled != True: return True
 410
 411     return False
 412   ```
 413 </td>
 414   </tr>
 415
 416
 417   <tr>
 418     <td valign="top"> getAppInfo </td>
 419     <td valign="top"> Get application info </td>
 420     <td valign="top"> rpc getAppInfo(ReqGetAppInfo) returns (RspGetAppInfo) {}<br><br>ReqGetAppInfo :<br>isRunning<br>isInstalled<br>isFocused </td>
 421     <td valign="top">
 422
 423   ```python
 424  def getAppInfoTest(stub):
 425     return stub.getAppInfo(ReqGetAppInfo(packageName='org.example.uicomponents')).isRunning
 426   ```
 427 </td>
 428   </tr>
 429
 430
 431   <tr>
 432     <td valign="top"> launchApp </td>
 433     <td valign="top"> Launch application </td>
 434     <td valign="top"> rpc launchApp(ReqLaunchApp) returns (RspLaunchApp) {} </td>
 435     <td valign="top">
 436
 437   ```python
 438 def launchAppTest(stub):
 439     return stub.getAppInfo(ReqGetAppInfo(packageName='org.example.uicomponents')).isRunning
 440   ```
 441 </td>
 442   </tr>
 443
 444
 445   <tr>
 446     <td valign="top"> closeApp </td>
 447     <td valign="top"> Close application </td>
 448     <td valign="top"> rpc closeApp(ReqCloseApp) returns (RspCloseApp) {} </td>
 449     <td valign="top">
 450
 451   ```python
 452 #Add code
 453   ```
 454 </td>
 455   </tr>
 456
 457
 458   <tr>
 459     <td valign="top"> getDeviceTime </td>
 460     <td valign="top"> Get device time </td>
 461     <td valign="top"> rpc getDeviceTime(ReqGetDeviceTime) returns (RspGetDeviceTime) {}<br><br>ReqGetDeviceTime :<br>SYSTEM<br>WALLCLOCK </td>
 462     <td valign="top">
 463
 464   ```python
 465  def getDeviceTimeTest(stub):
 466     response1 = stub.getDeviceTime(ReqGetDeviceTime(type='SYSTEM'))
 467     response2 = stub.getDeviceTime(ReqGetDeviceTime(type='WALLCLOCK'))
 468     print(response1, response2)
 469
 470     return response2.timestampUTC > response1.timestampUTC;
 471   ```
 472 </td>
 473   </tr>
 474
 475
 476   <tr>
 477     <td valign="top"> sendKey </td>
 478     <td valign="top"> Generate key event </td>
 479     <td valign="top"> rpc sendKey(ReqKey) returns (RspKey) {}<br><br>ReqKey :<br>BACK<br>MENU<br>HOME<br>VOLUP<br>VOLDOWN<br>POWER<br>XF86<br>WHEELUP<br>WHEELDOWN<br> actionType :<br>STROKE<br>LONG_STROKE<br>PRESS<br>RELEASE<br>REPEAT<br><br>XF86KeyCode<br>durationMs<br>intervalMs</td>
 480     <td valign="top">
 481
 482   ```python
 483  def sendKeyTest(stub):
 484     response = stub.sendKey(ReqKey(type='WHEELUP', actionType='STROKE'))
 485     time.sleep(1)
 486
 487     response = stub.sendKey(ReqKey(type='WHEELUP', actionType='STROKE'))
 488     time.sleep(1)
 489
 490     response = stub.sendKey(ReqKey(type='WHEELDOWN', actionType='STROKE'))
 491     time.sleep(1)
 492
 493     response = stub.sendKey(ReqKey(type='POWER', actionType='STROKE'))
 494     time.sleep(1)
 495
 496     response = stub.sendKey(ReqKey(type='POWER', actionType='STROKE'))
 497     time.sleep(1)
 498
 499     response = stub.sendKey(ReqKey(type='BACK', actionType='STROKE'))
 500     time.sleep(1)
 501
 502     return True
 503   ```
 504 </td>
 505   </tr>
 506
 507
 508   <tr>
 509     <td valign="top"> takeScreenshot </td>
 510     <td valign="top"> Take screenshot </td>
 511     <td valign="top"> rpc takeScreenshot(ReqTakeScreenshot) returns (stream RspTakeScreenshot) {} </td>
 512     <td valign="top">
 513
 514   ```python
 515  def takeScreenshotTest(stub):
 516     responses = stub.takeScreenshot(ReqTakeScreenshot())
 517     image = open("screenshot.png", "wb")
 518
 519     for response in responses:
 520         image.write(response.image)
 521     image.close()
 522
 523     return True;
 524   ```
 525 </td>
 526   </tr>
 527
 528
 529   <tr>
 530     <td valign="top"> dumpObjectTree </td>
 531     <td valign="top"> Get object infomation </td>
 532     <td valign="top"> rpc dumpObjectTree(ReqDumpObjectTree) returns (RspDumpObjectTree) {} </td>
 533     <td valign="top">
 534
 535   ```python
 536  def dumpObject(stub):
 537     response = stub.findElements(ReqFindElements(textField='Bg', isShowing=True))
 538     if len(response.elements) <= 0: return False
 539
 540     for ei in response.elements:
 541         response = stub.dumpObjectTree(ReqDumpObjectTree(elementId=ei.elementId))
 542
 543     print(response.roots)
 544   ```
 545 </td>
 546   </tr>
 547
 548
 549   <tr>
 550     <td valign="top"> setFocus </td>
 551     <td valign="top"> Set focus to specific UI Object </td>
 552     <td valign="top"> rpc setFocus(ReqSetFocus) returns (RspSetFocus) {} </td>
 553     <td valign="top">
 554
 555   ```python
 556
 557 def setFocus(stub):
 558     response = stub.findElements(ReqFindElements(textField='Button'))
 559     if len(response.elements) <= 0: return False
 560
 561     stub.setFocus(ReqSetFocus(elementId=response.elements[0).elementId)
 562   ```
 563 </td>
 564   </tr>
 565
 566
 567   <tr>
 568     <td valign="top"> actionAndWaitEvent </td>
 569     <td valign="top"> Do action and wait event </td>
 570     <td valign="top"> rpc actionAndWaitEvent(ReqActionAndWaitEvent) returns (RspActionAndWaitEvent) {}<br><br>ActionType :<br>CLICK<br>KEY<br>EventType<br>  EVENT_WINDOW_ACTIVATE<br>EVENT_WINDOW_DEACTIVATE<br>EVENT_STATE_CHANGED_FOCUSED</td>
 571     <td valign="top">
 572
 573   ```python
 574 def actionAndWaitEvent(stub):
 575     stub.actionAndWaitEvent(ReqActionAndWaitEvent(type=KEY, XF86keyCode='Return',
 576                                              eventType=EVENT_STATE_CHANGE_FOCUSED, timeoutMs=1))
 577
 578     response = stub.findElements(ReqFindElements(textField='Button'))
 579     if len(response.elements) <= 0: return False
 580
 581     stub.actionAndWaitEvent(ReqActionAndWaitEvent(type=CLICK, elementId=response.elements[0).elementId,
 582                                              eventType=EVENT_WINDOW_ACTIVATE, timeoutMs=5))
 583   ```
 584 </td>
 585   </tr>
 586
 587 </table>
 588
 589 ### Running on TV
 590 - Known issue
 591
 592   grpc not support environment that uses proxy on host PC
 593
 594 - Pre-condition
 595
 596   The TV device and Host PC should be already connected through SDB
 597
 598 - Gets Aurum latest version and checkout to tizen branch
 599
 600   [Aurum Github](https://github.sec.samsung.net/tizen/aurum)
 601
 602 - Set up a python virtual environment and run UI Automation (working directory: aurum/)
 603
 604   (host) cd ui_automation/python/tv
 605
 606   #### Create virtual env
 607   (host) python3 -m venv v
 608
 609   #### Activate a virtual env
 610   Linux
 611   (host) source v/bin/activate
 612   Window
 613   (host) v/Scripts/activate.bat
 614
 615   #### Install required pkg (only once)
 616   (python_virtual) pip3 install -r ../../../protocol/resources/python/requirements.txt
 617
 618   #### Generate aurum.proto file for python (only once)
 619   (python_virtual) python3 -m grpc_tools.protoc --python_out=./ --grpc_python_out=./ -I ./../../../protocol/ ../../../protocol/aurum.proto
 620
 621   #### Target setup such as sdb forward, bootstrap execution
 622   (python_virtual) python3 ../../../protocol/resources/python/tv/tvSetup.py
 623
 624   #### Then add your script then run it (Please refer sample scripts in 'aurum/protocol/resources/python/tv')
 625   (python_virtual) python3 sampleWithUtils.py
 626
 627   #### Deactivate a virtual env
 628   (python_virtual) deactivate
 629
 630 ### Reference and Tip
 631 - [How to run aurum on TM1](https://code.sec.samsung.net/confluence/display/GFX/04.+NUITizenGallery+Test+Script+Guide)
 632 - [NUI automation test](https://github.com/nui-dali/NUIAutomationTest)
 633 - [Aurum test sciprts(samples)](https://github.sec.samsung.net/NUI/AurumTestScript)
 634 - [Automation application(C++) in target](https://github.sec.samsung.net/TizenUIFW/aurum-test-sample)
 635    * Aurum API is not public. we dont recommand direct calling aurum API in target.