OD(Object Detection)

Screen analysis tool of a primary tool.

The AI engine, which has learned data such as icons, components, and screen areas commonly used in mobile apps, analyzes a given screen and shows 15 types of available screen elements.

The types of screen elements recognized by the AI engine are as follows.

  • Home Icon

  • Back Button

  • HamburgerMenu

  • Search Icon

  • Clickable Object

  • AD Icon

  • More Icon

  • Spinner Icon

  • SearchArea

  • Swipe Icon

  • EditText Area

  • Keyboard

  • Tabs

  • Close Icon

  • Switch

  • Thirdpartykeyboard

OD Result Inspector

Specific information about the screen elements recognized by AI as Object Detection can be found by clicking "Inspector" at the bottom of the screen analysis panel.

Click "Inspector" to display the recognized screen elements in a table format.

  1. LABEL: Indicates the type of screen element. If it is not of a particular type, it is often marked as "Clickable", which is a touchable element.

  2. TEXT: If the screen element contains a string, OCR recognizes and outputs the string.

  3. BOX: On-screen location information (coordinates) of the screen element is output.

Clicking an item in the Inspector table activates the yellow portion of the screen analysis results. Conversely, when you click a specific screen element in the screen analysis results, the items in the Inspector table become yellow.

OD Screen Elements Attributes

You can set various OD properties by clicking on the screen element in the step using OD.

Refer to the link below for instructions on how to access the OD screen element attributes.

To check and modify screen element properties

  1. Text: You can set comparison conditions when testing with character information on screen elements.

    • = : Check that the text and value of the screen element are the same.

    • *= : Checks whether the value set to value is included in the text of the screen element.

    • ^= : Check that the value set to value matches the beginning of the text on the screen element.

    • $= : Check that the value set to value matches the end of the text on the screen element.

    • match: Use regular expressions. Check that the text of the screen element matches the regular expression.

    Reference: https://docs.python.org/3/library/re.html#re.match

    • search: Use regular expressions. Check that the text of the screen element contains a regular expression.

    Reference: https://docs.python.org/3/library/re.html#re.search

    • not used: Does not use Text information as a comparison.

  2. Text Similarity(threshold):

    Indicates string similarity (available only when comparison condition is '=') You can set how identical the character information must be at the time of scenario creation and at the time of test execution to pass through. Due to the nature of OCR, recognition of the same string may result in some misrecognition depending on the resolution and other environments, in which case similarity can be adjusted. The higher the value, the higher the comparison accuracy. - Default value: 0.8 - Minimum value: 0 - Maximum value: 1

  3. Case Sensitive: For alphabetic characters, you can set case sensitive.

  4. Selector: Specifies which element to use when multiple elements match the screen element during the test. It can be set to an integer value other than zero. (Note: If you enter '-1', use the last element.)

PreviousScreen Analysis ToolsNextOCR(Optical Character Recognition)

Last updated