Help files

Move Mouse to Text on Screen (OCR) Action

This Action allows you to move the mouse over a specific image some specific text [2] found on screen or a foreground window [5] using an OCR Engine of your choice [1]. If this text is found more than once on that surface, you can target a specific occurrence [4] (2nd, 3rd, etc.) given the fact that the OCR is adding these matches first to last, top to bottom, left to right. Just like in other Actions that use OCR, you can work with a variety of options that allow you to search on very specific subregions of your screen [6], but the Action also gives you the option to select the speed with which the mouse [7] will be moved on the spot.

The Action returns co-ordinates [8] of where the text is found relative to the top left of your screen or the top left corner of the window you are working with. Along with the position of the text, it returns the width and height of that image in two [9] corresponding Variables.    


wc.png OCR Engine:

This text field with drop-down menu options invites you to enter or choose the instance of the OCR Engine you want to work with.

ea.png Text to Find:

This text field allows you to enter the text you want the OCR Engine to search for as text or as a Variable.

ea3.png Is Regular Expression:

Check this checkbox if you want to use a Regular Expression to find the text on the screen. A Regular Expression creates a range of possibilities and can return a number of results that match your search. The order that the Engine is adding these matches is from top to bottom, left to right as it finds them on target!

ea4.png Occurrence:

This positive integer should point to the match that interests you, if your search returns more than one occurrences or a collection of values (through a regular expression for example).

ea5.png Search for Text on:

Two options on this drop-down menu, Entire Screen or Foreground Window only.

ea6.png Search Mode:

While the first option of this drop-down menu is pretty straightforward (Search Whole Screen or Foreground Window) the other two options that allow you to target a specific subregion of your window or screen produce a whole different set of additional Properties which will see in detail here:

-Search on specified subregion of Screen or Foreground window:


By clicking the "Select a Different Subregion" button on the right of the red box, you will be prompted to select firstly the window you want to work:


and then you will be given the opportunity through an amazing interactive feature to actually select the area you want to narrow down your scan (drag your mouse to select the area). The X1 X2 Y1 Y2 coordinates are denoting the start and finish of your dragging maneuver.

However, using the  "Select a Different Subregion" button is not compulsory. You can enter the values of the coordinates by hand or as a Variable (notice the Gear Icon next to the text fields).

-Subregion relative to image:


This search mode enables you to narrow down your scan through a region that is defined relative to the top-left corner of an image from the Image Repository.

ea7.png Move Mouse From Previous Position:

This drop-down menu gives you a number of options in case the speed with which the mouse will move is critical to your process.

ea8.png Location of Text Found (co-ordinates):

It is important that you understand that OCR understands text as an Image. These coordinates pinpoint the position of that image on the surface (screen or foreground window) you are currently working with.

ea9.png Location of Text Found (dimensions):

For example, consider the following Print Screen:


If we run the process shown on Robot Designer, which is designed to move the mouse on the center of the image text "TeXt", the co-ordinates we will get for that image are (1, 336). This makes sense since the "TeXt" text on Word is positioned at the very left (%LocationOfTextFoundX% = 1) of our screen!

The width (145) and height (52) values also make sense, accurately describing the image the OCR Engine has found.