Image recognition

RE: Image recognition

gopalt0943 — Thu, 23 Nov 2023 10:23:48 GMT

Variable Setup:
- Create a variable named "calculator result."
Open Calculator Application:
- In the "Open Application" activity:
  - Set the executable name to "calc.exe."
  - Use "Find by Window Title" and set the expected window title to "Calculator."
Create a Support Image:
- Manually open the calculator in the Appian RPA agent.
- From the Appian RPA agent menu in the taskbar, select "Create Support Image."
Support Image Marking:
- Mark the new image for the number 8, save the image, rename it, and click "Save Image."
- Repeat this process for the "+" icon, saving images accordingly.
- Similarly, save images for values you know, like 5 and "=".
Configure Windows:
- For the "Eight" configuration window:
  - Select "Full Screen" in the search section.
  - Set the max wait time to 2 seconds.
  - Choose "Exact Match" for pixel and color tolerance.
  - Enable mouse actions.
  - For the location image, select "Throw Exception if Not Found."
  - After completion, set "Wait Before Executing Next Actions" to 4 seconds.
- For the "Plus" configuration window:
  - Set "Don't Wait" for wait time.
  - Choose "Exact Match" for pixel and color tolerance.
  - Enable mouse actions.
  - For the location image, select "Throw Exception if Not Found."
- Repeat the same configuration process for the "Five" value.
Keyboard Shortcuts:
- Use the "Press Keyboard Shortcuts" activity.
  - Change the display name to "Copy the Result."
  - In modifier keys, select "Ctrl."
  - In additional keys, select "Characters" and provide "c."
  - In interaction, select "Press and Release Keys."
Get Clipboard Value:
- Use the "Get Clipboard Value" activity.
  - In the configuration output section:
    - Select "Is Stored As" in the dropdown.
    - In the target, provide the variable created, i.e., "calculator result."
    - After completion, select the "Take a screenshot" option.

Follow these steps carefully, and you should obtain the correct result.
Follow these screenshots for a better understanding.

RE: Image recognition

saikiranb654094 — Wed, 22 Nov 2023 16:22:40 GMT

You have to take the screenshot clearly then only bot will reconize the image correctly

RE: Image recognition

Aparajita Singh — Mon, 30 Aug 2021 15:16:44 GMT

Hi Gavin, Have you tried searching for other kinds of images? Also, are you searching full screen or part of the screen? I have been facing issue in my bot too. I have tried with small and big images. I think I would also have to test every possible way to see what works.

RE: Image recognition

Gavin Hume — Thu, 12 Aug 2021 09:27:34 GMT

I finally managed to get this to work by keeping the Pixel Tolerance when locating the images to Exact Match but setting the Color Tolerance to Medium (5%) for the numbers and Very High (15%) for the operators. I have no idea why that might have been necessary as the images were captured on the same resource as the process was run on.

RE: Image recognition

Gavin Hume — Wed, 11 Aug 2021 10:10:42 GMT

The Execution Log just shows

[WARN] 2021-08-11 10:07:50 05s (0h) - Image not found

But the screenshot taken before the steps clearly shows that Calculator is open and visible (I added a step to show the desktop before starting Calculator)

RE: Image recognition

Gavin Hume — Wed, 11 Aug 2021 09:40:57 GMT

I have previously built a POC for Image Recognition where it searched for a shortcut on my desktop and opened the document and that worked fine (even if I moved the shortcut) and continues to work but I was trying to complete all the RPA tutorials to make sure I hadn't missed anything important

RE: Image recognition

Gavin Hume — Wed, 11 Aug 2021 09:38:33 GMT

I'm having a similar issue, I've really been struggling to get the RPA process to recognise the buttons based on the support images. I've tried changing the pixel precision and sometimes it recognises 8 and 5 but then uses the divide button instead of the add button. I've tried zooming in when creating the support images as well as including a reasonable border around them but nothing seems to work consistently

RE: Image recognition

amrutah0001 — Thu, 08 Apr 2021 04:11:01 GMT

It is regarding image recognition I store image of 8,+,5,= so as per pdf it should be excused in calculator and answer would be 13 but it is not executing and that why I am not able to complete this lesson

RE: Image recognition

Danny Verb — Wed, 07 Apr 2021 17:58:48 GMT

To help, you'll need to provide some more details. What step are you stuck on and what are you expecting?