Video exhibits how easily can any Windows GUI Application be automated with the help of Microsoft® VisualUIVerify + UI Automation and using QAthing library.
For more sample videos visit Youtube channel (QAthing Club).
The project, QAthing, was started as for educational purpose. Only basic or general scenarios have been addressed in the videos. For complex scenarios you may send your queries at email@example.com or visit FAQ page.
Automating any Windows application is not a big deal!, a lot of help is available over the internet.
Idea was to achieve Automation using a simple and intuitive way. In this regards a few changes have been made to VisualUIVerify (its code is available – The Tool); and the changes are made to grab features and properties of any UI Control (of any active application). In this idea, VisualUIVerify is used as a UI Control inspector accompanied by a library (QAthing.dll) which actually, with the help of KB files, imitates user activity.
Automation via this technique is like handling with building blocks where you design your own tests or scripts using Keywords. Keyword is a basic building block which defines any UI Control – it is like a contact-card (or descriptor) for any UI Control. At first, you define Keywords for the UI Controls that you want to include in your test script (or scenario), once all the Keywords are recorded, you then use them in your scripts and, with the help of QAthing.dll, you impersonate user operations, ‘Mouse Clicks’, for instance.
The code has been run through many test cases in that all the typical UI Controls have been accessed, and with those results indicating that almost any Windows application can be automated using this idea (The Tool and Library). The work that has been done so far is more than POC.
2 Step Automation
- Build KeywordBook (Phone book) file by creating Keywords (UI Controls’ contact-card) for any application – this is to let the process know about the application and its UI Controls.
- With the help of the library, QAthing.dll, access UI Control(s) using their respective Keyword(s), and imitate user operations (or actions).
UIA Verify is a Microsoft® product.
It’s a test automation framework that features the User Interface Automation Test Library (UIA Test Library) and Visual UI Automation Verify (Visual UIA Verify), the graphical user interface tool – Details.
Its source code is available at this Link.
It populates all the UI Controls in a tree form and provides their in-depth details including the supported patterns. Its tree and inspection technique is the inspiration behind this idea. The difficulty I see in automating any application is getting its UI Controls’ information. VisualUIAVerify already has this feature enabled in that it unwraps detailed information for any accessible UI Control. A few changes have been made which simply records some key information regarding UI Control, and additionally, coordinates (x and y points) and image technique (image processing) is added to access the UI Controls which do not share the standard (or any) Action patterns.
In 2-step Automation Idea, this altered Tool generates KB file. (KB stands for Keyword Book, and it holds Keywords which represent UI Controls.)
(Nothing Special) – It simply provides an interface to imitate user actions.
In 2-step Automation Idea, with the help of KB file, this libraries, QAthing.dll, sends commands to UI Controls.
There are some features that will boost the automation process. They are in pipeline, and as follows:
- Form-Level Operations – A designer (in VisualUIAVerify) to perform form-level or combined operations/actions.
Currently, there is no way to perform combined or grouped operations (e.g. form fill) that includes multiple controls (except calling them individually). To provide such feature a designer is required so that generic/form-level operations can be defined using the available Keywords. However, as demonstrated in the videos, Groups can be defined in KB file, and group or combined operations can be achieved via a Custom DLL.
- Python shell integration (to VisualUIAVerify) to execute/verify Form-level operations.