Introduction

Welcome to the exciting world of software testing! If you’re a beginner contributor looking to delve into the realm of end-to-end testing for GNOME OS, you’ve come to the right place. In this post, I will walk you through the process of implementing end-to-end tests using a powerful open-source testing tool called openQA .This is my Outreachy project and am still on the journey as I write this.

What is openQA?

Simply openQA is an automated test tool for operating systems and the applications they run. It allows you to simulate a user’s interaction with your application and ensure that the entire application, including its user interface, works as expected. This is particularly useful for a complex environment like GNOME OS, where ensuring a smooth user experience is crucial.I promise you i still didn’t understand this at first until later in the process!

Step 1: Understanding the Basics of GNOME OS

Before diving into testing, it’s important to have a fundamental understanding of GNOME OS. GNOME OS is not a standalone operating system; rather, it’s a reference system for the GNOME desktop environment. It’s used by developers and testers to ensure that the latest codebase is functioning as expected.For this particular project I wondered how i was supposed to install it and realised i need a virtual environment to so this.Unfortunately i had an IOS system that was not supported to install it on real hardware even when i tried to install the GNOME OS using UTM for IOS.I was later advised to use Boxes from flathub.

Step 2: Setting up the environment

1. You must have the right hardware running on Modern Linux-based OS such as Fedora or Ubuntu At least 20GB of disk space free At least 4GB of RAM free Support for x86_64 hardware virtualization.
2. Install Boxes , download and install the GNOME OS installer and run the initial setup on there.
3. Follow steps on the README guide that also has a contributing guide to make your first contribution.Another major highlight not to forget is to enable KVM follow the steps here on Virtualization here.Without this you will not be able to proceed because when you run the end-to-end tests, openQA creates a x86_64 virtual machine using QEMU and KVM .
4. Don’t forget this part was really the most challenging to get environment ready so first take it seriously and follow the attached links and incase of any issues you can reach out to the GNOME OS community here , you might probably find me there too.

Step 3: Writing Test Scripts

openQA tests are written in Perl. Don’t worry if you’re not familiar with Perl; basic scripting skills and a willingness to learn are enough to get started.

1. Understand the API : Familiarize yourself with the openQA API. The openQA documentation is a great resource that highlights the TEST API showing you all methods exposed by the os-autoinst backend to be used within tests that you will be writing.
2. Start Simple : Begin with a basic test, now in the beginning all i know is i dived deep into documention and I was worried on how to start but this CONTRIBUTING GUIDE helped me narrow down on how to start contributing.In my case i started with adding a test in settings app extending to Search as i progressed slowly to more advanced tests like the one am currently writing, Using the Gnome On Screen Keyboard test. see a11y_screen_keyboard.pm file screenshot below.refer to line 13 in the file.
3. Needles :You will create screenshots for several steps simulating user actions taken called needles and you will reference them as you follow the contributing guide here .Line 13 above refers to the needle files a11y_typing.png and json files attached.The png highlights the screenshot showing the typing field and json file shows the exact coordinates of that typing field.

Step 4: Running Your Tests

Once you’ve written your tests, it’s time to run them:

1. Make sure you have the latest Podman as highlighted in the README.md file .
2. Ssams openqa tool:This is a small command line helper for working with the test tool OpenQA. This is what we used for working with GNOME’s OpenQA tests.This will help you to run the test.

Step 5: Analyzing Test Results

After the tests have run, analyze the results:

1. Review Output : openQA provides screenshots and videos of each step of your test. Review these to understand what happened during the test run. Use save_screenshot; to capture screenshots ,this is highlighted in the TEST API highlighted above and check the output directory for test results ,logs and the .ogv video.
2. Debug Failures : If a test fails, use the output to debug. It could be an issue with the test script highlighted in “_isotovideo.stderr.log” or an actual bug in GNOME OS.

Step 6: Iterating on Your Tests

1. Refine Scripts : Based on your test results, refine your scripts to cover more scenarios or improve reliability.
2. Continuous Learning : Keep learning more about openQA Test API and how to extend more useful tests.

Conclusion

Implementing end-to-end tests with openQA for GNOME OS might seem daunting at first, but with patience and practice, it becomes an invaluable part of the development process. Your contribution will not only enhance the quality of GNOME OS but also give you a strong foundation in software testing. Happy testing!

Remember, this is a journey of continuous learning and improvement. Don’t hesitate to seek help from the GNOME and openQA communities – they are incredibly supportive and a treasure trove of knowledge. Good luck!