Eric TerryThe Chairwoman of the Comrade Cola Company is looking to the future and her legacy. She wants to move into the augmented and virtual reality spaces. That is both fantastic and terrifying news. Fantastic in the sense of using new technology to stay relevant. Our developers have their marching orders. Terrifying because we don’t have a solid plan to test – or do we?
Functional testing aside, we may be able to lean on other tried-and-true methods and techniques. Let’s take a look at some common areas to look at.
Performance testing
Performance testing is an important aspect of developing and deploying augmented reality (AR) and virtual reality (VR) applications. These applications place significant demands on hardware and software. It is important to ensure that they can deliver a smooth and immersive experience to users.
This testing involves evaluating the performance of an AR or VR application under various conditions, such as different hardware configurations and network conditions. This helps developers identify and address potential issues that could impact the user experience, such as slow loading times, lag, or jitter.
A performance tests answer questions like:
- What is the anticipated load? How does your solution perform under the load?
- Can the user load be sustained over a long period of time?
- What do peaks and surges do to performance?
- How does it manage data processing during high traffic?
- Behavior in changing and low network conditions.
Usability testing
Usability testing involves evaluating how well users can perform specific tasks with an AR or VR application. This is done using a variety of methods, such as user interviews, focus groups, and usability studies. The purpose is to gather feedback on the user experience and identify areas for improvement.
During usability testing, testers should focus on aspects such as the user interface (UI) design, navigation, and overall usability. They should also consider the context in which the application will be used, including the user’s level of expertise and the environment in which it will be used. This is one of the more subjective methods of testing. However, scorecards along with guidance can help add objective metrics.
Usability testing will provide answers to questions like:
- How intuitive is the product?
- How accurate is the user’s locomotion?
- Does it cause any discomfort or sickness?
- How does the user feel about using the product?
Audio and video quality testing
AR and VR applications rely heavily on audio and visual elements to create immersive (more on that later) and engaging experiences for users, so it is important to ensure that these elements are of high quality and function as intended.
Audio testing involves evaluating the quality and clarity of the audio in an AR or VR application. This should include evaluating the balance between different audio elements, such as music, dialogue, and sound effects, as well as testing the application under different volume levels and with different types of audio hardware.
Visual testing involves evaluating the visual quality and performance of an AR or VR application. This includes evaluating the resolution, frame rate, and overall visual fidelity of the application, as well as testing the application under different lighting conditions and on different display devices.
It will be impossible to test on every configuration out there. Start with devices used by your target audience.
Audio and video quality testing will give you insight into:
- Overall end user experience
- Audio quality
- Video quality
- Resource utilization
Hardware testing
Hardware testing involves evaluating the compatibility and performance of an AR or VR application on different hardware configurations. This includes testing the application on different types of devices, such as smartphones, tablets, and dedicated VR headsets, as well as testing the application on devices with different hardware specifications. Different processors, graphics cards, and memory configurations play an important role here just as they do in other software testing.
Testing can be performed using a variety of tools and techniques. A combination of device emulation, simulation, and real-world testing should be utilized.
This type of testing identifies critical flaws that can obstruct the user experience and gives answers to questions like:
- Does it work properly on common configurations?
- How much space does the user need to use the product?
- Is it compatible with other devices and software?
Immersive testing
Immersive testing involves evaluating the overall immersive experience of an AR or VR application. This includes evaluating the quality and believability of the virtual environment, the level of immersion and presence the application creates for the user, and the overall enjoyment of the experience. Remember, even a cartoonish environment can be immersive.
Much like usability testing, immersive testing can be performed using a variety of methods, such as user interviews and focus groups. These methods allow testers to gather feedback from users on their experience with the application and identify areas for improvement.
Immersive testing gives you a better understanding of things like:
- Does it cause motion sickness or headaches?
- Does it cause discomfort or user fatigue?
- How immersed is the user?
Security testing
Applications access sensitive data and perform sensitive actions, AR and VR apps are no different. It is important to ensure that they are secure and protect the privacy of users.
Security testing involves evaluating the security of an AR or VR application and identifying potential vulnerabilities that could be exploited by attackers. This can include testing the application’s authentication and access controls, as well as testing for vulnerabilities such as injection attacks, denial of service (DOS), and insecure data storage.
Penetration testing, code reviews, and static analysis security testing are common techniques for security testing. Penetration testing involves simulating an attack on the application to identify vulnerabilities, while code review involves manually reviewing the application’s code for potential vulnerabilities. Static analysis security testing involves using software to analyze the application’s code for potential vulnerabilities.
Performing security testing on your AR/VR product will answer questions like:
- Is user data safe from bad actors?
- Is the product shielded from unwanted and invisible intruders?
- Is the infrastructure safe from attack?
Accessibility testing
These types of applications can potentially be used by a wide range of users, and it is important to ensure that they are accessible and usable by all users. Accessibility testing involves evaluating the usability of an AR or VR application for users with disabilities. This can include testing the application for compliance with accessibility guidelines. The Web Content Accessibility Guidelines (WCAG) for web-based applications may be a good starting place.
A variety of testing strategies and tactics like automated testing, manual testing, and user testing with individuals with disabilities should be explored .
It answers accessibility questions like:
- Can the product be used by people with different capabilities?
- Does the product cause discomfort to specific groups of people?
- Does the product offer an alternative to motion controls?
- Is there a mix of audio and visual cues?
This looks like a great list of areas to test. Time to dig through those requirements and start planning.
Eric Terry
