Multi-device eLearning Testing with Tools and Simulators

A while back, I, along with my colleague Geera, presented a webinar – ‘Multi-device Testing: Challenges, Considerations and Solutions’. As a preparation for this webinar and, in general, being a part of the Quality Assurance team, we tried some tools and simulators to test multi-device eLearning to study to what extent they could help. Below is a quick summary of our experiences for some of the key ones.

Please note: These tools and simulators have been explored with a specific focus on their capability to test multi-device eLearning, and not websites or web applications.

1. Synchronous Testing Tools

Synchronised testing tools help in performing the same interaction automatically across a number of devices and browsers at the same time.

a. GhostLab

Ghostlab - Synchronous Multi-device Testing Tool

GhostLab is a tool that allows testers to wirelessly pair multiple iOS, Android, Windows, and BlackBerry devices to their computer and carry out synchronous testing. GhostLab involves concurrent testing on actual physical devices and configurations, and you can check real user experiences. Changes (e.g. changing a page or displaying content on clicking a button) can be simultaneously viewed on all connected devices.

Advantages

  • Allows us to check real user experiences
  • Results of change can be seen simultaneously
  • Visuals and alignment can be checked
  • Joint reviews can be conducted
  • Pre-release verification can be conducted
  • Allows you to immediately check code changes
  • Can be helpful for testing text-based courses

Below is an image of multi-device testing in progress using GhostLab.

Multi-device Testing Using Ghostlab

b. Adobe Edge Inspect

Adobe Edge Inspect - Synchronous Multi-device Testing Tool

Adobe Edge Inspect is another tool that displays synchronised output on physical devices. It allows testing on iOS and Android mobile devices, as well as Kindle Fire. With Adobe Edge, you need to install the main application on a desktop computer, and launch the course through a Chrome browser. You can then access the synchronized course through the Edge Inspect app on connected devices.

Advantages

  • Launches courses through the Chrome browser on the main desktop and through the Edge Inspect app on connected devices
  • Loads HTML pages
  • Allows synchronous control of course-level navigation and page-level interactivities for courses structured to have separate URLs for pages/content shown on click

Below is a snapshot of multi-device testing in progress using Adobe Edge Inspect.

Multi-device Testing Using Adobe Edge Inspect

Synchronous Testing tools do provide a multi-device testing interface, but it’s difficult to simultaneously keep one’s eyes and ears on so many devices!

Limitations of Synchronous Testing tools for eLearning

  • Difficult to simultaneously keep our eyes on so many devices!
  • Difficult – or pretty much impossible – to check audio synching on multiple devices simultaneously!
  • May need to do course-level navigation and page-level interactivities manually on connected devices
  • May not be able to check browser-related issues depending on how courses launch
2. Simulators

Simulators create a virtual environment or object which can be a physical system or an object. Simulators come with both the hardware and the software which runs on it to create the real experience.

a. BrowserStack

BrowserStack - Cloud-based Simulator

BrowserStack is a cloud-based simulator that we found to be useful for checking text alignment on a specific device-OS-browser combination and catching visual issues.

In case of cloud-based simulators, content loading, as well as speed and smoothness of animations, can vary based on the connection, which may lead to inaccurate issues being logged. Remote simulators can also cause some difficulties when it comes to testing interactivities, as option can’t be selected unless content is fully and properly loaded on the remote machine.

b. SDK Simulators

SDK simulators, which are local simulators, can also be helpful for testing some parameters. With the iOS and Windows simulators, we could check content, alignment, and layouts quite reliably. However, we did encounter some inaccuracies with the Android emulator.

After trying some simulators for testing eLearning, we realised we cannot ignore the fact that actual learner experience cannot be completely checked, since physical devices are not being used – so there can be inaccuracies and performance mismatches based on the configuration and resolution of the machine running the simulator.

Limitations of Simulators for testing eLearning

  • Cannot test actual learner experience
  • Cannot check real-time physical device aspects, i.e. battery life and charging, and actual bandwidth speed
  • Cannot check content/functionality dependent on physical device features
  • Cannot check gestural navigation (custom interactivities)
  • Inaccuracies and performance mismatches based on simulator machine configuration and resolution

While tools and simulators help in testing, they just aren’t quite the real thing. As mentioned in an earlier blog post – The Proof Of The Pudding Is In Multi-device ‘Physical’ Testing, there is no alternative to testing on physical (real) devices. Physical ‘real’ devices are an absolute must for you to experience your eLearning exactly as your users will.

Write a Comment

Leave a Reply

Your email address will not be published.

GET INSIGHTS AND LEARNING DELIGHTS STRAIGHT TO YOUR INBOX, SUBSCRIBE TO UPSIDE LEARNING BLOG.

    Published on

    Don't forget to share this post!

    WANT TO FIND OUT HOW OUR SOLUTIONS CAN IMPACT
    YOUR ORGANISATION?
    CLICK HERE TO GET IN TOUCH