Fundstück der Woche – MOVR report

2014-11-28 21_59_43-Mobile-Overview-usw-MOVR_2014_q3ScientiaMobile uses its WURFL products to collect and analyze the device intelligence contained in the MOVR report. WURFL is
a Device Detection Repository (DDR) that integrates an API and XML to provide an always-updated source for detecting devices and their capabilities. For more than 10 years, WURFL has been the industry standard for device detection. Today, ScientiaMobile offers a number of WURFL products to suit a range of needs, from small developers to large enterprises.

ScientiaMobile has published MOVR to provide the mobile Web community with timely information on mobile Web device usage.


The Mobile Test Pyramid (Guestpost by Daniel Knott)

Anyone who is involved in software testing and software test automation should know the test automation pyramid introduced by Mike Cohn.

As you can see in the following image, the typical pyramid consists of three layers. At the bottom, there is the automated unit-testing layer, in the middle the automated integration testing layer and at the top there is the automated end-to-end testing layer (including the user interface tests). Each layer has a different size, indicating the number of tests that should be written within each stage. Manual testing is not part of the test pyramid, hence it is shown as a cloud for additional testing work.


But this pyramid is not applicable to mobile apps and mobile test automation.
Mobile testing requires a totally different set of testing activities like movement, sensors, different devices and networks compared to other software like desktop or web applications. Lots of manual testing is required to be sure that a mobile app is working as expected in the different usage scenarios.

Mobile test automation tools are not currently as mature as their counterparts for web and desktop applications, which leads to a flipped test automation pyramid. As the tools become increasingly mature, this pyramid is likely to flip back again because the default test automation pyramid is based on a more stable foundation (see the pyramid image below).

The default pyramid therefore can’t be used as an indicator for test automation and manual testing in the mobile world.

The flipped testing pyramid looks like this:


In this version of the pyramid, the automated unit testing layer is the smallest one. This is the case because not every unit or method of mobile apps can be tested in an isolated manner. In some cases, different APIs, layers or systems may need to be faked or mocked in order to get the small unit to work. This is also the case for every other software application, but in some cases mocking or faking other systems for mobile apps is much more complex. This is not efficient from a technical or economic point of view. However, it’s no excuse for not writing mobile unit tests at all. The business logic of an app must be tested at unit level.

The next stage is the end-to-end test automation layer. Within this layer, the whole app is tested from a user perspective. It will be tested to make sure the whole system is working, from the app’s user interface through to the backend system via a wireless network, including integration testing with different libraries or APIs. The integration testing layer is therefore part of the end-to-end layer.

The biggest change in this pyramid is that manual testing is part of it. Mobile testing requires lots of manual testing, and this can’t yet be replaced by test automation or any other tools.

Nevertheless, mobile test automation is a really important topic and every mobile tester should be able to write automated regression tests that provide fast feedback about the current quality state of an app. Furthermore, test automation helps the team build a reliable and robust mobile app that makes the customers happy.

The Mobile Test Pyramid

The flipped testing pyramid has no stable foundation and mobile testing requires lots of manual testing, which is why I created my own mobile test pyramid consisting of four layers including manual and automated steps. The biggest layer of the pyramid is manual testing and forms the strong foundation for every mobile app project, followed by end-to-end testing, beta testing and a top layer comprising unit testing. The grey parts of the pyramid indicate the automated steps and the white parts are the manual testing steps. The beta-testing layer is new to the pyramid but essential to every mobile app project. Keeping the high expectations of the mobile users in mind requires this layer to be part of every mobile project to get early feedback from your mobile customers. You can either use a crowd testing approach for your beta testing or you can ask your colleagues to beta test early versions of your app to provide important feedback.


Keep the problem with the flipped pyramid in mind and use the mobile test pyramid in your project to have a good mix of manual and automated testing. I’ve used the pyramid in several projects and it has helped me set up a reliable, effective and valuable testing process.

This article is an excerpt from my book „Hands-On Mobile App Testing“ which is available at Hands-On Mobile App Testing.

About the author

Daniel Knott has been working in the field of software development and software testing since 2003. He started his career as a trainee at IBM where he was involved in enterprise software development and testing. After his time at IBM, Daniel studied computer science at the University of Applied Sciences in Wiesbaden, Germany. Software testing soon became a passion during his time at university and the reason why he chose a career in software testing. Daniel has worked at several companies in various different industries where he was responsible for testing web, desktop and mobile applications. During a number of projects he developed fully automated testing frameworks for Android, iOS and web applications.

Daniel is a well-known mobile expert, speaker at various conferences in Europe, and a blog author. Furthermore, Daniel is the founder and organizer of a local software testing user group in central Germany. If you want to learn more about mobile testing please have a look at his book, Hands-On Mobile App Testing, or get in touch via LinkedInXing,Twitter or email.

Virtualisierung von Testressourcen bei der App-Entwicklung

2014-11-19 17_27_03-TestObject - Test more. Worry less.Die zunehmende Komplexität von Business-Anwendungen und die Diversität der Endgeräte macht es Organisationen schwierig, notwendige Testumgebungen, in Time, in Scope, in Budget aufzubauen und zu pflegen.

Die Herausforderung von Unternehmen liegt unter anderem darin, dass mobile Apps und Webseiten auf allen auf dem Markt befindlichen Endgeräten, Browsern, Betriebssystemen und Softwareversionen einwandfrei laufen und dem Nutzer eine ansprechende Usability bieten müssen.

Aus diesem Grund muss sich jeder QA-Manager folgende Fragen stellen:

  • Wie stelle ich sicher, dass ich alle notwendigen Kombinationen an mobilen Testumgebungen (Hardware/Software) zur Verfügung stelle und auf neue Anforderungen schnell reagieren kann?
  • Wie stelle ich sicher, dass diese Testumgebungen hoch verfügbar sind und neben dem manuellen Testen auch die eingesetzten Testautomationswerkzeuge unterstützen?
  • Wie organisiere ich Usabilitytests für meine Anwendungen, dessen Zielgruppe heterogene Skill-Profile, unterschiedliche berufliche Hintergründe und verschiedenartiges kulturelles Nutzerverhalten aufweisen muss?

3 Fragen, 3 Antworten: Virtualisierung der Testressourcen, Virtualisierung der Testressourcen, Virtualisierung der Testressourcen.

Virtualisieren Sie durch CrowdTesting, virtualisieren Sie Ihre mobilen Devices/Testumgebungen durch Cloudanbieter.

Virtualisierung von Testressourcen in der Cloud bietet niedrige Kosten, eine Reduzierung der Investitionen, ein On-Demand-Zahlunsmodell, eine reduzierte Wartung, ein höheres Maß an Effizienz und vor allem einen schnelleren Markteintritt für wichtige Business Anwendungen.

Dieser Vortrag auf den Software Quality Dyas 2015 zeigt an Hand eines Erfahrungsberichtes wie Jumio Inc. diese Anforderung durch Virtualisierung von mobile Devices/Testservices in der Cloud gelöst hat.

Re-Blog: Testing JavaScript on Various Platforms (iPhone 7.1 on OS X 10.9) with Karma and Sauce Labs

2014-11-07 13_14_37-Testing JavaScript on various platforms with Karma and SauceLabs - codecentric BIm Codecentric Blog hat Ben Ripkens einen tollen Artikel zum Thema Crossbrowsertesten (incl. iPhone) mit Karma und Sauce Labs geschrieben.

[… Testing JavaScript on various platforms with Karma and SauceLabs

The web can be a brutal environment with various combinations of browsers and operating systems (platforms). It is quite likely that your continuous integration setup only covers a small portion of your users’ platforms. The unfortunate truth is that testing on these platforms is necessary to accommodate for compliance differences and partial support of standards and technologies like HTML, JavaScript, CSS and network protocols.

SauceLabs is a service that can be used to test web applications by simulating user behaviour or executing JavaScript unit tests. SauceLabs eases testing by supporting 442 different platform combinations and additionally recording the test execution. This means that you can actually see what a user might have seen and therefore trace errors easily.

For the purpose of this blog post we will cover the following setup:

  • Sources and tests written using CommonJS modules. Modules are transpiled for the web using Browserify.
  • Mocha as our test framework.
  • Local test execution in the headless PhantomJS browser.
  • Testing against the following platforms in our CI environment:
    • Chrome 35 on Windows 7,
    • Firefox 30 on an arbitrary operating system,
    • iPhone 7.1 on OS X 10.9 and
    • Internet Explorer on Windows 8.1.

You can follow along this blog post or check out a demo setup that you can find on GitHub. The repository contains a list of all dependencies, the various configuration files, commonJS based tests and a Travis CI setup.

… ]

Hier geht’s zum gesamten Post im Codecentric-Blog.