Are you a “software tester” or a “mobile tester?”  If you identify with the latter and let’s face it, most of the testing nowadays is focused on mobile app development, then you need to stop thinking “I’m only a software tester.”  As software testers we’ve long focused on the software only, meaning the functionality of the software and not really considering a systems perspective.  But with mobile apps, depending on the architecture of said mobile app, a full systems integration approach gives a more complete test coverage approach.

In my career, one hard lesson I have learned is how the inter-dependencies affect software behavior.  Those inter-dependencies include hardware limitations, firmware or operating system functionality or a combination.  The most complex type of testing is the combination of hardware and firmware conditions which directly or indirectly affect the app’s behavior.  A decade ago, I discovered the charging of the battery in various states of charge affects the functionality of a native app, a medical device.  Since then, I’ve seen other, commercial, non-proprietary apps which rely on various states of the hardware or operating system but are not tested as combined states.   

Testing states of the device and app are important but the tester also must understand when the functionality is supposed to behave based on the state.  Adding the sequence of when the functionality is supposed to behave and taking into account the conditions or combination of conditions can be complex and quite overwhelming.  Due to the complexity, a smart, well-planned test strategy is vital to give high test coverage for any mobile app.  And that planned out test strategy needs to incorporate systems thinking. 

Now, what do I mean by systems thinking in mobile test development?  The best way to describe systems thinking for mobile tests is through examples.  But first, the tester and/or test manager should consider these questions:  the who, the how, the where and the when of usage of the mobile app on the mobile device.  Mobile testers need to ask these questions when planning out your regression and new testing.  I’ve often spoken about “testing beyond the GUI” but I have not given any specific examples.  Tests should include system boundary tests based on who will use the app, how the app will be used, where the app is used and especially when at any given point of usage.  The sequence of functionality is not seriously addressed by many testers and often times miss uncomfortable states of the app.  

Another example of a system level test to add to your test strategy is performance.  Now, most testers think of performance as speed or “how fast can I connect.”  That is indeed one test along with how responsive the app is to the commands a user inflicts on the app.  But let’s not forget performance also includes load, stress, and endurance to name a few concepts of tests.  So to illustrate, an example of a use of a mobile app while on an airplane along with many other passengers also trying to access the Internet.  Mobile testers need to not only consider the different functions of the app while accessing the Internet but also more than 100 other passengers trying to access Inflight Internet.  Can the app respond well to the commands based on the load in such a context?    Now, has the development team defined the boundaries or limitations regarding the load on a network and how the app will respond?  What if the app’s limit is degrading to the point of being unusable when 100 other users are trying to access the same network?  Even the use of different routers does not seem to help improve the performance speed.  Knowing how the app will be used, where the app will be used, by whom the app will be used and when can help to formulate your test strategy.   

Let’s further discuss the app on an airplane example. Ask yourself, if you are trying to connect, seeing others connected and streaming content and you try to do the same but the plane hits poor weather, kicking everyone off the network. Then all the passengers are all trying to reconnect to the network at the same time, how does your app handle the load?  How does your app handle the interruption?  Does your app give you a notification to inform you the network is again available?  How responsive is the app once reconnected?  Can you, access the content where you left off before losing connectivity?  Sometimes, an app cannot reconnect because it cannot get a unique MAC address ID which means firmware testing of the router the app is trying to connect.   

Mobile apps which rely on the cache to respond quickly to commands may create problems unforeseen by the testers.  Going back to the example of losing connectivity while on a plane, the app stored the location of where the user was while streaming content.  Once connectivity was reestablished, tests involving the state of the app, the state of the device is maintained without errors, meaning the content continues streaming seamlessly.  But reconnection causes the app to restart.  This may involve loss of battery charge where the user is able to recharge the device and how long of an interruption might affect the app’s behavior.  Tests involve what state can the app be in where the cache is utilized or not.  Testers need to know how important cache retention is to the user at any given point of the app’s usage.  

Mobile tests can be incredibly complex.  Mobile testers need to think “system” and not just focus on software testing.  If the planning is done, automated tests can be created but keep in mind whether your tests are a one-time test or repeatable test.  It’s not cost-effective to create automation tests for one-time tests, and sometimes it’s just faster to perform the test manually.  This is why it’s vital to understand the who, the how, the where and the when of your app’s usage. 

Author Bio