Mobile App Test Guideline

Test Approach

  1.  Cross Platform Testing
  2.  Functional Testing
  3.  UI  Testing
  4.  Accessibility Testing
  5.  External Factors Testing
  6.  Performance Testing
  7.  Security Testing
  8.  Automation Regression Testing

 

  1. Cross Platform Testing and Device Selection
  • Primary Device with Primary OS -Most frequently used and physical device
  • Secondary Device – Less frequently used by target audience and cloud based real device
  • Test on Emulator/Simulator– does not support switching network connection, taking pic or video stream storage and installation issue etc                  
  • Cloud Based/Inhouse Test
  1. Functional Testing
  • Manual System Testing and Integration Testing
  • Installation Testing, Upgrades, User account create/edit, configure settings,
  • integration with camera/microphone/speaker, handling error messages etc
  1. UI  Testing
  • Style, alignment and overall theme
  • Responsiveness
  • Font and Menu and buttons size, color
  • Compare the style with widely used pre-installed app
  • Changing Screen Orientation and resolution –smallest to largest screen, rotate the device from portrait to landscape etc
  • Multi-touch, single-touch, long-touch
  1. Accessibility Testing
  • Consistent layout, focus and visibility
  • Small screen consideration, zoom support
  • Touch menu/button place, size or spacing, orientation
  • Grouping and positioning primary element before page scroll
  1. External Factors Testing
  • Network Connection- Only Wi-Fi, No Connection, No SIM device etc
  • Interruption with notification, phone call, calendar event, text message, push notification by various app
  • Handling low battery, power saving mode,
  • Handling different device options like font, theme, time zone, screen orientation etc.
  1. Performance Testing
  • Response time is critical as the growing user of mobile app
  • Space or Storage issue is critical if the app generate files
  • Stress testing (Simulate different scenario running test concurrently from our cloud based option)
  1. Security Testing
  • Ensure encrypted login
  • Handling Sensitive information including login credentials and bank details
  • Application and device permission
  1. Automation Regression Testing
  • Repetitive Use Cases
  • Test fundamental functionalities
  • Impossible to test manually

 

Test Automation

  1. Mobile App Test Automation Tool
  2. Environment and Installation requirement  
  3. Cloud Based Testing using multiple OS and devices

 

  1. Mobile App Test Automation Tool
  • Appium
  • Selenium
  • Sauce Labs
  1. Environment, apps and installation requirement

       Android

  • Appium for Windows
  • node.js and NPM for Windows
  • Android SDK
  • Java/JDK
  • Eclipse
  • TestNG
  • Appium Client Library
  • Selenium WebDriver
  • Windows Machines and 2 Android devices for automation tester
  •  Automation Script source control with SVN

      iOS

  • Appium on Mac
  • Install XCode (Version 9 or 10)
  • XCUITest  Driver (from Latest version of Appium)
  • iOS Developer program with valid Apple Id account.
  • Create Provisioning profile (OS to verify the app to run of specific device) with Apple Id and development certificate (from Apple’s developer website)
  • Java and Java Client Libraries
  • Eclipse
  • Selenium
  • 2 Macs (OS 11 or 12) machine and 2 iOS devices for Automation tester

     

     3. Cloud Based Testing using multiple OS and devices

  • Sauce Labs account for Cloud based testing

 

Recommendation:  iPhone and Android devices and OS for testing

      iPhone App Test

  • iOS App development and testing on 10.3 (expected to support any OS version after that)
  • Testing on Simulator/Emulator to be done from iOS 10.3 to 11.4
  • Testing on Real (Physical) Device with iOS 11.4
  • iOS Physical Device:    (Best to have two real devices with one small screen and the other one large screen)
    • 1 . iPhone 7  (Screen size : 4.7 inch)
    • 2 . iPhone X Plus (Screen size : 6.5 inch)

       Android App Test

  • Android app development and testing on 6.0-Marshmallow (expected to support any OS version after that)
  • Testing on Simulator/Emulator to be done on OS 6.0,  7.0 and 8.0
  • Testing on Real (Physical) Device with 7.0 (Nougat)
  • Android Physical Device :  (Best to have two real devices with one small screen and the other one large screen)
    • 1. Samsung S7  (Screen size : 5.1 inch)
    • 2. Samsung Notes 8 (Screen size : 6.3 inch)

 

Why Automation with Appium and Sauce Labs

  • Appium is Open Source and no licence required
  • Can test Android as well as iOS devices with Appium . Support native app, a hybrid app and even a web app.
  • Sauce Labs supports both Web and Mobile test suit
  • Sauce Labs supports a large number of  mobile device emulators as well as real devices. Provide huge selection of OS/Browser/Device combinations
  • Sauce Labs has No significant Cons
  • Appium is freeware mobile application for functional, regression & UI testing in multiple browser
  • Developer and Tester with Java experience can write and execute test script from their local machine or from any server, thus increasing volume of test scripts with no additional cost or licence fees
  • Selenium and Appium support parallel execution using Selenium Grid and Sauce Labs
  • Can easily integrate with our CI/CD, provides possibility of early bug detection for the developers
  • Appium supports writing tests against iOS and Android platforms using the same API, enabling code reuse
  • Appium and Selenium are supported by Sauce Labs for Cloud execution in multiple browsers, OS, emulator and real devices.