I’ll give you a list of tools available for iOS.
Hermetic UI Tests
KIF has proven to be a powerful solution for writing Objective-C UI tests. It runs in-process which allows tests to be more tightly coupled with the app under test, making the tests inherently more stable. KIF allows iOS developers to write tests using the same language as their application. Following the same paradigm as Android UI tests, you want Objective-C tests to be hermetic. A good approach is to mock the server with pre-canned responses. Since KIF tests run in-process, responses can be built programmatically, making tests easier to maintain and more stable.
iOS has no equivalent native tool for writing monkey tests as Android does, however this type of test still adds value in iOS (e.g. we found 16 crashes in one of our recent Google+ releases). The Google+ team developed their own custom monkey testing framework, but there are also many third-party options available.
A mobile testing strategy is not complete without testing the integration between server backends and mobile clients. This is especially true when the release cycles of the mobile clients and backends are very different. A replay test strategy can be very effective at preventing backends from breaking mobile clients. The theory behind this strategy is to simulate mobile clients by having a set of golden request and response files that are known to be correct. The replay test suite should then send golden requests to the backend server and assert that the response returned by the server matches the expected golden response. Since client/server responses are often not completely deterministic, you will need to utilize a diffing tool that can ignore expected differences.
To make this strategy successful you need a way to seed a repeatable data set on the backend and make all dependencies that are not relevant to your backend hermetic. Using in-memory servers with fake data or an RPC replay to external dependencies are good ways of achieving repeatable data sets and hermetic environments. Google+ mobile backend uses Guice for dependency injection, which allows us to easily swap out dependencies with fake implementations during testing and seed data fixtures.
Diagram: Normal flow vs Replay Tests flow
Mobile app testing can be very challenging, but building a comprehensive test strategy that understands the nature of different platforms and tools is the key to success. Providing a reliable and hermetic test environment is as important as the tests you write.
Finally, make sure you prioritize your automation efforts according to your team needs. This is how we prioritize on the Google+ team:
Unit tests: These should be your first priority in either Android or iOS. They run fast and are less flaky than any other type of tests.
Backend tests: Make sure your backend doesn’t break your mobile clients. Breakages are very likely to happen when the release cycle of mobile clients and backends are different.
UI tests: These are slower by nature and flaky. They also take more time to write and maintain. Make sure you provide coverage for at least the critical paths of your app.
Monkey tests: This is the final step to complete your mobile automation strategy.
Target device selection: Create an optimal mix of simulator testing and physical device testing on different models to maximize test coverage
Test automation: Select an effective test automation tool and maximize the use of automation to reduce the cost of regression testing
Network environment: Consider testing primarily on Wi-Fi networks and using network simulation tools to simulate cellular connectivity and various network conditions
Types of testing: Consider different types of testing required (functional, performance, security, compliance, etc.)
- Target device orientation selection: Landscape and portrait modes are available but we are using just portrait mode orientation.