Android and iOS UI Testing with Calabash

Testing is an essential part of any mobile app development process. Whether you are automating such tests or not, no sane developer considers their work to be done unless they have tested their app.

A well-tested app usually goes through multiple steps of testing: Unit testing, integration testing, acceptance testing, and so on. As your app grows, the importance of testing increases and automation in testing becomes a necessity.

While other platforms, such as the web, have advanced significantly in terms of testing mechanisms and frameworks, the mobile realm is not lagging behind. In this article, you will learn how you can use Calabash to automate the UI for your Android and iOS apps using plain English instructions and make acceptance testing them as painless as possible.

What Is UI Testing All About?

If you have been testing your apps manually, you are probably wasting a big chunk of your time performing the same tasks over and over again. You make some changes to the code, build the app, run it in a device or an emulator, and fiddle with the app to figure out if it works as expected.

By automating UI testing, you can perform those same manual steps automatically. If your app is of a decent size, this can save a lot of your time and also save your app from being riddled with embarrassing bugs, especially the regression ones.

“That sounds awesome,” you say, but how do you do it for your Android or iOS app?

UI Testing Frameworks for Android and iOS

If you read the official documentation for Android and iOS, they suggest you write and run UI tests in their official IDEs. For Android, it’s Android Studio, and for iOS, it’s Xcode.

The official documentation goes as far as to recommend specific frameworks for testing. The official Android documentation covers some topics about Espresso, the Android UI testing framework. Similarly, Apple suggests using the XCTest framework.

And if you are going to work seriously on UI tests, you may be following these suggestions, which makes sense since Espresso is maintained by Google and is a part of Android Support Repository. It is very likely that Espresso will support all the new features that Google will introduce for Android in the future. You could say the same about XCTest framework for iOS.

However, it is worth keeping in mind that despite the numerous benefits of automated testing, many developers simply don’t write them at all.

Every developer that is aware of test automations, deep inside, knows it is a great idea. But, when it comes to sitting down and writing these tests, many developers start questioning if it is worth their time, because “touch the button” manually turns out to be a much faster operation than writing a code that will “touch this button” automatically. Sometimes, clients and managers, eagerly waiting to try the app, don’t help either.

Many developers, at that point, decide that it is better to continue working on new features of the application rather than write automated UI tests for the existing ones.

When the application grows, it becomes more and more time consuming to “touch these buttons” manually every time you’re updating the application.

But what if there was a framework that made UI testing easier, and didn’t give you any excuse to not write UI tests for your apps?

Calabash: Automated Acceptance Test for Mobile Apps

About a year ago, I started searching for a testing framework that will be easy to use for the people who are not software developers. And, that is when I found Calabash.

This open source testing framework, developed and maintained by the Xamarin team, works for both Android and iOS. It lets you write and execute automated acceptance tests for mobile applications.

Acceptance tests are generally what comes after system tests that determine if your app satisfies the business requirements. Given that it operates on the UI level, this works well as our choice of UI testing automation framework.

Calabash can interact with your app like Espresso or XCTest does. However, what makes Calabash an excellent choice here is its support for Cucumber.

Cucumber is a tool that can run automated tests written in plain English (if you want you can adjust it to use any other plain language). So to write automated tests on Cucumber, the tester doesn’t need to know Java, Objective-C, or any other programming language.

What Makes Calabash Tick?

Calabash framework consists of libraries that can interact with the Android and iOS apps. It can be run on real devices. So it can do things that the tester is doing manually.

There are two different projects on GitHub that make Calabash possible:

  • calabash-android – for Android
  • calabash-ios – for iOS

Calabash can work with any Ruby-based test framework. In this article, we will cover Cucumber – the most popular and convenient way of writing tests for Calabash.

Before continuing, if you want to try Calabash as you follow the rest of the article, be sure you have Ruby installed on your machine. You can find detailed instructions of installation here.

Next, install Calabash for your favorite platform by following the GitHub links above.

Writing Your First Test on Calabash

Writing tests on Calabash is quite easy. Let’s see how a simple test for an iOS app looks:

Feature: User Login

    Scenario: Unsuccessful user login
        Given the app has launched
        Then I wait for the "Login" button to appear
        When I enter "tstuser" into the "Username" field
        And I enter "qwerty" into the "Password" field
        And I touch "Login"
        Then I should see "Username you entered is incorrect"

    Scenario: Successful user login
        Given the app has launched
        Then I wait for the "Login" button to appear
        When I enter "testeruser" into the "Username" field
        And I enter "qwerty" into the "Password" field
        And I touch "Login"
        Then I should see "Hey testeruser!"

Here, an app is being tested with incorrect username and password, and then is being tested with correct username and password. The test expects the app to fail login for the first scenario, but succeed in the second one.

You can create as many scenarios as need, and all you need to do that is break down the steps/instructions into simple English sentences. Just like you would write a story!

Anyone who knows about behavior-driven development (BDD) will already find themselves familiar with this.

How Does Calabash Work?

To see what goes on behind the steps that tester is using, you can open the project on GitHub and check the following file:


Let’s see a definition of the following step:

When I enter "testeruser" into the "Username" field
Then /^I enter "([^\"]*)" into the "([^\"]*)" field$/ do |text_to_type, field_name|
  touch("textField marked: '#{field_name}'")
  keyboard_enter_text text_to_type

This small snippet of Ruby code looks for a specific field, touches it, waits for keyboard to appear, types the text from the text_to_type variable, and waits for a bit before switching to the next step.

The first word of the step can be “Given,” “When,” “Then,” “And,” or “But.” It does not matter what keyword you will use. You can use any of them to make the story clearer. 

How to Add Custom Steps

If you need a step that is not implemented in Calabash yet, you can write it by yourself. The syntax is exactly the same as it is in already predefined steps.

For example, if a tester needs to access the input field by placeholder, instead of field name:

Then /^I enter "([^\"]*)" into the field with placeholder "([^\"]*)"$/ do |text_to_type, placeholder|
	touch("textField placeholder:'#{placeholder}'")
	keyboard_enter_text text_to_type

This step definition is much the same as it was the previous one, but you’re accessing the field by placeholder instead of the field name. Given how your app looks, this may make things even easier for the tester.

And it’s easy for the developer, too. The developer is implementing the step once, and then the tester is using it whenever they need it. Moreover, you don’t need to know a lot of Ruby to implement your own custom steps.

You can find the Ruby functions you can use, here:

Xamarin Test Cloud

There is one more challenge when testing mobile applications. You should test them on as many devices as possible, because there are so many devices and so many OS versions.

This is where Xamarin Test Cloud helps a lot. There are about 2,000 real devices in the cloud and the good news is that it supports Calabash tests.

The same Calabash tests that helped you save time by saving you from doing repetitive work can now be used to test your application on many real devices.

Start Writing UI Tests

Whether Calabash is the testing solution your app needs, with the advantages it brings, it leaves no room for excuses when it comes to writing automated UI tests for your mobile apps. Calabash may fall short if your app heavily relies on certain device features (e.g., the camera), but it still makes writing tests for a majority of the apps a much easier feat.

Related posts

9 Replies to “Android and iOS UI Testing with Calabash
  1. 激光無創熱作用模式精準作用於陰道粘膜層、肌層,使陰道組織發生新生改變,修復由於順產而引起的損傷,解決一系列產後女性生殖系統常見問題。私密緊緻作用原理:刺激陰道粘膜固有層、粘膜肌層,使其膠原纖維、彈性纖維大量增生重塑;彈性纖維網修復,陰道收緊 SUI作用原理:恢復盆底正常解剖位置 陰道彈性纖維網得以修復,恢復到正常解剖位置避免了擠壓,牽拉尿道,使尿道角度恢復正常 盆底血供豐富,間接的使尿道括約肌得以修復進而停止漏尿或症狀減輕 私密敏感、潤滑作用原理:CO2微脈管作用,使血管重建、促進血液循環 血管活性腸肽(VIP)和神經肽Y(NPY) 表達增加,敏感度增加 血流量增加,陰道上皮細胞功能增加、陰道粘膜自分泌功能增強,潤滑度增加合1子宮頸癌疫苗-Gardasil

  2. #mc每日3件事 情侶間心理小測試 Marie Claire (HK) Edition 1. 想知他怎樣想 問這10條問題看穿他心理 Charlene Choi(@choisaaaa)分享的貼文 於 2017 年 6月 月 27 1:36上午 PDT 張貼 2.&nbs舒壓

  3. These three figures — William Rick Singer, Mark Riddell and Rudolph Meredith, respectively — are at the center of the nationwide scandal in which 33 parents are accused of using their wealth and means to help their children game the college admissions system.
    But those three names do not appear anywhere in the complaint text, though the document gives enough clues to make clear who is who. Instead, they are named as Cooperating Witness 1, 2 and 3, and each agreed to plead guilty and to cooperate with the government in the hope of obtaining leniency when they are sentenced, the complaint says.

Leave a Reply