Xcode 7 introduces new UI testing features fully integrated into the IDE. Learn about the new APIs and how UI testing fits in with existing testing features in Xcode. See how to get started by recording your app, and how to efficiently craft and maintain UI tests.
WIL TURNER: Good morning.
And welcome to UI testing in Xcode.
My name is Wil Turner.
With me is Brooke Callahan.
We both work on the Xcode developer tools.
And I am extremely excited today because we are sharing
with you a huge expansion of the testing technology
in the Xcode developer tools.
That is UI testing.
With UI testing you can find user interface elements,
interact with them, and validate the UI properties in state.
Along with UI testing, we've introduced UI recording
which will allow you to really rapidly set
up UI testing for your projects.
Finally, we've updated the test reports in Xcode,
which show the pass and fail and results of your test outcomes,
to accommodate new data we have with UI testing.
So I want to talk about the core technologies
for UI testing, the first is XCTest.
The second is accessibility.
So XCTest is Xcode's testing framework.
In it, you create subclasses for your test cases,
you implement test methods, and you use assertions to validate
that your expected outcomes are holding true.
XCtest is integrated with Xcode, which means you get everything
from the ID in terms of code completion, debugging,
and the ability to run your tests directly
from your source code, and see the results right there.
You also get continuous integration via Xcode Server
Finally, XCTest supports both Swift and Objective-C,
so you can choose the native coding language
that you're most comfortable with.
XCTest was introduced in Xcode 5 as a unit testing framework.
In Xcode 6, we expanded it to support performance testing.
This allows you to catch regressions in your code,
and ensure that it continues
to perform optimally, release to release.
Now in Xcode 7, we've introduced UI testing which you can use
for both correctness and performance testing.
So that's XCTest.
Now let's take a look at Accessibility.
Accessibility is the technology on our platforms,
that gives disabled people the same great experience
on our devices and with our applications
that all other users receive.
To make Accessibility work, it offers a rich set
of semantic data about the UI that technologies like Voice
Over can use to guide users through the application.
UI testing uses that, and Accessibility is integrated
with the UI kit, and app kit, so, when you use controls
from those frameworks, you get a lot of accessibility support
for free, right out of the box.
It also provides APIs that allow you
to fine tune the accessibility data that is exposed.
The key about this is that with UI testing,
your tests will interact
with the application just the way a user does.
UI testing has a few requirements
that you should understand.
The first is it depends on new features in the OS.
For iOS you need iOS 9 and for OS X you need OS 10.11.
UI testing protects your privacy.
And to do so it means your iOS devices need to be enabled
for development and connected to a trusted host running Xcode.
On OS X you'll need to grant permission
to a special Xcode helper app, and you will be prompted to do
so the first time you run UI tests.
Let's take a look at what you need to get started
with UI test in your project.
First of all there's a new Xcode target type.
Traditionally, unit tests were a specific target type in Xcode,
and now UI tests are target type as well.
We've also introduced a large set of set of new APIs
for UI test, and of course UI recording,
which will really get you started quickly.
So the Xcode testing targets support the special requirements
that UI tests have.
This includes executing in a separate process
from your application that you are testing.
And it also handles the permission to use Accessibility
in the privacy protection.
These targets have new templates for both Cocoa and Cocoa Touch
and the assistant for these will set everything
up the way you need it to get started.
There's a target to be tested setting for UI test bundles
that identifies the application that you are testing.
In the new APIs there are three key classes.
The first is applications.
The second is elements.
And the third is element query.
We will take a deep dive
into these APIs a little bit later in the presentation.
UI Recording lets you interact
with your application hands-on your device,
the simulator, or OS X Mac.
While you are doing so, it generates the code necessary
to recreate those interactions.
You can do this to create new tests or expand existing tests.
So let's take a look at what this is all about.
Brooke, let's see a quick demo.
BROOKE CALLAHAN: Thanks, Wil.
So, without further ado.
The project I'm going to be using
for the demo today is the lister application.
This is an example project that you can download
So, let's get started, now it's got my target configured just
the way I want, but the one part I want to point out is
that the target to be tested is the lister application.
This is the one application
that my tests will be able to interact with.
So now I've got my new test class here.
And there's a little stub test method here and a setup function
which is going to call, which is going
to launch the application before my test method is called.
And it's going to do that for all the test methods I add
to this class.
So Let's add a new test from the lister app.
I'm going to put the keyboard cursor in the method and click
on the record button down the debug bar.
Now, Xcode is launching my application.
And here it is.
The lister application allows me to manage a series of lists.
So probably the most common thing people do is add
and remove items from their list.
So I am going to click here, add item.
It looks like we have a lot
of health some things in this grocery list.
So I'm going to add cookies.
You can see that, as I'm typing, the source header is updating.
If I press delete, it also removes what I removed
from the text field.
Next thing I'm going to do is tap on the cookies item
to mark it as completed.
And another thing I can do
in this application is remove items from the list.
So I'll click on edit, and then delete cookies,
and the delete confirmation button, and finally done.
Great. Now I have a simple test that adds
and removes an item from the list.
So I'll click stop.
Let's see that in action.
It added the cookies and it is removing it.
And we are done.
Thanks. As the test interacts with UI elements,
we get implicit validation that those UI elements exist.
But What we don't get validation of,
are things like state changes in the application.
For example, when we tapped on that cookies button,
we don't get any validation that the state
of the button actually changed.
And later on in the test, when it taps
on the delete confirmation button,
we know that the delete confirmation button was tapped.
but running the test doesn't validate
that the cookies row actually is removed.
So to get validation of those for my test,
I want to add some explicit assertions.
First thing I am going to do is add an assertion
that the cookies button here actually changes its state.
To do that, I will add a new constant.
Call this let cookies button.
I'm going to change the test to tap on that constant.
Now, to add the assertion I need some state, some properties
of the element to assert on.
And cookies button is XUI element
and XUI elements have a value property.
What I'm going to do, is I'm going to set a breakpoint here,
and run the test to that point.
So here we have the test in the state
where it's added the cookies row.
And it has yet to tap on that button.
So I'm going to go in the debugger,
and print out the value of this cookies button.
-- there we go.
I can see that the value of the cookies button is the string
with the number zero in it.
Next I'm going to step over that line.
So now it's tap the cookies button.
I will print out the value of it again and now I can see
that the value of the button is the string with the number one.
Great. So now I have all the information that I need
to assert that the value of the button changes when it's tapped.
So I'm going to add an assertion using XCT assert equal.
I'll assert that the value
after the tap is a string with the number one.
The value is in any object.
So I am going to need to assert that this is a string.
So I'll use as string.
And assert that it's number one after the tap.
And I'll assert that it is the number string
with the number zero before the tap.
And lastly, after the delete confirmation button is tapped I
want to assert that that cookies row goes away.
I'll call XCT assert equal and assert
that the button no longer exists.
Now iIf I run this test again we should see it does all the same
thing but also passing these assertions.
Great, so now I have just added my first test.
Back to you, Wil.
WIL TURNER: That was pretty awesome.
You can see just how easily Brooke took an existing
application and used it, just like he would as a user,
and in a handful of minutes he created a test.
He could expand that test using XCT assert
to do some additional validation and he just added reliability
to his project with minimal effort.
That's pretty exciting.
So you can see we added UI testing target,
very straightforward just like all our other other targets,
with an assistant and templates.
Used recording, interacted with the app, and it creates the code
that uses the elements and synthesizes the events
and then the additional validation with XCT assert.
Let's take a look at this UI testing API.
I mentioned earlier that there are three classes.
They are XCUIApplication,
XCUIElement, and XCUIElementQuery.
How do they work?
Let's start with a very simple, something even simpler
than Brooke's example.
Line-by-line going through here first I am instantiating my
This is a proxy for my application.
Then I launch it.
That brings it up for me.
And then I use element and query to find the add button.
And then I synthesize the event that taps on it.
Finally, I add in an assertion just like Brooke did
to make sure that the UI had the expected state
at the end of the test.
So I mentioned that UIApplication is a proxy
for the tested application.
It is independent of the lifecycle of the launch
and termination lifecycle of your app.
Because Your tests are running in a separate process.
You get explicit control over when the app is launched
and when it is terminated.
When you launch we will always spawn a new process.
So that's something we do to kind
of not give you completely Clean Slate because you have a lot
of state in your application that it's up to you to control.
But by launching a process clean each time,
we help you minimize the number of variables that you have
to deal with in your test.
So if it's already running,
calling launch will terminate the previous existing instance.
Application is also the starting point for finding elements.
Let's talk about elements.
XCUIElement, just like Application, is a proxy object,
but this time for user interface elements
in the tested application.
Elements have types.
Types are like button, or cell, or window, and so forth.
They have what we call identifiers, strings of data
that we get from the accessibility system,
hich are an identifier or label or title, so forth.
Most of the time you'll find elements using a combination
of the type and the label.
For example, if you have a bullton which is add,
you'll look for the button type with the identifier add.
Elements form a hierarchy in your application.
The application is the root of a tree.
If you think back to your computer science days you'll
remember tree data structures.
With the lister application we have this very simple tree here
with the application at the top and we have
like the nav bar and the add button.
So on and so forth.
So each of these is an element
that you can reference in your tests.
These are used by queries.
This hierarchy, as well as type and identifier,
to find your elements.
Elements, when you work with them
in your test, must be unique.
So what does that mean?
Well, every one of these elements is backed by a query.
The query has to resolve to a single instance.
Otherwise when we go to synthesize the events
that you tell us to tap but there's multiple buttons
that match that, it is not deterministic.
We can't really know what you mean for us to do.
Similarly, if you ask us for a property of the element,
we don't know which one did you really mean?
Maybe there wasn't one at all.
So it's important to the queries for the elements
to resolve to exactly one match.
If they don't, when you access the element we'll raise a
failure for that.
There is one exception to this,
which is an API called the exists property on XCUIElement.
This allows you to test for the existence of the element safely.
You can use this as Brooke did to verify
that the element had been removed from the UI
and you can also use this to handle cases
where you have conditional UI
that might display in some context.
For example, if you are saving a file to a location
where another file already exists, you might get some kind
of confirmation sheet.
That wouldn't be present all the time.
Just the case where there's these conflicts.
You can use an exist check for those cases.
Elements are where event synthesis APIs live.
Event synthesis is how we simulate user interaction,
and do this at the lowest level of the system so everything goes
through the same channels it would when the user interacts.
The APIs for event synthesis are platform-specific
with some exceptions.
We have button click, for example in OS X
and the corresponding is button tap on iOS.
We have type text which is the same on both platforms
and takes a string of text.
XCUIElementQuery is our API for specifying elements.
Queries resolve to collections of accessible elements.
They can only find elements that are visible to Accessibility.
And they will resolve to a set of these.
That means you can get the number
of matches using the count property.
And you can also specify distinct elements
in that set using subscripting with an identifier
or using an element at index API.
We will look at these more in a moment.
So how do queries work?
So I mentioned that element hierarchy.
So the relationships in that hierarchy are one side
of how queries work.
The other side is by filtering.
Filtering is taking one set and reducing
that set according to certain criteria.
With our lister example again here is how we might express
certain relationships, the first of which is descendants.
In this case I'm showing the view.
All the gold cells are its descendents.
On the other hand children is a more restrictive relationship.
It's just the elements that are directly below the element
So, the table's children are just the cells.
The final relationship we use is containment.
This happens to be very useful when we have cases
where the elements, like the cells, they don't have a lot
of data that unique them from each other,
but they contain elements which are unique.
For example, the first cell contains the groceries label.
Filtering lets us take things like the element type,
or its identifiers, to create queries
that filter a previous query.
We can also do this with predicates which will allow us
to go beyond the identifiers to look at the values
or do partial matching such as "begins with"
and that sort of thing.
We combine relationships and the filtering in the APIs
and the first of which is descendantsMatchingType.
You can figure out what that does.
It find the descendants that match a certain type.
This turns out to be the most common query you'll use.
Some examples of this are, I can find all the buttons
in an application by calling app,
Similarly I can find all the cells in a table
by telling the table to give me descendentsMatchingType cell,
or, another example, with menu items.
This is such a common query that we provide convenience API
for every one of these types.
DescendentsMatchingType buttons, becomes just buttons.
DescendentsMatchingType cell becomes just cells,
and so on and so forth.
These convenience APIs help make your tests very expressive
ChildrenMatchingType is the other combination
of relationships in filtering.
So in that, that allows us to differentiate
between descendants, all the descendants that match a type,
and just those that match, that are children.
Again the all buttons example, is app.buttons,
if I want to find just the buttons that are children
of my nav bar, I can take my nav bar
and say childrenMatchingType button.
It is not as common a query, but there are cases
where it becomes very useful to differentiate.
Finally, containing type.
This allows us to find elements by describing their descendants.
So in the example we have the cells
that are each somewhat anonymous.
They don't have any identifying characteristics,
but they do contain unique labels.
Labels are also known as static texts.
So here I can form a cell query which takes the cells
and finds the one that is containing the type static text
with the identifier groceries.
That will find the first cell for me.
There's also predicate variant for this API.
So those are our three APIs
that combine relationships and filtering.
containingType, and of course all the convenience APIs
The other powerful thing about query is
that they can be chained together.
So we can take the output of one query and make it the input
of the next, just like you would pipe commands together
on a Unix command line.
This is ver powerful, and lets you build
up very complex queries, again in a way
that is concise and expressive.
So here we have our application and I want
to find just the labels in the table.
So I start with the app.
And then I get the tables.
And I ask for static texts and I'm done.
I have those three labels.
So queries are sometimes the end unto themselves.
You want to get the count of the query and maybe assert
that you have the right number of items in there.
But often the goal of a query is to find an element.
All of our elements are backed by a query.
To get an element from a query we provide several
The first is subscripting, which allows us to take a query
and then subscript using an identifier.
That would give me back an element
which is the groceries label.
It can also do this with element and index.
If I have a set, maybe the rows in the table
and I want to iterate over them.
I could one at a time call element and index on these.
If I have a query which I know resolves uniquely,
hat is to a single thing, maybe I only have one navigation bar
in my application, I can use the element property
to create a new element backed by that query.
So when are queries evaluated?
So they are not actually evaluated just
when you create them.
They are evaluated on-demand or as they are needed.
This means that with an element, the query will be evaluated
when you synthesize events or read property values.
You can create the element but until you use it,
the query won't be evaluated.
Similarly if you create a query directly it will be evaluated
when you get the number of matches or if you call one
of the APIs that returns all the matches.
It will have to be evaluated at that point
and we will reevaluate queries when the UI changes.
So you are always working with the most current view
of the application rather than data from ten seconds ago
or two minutes ago, depending on the length of your test.
So in this way you can think of queries
and elements being somewhat similar to URLs.
They are, with a URL you can create a URL
but it doesn't fetch a resource immediately.
It is not until you actually go to create your URL request
or session that the actual URL gets resolved.
And so even if the URL was invalid,
no error would be raised until that point.
Similarly, queries and elements, they are just specifications
for accessible elements in the tested application.
So creating them doesn't do anything until you need them
and at that point they are resolved.
So that's the API.
There's three classes: The application
for launching your application; elements,
which like the application are proxy objects for elements
in your app; and finally queries,
which are more complex ways to specify elements.
So now I want to talk a little bit
about Accessibility and UI testing.
I mentioned earlier
that accessibility data is what makes UI testing possible.
So given that, it is not hard to see how the quality
of the accessibility data really impacts your testing.
In fact, the better the accessibility data
for your application, the easier it is to write tests
and the more reliable those tests are over time.
So you get a double benefit
when you improve the accessibility
in your application.
You've not only made it easier for your own testing
but you've improved the experience for all
of our disabled users.
I would really encourage you to keep that in mind when you work
with UI testing and accessibility.
Sometimes up's need to do some debugging.
It's an element may not be accessible, may not be showing
up for you even when you are using recording.
That could be because of the custom view subclass
that may not be accessible by default.
Or it's actually not a view.
It's a graphics object in a lower level graphics subsystem
such as a layer and that sort of thing.
In other cases, the element is visible
but has poor accessibility data.
All those table cells I was looking
at in the containment query, partly that might have gone away
if the cells themselves had better accessibility data.
When this happens, there's a couple tools I would point
The first of which is UI recording itself
because UI recording will give you the closest view
into how the testing system sees the elements.
But beyond that there's also great accessibility inspectors
on our platform.
They'll let you see the raw accessibility data
as it is exposed by the application.
When you need to go and improve that data,
your first stop should be Interface Builder.
Interface Builder has a great accessibility inspector
which allows you to enable or disable accessibility,
set values for the various accessibility attributes,
and configure the traits which have a direct impact
on how the element is expressed as a type in UI testing.
There's also API if you're working with elements
that you can't access through Interface Builder.
You can use APIs in NSAccessibility
and UIAccessibility to directly control how the element is
expressed to Accessibility.
So with that in mind, let's see another demo.
This time Brooke is going to take us
through more complex test cases and also a little bit
of accessibility debugging.
BROOKE CALLAHAN: Thanks, Wil.
BROOKE CALLAHAN: So in the last demo we saw adding a test
that can add and remove an item from the list.
So while I'm here I would like to add some more tests
around this area of user interface.
First I'm going to add a test
that adds multiple items to the same list.
With this multiple items with the same name to a list.
So to do that I'm going to copy this code here
from the last test.
And I'll call this new test, add to cookies.
I'll just paste that code in.
Great. Now I have a test that is going to tap
on the groceries label.
Next it is going to add the new cookies item to the list,
and then it is going to tap on that cookies button
in that item, and verify that it actually gets
to that tapped state.
So to add the second item I'm just going
to copy this code here.
And then to verify that the new button exists and gets tapped,
I'm going to copy that section.
So let's run this test and see how that works.
Now let's add the first one.
Ahh, failed an assertion here.
So on this line here, where we are getting the value
of the cookies button for the second time around,
we are actually failing the assertion.
Looks like it's failing because multiple matches are found.
I think I have an idea of what went wrong here.
The way that the cookies button constant is specified,
it is just looking for all the buttons in the table,
and giving me one that's called cookies.
By this point in the test,
there are two buttons called cookies in the table.
It will find both of them and there is not going
to be one value that the element can return
because there's these two matches.
Now, I only know that because I just saw the test running.
Normally you are not going to be watching your test as it runs.
So we thought we should provide a way for you
to see what the test looks like when it last ran
and we've added this information to the test reports in Xcode.
So if I go to the report navigator and click
on the most recent test report,
you can see the test add to cookies test.
If I expand this item, you can see all of the activities
that happen during the test.
And down here I see this last find the cookies button item.
There's my failure.
Multiple matches found.
There's also a quick look button here, where I can look on that
and it will show me the exact state
of the application when that happened.
Just like we know happened, yeah,
there's two rows called cookies.
One of them is tapped and one of them is not tapped.
BROOKE CALLAHAN: I am going to close that and if I want
to see the complete assertion failure,
I can go to the log section
and on this line I can expand the log for that test.
And here we can see the complete assertion failure showing the
accessibility hierarchy of both those buttons.
And here on the side I can see that yeah, there's one button
that is unchecked and the other one is checked,
just like I expect.
Let's go back and fix that test.
The easiest way for me to fix this test is
to use recording again.
I'll set a breakpoint on this line right before the assertion
failure, and run the test at that point.
So now I've got the application in exactly the state I need it
to be to get the value of that unchecked button.
All I'm going to do is I'm going to click on recording,
tap on that button, and stop recording.
Now I have a way to refer to that second button.
I'm going to clean this
up by using the table constant in both.
I'm going to call this cookies button two,
a new constant in my test.
And now to fix the test I'm going to change each
of the next three lines to use that new reference.
And when I run the test again, we should see that it works.
Now I've got one test that can add and remove an item
from the list and another test that can add two items
to the same list, and verify they both exist.
Now I want to build a test
that will remove all the items from the list.
Once again I'm going to use recording.
I am going to tap on groceries, edit, and then I'm going
to remove the apples row from the test.
I'll tap stop recording.
So all this much of the test is pretty much how I want,
but what I want to do is, I want to make my test agnostic
to the data of the application.
I want it to remove all these items, but I don't want it
to actually refer to the items by their labels.
Because that is going to get kind of wordy here.
So first thing I'm going to do, you see these tokens here,
the tokens provide multiple ways to get to the same UI element.
In this case, for that first delete apples button,
I can get to this by calling table.button delete apples.
I can be more specific and say it's the delete apples button
in the cell called apples.
So I'm going to use that.
And I'm going to double click on it to convert it to text.
So now I've got two rows here
where we are getting the elements the same way.
What I would like to do is get the cell simply by index.
I will add a constant called let cell, table., and set this equal
to table.cells elements index.
Since the apples row was index one, I'm going to use that.
I can simply replace these with the reference to that constant.
I'm almost done.
The next part I need to change here is how I'm getting the
button out of the cell because the other rows are not going
to have a delete apples button.
They will have a delete oranges button or delete bread.
What I need is a way to find the button
where the label starts with the word delete.
To do that I'm going to use a predicate.
So here I'm using matching predicate
to find the button whose label begins with the word delete.
So the last thing I need to change here,
I want to add an assertion just like before.
I want to verify that that cell goes away
after we've removed it.
I am going to use XCTAssertEqual,
and assert that the cells exist property returns false
after we tap that delete confirmation button.
I'm going to run the test now.
It is removing the apples row but failed the assertion.
I have an idea what might be going on here.
To show you what is going on I'll use the debugger.
I'll set a breakpoint here and run the test again,
to that same breakpoint.
Now in the debugger, I'm going
to call debug description on the cell.
So the debug description has a lot of information
about how the cell is actually resolved.
So I can see here that when I call it,
this cell result first finds the application
and then it finds the table in the application.
Then all the cells.
And then the element and index in those cells.
I can see that it's actually finding the oranges row.
It looks like what might be happening here is
when we call the exist property on this element called cell,
it is actually re-resolving itself.
Even though we just removed the apples row
from the table there's now a new cell at index one.
So that is no problem.
It just means I need to use a different way to find
out whether or not we've removed the row.
So I'm going to add an assertion,
assert that the number of cells
in the table goes down at this point.
So I'm going to add a new constant called "count"
and I'll set this equal to the count of the cells in the table.
And then I'll simply assert that that's equal to count minus 1.
Now, the last thing I need to do, I said this was going
to be a remove all items test.
I'm going to change it to add a wild loop and simply do this
over and over again as long as there's more
than one cell in the table.
Let's run that.
So it's removing the apples row, and oranges.
BROOKE CALLAHAN: Last thing I would like to do is add a test
to use that color row that we saw in the edit UI.
And once again I'm going to use the recorder for this.
I'll tap on groceries and edit.
Now, the color UI here lets me change the color for a list.
It looks like right now this list is green.
So I'll try changing it to red or how about blue?
Okay, so when I tap on those buttons it looks
like it's not actually recording what I want.
It looks like it's recording a tap
on the static text called color, which you see over here.
So I think what might be going on here is
that these UI elements simply might not be visible
So I'm going to stop recording,
and I can actually use the accessibility inspector
to tell me what's going on.
I'll right click on Xcode and go to open developer tool,
And the accessibility inspector provides a lot of information,
but all I want to use it for is ths shortcut, command S7.
It will highlight the UI element that's underneath the
If I put the mouse cursor
over the word bread here, and press command F7.
it highlights bread.
If I put it over the delete bread button,
you can see that it highlights that.
Now, let's see what happens if I put the mouse cursor
over this yellow color button.
Aha! So it highlighted the whole row.
So that pretty much confirms
that this UI element is simply not visible to accessibility.
Luckily I can actually change this,
and fix this problem using the story board.
I'll open the story board now.
So here I've got the same buttons in my story board
and if I open the inspector, here I can see
that these buttons actually have a class of color tappable view.
I'm familiar with this class.
I know it's actually not UI button.
It's custom view.
If I go down to the accessibility part
of the inspector, I can see it's not enabled for them.
I have gone through and added labels for them.
To fix this all I need to do is select all of the buttons.
And then I'll check the enabled for accessibility check box.
Since they behave like buttons,
I am going to give these the button trait.
Now let's run the application again.
And now I'm going to record this test one more time.
Tap on groceries and edit.
Then red. And orange.
Yellow, green, blue, and gray.
Great. Now it actually recorded all those.
I'll stop recording and let's run it and see
if it can play it back as well.
Great. I've fixed accessibility in my application.
And I've also made it more testable.
If I was going to complete this test,
I would probably add some assertions to verify
that the state of these buttons changes.
For now, I'll hand the stage back to Wil.
WIL TURNER: So that's really quite awesome,
especially just how easy it was to make this view
that previously a Voice Over user would have had no luck
with at all and just with a few quick changes
in interface builder Brooke was able to make
that both accessible and UI testable.
So in the demo, some sort of more advanced UI testing.
You saw he had cases where he dealt with a conflict
and a query, and how to correct those queries,
and how to debug it, and also how he can loop over elements
and validate them, and how you can use the exists property
and also highlighted how queries are reevaluated.
They use the criteria that you created them with.
That specification is what gets reevaluated.
We used element at index one.
That pointed to the apples label the first time through.
As soon as that was gone and the UI changed,
now that was pointing to the oranges label.And then finally,
how to improve accessibility,
and the rewards are truly fantastic for doing that.
Brooke also gave you a peek at the test reports.
We have done some work in Xcode 7
to overhaul them for UI testing.
To recap are, the test reports are where you see the results
for all the tests that ran.
It shows the pass or the fail, the failure message,
for performance tests are shown in the reports.
You get the same UI in XCode and in Xcode server.
It's a consistent experience regardless
of whether you are looking at integrations or the run
that you just did on your local computer.
On Xcode server you also get per-device results,
because you can have the devices integrating multiple devices
at the same time.
The additions for UI testing,
we collected additional data during UI testing,
and this includes screen shots.
You saw how it helped Brooke debug the conflict in his query
by pulling up the screen shot at the time that query failed.
Also, we organized the API calls into these nested activities
that help you understand how the API call is working.
So let's take a look at an example of this.
This is not the list draft.
It is a different application.
But I'm going to show you how there are several steps
in the nested activities breakdown.
This example we're typing into a test field
so the high level you call the API type test.
But internally that breaks down.
In the first step where we wait for the application to idle.
We're actually observing the main run loop
of the tested application.
Because We don't want to send events to it,
when it's busy processing.
We want it to be responsive as possible.
Once it's idled we capture the data we need from Accessibility
and we resolve the query,
make sure it matches exactly one thing.
Next step, we synthesize the actual events
that insert the text into that field.
Finally, we wait one more time for the app to idle afterwards.
Because, again, we want to hand off control in a way
where things are reliable and deterministic.
The quick looks are provided for screen shots
that are captured during critical steps.
You can see here is the state right after I type that text in.
Make sure everything looked just the way you expected.
So UI testing is obviously a huge expansion of the kind
of testing you can do for your applications.
So when do you use it, right?
We've got unit testing already.
UI testing is a complement to UI testing, not a replacement.
You should continue to use unit testing for your model objects,
and your controller logic,
because unit testing will more precisely pinpoint failures
when they happen in your code.
UI testing allows you to cover much broader ranges
of functionality, but tracking
down the failures can be more challenging.
It's a balance in your project, of finding the right blend
between unit testing and UI testing.
Some great candidates for UI testing.
Well, think about your app.
You have customers, you show it to them,
and you have little demo sequences that you show to them,
and you walk them through, "well,
here's how you do this with the app."
Demo sequences are great candidates for UI testing.
You'll know nightly build after nightly build
that those demos are going to work for you.
Second, beyond that, common workflows.
What the app is used for, if it's an editing app,
how you edit the documents.
Any custom views you have in the application.
And finally document-based workflows, opening and saving.
These are all great things to automate.
They are really hard to capture with unit testing.
When they go wrong they have a huge impact on users.
So UI testing.
New in XCode 7.
Opens up a world of possibilities
for how you can test your applications.
In UI testing you find and interact
with user interface elements and you synthesize events
that drives them, just the way a user does.
You can validate the UI properties and state.
And UI recording lets you create these tests super quick,
Finally, we've overhauled the test reports
to make you better able to understand how your tests work
and collect additional data about them.
So that's UI testing.
WIL TURNER: So for more information we've got great
documentation on XCTest.
You can get to it through Xcode itself and on our website.
And Accessibility also has great documentation,
and I encourage you to check out.
Developer Forums are great places to raise questions
and trade tips with other users about how you're using things.
Our evangelist Stefan Lesser is a great contact
to get you started.
There's some related sessions if you want to dial back in time
and watch the accessibility session from yesterday morning,
probably watch the video on your lunch if you want,
and continuous integration and code coverage, new technology
in Xcode server, you can see tomorrow afternoon --
Have great conference, everybody.
Looking for something specific? Enter a topic above and jump straight to the good stuff.
An error occurred when submitting your query. Please check your Internet connection and try again.