Keynote by Jürgen Allgayer – YouTube Team at Google
Jürgen shared the journey (so far) YouTube has taken towards continuous delivery and the challenges they’ve faced including unpredictable launch intervals, device testing and cultural challenges. They’ve got some way to go, especially with having so many platforms to support, but I agree with Jürgen’s goal of having zero scripted manual regression tests as where they want to be, pushing to production daily led by passionate people.
The Uber Challenge of Cross-Application/Cross-Device Testing by Apple Chow and Bian Jiang
This was an interesting talk where Apple and Bian shared how they have multiple device emulators (Android and iOS) running the Uber apps to communicate with each other. It seems like a neat solution, but quite complex and prone to intermittent issues. They made it clear in the Q&A that this wasn’t the only type of tests they do, as there are a lot of dependencies and complexities.
Robot Assisted Test Automation by Hans Kuosmanen and Natalia Leinonen
This talk had the wow factor of seeing a robot drive a mobile phone and I can see how this could be useful for device manufacturers etc. But it’s most probably not relevant to me as someone who is involved with creating simple apps as it’s seems overkill for testing a simple app.
Juggling Chainsaws for Fun and Profit: Lessons Learned from Mobile Cross-Platform Integration Testing by Dan Giovannelli
Dan shared some insights into cross platform mobile testing, recommending to avoid real devices as much as possible with automated tests.
Mobile Game Test Automation Using Real Devices by Jouko Kaasila
Jouko shared his experience testing mobile game apps on a large number of mobile devices using public and private device clouds, and how this is important for the games industry where supporting an extra device (particularly devices particular to regions) can mean a lot of additional revenue, as almost all of the revenue apps in the app stores are from games.
How to Component Test Soup Dumplings by Toni Change
This was my favorite talk of the day where Toni used the analogy of testing a soup dumpling and how instead of testing every dumpling as a whole (end to end testing), the components of the dumpling can be comprehensively tested as it is constructed leading to much more effective testing and greater confidence in the final product. End to end testing is still important, but it’s not needed as much. I liked this one, I found it similar to the theme in my post about 100,000 e2e selenium tests and building an aeroplane. Great talk.
Chromecast Test Automation by Brian Gogan
This was another interesting talk by Dan from the Chromecast team. The emphasis here was on real device testing: he shared some neat looking device racks with lots of Chromecasts connected via Ethernet for reliability.
Using Robots for Android App Testing by Shauvik Roy Choudhary
Shauvik shared his research of automated Android robots programs which control your app and try to make it crash. He did a comprehensive analysis and comparison of the different tools in this space and the pros and cons of each, including how much coverage they attained. My takeaway was most of the coverage was achieved within about 5 mins, and barely increased up to 1 hour.
Large-Scale Automated Visual Testing by Adam Carmi
Adam demonstrated the Applitools as a way to ensure visual consistency between releases of your apps. This looked impressive with the way it could ignore false positives of dynamic data elements.
Hands Off Regression Testing by Puneet Khanduri
Puneet from Twitter shared the Diffy tool as a way to get more test coverage for free, by using production data captures and replaying against a new version of your app. It intelligently uses two baselines to work out what has changed and what is dynamic and has some cool reports.
Automated Accessibility Testing for Android Applications by Casey Burkhardt
Casey from the accessibility team at Google gave a great presentation about how Android approaches accessibility and how they have recently made it easy to perform automated accessibility checks if writing Espresso or Robotium automated tests. Whilst these automated checks don’t replace human accessibility testing, they do enable developers to find simple accessibility bugs in their Android apps much faster. A good insight into how seriously Android is taking accessibility. My favourite quote of the day came from this talk:
“Accessibility is about taking assumptions and making software that doesn’t have them”