Enter Automation, which reduced test time by allowing developers to automate test cases and run them on demand. This more frequent and more immediate feedback enabled them to write better code and freed up QA engineers to test with more freedom.

Then mobile came along, and we were back to square one. Years later, the first mobile automation tools were developed and released. These early tools all had major drawbacks. They all required you to modify your application to support testing, usually by embedding an HTTP server in your app. Most of them only supported coding in their APIs, which rarely supported multiple languages. They also were doing things that were hacks or workarounds and not sanctioned by the guys making the OS.

Appium was originally developed at Zoosk back in late 2011 when we were looking to reduce the cost of testing our iOS app. iOS testing presents a throwback sort-of challenge which harkens back to the days when software was released on disks and CDs. A mistake in an iOS release will take days to correct. If you have a crash, once you figure it out and fix it, there’s a multi-day app-store approval process on the other end. It’s different than the web because you can’t just do another deploy and fix the problem; you must suffer the consequences for days even with an expedited review.

There are 4 principles that guide Appium development

1. Test the same app you put in the marketplace.
2. Write tests in the language you want to.
3. Use a standard automation specification
4. Keep it open-source and free

Principles aside, what is Appium? Appium is an implementation of the Selenium-WebDriver Protocol (the most popular web automation toolset) that supports controlling Android and iOS applications on devices and simulators. Coding an Appium test is identical to coding a Selenium test. You don’t have to modify your application and can use a protocol that is a (pending) W3C standard. Selenium already has bindings in a dozen or so languages, so you can even use the language of your choice. Under the hood, it uses the sanctioned automation APIs, UIAutomation for iOS and UIAutomation and Instrumentation for Android. And of course, it’s all open source.

In closing, the future is here. Mobile automation is now as easy as web automation. Your mobile QA team can blister their fingers playing guitar instead of by performing 3 straight days of iPad test passes. You can submit your app to the app-store faster and with more confidence.

The less interactive a site becomes the more likely users are to click away and do something else. Latency is the mother of interactivity. Though it’s possible through various UI techniques to make pages subjectively feel faster, slow sites generally lead to higher customer defection rates, which lead to lower conversation rates, which results in lower sales. Yet for some reason latency isn’t a topic talked a lot about for web apps. We talk a lot about about building high-capacity sites, but very little about how to build low-latency sites. We apparently do so at the expense of our immortal bottom line.

Many techniques have been developed over the years to address the issue of latency and interactivity, the most obvious being the rise of Web 2.0 and the dynamic loading of content after page render. Ajax has been the panacea of low latency websites, allowing for non blocking page loads and deferring high cost operations until such time as that content is needed or requested. This has gotten us far, but we can do better.

Enter BigPipe

In 2010, Facebook posted a blog entitled, BigPipe: Pipelining web pages for high performance. In it, they outline a rather novel solution for delivering highly dynamic and content driven websites with low latency. I will leave you to read the article for the specifics, but to summarize, a page is broken down into highly decoupled widgets, or pagelets as they call them. Each pagelet is represented on the page by a div, which is a placeholder for where the content will be displayed once it has been processed. This skeleton is flushed immediately to the user, however instead of closing the connection, the server keeps the request alive. In the background, asynchronous processes are triggered which fetch and process the data  for each pagelet. Upon completion, each process will flush some data to the page as JSON, which initializes the pagelet with the desired content. Once all processes have competed, the request is closed and the page is “complete.” Facebook found that by moving to BigPipe, they were able to decrease the time to interact latency by about half. This is a significant win, and one that I wanted to see if we could replicate here at Zoosk.

Javascript to the rescue

In order to test BigPipe with our product, I decided to try out a new stack, one which lent itself more readily to asynchronous programming. We have typically been a LAMP shop here at Zoosk, but for this, I decided to use Node.js. Having most recently rebuilt our site as an HTML5 single page application targeting smart phones, I found the Javascript event loop ideally suited for the task at hand. For the test case, I chose to rewrite our internal admin tool. While not a user facing application, the admin tool is the backbone of our user operations team and an ideal candidate for optimization. The end result? We were able to reduce TTFB (time to first byte) by 50 percent, and the time to page load was reduced by 35-50 percent. For an initial, unoptimized proof of concept, this was a huge win. I am convinced that with further tuning, we could easily drop the latency even further.

The devil is in the details

While the Facebook article does a good job outlining the BigPipe methodology at a high level, I want to draw attention to a few items that were not as immediately obvious as candidates for optimization:

• SSL - Here at Zoosk, we take user privacy extremely seriously, which is why we host an all-SSL experience. While SSL keeps your data secure, it does come at a price. SSL can increase the overhead of serving a request, with most of the overhead coming from the handshake protocol. Every external network resource, be it CSS, Javascript, an image tag, an AJAX request, etc, will incur this overhead, which can lead to a higher latency experience. By moving what would typically be an Ajax heavy experience to BigPipe, you are able to avoid incurring this cost. Instead of several Ajax requests having to negotiate the SSL handshake, you are able to make ONE request to the server which you can auth and terminate SSL at the edge of your network. All internal processing can then use more performant transport protocols while within the confines of your network. 
• Physical Locality – dovetailing on the above point, BigPipe is able to take advantage of the principle of locality. The location of a user and the quality of their connection can play a large role in the latency required to serve a request. How fast is their DNS server, how close are they to their local exchange, is their ISP performing any bandwidth shaping/throttling, how many hops total before reaching our cage, are they mobile? Every connection a user’s browser makes is affected by these factors, and more. However, requests within our cage take just a few milliseconds. In a typical Ajax heavy application, the page has to wait until DOMContentLoaded before firing off any additional requests. This initial request has an overhead which must be paid, and introduces a stall in our request pipeline. Then, each additional Ajax request incurs an additional cost. By switching to BigPipe, a user just has to make one (potentially) costly request to our servers. At that point, we are able to make highly performant requests within our cage, flushing the results as soon as we get them, thus keeping the request pipeline from stalling.
• Browser connections – speaking of pipeline stalls, one source of stalls in the rendering pipeline is the max number of simultaneous connections a browser can make per host. Currently, most browsers have an upper bound of 4-6 connections (this drops down to 1-2 connections on mobile devices over 3G). As pages have become more dynamic and interactive, it has become increasingly easy to hit these hard limits. Once that magic threshold has been reached, no new requests can be made to that domain until a previous request has been completed, and your page stalls out. By using BigPipe, you release these connections that would have been used for Ajax requests back into the available pool.

Over the coming months, we will be applying the lessons learned to our core product, resulting in a snappier, more delightful Zoosk.

• Represent a new feature that would grow the Zoosk business
• Generate an efficiency that would lower the cost/time to get things done
• Provide a new means of driving users to Zoosk products

The event ran from Thursday morning straight through to 4pm on Friday.  We had 24 teams with 40 individual participants working on projects.  We concluded the event with presentations by the teams to our judging committee, which consisted of our executive team.  The judges had some difficult decisions to make.  After much deliberation, the judges selected the following winners:

1st Place:  Mobile Chat… better (Chris K.)
2nd Place:  Ravamping Help and Customer Support   (Khyati D.)
3rd Place:  Marketing Search   (Craig A.)

We also had a People’s Choice award, which was voted on by all participants.  The winner was Better, More Violent Search  (Keith K.)

Overall, it was a great event.  Several of the ideas will be pipelined into our product development funnel.  We are already looking forward to our next Hack Day event in six months.

One of the sessions I attended was titled “Automation in the Technically Challenging World of Game Development”, where an hour long panel discussion ensued between notable QA staff from various companies such as VMC Game Labs, Sony Computer Entertainment America, LocalizeDirect, Disney Interactive, Atari inc., and The Walt Disney Company. One key point was the importance of a tightly coupled working relationship between the Engineering and QA teams. Some of the participants even commented on the value of QA staff having the ability to understand and write code, if necessary. Another key point was the growing need for Automation testing and knowing when to manually test versus when to automate a test case. I was proud of our QA team at Zoosk for already demonstrating previous knowledge of these key points by having implemented their own QA/Engineer cross-training program in addition to having contributed to the Appium( http://appium.io ) open source test automation tool in its early stages.

In Conclusion, attending GDC was a motivating learning experience that I would recommend to anyone interested in cutting-edge software development practices in any industry!

As Doug mentioned, since each graduate of the Zoosk R&D program became familiar with the tools, processes, and environment at Zoosk, they were able to assimilate into one of our existing engineering teams and quickly start contributing to it.  Zoosk R&D was originally targeted towards more junior developers, but we realized from the first session that any engineer no matter their level, could come in and benefit from it.  That being said, my team consisted of myself along with three new Zoosk developers with a varying range of experience.  I also had a part time product manager and graphic designer borrowed from the Web Team at Zoosk, so there were six people in total working on the project.  Without giving away the next big thing ;D, the team began work on Project ‘X’.  I was lucky enough to be able to lead the second R&D session which went from late August to early November and these are a few thoughts from it.

1. On-boarding rules now.

I think the most obvious benefit from Zoosk R&D is that the new on-boarding process is much more improved for everyone involved.  Having been the lead engineer of the web team for a couple of years now, I’ve on-boarded a fair number of new developers.  We typically would assign the new developer a mentor and slowly start by wading into the code with small bug fixes and minor changes while going over processes, tools, and other technologies we use.  With Zoosk R&D, we really turn that methodology upside down.

Zoosk R&D moves fast!  After meeting with the new team, laying out the new idea, and finally getting everyone’s buy-off on it, we typically end the meeting with, “OK, we have three months to build this, let’s go!”  For newly hired Zoosk engineers and any engineer in general, there’s nothing like building new software and Zoosk R&D allows you come in and immediately do that.

Also, instead of just explaining what a particular team or technology does, the new hires have an opportunity to get hands on with the code in most cases.  At the end of the term, the new developers not only have an abstract idea of what the other teams in the company build, they now have been given the  time and resources to actually dive into how they build it.

2. Cross training for current employees.

Another benefit I noticed after concluding my session of R&D was that current Zoosk employees were able to experiment in roles that were different than their primary job responsibilities.  One of the former UI developers on the web team was interested in product management and was able to assume the role as the part time PM for the R&D team.  Since then, she’s brought her passion for the product with her as the lead of the User Experience team at Zoosk, concentrating on our end user satisfaction.  Another developer was able continue his interest in graphic design as the main designer of the R&D product for that term.

3. Team building

Starting at a new company can often be intimidating for a lot of people, and I felt that the R&D program really took that extra step to help put new employees at ease.  Since the team consists mainly of new hires, you’re starting alongside people who are in the same situation as you.

In the old on-boarding model, after starting on a particular team, you end up interacting mainly with that specific team as opposed to members of other teams throughout the engineering organization.   While you eventually get to know people in the other teams, it naturally can take a bit longer.  With Zoosk R&D, it really helped to accelerate the team building process throughout the organization.  Since the members of the team you just spent your first three months at Zoosk with will also be transitioning into other teams throughout the company, it’s nice to have already built up relationships and a camaraderie with the other engineers.  This wasn’t one of the main reasons we started Zoosk R&D, but it just turned out to be a very nice side effect.

Overall, Zoosk R&D was a great experience for me and everyone involved in the project.  With our third term of R&D underway, we have already expanded the program to six developers, a graphic designer, and a product manager.  After the success of our previous R&D sessions, I can’t wait to see what the new team  comes up with!

This drive for change causes our engineers to be continually moving through development projects.  They will work on one feature in a two week sprint, launch it, and then move on to the next one.  This allows them to be exposed to a number of technologies and frameworks within our infrastructure very quickly.  Their technical skill set grows rapidly and within a short time are often making significant contributions to the code base.  In many cases, individual engineers are the primary maintainers of major software systems and product features.

From a professional development point of view, I think this intensity makes Zoosk engineers enter into some sort of accelerated time warp, where they grow their skill set more quickly than their peers at other, usually larger, companies.  An easy way to describe this phenomenon is by using the analogy of dog years.  They say that one dog year is equal to about 7 human years.  Applied to Zoosk, I think that one year as an engineer at Zoosk is equal to about 4 years at another technology company.  This assertion has been loosely supported through information gathered in interviews.  I will often ask a senior engineer who spent several years at company Y to describe for me their accomplishments at that company.  Usually, they name one or two large projects, with some general maintenance work.  Then, I will look at their resume again and note that they spent 4 years at company Y.  I think to myself – “Wow, that’s what you accomplished in 4 years. The average Zoosk engineer would blow through those projects in a year or less.”  Similarly, when Zoosk engineers conduct interviews, this multiplier kicks in.  A Zoosk engineer 2-3 years out of college can give a “seasoned” engineer with 8-10 years of experience at other companies a tough interview.

When filling leadership positions within the engineering group, this same accelerated growth effect applies as well.  Many of our team managers were promoted from within the organization.  Usually, they had held no prior engineering management roles before joining Zoosk or were recruited right out of college.  However, through the intensity of the Zoosk experience and a willingness to take ownership over outcomes, they demonstrated basic leadership skills. We then put them in a formal leadership position, gave frequent coaching and let them learn quickly through real-world application.  Within 6-12 months, these individuals become very competent leaders that I would match against long-time managers at bigger companies.  They are practiced problem solvers who know how to direct their teams to get projects done.  And because of the Zoosk time warp, they can hone their leadership skills very quickly.

The environment of constant change at Zoosk isn’t for everyone and certainly brings its share of stress and discomfort.  But, for those willing to put in the effort, I think the Zoosk experience is invaluable for professional development.  If you would like to start working in “Z” years, we are hiring.  Check out our Careers page for open positions.

There are a few typical approaches to answer this question.

Traditionally to gain business intelligence, one can leverage a star schema data warehouse with a multi-dimensional OLAP engine, to provide the business with a user-friendly toolset to quickly “slice and dice” data to identify trends and patterns.  These toolsets can be something that users are familiar with, such as Microsoft Excel and web dashboards. However, traditional data warehouses are typically not as flexible with semi-structured data, especially when data volume gets very large.

More recently, one can leverage what is known as “Big Data”, using tools such as Hadoop and Hive, to cost effectively deal with large amounts of data in unprecedented variety, velocity and volume.   Using MapReduce, Hadoop can effectively process large amounts of data cheaply by splitting the data into smaller nodes.  It also provides the advantage of being able to store unstructured data.  However, unlike traditional DW, analytics via Hadoop/Hive is still relatively primitive and currently lacks very user friendly tools for the business user, who may not want to write their own Hive-SQL queries.

So what do we do at Zoosk?  Simple, do both!

The overall Zoosk Analytics Platform high level architecture is illustrated as follows:

1)      We first start with collecting as much as data as we need.  Our website writes user behavior data to a scribe log, while the web server collects vital web statistics in its web logs.  The web logs are considered our “Big Data” portion of our source data.

2)      In parallel, we also pull mission critical, structured, production data directly from our user cluster database.  These database clusters include user profile data, financial information and other transaction based information.  These are considered the “Small Data” portion of our source data.

3)      “Big Data” such as scribe logs / web logs are staged in Hadoop, where we also have Hive on top of it to provide ad hoc analytical access to the underlying data.  It also acts as a first stage data integration process to structure and aggregate our “Big Data” into more manageable information.

4)      Database scripts and Extract, Transform, Load (ETL) programs are used to load both our “Big Data” and “Small Data” sets into our RDBMS staging tables.

5)      By using industry proven methodologies in data warehouse / dimensional modeling, we conform all of our data with over 100 dimensions across 20+ fact tables, and store them in a centralized Enterprise Data Warehouse (EDW).  The EDW can be used for ad hoc query analysis as well, but its main purpose is to populate our OLAP multidimensional data marts.

6)      OLAP data marts have all the dimensional structures, KPIs, and metrics defined so users can simply browse our cube via Excel or web reports, easily navigate our user friendly BI universe, or create their own reports based on a single version of truth.  They are aggregation aware, and can automatically direct users’ traffic on most used reports to our aggregation cache providing them with sub second query response time.

7)      The Zoosk analytical platform enables our users to gain insights from the data via different complimentary vehicles.  Power users can perform ad hoc analysis directly from Hive or SQL EDW.  Further, power users can connect to the OLAP cube directly via toolsets such as MS Excel and perform data pivot analysis.    Advance visualization techniques, such as charts, gauges, sparklines, in-cell data bars and conditional formatting, can be employed to provide executive dashboard reporting.   Standard template reports are also available online and in mobile platform for 24/7 anywhere data availability.
Below is a few interesting statistics of our analytics platform:

8 OLAP cubes

20+ Fact tables

90 conformed dimensions in the main cube alone

150+ cube dimensions

450+ measures across the measure groups in the main cube alone

3.6 billion session logs processed a year

10 billion tracked interactions tracked a year

80 billion web traffic count tracked a year

130 billion rows of data ingest to Hadoop a year

40 Terabytes of data ingest to Hadoop a year

By combining traditional DW / BI architecture with newer technologies such as Hadoop / Hive, we are able to provide:

1. Highly scalable infrastructure that easily handles unstructured data, as well as scale to large amounts of data typical to websites
2. Structured, time proven, enterprise-wide information repository consolidating all incoming data and provide a single version of truth to all of our users.
3. Conformed analytic platform which enable cross analysis to be performed across various different fact metrics via conformed dimensions.
4. User friendly analytical platform which allow browsing and creation of custom reports without deep technical knowledge
5. Performance optimized analytical platform where OLAP / Usage based aggregations help provide sub-second query response time for standard queries / reports

Check out the article here:  Zoosk – The Engineering Behind Real Time Communications.  Look for more articles in the future about our evolving use of XMPP based technologies.

Our general format is that we’ll all eat dinner together (thanks, Zoosk!), and then one woman will give a casual 15-minute presentation on some front-end tip or trick, could be Javascript, HTML, or CSS. Afterwards, we’ll spend a couple hours on JS tutorials. We’re currently using Code Academy, but might try other sites, too. When one person gets stuck, we can pitch in to help, or if someone has a side project they need any front-end help with, we can do that too.

I’m proud to work at a company that not only values employee training, but also community outreach and support of women in technology.  So… if you or anyone you know is a woman who’d like to learn Javascript or improve her skills, come join us tonight or any other Wednesday evening!