Credit Karma serves more than 60 million members as a broker and source of information for financial services. The site tailors recommendations for loans for people’s particular situations, and the business revenue comes in from approved financial offers that people discovered through the site’s service.
However, if consumers can’t get to the features or products on the site, the possibility of closing the deal and earning money for the company is lost. Though Credit Karma does its own checks on new feature and product rollouts, there’s far too much going out to be handled by manual tracking.
Pedro Silva, senior product manager, explains that problems can result from something in “the rollout of a new feature (or business logic) that has not been captured in the QA process.” Though they do their own checks, they “have many scrum teams shipping hundreds of features per week in a nearly continuous release cycle.”
On top of that, a single small change in a feature could have ramifications for a number of other Web features and business metrics. Consequently, Silva says, it is virtually impossible for an individual to trace “the impact of the release of each feature on every page, vertical, platform, partner in a manual way.”
Finding it too costly for Credit Karma to develop its own user interface to provide the visualization and metrics necessary to create alerts for anomies in real time, the company sought an external solution. It found it in Anodot, a real-time analytics and anomaly detection system.
Analytics Tool Spots Needles in Haystacks in Real Time
“Anodot applies time-series-based machine learning to determine what a normal pattern is and what represents an anomaly in an automatic way, across all metrics,” Silva explains. Plus, it integrates well with the other communication tools used by Credit Karma. Currently, three different teams use it, and more teams are slated to be onboarded to the platform.
The first team to use this product was the personalization team, which ran the pilot project. “They added all the internal tooling for Anodot within Credit Karma, and are currently leading the adoption with the organization,” he reports.
The platform has also been embraced by the mobile team, which uses it to visualize certain performance metrics and to get alert of anomalies. That way, they can ascertain that the mobile API is functioning as it should. “This is essential, as members rely on us to be available 24/7,” Silva points out.
The team got to appreciate how quickly they were able to resolve problems when Anodot detected a mobile issue within the first four days of its use. That “definitely spurred the team’s confidence in the product,” he says.
The build-and-release team uses Anodot as a final checkpoint. If a problem is indicated, they would make the call on whether or not to hold back the release. “Being able to, at a glance, see all the abnormalities of our monolithic application for any time period greatly facilitates the process for them, saves time and money, and results in a smoother user experience.” Silva says,
Credit Karma’s goal is to get more teams to use Anodot as part of fostering an internal culture that demands the standard of real-time measurement and alerts. Silva likens Anodot’s visualization and alert to a car’s windshield and dashboard: “How can you know where you’re going, and how fast you’re going, if you don’t have the right visibility? For us, it’s the same thing, except we have 50 million-plus ‘passengers.'”
