Skyscanner, today one of the world’s best-known flight aggregators and a metasearch engine, grew from
a 3-person start-up to a billion-dollar company with millions of users across the world.
Apart from software development and testing projects delivered by Order of Code for Skyscanner, the team supported the company with R&D projects.
While growing their business, Skyscanner encountered several problems, each generating significant costs or performance issues. As some of these problems required finding a tailored solution, Order of Code supported Skyscanner with R&D projects.
R&D projects, including bot detection and elimination, system optimisation, and NLP implementation, eventually helped to exclude unauthorised website traffic, increased the website’s speed for their users, improved database, and allowed for NLP’s use on the website. At the same time, solutions provided by OoC cut the cost of website maintenance and improved customer satisfaction.
Bot detection and elimination
With the growing userbase (at the time around 25 million visits per month), Skyscanner became prone to automated bots visiting the website and retrieving data from servers. These visits, in turn, increased the cost of equipment and Internet traffic significantly.
The OoC designed and developed a prototype for traffic analysis, that helped to detect and block bots entering the website. Based on behavioural patterns analysis, suspicious IP addresses were blocked automatically.
The implementation of this prototype helped to significantly reduce the amount of unwanted traffic generated by bots, saving Skyscanner millions of dollars in the long run, increasing website speed for regular users at the same time.
Growing rapidly, Skyscanner had to overcome performance issues regarding data aggregation. Due to increasing amount of data, price information hadn’t displayed properly. As data inconsistency meant revenue loses and lower user satisfaction, the OoC team rebuilt the system dividing it into several smaller, individually scalable tasks.
This solution improved overall performance of the website and eliminated a single point of failure. The new system also allowed for more relevant search results. As a result, the database shrank 10x, and the time needed to obtain information by the user was reduced by half, improving response time.
NLP-powered search engine
To improve how users interact with the website, Skyscanner decided to implement the ability to parse and interpret natural language queries. Apart from returning key/value pairs based on users’ queries, the system had to support multiple languages and recognize common mistakes and incorrect grammar forms.
The OoC team developed algorithms for recognizing text queries, linked with Skyscanner’s flight data API. The system, first developed for English, Polish, and Italian, was later expanded for Chinese Traditional, Spanish, and Russian. This way, Skyscanner’s users could as questions like: “How much is the cheapest airline ticket to London for me and my daughter for next weekend”, and receive an answer based on Skyscanner’s data.