The System continuously collects information from more than 50 000 sites in the English part of the Web and processes at the average 80 000 new articles per day. Besides, it updates data on more than 1 million of already existing articles.
The client requested expert review from Byndyusoft after one of the leading developers of custom designed software in the Eastern Europe failed to design and implement a scalable solution with sufficient performance.
Works on version 1.0 were performed by a team of 4–6
After Byndyusoft took over the development, the tasks for the first six months were stated as follows:
Conclusions, drawn by the Byndyusoft team from mistakes, which had been made by the previous team in the design of the system, wide experience in development of high load systems and professional usage of flexible development techniques allowed to create a new version of the System in compliance with all client’s requirements in the shortest time possible.
Within 6 months, a team of 6 persons developed an operational project, meeting all the assigned objectives.
Byndyusoft team has designed and created the required conveyor of full article processing cycle:
Scalable model of services, interacting through common data bus (RabbitMQ). Processes of reading and writing to database are carefully assigned between the services to prevent interlocking. All information for content output to the user is copied to MongoDB cluster to increase the speed of data output.
Created mechanism of data output to the UI was designed and developed from scratch. In the
process of designing main focus was on fast and relevant output of content to users. All
Based on zabbix and
API provided by web servers for the site UI allowed to create Android application without any changes on the server part. The net result is that both main and mobile versions of site UI, and Android application use the same API.
Results of work, provided by the Byndyusoft team had the following
advantages for the client: