Case Studies

Importing Data from Any Data Source to a Data Warehouse

CHALLENGE
A federal retail network faced a problem with collecting information from different Excel-files and data sources and importing them to a Data Warehouse.

The client had no single data source for business intelligence and data visualization tools, because employees collected data about sales, competitors’ activities, promotions, etc in non-structured Excel-files and difference data bases on their computers.

The client asked Byndyusoft to develop a tool for continuously importing any data source to a Data Warehouse and tune it through a web interface.

SOLUTION

Byndyusoft worked together with the client’s Reporting and Analysis Department. We found that almost all data sources can export data in an Excel format so this format was chosen as a default for the data transfer.

In our system, an uploaded Excel file or a linked data source is parsed using a particular template. The template contains rules with constraints and metadata about internal data. The system can check an Excel file, show errors and suggest solutions for fixing the problem. After uploading and checking, the system put the data in a structured format and sent it to a Data Warehouse for further analysis and reporting.

Byndyusoft developed a web interface on React.js and Redux.js and solved the problem of processing more than 500,000 rows in a web browser. A back-end was developed on .NET Core by using Docker.

RESULTS

Byndyusoft designed and developed a system that can upload, analyze and import data from any data source.

Any employee can link his data source to a Data Warehouse which will lead to quickly sharing important information within the company.

500,000+ rows
of data can be parsed in a web browser
50 data sources
have already been added by the client’s team
Infinite numbers
of further connected data sources
Data Warehouse · Reporting · Analysis
Byndyusoft launched a GUI importer for any data source to a DWH in a short period of time. A working prototype was developed in just two weeks. They worked according to an established roadmap and after a month we started to use the new system on production servers.
Sergey Oreshkin, Director of Reporting and Analysis Systems at 585 Gold, a federal network of jewelry shops
SHARE