Moving to Modern Data Architecture
WATI is optimizing the ARIES Enterprise Data Warehouse to Modern Data Architecture using Apache Hadoop
- Modern Data Architecture (MDA) implementation with Hadoop
- Big Data solution to augment the current Microsoft Enterprise Data Warehouse (EDW)
- Rapid integration of data sources into ARIES environment using ETL
ARIES 2.0 will be in production offering the law enforcement officers improved user experience with a modern user interface that is adaptable to any mobile device. ARIES user experience can only be better when the response in the UI is rapid across the multiple agencies that ARIES caters to. ARIES user experience can only be better when the response in the UI is rapid across the multiple agencies that ARIES caters to. The workload on already constrained ARIES team will significantly increase in coordinating and processing data extractions, transformations and loading to ARIES.
In the current approach, each law enforcement agency is responsible for writing the necessary views, and the necessary queries for ARIES to retrieve the data from their database(s). The efficiency of these queries and the proper usage of indexes will significantly affect the impact the ARIES system has on the performance of the agency’s database. The process causes significant burden on each of the agencies’ staff and can also slowdown the integration into the ARIES data warehouse.
After supporting ARIES on the UI, West Advanced Technologies, Inc., (WATI), continues its efforts to support, improve and enhance ARIES by improving the efficiency in data integration and transformation. The goals for this engagement are:
- Update and Optimize ARIES Data warehouse to prepare for future growth
- Improve new source integration to reduce overall time and effort
The solution enables rapid data integration to ARIES using modern technologies like Big Data and Hadoop at a fraction of the current costs and efforts, while still protecting the existing investments of ARIES in the Microsoft SQL Server data warehouse.
WATI implements Modern Data Architecture (MDA), where WATI will implement Hadoop Big Data solution to augment the current Microsoft Enterprise Data Warehouse (EDW).
Hadoop with its distributed compute framework can process large volumes ARIES data across multiple agencies and can provide near real-time response on warrant and inmate searches. Users will benefit from live data and the access speed, instead of having to choose between them.
The new data sources will leverage MDA to load data efficiently into the ARIES data warehouse through Hadoop. These new integrations will happen at a much rapid pace in an efficient manner with minimal disruption.
Some of the significant benefits that ARIES realize from this project implementation include:
- Rapid integration of data sources into ARIES environment
- Improved efficiency in integration reducing cost, time and efforts
- Opportunity to enable innovative new data models
- Potential for new insights into the data that provide predictive and preventive analysis
- Ability to have larger data sets for analysis as the data collected and stored continues to grow exponentially
- Cost advantages of commodity hardware & open source software
This project is currently under implementation.