Projeto e implementação de subsistema para agendamento, execução e monitoramento de processos ETL

Abstract

During the creation of a data warehouse environment, one of the most important steps is the development of the ETL system (Extract, Transform, Load), which is responsible for the extraction of data in operational systems and external sources, the transformations that these data have to suffer and the load of facts and dimensions. Although crucial, the processes of extraction, transformation and load are only a part of the life cycle of an ETL system development. As important as the process, is the correct scheduling, the precise execution and the continuous monitoring. These activities are responsibility of an ETL subsystem, known as Scheduler. This paper proposes a Scheduler for ETL processes. With the use of metadata, the used approach tries to simplify the processes of scheduling, execution and monitoring of ETL processes in a data warehouse environment.