The use of declarative transformational languages to enhance the ETL process
Keywords:
declarative transformation languages, ETL, data processing, data engineeringAbstract
The Extract, Transform, Load (ETL) method helps businesses in the dynamic data management environment to efficiently control and apply their data. Conventions in ETL might show inefficiencies and complexity that compromise data integration and quality. This work looks at how declarative transformation languages may be used to enhance the ETL process. Moreover, by abstracting the fundamental implementation details, declarative transformation languages enable companies to quickly adapt to evolving data needs and hence provide a more flexible ETL technique. This work will investigate numerous declarative languages and their impact on the ETL process by means of case studies highlighting their applicability in pragmatic implementations.
References
1. Raminhos, R. F., & Moura-Pires, J. (2007, June). Extraction and transformation of data from semi-structured text files using a declarative approach. In Ninth International Conference on Enterprise Information Systems, Madeira, Portugal.
2. Theodorou, V., Abelló, A., Thiele, M., & Lehner, W. (2014, November). A framework for user-centered declarative etl. In Proceedings of the 17th international workshop on data warehousing and OLAP (pp. 67-70).
3. Jörg, T., & Deßloch, S. (2008, September). Towards generating ETL processes for incremental loading. In Proceedings of the 2008 international symposium on Database engineering & applications (pp. 101-110).
4. Bansal, S. K. (2014, June). Towards a semantic extract-transform-load (ETL) framework for big data integration. In 2014 IEEE International Congress on Big Data (pp. 522-529). IEEE.
5. El-Sappagh, S. H. A., Hendawi, A. M. A., & El Bastawissy, A. H. (2011). A proposed model for data warehouse ETL processes. Journal of King Saud University-Computer and Information Sciences, 23(2), 91-104.
6. Vassiliadis, P., & Simitsis, A. (2009). Extraction, Transformation, and Loading. Encyclopedia of Database Systems, 10, 14.
7. Deufemia, V., Giordano, M., Polese, G., & Tortora, G. (2014). A visual language‐based system for extraction–transformation–loading development. Software: Practice and Experience, 44(12), 1417-1440.
8. Vassiliadis, P., Simitsis, A., Georgantas, P., Terrovitis, M., & Skiadopoulos, S. (2005). A generic
and customizable framework for the design of ETL scenarios. Information Systems, 30(7), 492-525.
9. Chakraborty, J., Padki, A., & Bansal, S. K. (2017, January). Semantic etl—State-of-the-art and open 16.research challenges. In 2017 IEEE 11th International Conference on Semantic Computing (ICSC) (pp. 413-418). IEEE.
10. Sellis, T. K., & Simitsis, A. (2007, September). Etl workflows: From formal specification to optimization.In East European Conference on Advances in Databases and Information Systems (pp. 1-11). Berlin, Heidelberg: Springer Berlin Heidelberg.
11. Vassiliadis, P., Vagena, Z., Skiadopoulos, S., Karayannidis, N., & Sellis, T. (2001). ARKTOS: towards the modeling, design, control and execution
of ETL processes. Information Systems, 26(8), 537-561.
12. Samimi-Dehkordi, L., Khalilian, A., & Zamani, B. (2016). Applying Programming Language EvaluationCriteria for Model Transformation Languages. International Journal of Software & Informatics, 10(4).
13. Schubert, L. (2010). An evaluation of model transformation languages for uml quality engineering (Doctoral dissertation, Master’s thesis, Masterarbeit im Studiengang Angewandte Informatik am Institute für Informatik, ZFI-MSC-2010-01, ISSN 1612-6793, Zentrum für Informatik, Georg-August-Universität Göttingen).
14. Albrecht, A., & Naumann, F. (2009, August). METL: Managing and Integrating ETL Processes. In VLDB PhD workshop.
15. dos Santos, V. N. C. (2015). A Relational Algebra Approach to ETL Modeling (Doctoral dissertation, Universidade do Minho (Portugal)).
16. Gade, K. R. (2017). Integrations: ETL vs. ELT: Comparative analysis and best practices. Innovative Computer Sciences Journal, 3(1).
17. Gade, K. R. (2017). Integrations: ETL/ELT, Data Integration Challenges, Integration Patterns. Innovative Computer Sciences Journal, 3(1).
18. Komandla, V. Transforming Financial Interactions: Best Practices for Mobile Banking App Design and Functionality to Boost User Engagement and Satisfaction.
19. Gade, K. R. (2018). Real-Time Analytics: Challenges and Opportunities. Innovative Computer Sciences Journal, 4(1).
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
