Batch vs. Stream Processing: A Comprehensive Technology Comparison with Guidance on Selecting the Best Method for Particular Applications
Keywords:
Batch processing, stream processing, resource allocation, data processingAbstract
Two important data control techniques that are appropriate for various purposes are batch processing & stream processing. Batch processing is a perfect for jobs like reporting, the historical analysis & ETL procedures that because it can be handled data in big chunks that have been gathered close time. It works well in situations when processing huge amounts of data is necessary but actual time speed is not crucial. Stream processing emphasizes instantaneous processing & actual time data intakes. Applications where prompt decision-making is very crucial, such as fraud detection, live analytics & monitoring systems, are ideal for it. Stream processing guarantees that the fast insights & quick reactions to be changed by the continually processing data as it comes in. Huge non-time-sensitive jobs are better served by the batch processing, while time-sensitive, event-driven situations are the best served by a stream processing. These kind of data, latency constraints & throughput the demands of all influences the best course of action. Knowing each's advantages may be improves performance, cost-effectiveness & responsiveness by assisting businesses in making well-informed judgments for jobs like data warehousing, actual time analytics or event processing.
References
1. Andrade, H. C., Gedik, B., & Turaga, D. S. (2014). Fundamentals of stream processing: application design, systems, and analytics. Cambridge University Press.
2. Chakravarthy, S., & Jiang, Q. (2009). Stream data processing: a quality of service perspective: modeling, scheduling, load shedding, and complex event processing (Vol. 36). Springer Science & Business Media.
3. Belhadi, A., Zkik, K., Cherrafi, A., & Sha'ri, M. Y. (2019). Understanding big data analytics for manufacturing processes: insights from literature review and multiple case studies. Computers & Industrial Engineering, 137, 106099.
4. Zhang, B., Jin, X., Ratnasamy, S., Wawrzynek, J., & Lee, E. A. (2018, August). Awstream: Adaptive wide-area streaming analytics. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (pp. 236-252).
5. Goudarzi, M. (2017). Heterogeneous architectures for big data batch processing in map reduce paradigm. IEEE Transactions on Big Data, 5(1), 18-33.
6. Pääkkönen, P., & Pakkala, D. (2015). Reference architecture and classification of technologies, products and services for big data systems. Big data research, 2(4), 166-186.
7. Narkhede, N., Shapira, G., & Palino, T. (2017). Kafka: the definitive guide: real-time data and stream processing at scale. " O'Reilly Media, Inc.".
8. Chanda, A., Daly, A. M., Foley, D. A., LaPack, M. A., Mukherjee, S., Orr, J. D., ... & Ward, H. W. (2015). Industry perspectives on process analytical technology: tools and applications in API development. Organic Process Research & Development, 19(1), 63-83.
9. Banerjee, A. (2018). Blockchain technology: supply chain insights from ERP. In Advances in computers (Vol. 111, pp. 69-98). Elsevier.
10. Besnard, L., Fabre, V., Fettig, M., Gousseinov, E., Kawakami, Y., Laroudie, N., ... & Pattnaik, P. (2016). Clarification of vaccines: An overview of filter based technology trends and best practices. Biotechnology advances, 34(1), 1-13.
11. Hanes, D., Salgueiro, G., Grossetete, P., Barton, R., & Henry, J. (2017). IoT fundamentals: Networking technologies, protocols, and use cases for the internet of things. Cisco Press.
12. Duflou, J. R., Sutherland, J. W., Dornfeld, D., Herrmann, C., Jeswiet, J., Kara, S.,... & Kellens, K. (2012). Towards energy and resource efficient manufacturing: A processes and systems approach. CIRP annals, 61(2), 587-609.
13. Wilderer, P. A., Irvine, R. L., & Goronszy, M. C. (Eds.). (2001). Sequencing batch reactor technology. IWA publishing.
14. Fung, H. P. (2014). Criteria, use cases and effects of information technology process automation (ITPA). Advances in Robotics & Automation, 3.
15. Singh, N., Arunkumar, A., Chollangi, S., Tan, Z. G., Borys, M., & Li, Z. J. (2016).
Clarification technologies for monoclonal antibody manufacturing processes: Current state and future perspectives. Biotechnology and bioengineering, 113(4),698-716.
16. Gade, K. R. (2017). Integrations: ETL vs. ELT: Comparative analysis and best practices. Innovative Computer Sciences Journal, 3(1).
17. Gade, K. R. (2019). Data Migration Strategies for Large-Scale Projects in the Cloud for Fintech. Innovative Computer Sciences Journal, 5(1).
18. Gade, K. R. (2017). Migrations: Challenges and Best Practices for Migrating Legacy Systems to Cloud-Based Platforms. Innovative Computer Sciences Journal, 3(1).
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
