Performance and Energy Efficiency Analysis of Language Models of Different Sizes

Abstract

The impact of artificial intelligence on performance is being increasingly studied. Research is being conducted on the uses, benefits, shortcomings, and performance of language models in various fields. The differences between small and large language models are beginning to gain prominence in current research. However, there are still areas for improvement in terms of hardware efficiency and energy consumption. To contribute to this field, this study analyzes various large and small language models based on energy efficiency, measuring their performance and hardware metrics. Seven different language models, selected according to their size, were analyzed based on various metrics. The RAG method, which involves providing the AI with a dataset and having it respond only based on that data, was used in the applications. A mini-dataset was created, and measurements were made using this dataset. The same dataset was used for each language model. In this analysis, conducted using the Python programming language and packages, carbon emissions of the models while running were measured via CodeCarbon. This metric is important for measuring energy efficiency. Additionally, the general intelligence, honesty, speed, TPS, GPU memory usage, and average taken values of the models were measured. Inferences were drawn from the analysis results based on these values. For each metric and for each language model analyzed, the analysis results were specifically evaluated. The results show how energy efficiency can vary depending on the size of the language models and performance metrics. The aim was to conduct research on how models can operate more efficiently and how to improve in the field of sustainable artificial intelligence.