Machine learning and statistics with python

Introduction: So there has been a lot of developments in LLM and I have not gone through any of it. In the coming few parts, I will talk about LLM and its related eco-system that has developed and will try to reach to the cutting or more like bleeding edge. Lets go through the main concepts first. What is LLM? LLM[1] refers to large language models; that refer to mainly deep learning based big transformer models that can perform the natural language understanding and natural language generation tasks much better than the previous versions of the models generated in NLP history. LLM models are generally quite big, in terms of 10-100GBs and they can't fit in even in one machine's ram. So, most LLMs are inferenced using bigger GPU cluster systems and are quite computationally exhaustive. What was the first true LLM? The BERTs Transformers were invented on 2017 by vaswani et al in their revolutionary paper called "attention is all you need". After that we had the BER...

 Introduction: Large Language models are now part of the latest data science and machine learning craze. Since the invent of transformers and first efficient discussion of it is vaswani et al. paper "attention is all you need" and the starting of training ever big models by organizations such as openai, google, microsoft, mistral, etc; we have come across models that are very large deep neural network models with a transformer architecture underlying. These models are generally having 1 Billion or more parameters ; and they perform quite well in generative AI tasks such as comprehensive text generation, instructed text creation, task completion and others.  In this article, we are going to talk about how we have landed in this genre, where did we come from; and also we will finish with providing you ways to start using these models from both huggingface and openai. The evolution of NLP models: Large Language Models (LLMs) have seen significant development and progress in...