{"id":54809,"date":"2024-06-04T13:58:41","date_gmt":"2024-06-04T17:58:41","guid":{"rendered":"https:\/\/sdtimes.com\/?p=54809"},"modified":"2024-06-05T11:00:16","modified_gmt":"2024-06-05T15:00:16","slug":"rag-is-the-next-exciting-advancement-for-llms","status":"publish","type":"post","link":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/","title":{"rendered":"RAG is the next exciting advancement for LLMs"},"content":{"rendered":"

One of the challenges with generative AI models has been that they tend to hallucinate responses. In other words, they will present an answer that is factually incorrect, but will be confident in doing so, sometimes even doubling down when you point out that what they\u2019re saying is wrong.<\/span><\/p>\n

\u201c[Large language models] can be inconsistent by nature with the inherent randomness and variability in the training data, which can lead to different responses for similar prompts. LLMs also have limited context windows, which can cause coherence issues in extended conversations, as they lack true understanding, relying instead on patterns in the data,\u201d said Chris Kent, SVP of marketing for <\/span>Clarifai<\/span><\/a>, an AI orchestration company.\u00a0<\/span><\/p>\n

Retrieval-augmented generation (RAG) is picking up traction because when applied to LLMs, it can help to reduce the occurrence of hallucinations, as well as offer some other additional benefits.<\/span><\/p>\n

\u201cThe goal of RAG is to marry up local data, or data that wasn’t used in training the actual LLM itself, so that the LLM hallucinates less than it otherwise would,\u201d said Mike Bachman, head of architecture and AI strategy at <\/span>Boomi<\/span><\/a>, an iPaaS company.<\/span><\/p>\n

He explained that LLMs are typically trained on very general data and often older data. Additionally, because it takes months to train these models, by the time it is ready, the data has become even older.\u00a0\u00a0<\/span><\/p>\n

For instance, the <\/span>free version of ChatGPT<\/span><\/a> uses GPT-3.5, which cuts off its training data in January 2022, which is nearly 28 months ago at this point. The paid version that uses GPT-4 gets you a bit more up-to-date, but still only has information from up to April 2023.<\/span><\/p>\n

\u201cYou’re missing all of the changes that have happened from April of 2023,\u201d Bachman said. \u201cIn that particular case, that\u2019s a whole year, and a lot happens in a year, and a lot has happened in this past year. And so what RAG will do is it could help shore up data that’s changed.\u201d<\/span><\/p>\n

As an example, in 2010 Boomi was acquired by Dell, but in 2021 Dell divested the company and now Boomi is privately owned again. According to Bachman, earlier versions of GPT-3.5 Turbo were still making references to Dell Boomi, so they used RAG to supply the LLM with up-to-date knowledge of the company so that it would stop making those incorrect references to Dell Boomi.\u00a0<\/span><\/p>\n

RAG can also be used to augment a model with private company data to provide personalized results or to support a specific use case.\u00a0<\/span><\/p>\n

\u201cI think where we see a lot of companies using RAG, is they’re just trying to basically handle the problem of how do I make an LLM have access to real-time information or proprietary information beyond the time period or data set under which it was trained,\u201d said Pete Pacent, head of product at Clarifai.<\/span><\/p>\n

For instance, if you\u2019re building a copilot for your internal sales team, you could use RAG to be able to supply it with up-to-date sales information, so that when a salesperson asks \u201chow are we doing this quarter?\u201d the model can actually respond with updated, relevant information, said Pacent.\u00a0\u00a0<\/span><\/p>\n

The challenges of RAG<\/b><\/h5>\n

Given the benefits of RAG, why hasn\u2019t it seen greater adoption so far? According to Clarifai\u2019s Kent, there are a couple factors at play. First, in order for RAG to work, it needs access to multiple different data sources, which can be quite difficult, depending on the use case.\u00a0<\/span><\/p>\n

RAG might be easy for a simple use case, such as conversation search across text documents, but much more complex when you apply that use case across patient records or financial data. At that point you\u2019re going to be dealing with data with different sources, sensitivity, classification, and access levels.\u00a0<\/span><\/p>\n

It\u2019s also not enough to just pull in that data from different sources; that data also needs to be indexed, requiring comprehensive systems and workflows, Kent explained.\u00a0<\/span><\/p>\n

And finally, scalability can be an issue. \u201cScaling a RAG solution across maybe a server or small file system can be straightforward, but scaling across an org can be complex and really difficult,\u201d said Kent. \u201cThink of complex systems for data and file sharing now in non-AI use cases and how much work has gone into building those systems, and how everyone is scrambling to adapt and modify to work with workload intensive RAG solutions.\u201d<\/span><\/p>\n

RAG vs fine-tuning<\/b><\/h5>\n

So, how does RAG differ from fine-tuning? With fine-tuning, you are providing additional information to update or refine an LLM, but it\u2019s still a static mode. With RAG, you\u2019re providing additional information on top of the LLM. \u201cThey enhance LLMs by integrating real-time data retrieval, offering more accurate and current\/relevant responses,\u201d said Kent.\u00a0<\/span><\/p>\n

Fine-tuning might be a better option for a company dealing with the above-mentioned challenges, however. Generally, fine-tuning a model is less infrastructure intensive than running a RAG.\u00a0<\/span><\/p>\n

\u201cSo performance vs cost, accuracy vs simplicity, can all be factors,\u201d said Kent. \u201cIf organizations need dynamic responses from an ever-changing landscape of data, RAG is usually the right approach. If the organization is looking for speed around knowledge domains, fine-tuning is going to be better. But I\u2019ll reiterate that there are a myriad of nuances that could change those recommendations.\u201d<\/span><\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

One of the challenges with generative AI models has been that they tend to hallucinate responses. In other words, they will present an answer that is factually incorrect, but will be confident in doing so, sometimes even doubling down when you point out that what they\u2019re saying is wrong. \u201c[Large language models] can be inconsistent … continue reading<\/a><\/p>\n","protected":false},"author":752,"featured_media":54810,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"om_disable_all_campaigns":false,"cybocfi_hide_featured_image":"","footnotes":"","_links_to":"","_links_to_target":""},"categories":[1],"tags":[143,16325,16974],"coauthors":[11687],"acf":[],"yoast_head":"\nRAG is the next exciting advancement for LLMs - SD Times<\/title>\n<meta name=\"description\" content=\"RAG can help reduce LLM hallucinations and provide LLMs access to private data and up-to-date, current information.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"RAG is the next exciting advancement for LLMs - SD Times\" \/>\n<meta property=\"og:description\" content=\"RAG can help reduce LLM hallucinations and provide LLMs access to private data and up-to-date, current information.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/\" \/>\n<meta property=\"og:site_name\" content=\"SD Times\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/SDTimesD2\" \/>\n<meta property=\"article:published_time\" content=\"2024-06-04T17:58:41+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-06-05T15:00:16+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/sdtimes.com\/wp-content\/uploads\/2024\/06\/pexels-cottonbro-5473955.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1280\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Jenna Barron\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@sdtimes\" \/>\n<meta name=\"twitter:site\" content=\"@sdtimes\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Jenna Barron\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/\"},\"author\":{\"name\":\"Jenna Barron\",\"@id\":\"https:\/\/sdtimes.com\/#\/schema\/person\/f2524e55ae19da07ea3613577da9f786\"},\"headline\":\"RAG is the next exciting advancement for LLMs\",\"datePublished\":\"2024-06-04T17:58:41+00:00\",\"dateModified\":\"2024-06-05T15:00:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/\"},\"wordCount\":883,\"publisher\":{\"@id\":\"https:\/\/sdtimes.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/sdtimes.com\/wp-content\/uploads\/2024\/06\/pexels-cottonbro-5473955.jpg\",\"keywords\":[\"AI\",\"LLM\",\"RAG\"],\"articleSection\":[\"Latest News\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/\",\"url\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/\",\"name\":\"RAG is the next exciting advancement for LLMs - SD Times\",\"isPartOf\":{\"@id\":\"https:\/\/sdtimes.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/sdtimes.com\/wp-content\/uploads\/2024\/06\/pexels-cottonbro-5473955.jpg\",\"datePublished\":\"2024-06-04T17:58:41+00:00\",\"dateModified\":\"2024-06-05T15:00:16+00:00\",\"description\":\"RAG can help reduce LLM hallucinations and provide LLMs access to private data and up-to-date, current information.\",\"breadcrumb\":{\"@id\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#primaryimage\",\"url\":\"https:\/\/sdtimes.com\/wp-content\/uploads\/2024\/06\/pexels-cottonbro-5473955.jpg\",\"contentUrl\":\"https:\/\/sdtimes.com\/wp-content\/uploads\/2024\/06\/pexels-cottonbro-5473955.jpg\",\"width\":1920,\"height\":1280},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/sdtimes.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"RAG is the next exciting advancement for LLMs\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/sdtimes.com\/#website\",\"url\":\"https:\/\/sdtimes.com\/\",\"name\":\"SD Times\",\"description\":\"Software Development News\",\"publisher\":{\"@id\":\"https:\/\/sdtimes.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/sdtimes.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/sdtimes.com\/#organization\",\"name\":\"SD Times\",\"url\":\"https:\/\/sdtimes.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/sdtimes.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/sdtimes.com\/wp-content\/uploads\/2014\/05\/deafaultlogo.png\",\"contentUrl\":\"https:\/\/sdtimes.com\/wp-content\/uploads\/2014\/05\/deafaultlogo.png\",\"width\":225,\"height\":90,\"caption\":\"SD Times\"},\"image\":{\"@id\":\"https:\/\/sdtimes.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/SDTimesD2\",\"https:\/\/x.com\/sdtimes\",\"https:\/\/www.linkedin.com\/company\/sdtimes\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/sdtimes.com\/#\/schema\/person\/f2524e55ae19da07ea3613577da9f786\",\"name\":\"Jenna Barron\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/sdtimes.com\/#\/schema\/person\/image\/b4be3423b187642936e62f121111345e\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b128943929626cdcafccbac86bd306f9?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b128943929626cdcafccbac86bd306f9?s=96&d=mm&r=g\",\"caption\":\"Jenna Barron\"},\"description\":\"Jenna Barron is News Editor of SD Times.\",\"url\":\"https:\/\/sdtimes.com\/author\/jennifer-sargent\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"RAG is the next exciting advancement for LLMs - SD Times","description":"RAG can help reduce LLM hallucinations and provide LLMs access to private data and up-to-date, current information.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/","og_locale":"en_US","og_type":"article","og_title":"RAG is the next exciting advancement for LLMs - SD Times","og_description":"RAG can help reduce LLM hallucinations and provide LLMs access to private data and up-to-date, current information.","og_url":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/","og_site_name":"SD Times","article_publisher":"https:\/\/www.facebook.com\/SDTimesD2","article_published_time":"2024-06-04T17:58:41+00:00","article_modified_time":"2024-06-05T15:00:16+00:00","og_image":[{"width":1920,"height":1280,"url":"https:\/\/sdtimes.com\/wp-content\/uploads\/2024\/06\/pexels-cottonbro-5473955.jpg","type":"image\/jpeg"}],"author":"Jenna Barron","twitter_card":"summary_large_image","twitter_creator":"@sdtimes","twitter_site":"@sdtimes","twitter_misc":{"Written by":"Jenna Barron","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#article","isPartOf":{"@id":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/"},"author":{"name":"Jenna Barron","@id":"https:\/\/sdtimes.com\/#\/schema\/person\/f2524e55ae19da07ea3613577da9f786"},"headline":"RAG is the next exciting advancement for LLMs","datePublished":"2024-06-04T17:58:41+00:00","dateModified":"2024-06-05T15:00:16+00:00","mainEntityOfPage":{"@id":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/"},"wordCount":883,"publisher":{"@id":"https:\/\/sdtimes.com\/#organization"},"image":{"@id":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#primaryimage"},"thumbnailUrl":"https:\/\/sdtimes.com\/wp-content\/uploads\/2024\/06\/pexels-cottonbro-5473955.jpg","keywords":["AI","LLM","RAG"],"articleSection":["Latest News"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/","url":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/","name":"RAG is the next exciting advancement for LLMs - SD Times","isPartOf":{"@id":"https:\/\/sdtimes.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#primaryimage"},"image":{"@id":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#primaryimage"},"thumbnailUrl":"https:\/\/sdtimes.com\/wp-content\/uploads\/2024\/06\/pexels-cottonbro-5473955.jpg","datePublished":"2024-06-04T17:58:41+00:00","dateModified":"2024-06-05T15:00:16+00:00","description":"RAG can help reduce LLM hallucinations and provide LLMs access to private data and up-to-date, current information.","breadcrumb":{"@id":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#primaryimage","url":"https:\/\/sdtimes.com\/wp-content\/uploads\/2024\/06\/pexels-cottonbro-5473955.jpg","contentUrl":"https:\/\/sdtimes.com\/wp-content\/uploads\/2024\/06\/pexels-cottonbro-5473955.jpg","width":1920,"height":1280},{"@type":"BreadcrumbList","@id":"https:\/\/sdtimes.com\/ai\/rag-is-the-next-exciting-advancement-for-llms\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sdtimes.com\/"},{"@type":"ListItem","position":2,"name":"RAG is the next exciting advancement for LLMs"}]},{"@type":"WebSite","@id":"https:\/\/sdtimes.com\/#website","url":"https:\/\/sdtimes.com\/","name":"SD Times","description":"Software Development News","publisher":{"@id":"https:\/\/sdtimes.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sdtimes.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/sdtimes.com\/#organization","name":"SD Times","url":"https:\/\/sdtimes.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/sdtimes.com\/#\/schema\/logo\/image\/","url":"https:\/\/sdtimes.com\/wp-content\/uploads\/2014\/05\/deafaultlogo.png","contentUrl":"https:\/\/sdtimes.com\/wp-content\/uploads\/2014\/05\/deafaultlogo.png","width":225,"height":90,"caption":"SD Times"},"image":{"@id":"https:\/\/sdtimes.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/SDTimesD2","https:\/\/x.com\/sdtimes","https:\/\/www.linkedin.com\/company\/sdtimes\/"]},{"@type":"Person","@id":"https:\/\/sdtimes.com\/#\/schema\/person\/f2524e55ae19da07ea3613577da9f786","name":"Jenna Barron","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/sdtimes.com\/#\/schema\/person\/image\/b4be3423b187642936e62f121111345e","url":"https:\/\/secure.gravatar.com\/avatar\/b128943929626cdcafccbac86bd306f9?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b128943929626cdcafccbac86bd306f9?s=96&d=mm&r=g","caption":"Jenna Barron"},"description":"Jenna Barron is News Editor of SD Times.","url":"https:\/\/sdtimes.com\/author\/jennifer-sargent\/"}]}},"_links":{"self":[{"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/posts\/54809"}],"collection":[{"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/users\/752"}],"replies":[{"embeddable":true,"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/comments?post=54809"}],"version-history":[{"count":2,"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/posts\/54809\/revisions"}],"predecessor-version":[{"id":54813,"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/posts\/54809\/revisions\/54813"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/media\/54810"}],"wp:attachment":[{"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/media?parent=54809"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/categories?post=54809"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/tags?post=54809"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/sdtimes.com\/wp-json\/wp\/v2\/coauthors?post=54809"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}