Are AI’s wordsmiths right for your business? The possibilities and the risks
by Julie Ripplinger, András Szabó, Jonathan Leslie
Artificial Intelligence has gained a new, powerful voice with the emergence of multimodal Large Language Models (LLMs). In just the last few months, these cutting-edge systems have mesmerised the world with their ability to conjure up language and imagery almost indistinguishable from human creations. Beyond reading text, they’ve shown that AI can understand image-rich texts, build computer programs, and even pass the Bar exam. Setting the hype aside, LLMs and multimodal LLMs raise crucial questions for businesses: Are they a transformative opportunity or just another passing fad? And are they right for me?
This post delves into the practical applications of multimodal LLMs, exploring their potential to revolutionise industries like marketing, customer service, and content creation. We’ll also examine the technical underpinnings of these models, including their strengths and limitations. Importantly, we’ll address the ethical considerations surrounding LLMs, such as bias and misinformation.
By the end of this exploration, you’ll have a clear understanding of LLMs and their potential impact on your business. We’ll highlight a few of their more common applications. And you’ll be equipped to take informed decisions about whether and how to integrate LLMs into your strategies, ensuring you don’t get swept away by the hype and instead leverage the true power of this technology.
Background
LLMs have shown to be powerful tools for a wide range of use-cases that largely fall under the umbrella of Natural Language Processing (NLP). NLP is at the crossroads of computer science and linguistics. It aims to equip computers with the ability to comprehend and process human language. Until recently NLP was an area in which computers struggled to match human capabilities. However algorithmic advancements, boosted processing power, and the volume of relevant training data paved the way for the creation of powerful LLMs. Unlike their predecessors, LLMs excel at both extracting meaning from existing text and generating entirely novel text.
Opportunities of LLMs and Multimodal LLMs
“Some people call this artificial intelligence, but the reality is this technology will enhance us. So instead of artificial intelligence, I think we’ll augment ourintelligence.”— Ginni Rometty, former CEO of IBM
Until relatively recently, human cognition was unrivalled for any task requiring natural language processing, like language translation or topic modelling. The advantage of the human brain over computers stemmed from the fundamental difficulty of equipping machines with the ability to grasp the intricate context, semantic content, and subtle nuances intrinsic to human language. Similarly natural language generation, historically a labour-intensive process performed by experts in their craft, remained beyond machine capabilities until recent breakthroughs.
Automating these activities is an alluring prospect to the business owner: compared to humans, machines are less expensive, better equipped for high-throughput work and, if asked the correct question, more consistent. The ability to automate such activities could be transformative to a business’s ability to reduce costs, offer more responsive services and streamline operations. Some uses of LLMs will appear quite obvious and visible (think: chatbots, image generation) while others will be less obvious, perhaps operating behind the scenes (think: employee support, writing computer code).
Let’s look at a few examples of where LLMs will bring value to a business, “visible” for the more obvious use-cases and “hidden” for internal uses:
Visible LLMs
Customer Support. LLMs can power chatbots that answer customer questions, summarise and route support requests, and even provide basic troubleshooting. This can improve customer experience and free up human agents for complex issues.
Business Intelligence. Natural language querying empowers non-technical users to explore datasets and unearth insights directly through intuitive questions, democratising data analysis and driving informed decision-making.
Text summarisation. LLMs generate concise and engaging summaries of complex articles and reports, product review, and more, making them easily digestible for target audiences and boosting content reach and driving traffic to key content.
Hidden LLMs
Marketing. LLMs can analyse customer data (search queries, social media interactions, purchases) to identify behavioural patterns and support marketing decisions. Could include personalised marketing materials like eye-catching advertisements, or product recommendations and offers. This can lead to higher engagement and conversion rates.
Fraud Detection. LLMs can scan financial records for anomalies and potential fraud. This can protect consumers and businesses from financial losses.
HR Assistant. LLMs can analyse resumes and applications to identify qualified candidates and streamline the recruitment process.
Report Automation. LLMs can generate reports (annual reviews, audits) automatically, saving time and resources for large corporations and financial institutions.
Market Trend Analysis. LLMs can process online data in real-time to identify trending topics and inform business decisions.
It’s worth noting that some use cases could potentially fall into both categories, depending on the specific implementation. For example, a chatbot might be considered “visible” when interacting with customers but “hidden” if it’s used internally for employee support.
Limitations and Challenges of LLMs
While easy public access to LLMs like ChatGPT and Google Gemini paints a rosy picture, integrating them into real-world applications is far from plug-and-play. Let’s look into some hidden challenges — from training biases to data staleness — that lurk beneath the surface of LLM accessibility. Understanding these challenges is crucial before diving in to ensure your LLM journey is successful.
Training bias. Careful planning is crucial for LLMs, as both internal and client-facing products depend on them. These models work best when they have domain familiarity, gained through targeted training and error correction on a particular topic or corpus of text. Out-of-the-box, LLMs are very good at generating seemingly-convincing text that can be susceptible to factual errors due to their statistical nature. They prioritise oft-repeated patterns, not necessarily accuracy. In short, LLMs don’t necessarily learn the information that is most accurate, but rather they learn what is repeated the most in their training data. Therefore, vigilant monitoring and domain-specific training are essential for reliable outputs.
Statistical conformity. For this same reason, LLMs are susceptible to a phenomenon called “regression to the mean”. Because LLMs generate language based on what is statistically most probable, concepts and opinions that are widely repeated across the internet have a selective advantage over those that are not. Conceptual edge cases — statistically speaking — are less attractive to an LLM compared to those that are more widely shared online.
Data staleness. The relevance of outputs generated by LLMs is limited by how current the text used to train it is. ChatGPT, for instance, was trained on a static dataset that was two years old, and now a recent development offers paid subscribers access to real-time information through the internet. This potentially addresses the recency issue for ChatGPT but does not completely solve the issue. Language models still lack inherent understanding of temporal context, struggling with dynamic information such as the current stock prices or weather out-of-the-box. Injecting supplemental information, like real-time data feeds during model fine-tuning can help build domain intelligence and improve performance in specific areas. Responsible implementation requires acknowledging limitations while leveraging new advancements like ChatGPT’s internet access to push the boundaries of LLM capabilities.
Opaque integration. The carefully curated multimodal LLM demos we see can mask the messy reality of diverse user inputs, unpredictable edge cases, and the lack of explainability behind LLM responses. Debugging errors becomes a frustrating guessing game, and potential for biassed or offensive outputs due to data disparities demands constant vigilance and mitigation strategies. Domain experts in AI can help mitigate the challenges of LLM integration by guiding mitigation strategies and ensuring responsible deployment for robust LLM performance.
Risks
Any emerging technology presents some risks for adoption. If you’re considering operationalising LLMs, this is the time to start asking questions about risk, preparing your business, and finding the right partners for your journey. It’s important to appraise would-be risks early in the process so you can discuss with your team or clients how you plan to use LLMs responsibly.
Security vulnerabilities. LLMs raise important questions about user privacy. LLMs use large amounts of data for training, and the companies behind them retain access to user prompts and data shared as part of those prompts. There is a risk that user prompts will share proprietary information or trade secrets to the provider. Many enterprise operations are banning LLMs amid privacy fears to minimise (client) risks and concerns. There are other, more proactive ways of handling data leaks, like training and hosting your own private LLM on-premises. Techniques such as transfer learning can be a powerful solution, in which previously trained models are fine-tuned to specific domains. For a good description of this technique, we recommend Neri Van Otten’s plain-language breakdown of the topic (https://spotintelligence.com/2023/03/28/transfer-learning-large-language-models/).
Data bias and fairness. Across LLMs and AI in general, training datasets are biassed, and model outputs will reflect that bias. Depending on the use-case this trained bias may result in discrimination and exclusion. Assessment and planning are critical to bias mitigation strategies.
Judgement calls. Some ethicists worry that professionals will turn to LLMs for advice when confronted with challenging ethical decisions, like whether to hire one of two candidates from an underrepresented minority. Current evidence suggests that LLM algorithms lack the dexterity to balance different ethical principles. For the time being, these are decisions best handled by people thinking through the delicate decisions themselves.
Hallucinations. Users need to be aware of how LLMs can return a response amalgamated from multiple training sources. In short, they may make things up or “hallucinate”, producing mis- and disinformation. An example of this is of a fictitious article or book title giving attribution to a real author. Savvy users will adopt practices that are resilient to disinformation, like training on contextually relevant datasets and fact-checking the algorithm’s responses.
In addition to the risks listed above, it is worth noting that LLMs can potentially pose a more existential risk to our knowledge base. The phenomenon of regression to the mean discussed above could have longer-term consequences. Consider a scenario in which an LLM is trained on a large corpus of text, for instance text on the internet. When asked to generate new text, it will, by design, choose concepts that are statistically advantageous (in other words, common) based on what it has seen. If turned loose, the machine could generate additional data that amplifies the common concepts at the expense of a broad representation of the concepts available, which could in-turn be used to train a future model. The result would be a positive feedback loop, in which a few popular concepts win out over all the others. An important question facing the field, then, is how do we protect our collective knowledge against this? Because AI is a cutting-edge field that is moving incredibly fast with limited regulation, it is critical to be aware of the possible consequences of its use.
Best practices for responsible LLM implementation
Responsible LLM implementation ensures trust and mitigates potential risks through robust data security measures, clear explainability of outputs, and collaboration with experts to mitigate risks and foster responsible development.
Some considerations to discuss with your collaborators before pursuing an LLM project:
What steps will you take for mitigating bias and ensuring fairness?
Discuss approaches to improve explainability and interpretability.
Suggest security best practices for LLM deployments.
Provide guidance on data collection, management, and governance.
Outline strategies for building technical expertise and partnerships.
Closing thoughts
While LLMs offer immense potential, their true value lies in serving real business needs, not just chasing the latest tech trends. Root your explorations of LLMs in genuine business needs. Answer the question of “Why” before moving on to “How”.
Remember, these are complex tools, not magic spells (or as a former Pivigo CEO used to say, “it’s maths, not magic”).
Address potential biases, prioritising responsible data practices, and remember, human expertise remains irreplaceable.
When you deploy LLMs strategically, with a clear understanding of both their capabilities and limitations, you’ll unlock their true potential, transforming them from fleeting fads into enduring business assets.