Executives and managers are grappling with the challenge of monetizing generative artificial intelligence (AI) and large language models (LLMs) in their businesses. While AI often showcases impressive proofs of concept, realizing returns on investment remains a significant hurdle. According to an executive VP at Capgemini, proving the ROI is the biggest challenge when implementing multiple GenAI solutions into production.
To successfully monetize LLMs, investments need to be made in testing and monitoring. Testing is crucial to ensure the accuracy and reliability of LLMs. Experts recommend intentionally “poisoning” the models during testing to assess their handling of erroneous information. For instance, a business model was prompted with the idea of using dragons for long-distance haulage, and the model responded affirmatively, providing training recommendations involving dragons and princesses. This highlights the need for caution when integrating generative AI into existing applications, as it can be tempting to add superficial features without considering security and risk implications.
While generative AI is rapidly advancing, it is estimated to take another two to five years before it becomes mainstream. The market is expected to witness intense competition among vendors, software platforms, and cloud providers. However, organizations must be cautious about using a single LLM for all tasks, as it can lead to unnecessary expenses. Instead, they should explore cost-effective ways to leverage LLMs and be prepared to decommission solutions swiftly if needed.
To measure performance and quality of responses, AI users should apply their queries against multiple models. Having a common mechanism to capture metrics and replay queries against different models enables organizations to identify more efficient approaches. Guardrails are essential to prevent generative AI from producing unexpected or irrelevant outputs, such as generating a 4,000-word essay on President Andrew Jackson in response to an invoice.