Inference: The Core Component of Future Applications

The year 2023 marked a significant leap in the evolution of generative AI, captivating the attention of diverse users across the globeAs we look ahead to 2024, this timeline is poised to become a validation era for generative AI, where diverse sectors are beginning to test the technology in various business contexts, gauging its effectiveness and potential.

Over the past two years, generative AI has emerged as a transformative technology, showcasing its immense application potentialThis leads us to consider critical questions: what substantial impacts will generative AI have on business applications? How can companies effectively integrate generative AI into their operational frameworks? What key trends can we anticipate for large models in the future?

As a leading event in the cloud computing realm, the re:Invent conference organized by Amazon Web Services (AWS) has long served as a bellwether for advancements in cloud computing and artificial intelligence

The insights shared at the 2024 re:Invent conference present a fresh perspective on the profound influence generative AI is heralding across sectors.

In the words of Amazon's new CEO, Matt Garman, generative AI is poised to become a transformative force in industries around the worldAs this technology becomes more entwined with business applications, inference will emerge as a critical component of future applications.

The significance of inference's role was underscored during the conference, particularly in comments made by Andy Jassy, the CEO of AmazonReturning to the spotlight after a three-year hiatus, Jassy unveiled Amazon's latest multi-modal model named Nova, which contains six foundational models designed to handle text, images, and video processing seamlesslyJassy believes that Amazon Nova stands to provide an intelligent and cost-effective new foundation, offering substantial advancements in various applications.

Garman described a transformational shift in application dynamics

• Chery's Top Seller Hit by Cancellation Surge
• Inference: The Core Component of Future Applications
• Debate Over U.S. Inflation Outlook
• Federal Reserve Plans Changes to Bank Stress Tests
• AI Toy Craze: ByteDance and Tesla Join the Race

Traditionally, the trifecta of computing, storage, and database were recognized as the core pillars of application developmentHowever, with the advent of generative AI, inference is emerging as a pivotal element in constructing applicationsThat relevance of inference is echoed in Garman's belief that generative AI has the potential to disrupt all fields, reshaping processes, and enhancing user experiences.

To facilitate this transition, AWS emphasized the importance of their Bedrock product at this year’s conferenceDesigned to provide high-performance, flexible, and secure model inference services, this product aims to lower the barriers of AI inference significantlyUltimately, the goal is to empower users to construct and scale generative AI applications with greater ease.

For instance, sparked by scaling laws, model parameter sizes are gradually increasing; however, larger models aren’t universally better

Some specific fields or scenarios may require smaller and faster models, underscoring the increasing relevance of model distillationUnfortunately, many users previously relied on open-source large models, necessitating frameworks and extensive coding, lacking efficiency and ease of use.

In response to these challenges, AWS introduced the Amazon Bedrock Model Distillation feature at the conferenceThis functionality allows users to select their desired models tailored to specific use cases, leading to enhancements in model runtime speeds and reduced costs.

Even the best large models face challenges such as hallucinations— a phenomenon where AI generates inaccurate or nonsensical informationWhen generative AI technology is deployed in practical settings, users inevitably confront hallucination concernsAWS's approach to this issue involves automated reasoning checks through Amazon Bedrock Automated Reasoning

This ensures the accuracy of inferences, mitigating potential factual errors and issues arising from hallucinations.

The immense value of AI Agents in generative AI applications cannot be overlookedWhile AI Agents can handle simple tasks independently, the intricacies presented by complex workflows with hundreds of tasks require coordinated actions by multiple agentsHence, AWS has developed Amazon Bedrock multi-agent Collaboration, which supports intricate workflows involving multiple agents.

Beyond these capabilities, AWS also introduced several additional features related to Amazon Bedrock during the conferenceIt is clear that as generative AI continues to be seamlessly integrated into production applications, inference will be omnipresent, making its management significantly valuableAWS is continuously enriching Amazon Bedrock with various functionalities that cater to users' diverse needs in inference, enhancing the momentum for generative AI's practical applications.

The reappearance of Andy Jassy at the re:Invent conference was a notable highlight, marking his return after a three-year absence.

As a titan in the e-commerce sector, Amazon is also keenly adopting generative AI advancements

Jassy asserted that no singular tool would dominate the technological landscape; instead, the key to propelling AI applications lies within lowering costs and enhancing productivity, rather than merely competing in assessments and contests.

Jassy revealed that Amazon is currently operating around 1,000 generative AI applications internally and has developed new models designed for internal use, which are now being shared publicly, leading to the reveal of Amazon Nova— the latest generation of multi-modal foundational models.

Amazon Nova includes four models: the Amazon Nova Micro, which processes text with minimal latency and cost; Amazon Nova Lite, a cost-effective multi-modal model for swiftly handling text, image, and video inputs; Amazon Nova Pro, a robust multi-modal model that balances accuracy, speed, and cost for various tasks; and the premium Amazon Nova Premier, specifically tailored for complex reasoning tasks and capable of acting as a "teacher model" for distilling custom models.

Furthermore, Amazon launched two models focused on the generation of creative content: Amazon Nova Canvas, intended for crafting high-quality images, and Amazon Nova Reel, aimed at producing high-quality video content.

Jassy disclosed that all models from the Amazon Nova family—Micro, Lite, Pro, Canvas, and Reel— are now fully available on Amazon Bedrock, with the Premier model anticipated to launch in the first quarter of 2025. Notably, Amazon plans to introduce Amazon Nova Speech-to-Speech and Amazon Nova Any to Any models— enabling versatile generative functions between various modalities— in the coming year.

Benchmark testing results demonstrate that Amazon Nova Micro, Nova Lite, and Nova Pro exhibit strong competitive performance

For instance, Nova Micro performed comparably or better than Meta's LLaMa 3.1 8B across 11 applicable benchmarking tests, while Nova Pro achieved similar scores to OpenAI's GPT-4o in 17 out of 20 benchmark tests.

Indeed, AWS is solidifying its stature in the landscape of large models, following its initial Amazon Titan modelWith the unveiling of the Amazon Nova suite of foundational models, Amazon is further bolstering its capabilities in this spaceAdditionally, the partnership with Anthropic is poised to enhance AWS's offerings in generative AI services, catering to a variety of user needs.

The principle of "empowering customer choice" is one that AWS frequently emphasizesAccording to Jassy, insights and practices within Amazon's internal teams reveal a strong preference among developers for a broad selection of AI modelsThis trend is expected to continue.

Currently, the domains of generative AI and large models are still in their early stages of industrial development, witnessing high levels of innovation and activity

This results in an array of emerging models that continuously iterate, upgrade, and evolve, with new versions and functionalities regularly surfacingConversely, users are similarly contending with the early phases of aligning generative AI with specific scenariosVarious fields and contexts often reveal distinct requirements that necessitate careful consideration of performance, cost, and reliability among large models.

As such, AWS believes that empowering users with adequate choice is essential for effectively integrating generative AI into business landscapes.

For instance, Amazon Bedrock now offers a marketplace featuring over 100 large models for users to choose fromFurthermore, given that many users need to distill models based on specific business needs, Amazon Bedrock Model Distillation allows users to elect from cutting-edge models like LLaMa, Claude, and others, facilitating the distillation of models best suited for their particular scenarios.

In summary, if the previous year saw AWS presenting a layered technology stack aimed at propelling generative AI's applications, this year’s conference revealed an array of dazzling new functionalities that significantly enrich the technological framework for generative AI applications