The Building Blocks of AI: Decoding the Role and Significance of Foundation Models
10/04/2024
Skyscrapers start with strong foundations. The same goes for apps powered by AI.
A foundation model is an AI neural network trained on immense amounts of raw data, generally with unsupervised learning.
It's a type of artificial intelligence model trained to understand and generate human-like language. Imagine giving a computer a huge library of books to read and learn from, so it can understand the context and meaning behind words and sentences, just like a human does.
Foundation models. A foundation model's deep knowledge base and ability to communicate in natural language make it useful for a broad range of applications, including text generation and summarization, copilot production and computer code analysis, image and video creation, and audio transcription and speech synthesis.
ChatGPT, one of the most notable generative AI applications, is a chatbot built with OpenAI's GPT foundation model. Now in its fourth version, GPT-4 is a large multimodal model that can ingest text or images and generate text or image responses.
Online apps built on foundation models typically access the models from a data center. But many of these models, and the applications they power, can now run locally on PCs and workstations with NVIDIA GeForce and NVIDIA RTX GPUs.
Foundation Model Uses Foundation models can perform a variety of functions, including:
Language processing: understanding and generating text
Code generation: analyzing and debugging computer code in many programming languages
Visual processing: analyzing and generating images
Speech: generating text to speech and transcribing speech to text
They can be used as is or with further refinement. Rather than training an entirely new AI model for each generative AI application - a costly and time-consuming endeavor - users commonly fine-tune foundation models for specialized use cases.
Pretrained foundation models are remarkably capable, thanks to prompts and data-retrieval techniques like retrieval-augmented generation, or RAG. Foundation models also excel at transfer learning, which means they can be trained to perform a second task related to their original purpose.
For example, a general-purpose large language model (LLM) designed to converse with humans can be further trained to act as a customer service chatbot capable of answering inquiries using a corporate knowledge base.
Enterprises across industries are fine-tuning foundation models to get the best performance from their AI applications.
Types of Foundation Models More than 100 foundation models are in use - a number that continues to grow. LLMs and image generators are the two most popular types of foundation models. And many of them are free for anyone to try - on any hardware - in the NVIDIA API Catalog.
LLMs are models that understand natural language and can respond to queries. Google's Gemma is one example; it excels at text comprehension, transformation and code generation. When asked about the astronomer Cornelius Gemma, it shared that his contributions to celestial navigation and astronomy significantly impacted scientific progress. It also provided information on his key achievements, legacy and other facts.
Extending the collaboration of the Gemma models, accelerated with the NVIDIA TensorRT-LLM on RTX GPUs, Google's CodeGemma brings powerful yet lightweight coding capabilities to the community. CodeGemma models are available as 7B and 2B pretrained variants that specialize in code completion and code generation tasks.
MistralAI's Mistral LLM can follow instructions, complete requests and generate creative text. In fact, it helped brainstorm the headline for this blog, including the requirement that it use a variation of the series' name AI Decoded, and it assisted in writing the definition of a foundation model.
Hello, world, indeed. Meta's Llama 2 is a cutting-edge LLM that generates text and code in response to prompts.
Mistral and Llama 2 are available in the NVIDIA ChatRTX tech demo, running on RTX PCs and workstations. ChatRTX lets users personalize these foundation models by connecting them to personal content - such as documents, doctors' notes and other data - through RAG. It's accelerated by TensorRT-LLM for quick, contextually relevant answers. And because it runs locally, results are fast and secure.
Image generators like StabilityAI's Stable Diffusion XL and SDXL Turbo let users generate images and stunning, realistic visuals. StabilityAI's video generator, Stable Video Diffusion, uses a generative diffusion model to synthesize video sequences with a single image as a conditioning frame.
Multimodal foundation models can simultaneously process more than one type of data - such as text and images - to generate more sophisticated outputs.
A multimodal model that works with both text and images could let users upload an image and ask questions about it. These types of models are quickly working their way into real-world applications like customer service, where they can serve as faster, more user-friendly versions of traditional manuals.
Many foundation models are free to try - on any hardware - in the NVIDIA API Catalog. Kosmos 2 is Microsoft's groundbreaking multimodal model designed to understand and reason about visual elements in images.
Think Globally, Run AI Models Locally GeForce RTX and NVIDIA RTX GPUs can run foundation models locally.
The results are fast and secure. Rather than relying on cloud-based services, users can harness apps like ChatRTX to process sensitive data on their local PC without sharing the data with a third party or needing an internet connection.
Most recent headlines
04/08/2024
Dalet Appoints Santiago Solanas as CEO to Lead Next Era of Growth and Innovation
Dalet, a leading technology and service provider for media-rich organizations, is excited to announce Santiago Solanas as its new Chief Executive Officer (CEO)....
03/06/2024
Dalet and Veritone Reach Agreement to Distribute, Transact and Monetize Media Archives
Dalet, a leading technology and service provider for media-rich organizations, a...
16/05/2024
Power Provokes Vital Questions About the Role of Police
PARK CITY, UTAH - JANUARY 18: Director Yance Ford introduces the Power premiere at Library Center Theatre. (Photo by Chad Salvador/Shutterstock for Sundance F...
16/05/2024
Mental Health Matters: Embrace Self-Care With These Audiobook and Podcast Listens
May is Mental Health Awareness month, a time to recognize those living with ment...
16/05/2024
ABC and SBS bring digital radio (DAB+) services to the Gold Coast
ABC and SBS bring digital radio (DAB ) services to the Gold Coast 15 May, 2024 Media releases ABC and SBS today announced that audiences on the Gold Coast ...
16/05/2024
Clear-Com Communication Solutions Enhance Oregon State University's PRAx Building
eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({...
16/05/2024
6 New Clearmountain's Domain Presets
Share this article width=32 height=32 data-lazy-src=https://apogeedigital.com/app/themes/juniper-theme/src/img/icons/facebook-purple.svg /> width=32 height=...
16/05/2024
Updated: Netflix to Stream Live Christmas Day NFL Games
In a major example of how large streaming services are successfully battling for lucrative sports rights, Netflix has announced that it has secured global right...
16/05/2024
ESPN, Fox, WBD Unveil Name for New Sports Streaming Service
LOS ANGELES & NEW YORK More than three months after announcing plans to launch a major new sports streaming service, ESPN, Fox Corp. and Warner Bros. Discovery,...
16/05/2024
CTV/Streaming Top of Mind for Marketers in 2024
In a survey conducted late last year, marketing companies said their main focus for increasing advertising would be CTV/streaming. Generative AI, which was not ...
16/05/2024
CPB Issues More Grants for Next-Generation Warning
Three public radio and one public television outlet are the latest recipients of funding from the Corporation for Public Broadcasting to upgrade their emergency...
16/05/2024
A year in live HDR production
John Mailhot, senior vice president, product management at Imagine Communications, gives an insight into how HDR is impacting live production today, as well as ...
16/05/2024
Sony Pictures Post Production Services Completes Expansion of its Sound Facilities
Sony Pictures Post Production Services Completes Expansion of its Sound Faciliti...
16/05/2024
Telestream Taps Power of AI to Manage, Define Media Workflows
TV Tech recently sat down with Telestream CEO Rhonda Bassett-Spiers and CTO Simon Clark to discuss how the M&E industry is adapting the cloud, IP and AI to enha...
16/05/2024
1185 Films launches one stop shop' for industry talent
1185 Talent will include directors, editors, colourists, VFX artists and photographers By Jenny Priestley Published: May 16, 2024 1185 Talent will include...
16/05/2024
Updated: Presteigne Broadcast Hire enters administration
The administrators said they are focused on trading the business as a going concern while they assess options, including exploring the possibility of a sale By...
16/05/2024
AWS CEO Adam Selipsky steps down
Selipsky will leave the company on 3rd June, and will be succeeded by Matt Garman, a senior vice president who has overseen sales and marketing at AWS By Jenny...
16/05/2024
Sony Pictures Post Production Services Completes Sound Facilities Expansion
Sony Pictures Post Production Services has announced that the final phase of a multi-year effort to expand and modernize its motion picture and television sound...
16/05/2024
CNN, ABC to Televise Two Presidential Debates
President Joe Biden and Former President Donald Trump have agreed to participate in two televised debates, the first of which will be hosted by CNN on June 17....
16/05/2024
Disney+ Debuts First Concert Film to Stream with IMAX Enhanced Sound
BURBANK, Calif. and NEW YORK Disney+, IMAX Corporation, and DTS, a wholly-owned subsidiary of Xperi, have announced that Queen Rock Montreal is now streaming ...
16/05/2024
WBD Unveils Advanced Advertising Features for Max
NEW YORK During its upfronts presentation, Warner Bros. Discovery announced a suite of new advanced ad products available on Max, that include shoppable ads, ad...
16/05/2024
iZotope announces RX 11 audio software, introducing new machine learning for tackling noise, and new tools for music streaming
iZotope announces RX 11 audio software, introducing new machine learning for tac...
16/05/2024
Check Out the Cine Gear Expo June 7 9, 2024 at Warner Bros. Studios in Burbank, CA
Check Out the Cine Gear Expo June 7 9, 2024 at Warner Bros. Studios in Burbank...
16/05/2024
Nouvel Photo & Film Captures Cinematic Wedding Films with Blackmagic Cinema Camera 6K
Nouvel Photo & Film Captures Cinematic Wedding Films with Blackmagic Cinema Came...
16/05/2024
Replacing Surfaces Like a Pro with Mocha AE Planar Tracking
Replacing Surfaces Like a Pro with Mocha AE Planar Tracking Levi Borland May 15, 2024 0 Comments Welcome to our comprehensive tutorial on planar track...
16/05/2024
CBC's CPL Teams Salamanders, Yard Gnomes One Week Away From Season Opener
The world of baseball at Capitol Broadcasting is about to expand like fireworks on a Friday night. The Holly Springs Salamanders will open their season at home...
16/05/2024
Blockgraph Adopted by tvbeat, Enabling Audience-Based Planning
Blockgraph said that its identity and data collaboration platform has been adopted by tvbeat, a provider of campaign planning technology for media companies....
16/05/2024
VEON 1Q24 Trading Update: Accelerating USD Growth, Strong Operational Execution
16 May 2024 VEON 1Q24 Trading Update: Accelerating USD Growth, Strong Operational Execution Amsterdam, 16 May 2024 07:00AM CEST - VEON Q1 2024 Highlights Q1...
16/05/2024
Tribeca X Announces 2024 Speaker Lineup for Expanded Two-Day Event; Official Selections for Tribeca X Award
May 16th, 2024 Press Materials Available Here TRIBECA X ANNOUNCES 2024 SPEAKER...
16/05/2024
SVG Sit-Down: Beam Dynamics CEO Kaszycki on Maximizing Physical-Asset Usage, Tracking
SVG Sit-Down: Beam Dynamics CEO Kaszycki on Maximizing Physical-Asset Usage, Tra...
16/05/2024
SVG New Sponsor Spotlight: Aggreko's Don Gray on Advocating for Sustainability in Live Sports Production
SVG New Sponsor Spotlight: Aggreko's Don Gray on Advocating for Sustainabili...
16/05/2024
At PGA Championship, CBS Sports Bolsters Aerial, Bunker Camera Lineup; Takes Viewers Deeper with Optimotion Swing Tech
At PGA Championship, CBS Sports Bolsters Aerial, Bunker Camera Lineup; Takes Vie...
16/05/2024
Rohde & Schwarz shows the future of terrestrial broadcasting at CABSAT 2024
Rohde & Schwarz shows the future of terrestrial broadcasting at CABSAT 2024 Real innovations in services and sustainability Meet us at CABSAT 2024 in Dubai...
16/05/2024
Hierarchy' Teaser Unveils Jooshin High School's Elite World, Starring Next Generation of K-Teen Stars
Back to All News Hierarchy' Teaser Unveils Jooshin High School's Elite...
16/05/2024
Steamy Doctor Climax' Trailer Teases the Questions That Stirred Up 70s Thailand
Back to All News Steamy Doctor Climax' Trailer Teases the Questions That S...
16/05/2024
Getting moving for Mental Health Awareness Week
Posted on 16 May 2024 We're huge advocates of positively improving our mental wellbeing so going green for Mental Health Awareness Week was an obvious ward...
16/05/2024
Venu Sports Introduced as Name for Forthcoming Sports Streaming Service
Venu Sports Introduced as Name for Forthcoming Sports Streaming Service Details of ESPN, Fox and Warner Bros. Discovery Joint Venture Take Shape with Brand an...
16/05/2024
Irma Stern's 1930 portrait Cape Girl with Fruit headlines SA art auction
Irma Stern's 1930 portrait Cape Girl with Fruit headlines SA art auctionAn Irma Stern painting from 1930 leads Strauss & Co's diverse catalogue of moder...
16/05/2024
Prison Journalism: The role my mother played in my life
Prison Journalism: The role my mother played in my lifeWesley Leong was incarcerated at the age of 15 in 1996 at Pollsmoor Prison. He is currently part of Resto...
16/05/2024
Actress Sibongokuhle Cele joins Muvhango'
Actress Sibongokuhle Cele joins Muvhango'Model and former Durban Gen' actress Sibongokuhle Cele has secured a role in SABC2's soapie Muvhango'...
16/05/2024
Tonight on Skeem Saam: Carol sees Ntswaki getting up to no good with Toby
Tonight on Skeem Saam: Carol sees Ntswaki getting up to no good with TobyDon't miss Thursday, 16 May's riveting episode of South African soapie Skeem Sa...
16/05/2024
Fight for Honor in Men of War II' on GFN Thursday
Whether looking for new adventures, epic storylines or games to play with a friend, GeForce NOW members are covered. Start off with the much-anticipated sequel...
16/05/2024
Skeem Saam: Wednesday's episode, 15 May 2024 [video]
Skeem Saam: Wednesday's episode, 15 May 2024 [video]Missed an episode of Skeem Saam? No problem! Watch the latest episode of your favourite South African so...
16/05/2024
Monet painting fetches R640 million at New York auction
Monet painting fetches R640 million at New York auctionA Monet has sold for nearly R640 million at auction, Sotheby's said, marking a solid start to New Yor...
16/05/2024
Cruinni na ng 2024 - a day of free creative activity for young people
RT is supporting Cruinni na n g with a nationwide campaign on television, radio, online and digital. For a full listing of all activities and information on h...
15/05/2024
UKTV Ventures partners with The Farewell Guide in 1million airtime-for-equity deal
12th March 2024 - Award-winning broadcaster UKTV has today announced it has inve...
15/05/2024
Give Me the Backstory: Get to Know Jianjie Lin, the Writer-Director Behind Brief History of a Family
By Bailey Pennick One of the most exciting things about the Sundance Film Festi...
15/05/2024
De viktigaste insikterna frn Spotifys rliga Loud & Clear-rapport
Med sin rliga royaltyrapport Loud & Clear kar Spotify transparensen i musikindustrin, s att artister, l tskrivare, fans och hela musikbranschen f r en st rre...
15/05/2024
Chegaram as ltimas estatsticas: Artistas brasileiros geraram mais receita e alcanaram mais fs no Spotify - basta perguntar Anitta
Nos ltimos 10 a 15 anos, a mudan a para o streaming de udio transformou comple...
15/05/2024
The Latest Stats Are In: Brazilian Artists Generated More Revenue and Reached More Fans on Spotify-Just Ask Anitta
Over the last 10-15 years, the move toward streaming audio has completely transf...