Sony Pixel Power calrec Sony

Brave New World: Leo AI and Ollama Bring RTX-Accelerated Local LLMs to Brave Browser Users

02/10/2024

Editor's note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and showcases new hardware, software, tools and accelerations for GeForce RTX PC and NVIDIA RTX workstation users.

From games and content creation apps to software development and productivity tools, AI is increasingly being integrated into applications to enhance user experiences and boost efficiency.

Those efficiency boosts extend to everyday tasks, like web browsing. Brave, a privacy-focused web browser, recently launched a smart AI assistant called Leo AI that, in addition to providing search results, helps users summarize articles and videos, surface insights from documents, answer questions and more.

Leo AI helps users summarize articles and videos, surface insights from documents, answer questions and more. The technology behind Brave and other AI-powered tools is a combination of hardware, libraries and ecosystem software that's optimized for the unique needs of AI.

Why Software Matters NVIDIA GPUs power the world's AI, whether running in the data center or on a local PC. They contain Tensor Cores, which are specifically designed to accelerate AI applications like Leo AI through massively parallel number crunching - rapidly processing the huge number of calculations needed for AI simultaneously, rather than doing them one at a time.

But great hardware only matters if applications can make efficient use of it. The software running on top of GPUs is just as critical for delivering the fastest, most responsive AI experience.

The first layer is the AI inference library, which acts like a translator that takes requests for common AI tasks and converts them to specific instructions for the hardware to run. Popular inference libraries include NVIDIA TensorRT, Microsoft's DirectML and the one used by Brave and Leo AI via Ollama, called llama.cpp.

Llama.cpp is an open-source library and framework. Through CUDA - the NVIDIA software application programming interface that enables developers to optimize for GeForce RTX and NVIDIA RTX GPUs - provides Tensor Core acceleration for hundreds of models, including popular large language models (LLMs) like Gemma, Llama 3, Mistral and Phi.

On top of the inference library, applications often use a local inference server to simplify integration. The inference server handles tasks like downloading and configuring specific AI models so that the application doesn't have to.

Ollama is an open-source project that sits on top of llama.cpp and provides access to the library's features. It supports an ecosystem of applications that deliver local AI capabilities. Across the entire technology stack, NVIDIA works to optimize tools like Ollama for NVIDIA hardware to deliver faster, more responsive AI experiences on RTX.

Applications like Brave's Leo AI can access RTX-powered AI acceleration to enhance user experiences. NVIDIA's focus on optimization spans the entire technology stack - from hardware to system software to the inference libraries and tools that enable applications to deliver faster, more responsive AI experiences on RTX.

Local vs. Cloud Brave's Leo AI can run in the cloud or locally on a PC through Ollama.

There are many benefits to processing inference using a local model. By not sending prompts to an outside server for processing, the experience is private and always available. For instance, Brave users can get help with their finances or medical questions without sending anything to the cloud. Running locally also eliminates the need to pay for unrestricted cloud access. With Ollama, users can take advantage of a wider variety of open-source models than most hosted services, which often support only one or two varieties of the same AI model.

Users can also interact with models that have different specializations, such as bilingual models, compact-sized models, code generation models and more.

RTX enables a fast, responsive experience when running AI locally. Using the Llama 3 8B model with llama.cpp, users can expect responses up to 149 tokens per second - or approximately 110 words per second. When using Brave with Leo AI and Ollama, this means snappier responses to questions, requests for content summaries and more.

NVIDIA internal throughput performance measurements on NVIDIA GeForce RTX GPUs, featuring a Llama 3 8B model with an input sequence length of 100 tokens, generating 100 tokens. Get Started With Brave With Leo AI and Ollama Installing Ollama is easy - download the installer from the project's website and let it run in the background. From a command prompt, users can download and install a wide variety of supported models, then interact with the local model from the command line.

For simple instructions on how to add local LLM support via Ollama, read the company's blog. Once configured to point to Ollama, Leo AI will use the locally hosted LLM for prompts and queries. Users can also switch between cloud and local models at any time.

Brave with Leo AI running on Ollama and accelerated by RTX is a great way to get more out of your browsing experience. You can even summarize and ask questions about AI Decoded blogs! Developers can learn more about how to use Ollama and llama.cpp in the NVIDIA Technical Blog.

Generative AI is transforming gaming, videoconferencing and interactive experiences of all kinds. Make sense of what's new and what's next by subscribing to the AI Decoded newsletter.
LINK: https://blogs.nvidia.com/blog/rtx-ai-brave-browser/...
See more stories from nvidia

Most recent headlines

05/01/2027

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be demoed at CES 2026

Worlds first 802.15.4ab-UWB chip verified by Calterah and Rohde & Schwarz to be ...

17/11/2026

Fabric Insights and Fonn Group Partner to Connect Newsroom Planning and Media Operations

Fabric Insights and Fonn Group have announced a partnership to connect editorial...

17/11/2026

NABJ Online Seminar Highlights Black Sports Production Legends on Nov. 25

The National Association of Black Journalists is offering up a Thanksgiving Eve treat on Wednesday, Nov. 25, in the form of a live streaming event featuring for...

17/11/2026

Bridge Technologies Adds Apple ProRes Support to VB440 Production Probe

Bridge Technologies has announced that its VB440 production probe now supports both monitoring and signal generation for Apple ProRes video streams. The additio...

17/11/2026

Canon U.S.A. Hosts Jericho High School Journalism Students at Melville Headquarters

Canon U.S.A., Inc. recently welcomed 21 Jericho High School students to its Melv...

17/11/2026

Amagi to Power Live Newscasts for Spectrum News

Spectrum News has deployed the Amagi platform to support its live news operations. Amagi says the platform lets Spectrum News streamline newsroom workflows, red...

17/11/2026

Microsoft Expands Haivision Video Contribution Deployment, Adds Makito ONE for Global Live Events

Microsoft has expanded its use of Haivision's live video contribution ecosys...

17/11/2026

Setanta Sports Deploys Ateme TITAN Live for OTT Sports Streaming

Setanta Sports has deployed Ateme's TITAN Live platform to support its OTT video delivery operations. Ateme's transcoding technology is integrated into ...

17/11/2026

Media Links Joins Panasonic's KAIROS Alliance Partner Program

Media Links has joined the KAIROS Alliance Partner program for Panasonic's KAIROS IT/IP live video production platform. The company completed interoperabili...

17/11/2026

Behind The Mic: Dan Hicks, Iconic NBC Sports Broadcaster, Passes Away at 64

Behind The Mic provides a roundup of recent news regarding on-air talent, including new deals, departures, and assignments compiled from press releases and repo...

17/11/2026

2026 Sports Broadcasting Hall of Fame: Jim Dove, an Innovator Offering Second Looks at the Greatest Plays in Sports

In live sports and in life, things go by in the blink of an eye. Savoring the mo...

17/11/2026

SVG Sit-Down: Eutelsat's Gonzalez, Carroccia on C-Band Clearing, LEO Contribution, the Future of Satellite in Sports

Satellite operators are adding a new generation of low-Earth-orbit (LEO) network...

17/11/2026

Clear-Com Delivers Portable Wireless Systems for Aircraft Communications Security Testing

Clear-Com has delivered five customized test systems for electromagnetic securit...

17/11/2026

Stephen Arnold Music Creates Sonic Identity for WTGL

Stephen Arnold Music (SAM) has created the sonic identity for WTGL, the women's team golf league from TMRW Sports, created in partnership with the LPGA and ...

17/11/2026

PAMA Introduces Give Sound a Voice Initiative to Promote the Value of Sound

The Professional Audio Manufacturers Alliance (PAMA) has introduced Give Sound a Voice, an initiative aimed at raising awareness of the value of sound among the...

17/11/2026

Fanatics Studios Names Jamal Henderson First President of Film & TV

Fanatics Studios, the sports entertainment joint venture between Fanatics and OBB, has named Jamal Henderson its first President of Film & TV, announced Michael...

17/11/2026

NFL Thanksgiving Week Halftime Shows to Feature U2, Wu-Tang Clan, Miranda Lambert and Goo Goo Dolls

The NFL's Thanksgiving Week games will feature halftime performances by U2, ...

17/11/2026

FanSided Launches Redesigned App with Live Scores and Personalized Feeds

FanSided, a Minute brand with more than 250 fan communities, has launched a redesigned mobile app that combines team news, analysis, live game coverage, stats a...

17/11/2026

Major League Volleyball Announces 2027 Media Lineup with NBC Sports, ESPN, ION and CBS Sports

Major League Volleyball (MLV), the longest-running indoor professional volleybal...

17/11/2026

Prime Video to Present Black Friday Sports Lineup in 200+ Countries, No Subscription Required

Prime Video will present a 15-hour slate of live sports on Black Friday, Nov. 27...

17/11/2026

DIOR X SUNDANCE INSTITUTE PARTNERSHIP ANNOUNCED AHEAD OF 2027 SUNDANCE FILM FESTIVAL

Catherine Dior was more than a muse. A Resistance fighter, a survivor of deporta...

17/11/2026

Freqport introduce FreqLift plug-in

Multiband dynamics plug-in boasts FT1 integration Freqport's latest plug-in combines multiband upward and downward dynamics with the company's renow...

17/11/2026

Studio 5X: Debut monitors from Teufel Audio

Hi-fi experts launch two pro-audio models Berlin-based hi-fi experts Teufel Audio have recently branched out into the pro-audio world with their first pairs...

17/11/2026

EastWest Black Friday 2026

A year of ComposerCloud+ for $99 EastWest's annual Black Friday Sale is already live, and new users are being offered one year of the all-encompassing C...

17/11/2026

SOS Awards: Make Your Vote Count!

Pick your favourite gear from 2026 The annual SOS Awards are now underway, and as always, we're asking readers to help us choose the best pro-audio rele...

17/11/2026

Production capacity expanded at SGL Carbons Ried im Innkreis site

As part of an ongoing series production program for a German automaker, production capacity has been increased at SGL Carbon's Austrian site in Ried im Innk...

17/11/2026

Clear-Com Delivers Portable Wireless Systems for Secure Aircraft Communications Testing

eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({...

17/11/2026

Signiant Acquires Media Intelligence Technology

Signiant has acquired the media intelligence technology, Ceivo, developed by Burnlab with founder Aaron Sloman joining the company as Vice President, Product Ma...

17/11/2026

Broadcasters to Conduct EAS Test Today

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

17/11/2026

Microsoft Taps Haivision Contribution, Video Transport Solutions for Live Events

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

17/11/2026

NAB Promotes Broadcast Positioning System Amid Tech Outages

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

17/11/2026

Colorfront's Color Management Added to RED Camera Feeds

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

17/11/2026

Berklees 2027 Alumni Grammy Nominees: See the Full List

Berklees 2027 Alumni Grammy Nominees: See the Full List From pop and R&B to jazz, musical theater, and video game scores, Berklee talent is represented across...

17/11/2026

Sinclair's Bill Fanshawe Sets Retirement

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

17/11/2026

WCVB Boston Names Michael Murgia Director of Engineering

Share {alert(Link copied)}).catch(err => {console.error(Failed to copy: , err)}) data-platform=copy_link data-action= class=font-ui-heading py-2 text-sm tex...

17/11/2026

Two Scripps Research faculty named inaugural recipients of the Schultz Innovative Investigator Award

LA JOLLA, CA-Two Scripps Research faculty, Assistant Professor Christian Diercks...

17/11/2026

Synthogy Black Friday Savings - 33% on ALL Ivory 3 products

Our Biggest Sale of the Year on Ivory 3! From November 17th through November 30th, enjoy our best savings of the year on Ivory 3 Products! Whether it's ou...

17/11/2026

Sky announces major free upgrade to the Sky Glass and Sky Stream experience with the launch of the new Sky OS

Tuesday 17 November 2026 Sky announces major free upgrade to the Sky Glass and ...

17/11/2026

Passion, Betrayal and Scandal: Sky Arts Reveals the Man Behind the Music in Dangerous Debussy, Premiering 1 December

Tuesday 17 November 2026 Passion, Betrayal and Scandal: Sky Arts Reveals the Ma...

17/11/2026

Exact 3D Video launches AI service that turns product documents and CAD files into narrated 3D explainer videos

Exact 3D Video, a web service based in Japan, has launched its English version, ...

17/11/2026

Apple Podcasts names Baby, this is Keke\u00A0Palmer the 2026 Show of the Year - UPDATE - Posted on 17 November 2026

Today, Apple Podcasts named Baby, this is Keke Palmer its Show of the Year, hono...

17/11/2026

Ian Buck Kicks Off NVIDIA's Week at SC26 as Science Enters the Age of Agents

NVIDIA's Ian Buck kicked off the company's week at SC26 in Chicago on Monday with new tools for AI agents, software that can plan experiments, run code ...

16/11/2026

ESPN Edge Partners Chart a Future Built on Data, AI and a New Generation of Fans

The ESPN Edge Innovation Conference once again gave key ESPN technology partners as a chance to take to the stage and discuss the future of sports fan experienc...

16/11/2026

IMGs Ashley Shaner: How Micro-Dramas Can Build Sports Media Audiences and Why Content Creators Matter

As rights holders rethink who tells their stories and where, IMG is preparing th...

16/11/2026

Heard' at SVG Live!: Audio Needs To Make More Noise

From the sidelines, sonic savants advocate for greater emphasis on sound as sports and entertainment increasingly merge SVG Live!, a one-day event on Nov. 10 i...

16/11/2026

Yamaha To Acquire Disguise, Expanding Into Live Sports, Venue Production, and Immersive Entertainment

Deal combines Yamaha's audio technology with Disguise's visual-productio...

16/11/2026

New plug-in suite from Berry Effects

Creative effects designed to encourage experimentation Berry Effects is the brainchild of Abe and Ralph, a pair of multi-instrumentalists and audio engineer...

16/11/2026

Anukari 1.1 now available

Physics-based synth gets first major update Anukari have just released a new version of their innovative physics-based software synth, marking the instrumen...

16/11/2026

Hit'n'Mix RipX DAW 9 arrives

Both versions kitted out with new tools & improvements Hit'n'Mix have just launched the latest major version of their innovative DAW software, with ...

16/11/2026

Eventide to launch SplitEQ 2

Dynamic bands, unmasking features & new stereo tools Launched in 2021, Eventide's SplitEQ plug-in introduced an interesting new approach to EQ, with the...