Sony Pixel Power calrec Sony

NVIDIA Launches NIM Microservices for Generative AI in Japan, Taiwan

26/08/2024

Nations around the world are pursuing sovereign AI to produce artificial intelligence using their own computing infrastructure, data, workforce and business networks to ensure AI systems align with local values, laws and interests.

In support of these efforts, NVIDIA today announced the availability of four new NVIDIA NIM microservices that enable developers to more easily build and deploy high-performing generative AI applications.

The microservices support popular community models tailored to meet regional needs. They enhance user interactions through accurate understanding and improved responses based on local languages and cultural heritage.

In the Asia-Pacific region alone, generative AI software revenue is expected to reach $48 billion by 2030 - up from $5 billion this year, according to ABI Research.

Llama-3-Swallow-70B, trained on Japanese data, and Llama-3-Taiwan-70B, trained on Mandarin data, are regional language models that provide a deeper understanding of local laws, regulations and other customs.

The RakutenAI 7B family of models, built on Mistral-7B, were trained on English and Japanese datasets, and are available as two different NIM microservices for Chat and Instruct. Rakuten's foundation and instruct models have achieved leading scores among open Japanese large language models, landing the top average score in the LM Evaluation Harness benchmark carried out from January to March 2024.

Training a large language model (LLM) on regional languages enhances the effectiveness of its outputs by ensuring more accurate and nuanced communication, as it better understands and reflects cultural and linguistic subtleties.

The models offer leading performance for Japanese and Mandarin language understanding, regional legal tasks, question-answering, and language translation and summarization compared with base LLMs like Llama 3.

Nations worldwide - from Singapore, the United Arab Emirates, South Korea and Sweden to France, Italy and India - are investing in sovereign AI infrastructure.

The new NIM microservices allow businesses, government agencies and universities to host native LLMs in their own environments, enabling developers to build advanced copilots, chatbots and AI assistants.

Developing Applications With Sovereign AI NIM Microservices Developers can easily deploy the sovereign AI models, packaged as NIM microservices, into production while achieving improved performance.

The microservices, available with NVIDIA AI Enterprise, are optimized for inference with the NVIDIA TensorRT-LLM open-source library.

NIM microservices for Llama 3 70B - which was used as the base model for the new Llama-3-Swallow-70B and Llama-3-Taiwan-70B NIM microservices - can provide up to 5x higher throughput. This lowers the total cost of running the models in production and provides better user experiences by decreasing latency.

The new NIM microservices are available today as hosted application programming interfaces (APIs).

Tapping NVIDIA NIM for Faster, More Accurate Generative AI Outcomes The NIM microservices accelerate deployments, enhance overall performance and provide the necessary security for organizations across global industries, including healthcare, finance, manufacturing, education and legal.

The Tokyo Institute of Technology fine-tuned Llama-3-Swallow 70B using Japanese-language data.

LLMs are not mechanical tools that provide the same benefit for everyone. They are rather intellectual tools that interact with human culture and creativity. The influence is mutual where not only are the models affected by the data we train on, but also our culture and the data we generate will be influenced by LLMs, said Rio Yokota, professor at the Global Scientific Information and Computing Center at the Tokyo Institute of Technology. Therefore, it is of paramount importance to develop sovereign AI models that adhere to our cultural norms. The availability of Llama-3-Swallow as an NVIDIA NIM microservice will allow developers to easily access and deploy the model for Japanese applications across various industries.

For instance, a Japanese AI company, Preferred Networks, uses the model to develop a healthcare specific model trained on a unique corpus of Japanese medical data, called Llama3-Preferred-MedSwallow-70B, that tops scores on the Japan National Examination for Physicians.

Chang Gung Memorial Hospital (CGMH), one of the leading hospitals in Taiwan, is building a custom-made AI Inference Service (AIIS) to centralize all LLM applications within the hospital system. Using Llama 3-Taiwan 70B, it is improving the efficiency of frontline medical staff with more nuanced medical language that patients can understand.

By providing instant, context-appropriate guidance, AI applications built with local-language LLMs streamline workflows and serve as a continuous learning tool to support staff development and improve the quality of patient care, said Dr. Changfu Kuo, director of the Center for Artificial Intelligence in Medicine at CGMH, Linko Branch. NVIDIA NIM is simplifying the development of these applications, allowing for easy access and deployment of models trained on regional languages with minimal engineering expertise.

Taiwan-based Pegatron, a maker of electronic devices, will adopt the Llama 3-Taiwan 70B NIM microservice for internal- and external-facing applications. It has integrated it with its PEGAAi Agentic AI System to automate processes, boosting efficiency in manufacturing and operations.

Llama-3-Taiwan 70B NIM is also being used by global petrochemical manufacturer Chang Chun Group, world-leading printed circuit board company Unimicron, technology-focused media company TechOrange, online contract service company LegalSign.ai and generative AI startup APMIC. These companies are also collaborating on the open model.

Creating Custom Enterprise Models
LINK: https://blogs.nvidia.com/blog/nim-microservices-generative-ai/...
See more stories from nvidia

More from Nvidia

17/10/2025

Open Source AI Week - How Developers and Contributors Are Advancing AI Innovation

NVIDIA's on the ground at Open Source AI Week. Stay tuned for a celebration ...

17/10/2025

The Engines of American-Made Intelligence: NVIDIA and TSMC Celebrate First NVIDIA Blackwell Wafer Produced in the US

AI has ignited a new industrial revolution. NVIDIA and TSMC are working togethe...

16/10/2025

Ready, Set, Reward - GeForce NOW Membership Rewards Await

GeForce NOW is more than just a platform to stream fresh games every week - it offers celebrations for the gamers who make it epic, with member rewards to sweet...

14/10/2025

NVIDIA and Oracle to Accelerate Enterprise AI and Data Processing

AI is transforming the way enterprises build, deploy and scale intelligent applications. As demand surges for enterprise-grade AI applications that offer speed,...

14/10/2025

Oracle and NVIDIA Accelerate Sovereign AI, Enabling Abu Dhabi's AI-Native Government Transformation

At Oracle AI World, NVIDIA and Oracle announced they are deepening their collabo...

13/10/2025

Elon Musk Gets Just-Launched NVIDIA DGX Spark: Petaflop AI Supercomputer Lands at SpaceX

The future of AI took flight at Starbase, Texas - where NVIDIA CEO Jensen Huang ...

13/10/2025

NVIDIA, Partners Drive Next-Gen Efficient Gigawatt AI Factories in Buildup for Vera Rubin

At the OCP Global Summit, NVIDIA is offering a glimpse into the future of gigawa...

09/10/2025

NVIDIA Blackwell Raises Bar in New InferenceMAX Benchmarks, Delivering Unmatched Performance and Efficiency

NVIDIA Blackwell swept the new SemiAnalysis InferenceMAX v1 benchmarks, deliveri...

09/10/2025

Microsoft Azure Unveils World's First NVIDIA GB300 NVL72 Supercomputing Cluster for OpenAI

Microsoft Azure today announced the new NDv6 GB300 VM series, delivering the ind...

09/10/2025

Incoming: Battlefield 6' Lands on GeForce NOW at Launch

Lock, load and stream - the battle is just beginning. EA's highly anticipated Battlefield 6 is set to storm the cloud when it launches tomorrow with GeForce...

08/10/2025

How AI-Powered Wireless Networks Will Revitalize US Global Leadership in Communications

Telecommunication networks are critical infrastructure for every nation, underpi...

02/10/2025

GeForce NOW Brings 17 Games to the Cloud in October for a Spooky Good Time

Editor's note: This blog has been updated to include an additional game for October, The Outer Worlds 2. October is creeping in with plenty of gaming treat...

01/10/2025

How to Get Started With Large Language Models on NVIDIA RTX PCs

Many users want to run large language models (LLMs) locally for more privacy and control, and without subscriptions, but until recently, this meant a trade-off ...

30/09/2025

How Quantum Computing's Biggest Challenges Are Being Solved With Accelerated Computing

Quantum computing promises to reshape industries - but progress hinges on solvin...

30/09/2025

Into the Omniverse: Open-Source Physics Engine and OpenUSD Advance Robot Learning

Editor's note: This blog is a part of Into the Omniverse, a series focused o...

25/09/2025

Pilots Wanted: Stream Mecha BREAK' on GeForce NOW

Suit up and head for the cloud. Mecha BREAK, the popular third-person shooter, is now available to stream on GeForce NOW with NVIDIA DLSS 4 technology. Catch i...

24/09/2025

Canada Goes All In on AI: NVIDIA Joins Nations' Technology Leaders in Montreal to Shape Sovereign AI Strategy

Canada's role as a leader in artificial intelligence was on full display at ...

24/09/2025

Open Secret: How NVIDIA Nemotron Models, Datasets and Techniques Fuel AI Development

Open technologies - made available to developers and businesses to adopt, modify...

23/09/2025

At Climate Week NYC, NVIDIA Details AI's Key Role in the Sustainable Energy Transition

Energy efficiency in large language model inference has improved 100,000x in the...

22/09/2025

NVIDIA, OpenAI Announce the Biggest AI Infrastructure Deployment in History'

OpenAI and NVIDIA just announced a landmark AI infrastructure partnership - an initiative that will scale OpenAI's compute with multi-gigawatt data centers ...

19/09/2025

AI On: How Onboarding Teams of AI Agents Drives Productivity and Revenue for Businesses

AI is no longer solely a back-office tool. It's a strategic partner that can...

18/09/2025

The UK's Goldilocks Moment for AI': NVIDIA, UK and US Leaders Highlight AI Infrastructure Investments

The U.K. was the center of the AI world this week as NVIDIA, U.K. and U.S. leade...

18/09/2025

GeForce NOW Unleashes Dying Light: The Beast' in the Cloud

GeForce NOW is packing a monstrous punch this week. Dying Light: The Beast, the latest adrenaline fueled chapter in Techland's parkour meets survival horror...

17/09/2025

Meet the Streamlabs Streaming Assistant, Accelerated by NVIDIA RTX

Today's creators are equal parts entertainer, producer and gamer, juggling game commentary, scene changes, replay clips, chat moderation and technical troub...

16/09/2025

The AI Makers: NVIDIA Partners in UK Advance Physical and Agentic AI, Robotics, Life Sciences and More

The U.K. is driving investments in sovereign AI, using the technology to advance...

13/09/2025

Reaching Across the Isles: UK-LLM Brings AI to UK Languages With NVIDIA Nemotron

Celtic languages - including Cornish, Irish, Scottish Gaelic and Welsh - are the U.K.'s oldest living languages. To empower their speakers, the UK-LLM sover...

10/09/2025

Paint It Blackwell: GeForce RTX 5080 SuperPOD Rollout Begins

GeForce NOW Blackwell RTX 5080-class SuperPODs are now rolling out, unlocking a new level of ultra high-performance, cinematic cloud gaming. GeForce NOW Ultima...

09/09/2025

NVIDIA Rubin CPX Accelerates Inference Performance and Efficiency for 1M+ Token Context Workloads

Inference has emerged as the new frontier of complexity in AI. Modern models are...

09/09/2025

NVIDIA Blackwell Ultra Sets New Inference Records in MLPerf Debut

As large language models (LLMs) grow larger, they get smarter, with open models from leading developers now featuring hundreds of billions of parameters. At the...

09/09/2025

NVIDIA Partners With AI Infrastructure Ecosystem to Unveil Reference Design for Giga-Scale AI Factories

At this week's AI Infrastructure Summit in Silicon Valley, NVIDIA's VP o...

09/09/2025

NVIDIA Blackwell Ultra Sets the Bar in New MLPerf Inference Benchmark

Inference performance is critical, as it directly influences the economics of an AI factory. The higher the throughput of AI factory infrastructure, the more to...

09/09/2025

Safety First, Always,' NVIDIA VP of Automotive Says, Unveiling the Future of AI-Defined Vehicles at IAA Mobility

At this week's IAA Mobility conference in Munich, NVIDIA Vice President of A...

09/09/2025

Get Started Using Generative AI for Content Creation With ComfyUI and NVIDIA RTX AI PCs

ComfyUI - an open-source, node-based graphical interface for running and buildin...

04/09/2025

NVIDIA Pledges AI Education Funding for K-12 Programs

NVIDIA today announced new AI education support for K-12 programs at a White House event to celebrate public-private partnerships that advance artificial intell...

04/09/2025

AI On: 6 Ways AI Agents Are Raising Team Performance - and How to Measure It

Editor's note: This post is part of the AI On blog series, which explores the latest techniques and real-world applications of agentic AI, chatbots and copi...

04/09/2025

Cloud Gaming to Reach New Heights: GeForce NOW's Blackwell RTX Upgrade Begins Next Week

NVIDIA Blackwell RTX is coming to the cloud on Wednesday, Sept. 10 - an upgrade ...

03/09/2025

Scene It to Believe It: Populate 3D Worlds Quickly With NVIDIA AI Blueprints

3D artists are constantly prototyping. In traditional workflows, modelers must build placeholder, low-fidelity assets to populate 3D scenes, tinkering and adju...

02/09/2025

It's the Humidity: How International Researchers in Poland, Deep Learning and NVIDIA GPUs Could Change the Forecast

For more than a century, meteorologists have chased storms with chalkboards, equ...

28/08/2025

Drop Into the Battle: Gears of War: Reloaded Unleashed' Launches on GeForce NOW

Brace yourself, COGs - the Locusts aren't the only thing rising up. The Coal...

28/08/2025

Game On: How Modders Reimagine Classic Games With NVIDIA RTX Remix and Generative AI

Last week at Gamescom, NVIDIA announced the winners of the NVIDIA and ModDB RTX ...

27/08/2025

How Do You Teach an AI Model to Reason? With Humans

AI models are advancing at a rapid rate and scale. But what might they lack that (most) humans don't? Common sense: an understanding, developed through rea...

25/08/2025

NVIDIA Jetson Thor Unlocks Real-Time Reasoning for General Robotics and Physical AI

Robots around the world are about to get a lot smarter as physical AI developers...

25/08/2025

Take It for a Spin: NVIDIA Rolls Out DRIVE AGX Thor Developer Kit to World's Automotive Developers

As autonomous vehicle systems rapidly grow in complexity, equipped with reasonin...

22/08/2025

Inside NVIDIA Blackwell Ultra: The Chip Powering the AI Factory Era

As the latest member of the NVIDIA Blackwell architecture family, the NVIDIA Blackwell Ultra GPU builds on core innovations to accelerate training and AI reason...

22/08/2025

Hot Topics at Hot Chips: Inference, Networking, AI Innovation at Every Scale - All Built on NVIDIA

AI reasoning, inference and networking will be top of mind for attendees of next...

21/08/2025

RIKEN, Japan's Leading Science Institute, Taps Fujitsu and NVIDIA for Next Flagship Supercomputer

Japan is once again building a landmark high-performance computing system - not ...

21/08/2025

Think SMART: How to Optimize AI Factory Inference Performance

From AI assistants doing deep research to autonomous vehicles making split-second navigation decisions, AI adoption is exploding across industries. Behind ever...

21/08/2025

Gearing Up for the Gigawatt Data Center Age

Across the globe, AI factories are rising - massive new data centers built not to serve up web pages or email, but to train and deploy intelligence itself. Inte...

21/08/2025

GeForce NOW Brings RTX 5080 Power to the Ultimate Membership

Get a glimpse into the future of gaming. The NVIDIA Blackwell RTX architecture is coming to GeForce NOW in September, marking the service's biggest upgrade...

20/08/2025

Into the Omniverse: How OpenUSD and Digital Twins Are Powering Industrial and Physical AI

Editor's note: This blog is a part of Into the Omniverse, a series focused o...