From the course: Generative AI: Working with Large Language Models
Unlock the full course today
Join today to access over 23,200 courses taught by industry experts.
PaLM
From the course: Generative AI: Working with Large Language Models
PaLM
- [Instructor] In April, 2022, Google released PaLM, or to give it its full name, the Pathways Language Model. Now there are a couple of key takeaways from this model. Comparing the number of parameters, we can see that PaLM is the largest of the dense parameter models with 540 billion parameters. It dwarfs the GPT-3's 175 billion parameters, Gophers, 280 billion, and just edges out Megatron-Turing NLG at 530 billion parameters. Now, Google used the pathway system, a new AI architecture that they revealed at the end of 2021. So using this framework allows for many more chips to be used for model training, with PaLM being trained on 6,144 hardware accelerators versus smaller numbers of chips being used for previous large language models. And finally, if we look at the Model Flops Utilization, you can see that the Model Flops Utilizations have increased going from GPT-3 to PaLM. PaLM has effectively doubled the Model Flops…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Contents
-
-
-
-
-
GPT-34m 32s
-
(Locked)
GPT-3 use cases5m 27s
-
(Locked)
Challenges and shortcomings of GPT-34m 17s
-
(Locked)
GLaM3m 6s
-
(Locked)
Megatron-Turing NLG Model1m 59s
-
(Locked)
Gopher5m 23s
-
(Locked)
Scaling laws3m 14s
-
(Locked)
Chinchilla7m 53s
-
(Locked)
BIG-bench4m 24s
-
(Locked)
PaLM5m 49s
-
(Locked)
OPT and BLOOM2m 51s
-
-