Yun (Maigo) Wang

Facebook just released a very different kind of LLMs. These are focused on code optimization and compiler optimization. This is super important because it can fundamentally change the way code is executed on any machine. Here's how it works right now: Programs written in many languages like C/C have intermediate steps before they are run on the computer. For example they are converted into Byte Code and then converted into machine code via LLVM (Low level virtual machine). But Facebook's new model can take in the byte code as input and convert it to into a more efficient representation so that when this new code is converted into machine code, it will be faster. "In theory", it could mean that all existing code might be made faster without any change by developers. Our games can have higher frame rates on the same GPUs. Photoshop might run faster on the same laptop. Our software might complete tasks faster without any changes at all. But that will probably take quite a long time. To be honest I don't expect these models to actually be used in production since unless they were 100% as good as compilers they will probably break the code. And are they 100% good? No, not even close. But the speed at which improvements are happening, they will probably reach that level eventually. And that's when we will see models like these everywhere. 

11

Tried to install NVIDIA Isaac Sim to run some local reinforcement learning experiments, but it turns out my (expensive at the time) TITAN V GPU is not compatible. Isaac Sim requires an RTX-branded GPU (which offer hardware-accelerated ray tracing). Fun fact: Even thee latest NVIDIA H100 data center GPUs don't support ray tracing - their flagship GPUs are optimized for AI training inference. For simulation workloads, you need something like an an L4, or one of the desktop-class GeForce (gaming) or A6000 (pro workstation) GPUs. NVIDIA refer to this as the "3 GPU architecture" (simulation, training/inference, and on-robot). So I've purchased a GeForce RTX 4090, and damn this thing is huge. "Titan" V for scale. Barely fits in my desktop case, and turns out I need to order some more power cables off Amazon before I can even plug it in. More of my RL journey to come.

325

44 Comments

Thanks Hamdi Amroun, PhD - this is a great paper to share. In the real world business context we often need AI to reason over large pieces of text, and often you need/want multi-document summarization (MDS). 'When testing five LLMs with benchmarks using news and conversation datasets, they found up to 75% of content in MDS summaries was hallucinated, with notable increases towards summaries' end. Alarmingly, even for non-existent topics, models like GPT-3.5-turbo and GPT-4 generated fabricated content 79% and 44% of the time, respectively. Analysis of 700 generated insights showed that hallucinations often arose from failures to follow instructions or overly generic content.' I agree with Hamdi's sentiments that despite improvements in post-processing techniques, robust solutions are urgently needed to reduce hallucinations, as inaccurate summaries can lead to business risks (notably misinformation and misrepresentation). I would add that there is still a lot of utility in LLMs/Multimodal models but we need to be vigilant - this is not about AI malevolence or intentional deception. LLMs do not possess consciousness or intent; they generate content based on patterns in data they were trained on.

4

1 Comment

It’s been a pleasure to talk alongside such great researchers from Google DeepMind and top Taiwanese universities at Google AI Taiwan Academic Forum. (Details at https://lnkd.in/gnbcWZyf) My topic — “Challenges of Applied ML” — covered 4 observations: - We ought to build product, not merely expose LLMs. - Making them useful and trustworthy requires handling input and output properly. (Read more at https://lnkd.in/gcNDC54R) - Making those model useful requires high-quality data for tuning (and evaluation). (More at https://lnkd.in/gyVYQaaD) - There’s many practical deployment considerations, one of the main ones being cost.

111

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

16

There is a nice video about CMake, if you want to get some overview about that build-system, it might be a good choice. #CMake #cpp #buildsystems #development #softwaredevelopment

31