Xiaolong Wang

🔥 Excited to share our work on speculative diffusion decoding! We use discrete diffusion as a drafting model to massively speed up #LLM inference---up to 8.7x faster over standard generation and 2.5x faster than traditional speculative decoding approaches. 🚀 In speculative decoding, the drafter does not need to be of the same type as the larger model, it only requires having the same tokenizer. We exploit this by taking advantage of the speed of novel discrete diffusion for language generation, resulting in impressive speedups. 🔥 Work with Jacob Christopher, Brian Bartoldson, and Bhavya Kailkhura 🔗 paper link: https://lnkd.in/eVgG6k7C

32

Just share two experiments with ChatGPT4o. First experiment, I uploaded a paper TOC as a GIT image with 20 KB with a schematic on the left and SFG curves on the right. GPT4o recognized the right part as SFG spectroscopic data and pointed out the key points from those curves correctly. It also recognized the schematic on the left and all organic molecules and presented the right information. In the second experiment, I uploaded a CSV file with nonfouling and fouling peptides. Then, I requested GTP4o summarize what it learned from these peptides and predict another group of peptides' nonfouling properties. Interestingly, GTP4o got 82.9% correct. This is a very nice value, considering we have not played any prompt or RAG yet.

12

Dr. Kripa Sundar asked me an interesting question today on my thoughts on #AI in education. My first thought was: Haven't we already been there in terms of the decades of research on cognitive tutors? Though not technically AI as we know it today but some form of it. We still have a long ways to go in terms of integrating AI into classroom instruction. My second thought was: How I can help promote the #ethical use of AI both as a researcher and as an educator? This is the list that I came up with: I can help people understand the limitations of using AI to teach students I can help people understand privacy standards and the ethical use of student data I can help promote practices around #digitalliteracy I can help promote practices around #researchliteracy on not only evaluating the effectiveness of AI but also on when to use AI as an analytical tool and to assist in data analysis. I am not worried about AI replacing teachers. I am not worried about AI replacing researchers. Would love to hear from others on where they see their role in AI!

26

3 Comments

What I take away from this new piece of our research is a reminder that the human brain has a strong tendency to create meaning maps and that the process of learning to understand computer code looks a heck of a lot like the process of learning to understand a second language. I wonder who might show up to an Intro to Programming class if this were better understood and instantiated in our teaching methods?

7

A comprehensive overview and synthesis of retrieval-enhanced machine learning models (RAG and beyond)! 👇

43

Let's have a look at the Best Paper Honorable Mention of ACM SIGGRAPH 2024! High quality asset generation "CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets" is a great paper published by DeemosTech, Huazhong University of Science and Technology and ShanghaiTech University. They introduce CLAY, a 3D geometry and material generator designed to effortlessly transform human imagination into intricate 3D digital structures. CLAY supports classic text or image inputs as well as 3D-aware controls from diverse primitives (multi-view images, voxels, bounding boxes, point clouds, implicit representations, etc). At its core is a large-scale generative model composed of a multi-resolution Variational Autoencoder (VAE) and a minimalistic latent Diffusion Transformer (DiT), to extract rich 3D priors directly from a diverse range of 3D geometries. Specifically, it adopts neural fields to represent continuous and complete surfaces and uses a geometry generative module with pure transformer blocks in latent space. They present a progressive training scheme to train CLAY on an ultra large 3D model dataset obtained through a carefully designed processing pipeline, resulting in a 3D native geometry generator with 1.5 billion parameters. For appearance generation, CLAY sets out to produce physically-based rendering (PBR) textures by employing a multi-view material diffusion model that can generate 2K resolution textures with diffuse, roughness, and metallic modalities. They demonstrate using CLAY for a range of controllable 3D asset creations, from sketchy conceptual designs to production ready assets with intricate details. Even first time users can easily use CLAY to bring their vivid 3D imaginations to life, unleashing unlimited creativity. You can test it with Rodin-Gen-1 already! https://lnkd.in/entaZjtR All links in comments below #generativeai #mesh #cad #assets #vfx #assetsgeneration #diffusionmodels #neuralradiance #machinelearning #deeplearning #futurism #computervision #graphics #graphicsengine

26

4 Comments

I am excited to share that our paper, "Silver Lining in the Fake News Cloud: Can Large Language Models Help Detect Misinformation?" has been accepted for publication in IEEE Transactions on Artificial Intelligence (IEEE TAI)! Abstract: In the age of advanced generative AI, distinguishing truth from deception is a growing societal challenge. Our research explores how Large Language Models (LLMs) can aid in detecting misinformation. We use a comprehensive approach, evaluating multiple LLMs with few and zero-shot prompts and incorporating sentiment and emotion annotations to understand their impact on misinformation detection. Additionally, we deliberately distort news using ChatGPT to examine various parameters like abstractness and concreteness, enhancing our framework for identifying authentic, human-generated, and LLM-induced misinformation. Impact Statement: As LLMs evolve, they play a dual role as both defenders and perpetrators of misinformation. This research delves into the conflicting nature of LLMs, evaluating their effectiveness across diverse datasets and prompting techniques. By corrupting authentic news and human-generated fake news with ChatGPT, we quantify parameters like concreteness and abstractness to improve misinformation detection tools. Our findings aim to enhance current frameworks and contribute to the responsible development of AI, fostering a more reliable and resilient information ecosystem in the face of misinformation. Key Contributions: LLM Evaluation: Assessed multiple LLMs' effectiveness in detecting rumor content across six datasets. Prompting Techniques: Investigated few-shot and zero-shot prompting and examined sensitivity to hyperparameters. Sentiment & Emotion: Explored the impact of sentiment and emotion on LLMs' rumor detection capabilities. Dual Corruption Approach: Intentionally distorting both authentic news and human-generated misinformation using ChatGPT. We analyzed parameters such as concreteness and named entity density to gain deeper insights into the impact of these distortions. Explore our research and access the code on GitHub: https://lnkd.in/gbY-9QcT A huge thank you to my co-authors Raghvendra Kumar, Bhargav Goddu, Adam Jatowt, and all those who supported this journey. Here's to advancing the fight against misinformation with cutting-edge AI technologies! #Research #IEEE #ArtificialIntelligence #MisinformationDetection #LargeLanguageModels #AI #FakeNews #IEEE_TAI

249

11 Comments