Nikhil Paliwal

Is GPT-4o good at table recognition? Not quite! OCR looks fine, but tripped by spanning cell table structure! 🤩 However, our UniTable, with only 124M param trained by self-supervision, shows great understanding! 👉 UniTable paper: arxiv.org/abs/2403.04822 👉 UniTable code: https://lnkd.in/e4a7YMxJ #MachineLearning #GPT-4o #Table #Document

83

7 Kommentare

🚨Paper Alert 🚨 ➡️Paper Title: Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control 🌟Few pointers from the paper 🎯Current face reenactment and swapping methods mainly rely on GAN frameworks, but recent focus has shifted to pre-trained diffusion models for their superior generation capabilities. 🎯However, training these models is resource-intensive, and the results have not yet achieved satisfactory performance levels. 🎯To address this issue, authors have introduced "Face-Adapter", an efficient and effective adapter designed for high-precision and high-fidelity face editing for pre-trained diffusion models. 🎯They observed that both face reenactment/swapping tasks essentially involve combinations of target structure, ID and attribute.   🎯They aimed to sufficiently de-couple the control of these factors to achieve both tasks in one model. 🎯Specifically, their method contains :  ➕A Spatial Condition Generator that provides precise landmarks and background. ➕A Plug-and-play Identity Encoder that transfers face embeddings to the text space by a transformer decoder. ➕An Attribute Controller that integrates spatial conditions and detailed attributes. 🎯Face-Adapter achieves comparable or even superior performance in terms of motion control precision, ID retention capability, and generation quality compared to fully fine-tuned face reenactment/swapping models. 🏢Organization: Zhejiang University, Tencent, VIVO, Nanyang Technological University Singapore 🧙Paper Authors: Yue Han, Junwei Zhu, Keke He, Xu Chen, Yanhao Ge, Wei Li, Xiangtai Li, Jiangning Zhang, Chengjie Wang, Yong Liu 1️⃣Read the Full Paper here: https://lnkd.in/gwkpW6TE 2️⃣Project Page: https://lnkd.in/gqDTQGQQ 3️⃣Code: https://lnkd.in/gtd9tGmP (Coming 🔜) 🎥 Be sure to watch the attached Demo Video-Sound on 🔊🔊 🎵 Music by Aleksey Chistilin from Pixabay Find this Valuable 💎 ? ♻️REPOST and teach your network something new Follow me 👣, Naveen Manwani, for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements.

63

Advances in Bayesian Deep Neural Network Ensembles and Active Learning for Preference Modeling Quick read: https://lnkd.in/gzZB2VGh Paper 1: https://lnkd.in/gBkwknBZ Paper 2: https://lnkd.in/gNrqnqg9

18

Discover the benefits of the Cerebras AI Model studio: - Train 1-175 billion parameter models 8x faster than most other publicly available GPU instances - Enable higher-performing models with longer sequence lengths (up to 50,000!) - Train your models from scratch or fine-tune open-source models with your data Learn more about our partnership with Cerebras Systems: https://hubs.la/Q02tw4KY0

9

Some exciting news from Google DeepMind. The Gemma 2 2B model has been added to the lightweight Gemma family! ⚡️ Model Details ⚡️ 🔷 Gemma 2B is optimized for on-device and edge inference via knowledge distillation. 🔷 It is a 2 billion (2B) parameter model. 🔷 Trained on 2 trillion tokens (web data, code, mathematics text) using the latest TPUs. 🔷 Memory Requirement: ~1GB in int4 quantization, optimized with the NVIDIA TensorRT-LLM library. 🔷 Improved balance between performance and efficiency 🔷 Available under the commercially friendly Gemma terms for research and commercial applications. 🔷 Gemma 2 2B surpasses all GPT-3.5 models on the LMSYS Chatbot Arena leaderboard. 💡 Why this matters? 💡 The work done by Google and Meta will play a significant role in the future. Almost 90% of the companies will use a hundred billion parameter models and distill them into much smaller models for their specific tasks. Google DeepMind further released Shield Gemma and Gemma Scope. Gemma Sield filters the input and outputs of AI models and keep the user safe, showing their commitment to Responsible AI. Gemma Scope on the other hand is a new model interpretability tool that which I believe can further enhance our knowledge and process about model distillation, resulting in further gains. 🔗 Links 🔗 Release blog: https://lnkd.in/gYRsbDWa Gemma Scopre: https://lnkd.in/gZRYmQZX

19

Multimodal foundation models have shown significant promise, with several attempts proposed since 2020 - https://lnkd.in/eEzTmTER. I look forward to reading this new development. AI at Meta

1

🚀 Supercharge LLMs with Buffer of Thoughts (BoT)! 🚀 This new prompting method enables Llama3-8B to exceed the performance of Llama3-70B: 🎯 Existing prompting methods can be divided into two main types - Single query: Methods like “Chain-of-thought” and Few-Shot Prompting which generally tackle the task directly. However, they require manually crafted examples / assumptions, making them hard to scale. - Multi query: Methods like “Tree of thoughts”, “Graph of thoughts” which solve the task using multiple queries to identify solution paths for every task. These are computationally expensive due to the underlying search operation. 🎯 The paper proposes a new framework called “Buffer of Thoughts”.   The following is the core idea: - A problem distiller extracts the relevant information from the task  - A lookup process is used to retrieve a “thought template” from a meta buffer. As the LLM processes new tasks over time, this meta buffer is kept up to date by  a buffer manager which extracts new thought templates from the provided tasks. - This thought template is then instantiated by the specific task inputs. - The LLM solves the task using the instantiated template. 🎯 The paper shows promising results, allowing Llama3-8B with BoT to exceed Llama3-70B (standard prompting). It also outperforms existing single and multi query #prompting methods over multiple tasks. While the direction is promising, there may be some efficiency limitations as the individual components like problem distiller, buffer manager and thought instantiator are also based on #LLMs. However, the paper claims the method to be at only 12% cost of multi-query methods on average. Check out the full paper at https://lnkd.in/g3q3i7vm

15

🚨Paper Alert 🚨 ➡️Paper Title: A Multimodal Automated Interpretability Agent 🌟Few pointers from the paper 🎯In this paper authors have described MAIA, “a  🤖 Multimodal Automated Interpretability Agent 🤖”. MAIA is a system that uses neural models to automate neural model understanding tasks like feature interpretation and failure mode discovery. 🎯It equips a pre-trained vision-language model with a set of tools that support iterative experimentation on subcomponents of other models to explain their behavior. 🎯These include tools commonly used by human interpretability researchers: for synthesizing and editing inputs, computing maximally activating exemplars from real-world datasets, and summarizing and describing experimental results. 🎯Interpretability experiments proposed by MAIA compose these tools to describe and explain system behavior. 🎯Authors evaluated  applications of MAIA to computer vision models. They first characterize MAIA’s ability to describe (neuron-level) features in learned representations of images. 🎯Across several trained models and a novel dataset of synthetic vision neurons with paired ground-truth descriptions, MAIA produces descriptions comparable to those generated by expert human experimenters. 🎯Then Authors showed that MAIA can aid in two additional interpretability tasks: reducing sensitivity to spurious features, and automatically identifying inputs likely to be mis-classified. 🏢Organization:  MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) 🧙Paper Authors: Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram, Evan Hernandez, Jacob Andreas , Antonio Torralba 1️⃣Read the Full Paper here: https://lnkd.in/gjmN_gaR 2️⃣Project Page: https://lnkd.in/gxPr_jpM 3️⃣Experiment Browser: https://lnkd.in/gGe7FyUY 🎥 Be sure to watch the attached Demo Video-Sound on 🔊🔊 Music by Sergii Pavkin from Pixabay Find this Valuable 💎 ? ♻️REPOST and teach your network something new Follow me 👣, Naveen Manwani, for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements.

2