Pamela Krzypkowska

The Chinchilla scaling paper by Hoffmann et al. has been highly influential in the language modeling community. So researchers tried to replicate a key part of their work and discovered discrepancies 👉 Chinchilla Scaling: A replication attempt Tamay Besiroglu, Ege Erdil, Matthew Barnett, Josh You Abstract Hoffmann et al. (2022) propose three methods for estimating a compute-optimal scaling law. We attempt to replicate their third estimation procedure, which involves fitting a parametric loss function to a reconstruction of data from their plots. We find that the reported estimates are inconsistent with their first two estimation methods, fail at fitting the extracted data, and report implausibly narrow confidence intervals--intervals this narrow would require over 600,000 experiments, while they likely only ran fewer than 500. In contrast, our rederivation of the scaling law using the third approach yields results that are compatible with the findings from the first two estimation procedures described by Hoffmann et al. 👉 https://lnkd.in/de7uEdEy #machinelearning

2

🚀 Exciting times at DeepLearn 2024! 🚀 Professor: Michalis Vazirgiannis University: École Polytechnique Course: Graph Machine Learning and Multimodal Graph Generative AI Level: Intermediate/Advanced Dates: 15th - 19th of July Location: Universidade da Maia - UMAIA 📅 Course Highlights: Graph Machine Learning (GML): Understand the challenges of GML, including graph similarity via graph kernels with applications in community detection and impact evaluation. GraKeL Python Library: Get introduced to the GraKeL library for enabling machine learning with graph kernels. Deep Learning for Graphs: Learn about Graph Neural Networks (GNNs) with node and graph embedding algorithms and their experimental evaluations. Message Passing GNN Model: Focus on the GNN model, while exploring alternatives such as RWNN for explainability and Hyperbolic GNNs for hierarchical relations. Graph Generative AI: Discover the emerging field of Graph Generative AI using deep neural networks and large language models, with applications in biomedicine and molecule generation. Multimodality for Graphs: Explore the topic of multimodality for graphs and its implications in various domains. Join us to dive deep into the fascinating world of graph machine learning and multimodal graph generative AI, and enhance your skills in applying these advanced techniques to real-world data structures. For more details, visit: https://lnkd.in/dxZ-S_EZ #DeepLearn2024 #MachineLearning #GraphML #GraphNeuralNetworks #AI #GraphKernels #TechInnovation #DeepLearning #GenerativeAI #MultimodalGraphs

4

From Universitat Polytecnica de Catalunya, CLCG, University of Groningen, FAIR, Meta A Primer on the Inner Workings of Transformer-based Language Models https://lnkd.in/eGJNPH_g The rapid progress of research aimed at interpreting the inner workings of advanced language models has highlighted a need for contextualizing the insights gained from years of work in this area. This primer provides a concise technical introduction to the current techniques used to interpret the inner workings of Transformer-based language models, focusing on the generative decoder-only architecture. We conclude by presenting a comprehensive overview of the known internal mechanisms implemented by these models, uncovering connections across popular approaches and active research directions in this area. This new paper presents a technical introduction to current techniques used to interpret the inner workings of Transformer-based language models. It also provides a detailed overview of the internal mechanisms implemented in these models. It explains why Transformer models have been so effective and transformative for the field.

34

1 komentarz

In this article, the authors prototype a sustainable big #artificial #intelligence (#AI)-native quantum anonymous #semantic #communication (#SC) system. In this framework, they leverage big-AI models for semantic retrieval processing, that is, semantic extraction and recovery, and employ quantum anonymous communication protocols to broadcast semantics. They detail the underlying functionalities, sustainable practices, and potential challenges of integrating big AI into a quantum anonymous semantic broadcast system. They also formulate case studies demonstrating the sustainability and reliability of the envisioned framework. This work provides a sustainable and quantum-safe semantic communication framework by integrating big AI and quantum anonymous communication. ---- Shehbaz Tariq, Uman Khalid, Brian Estadimas Arfeto, Trung Q. Duong, Hyundong Shin More details can be found at this link: https://lnkd.in/egJGsUyz

14

2 komentarze

What about #Java & #AI? Lots of opinions about where it is today but where is it going? That's an awesome question to ask. Nicolai Parlog gave an impressive briefing in his YouTube Newscast earlier this month ... If you're curious, check out the Newscast. #itnews #technews #softwareengineering #ml #machinelearning #softwaredevelopment #itmanagers #devops #datascience https://lnkd.in/gfSrxre6

1

Read the interview with Reinhard Heckel, Professor for Machine Learning, who talks about the role of data in the training of AI systems, how to avoid distortions, and how we can ensure that data is protected. #AI #MachineLearning #data #LLMs Munich Data Science Institute 📷A. Eckert go.tum.de/537884

157

2 komentarze

Fcdnet: Fuzzy Cognition-based Dynamic Fusion Network for Multimodal Sentiment Analysis https://lnkd.in/d6mstJya The Fcdnet dynamically integrates #sentiment #scores across different modalities using a fuzzy cognition fusion mechanism (FCM), significantly enhancing the accuracy of identifying divergent sentiments across modalities. Additionally, a disparity balancing module (DBM) is proposed to normalize the representations between different modality features by penalizing the similarity of sentiments with different #degrees and rewarding the #separability of sentiments with same degree. #Experimental results demonstrate that Fcdnet outperforms state-of-the-art methods on public datasets, validating the superiority and effectiveness. Marc Polo (Ph.D) Miriam Diez Bosch PhD Josep Maria Picola Meix (Ph. D.) Víctor Curto (Ph.D) Blanquerna - Universitat Ramon Llull

4