Sean (Xiang) Ren

The BayLearn 2024 abstract submission deadline has been extended to Aug 5, 2024. If you are working on cutting-edge ML research, please consider submitting an abstract. BayLearn 2024 will be an in-person event, held on Oct 10, 2024: https://baylearn.org This is a great opportunity for student researchers to showcase their latest research work and get good feedback. Submissions are non-archival and the review process is quite informal. #machinelearning #research

69

Great work here Rohan Sidankar . Thanks to Rohan and Justin, hushh is known for its work in optimizing neural network search through dimensionality reduction, specifically with ViT-B-32 CLIP for various search tasks as we build out our search index. This allows for maintaining recall while reducing vector size, enhancing search efficiency and effectiveness against large datasets like the LAION Benchmark Repository. Dimensionality reduction impacts neural network search by enabling the maintenance of recall while reducing vector size. This optimization, as seen in Rohan Sidankar's work at Hushh with ViT-B-32 CLIP, enhances search efficiency and effectiveness against large datasets like the LAION Benchmark Repository. ViT-B-32 CLIP can enhance search tasks by allowing for dimensionality reduction in neural network search, as demonstrated by Rohan Sidankar's work at Hushh. This approach enables the maintenance of recall while reducing the vector size, which is crucial for efficient and effective search tasks against large datasets like the LAION Benchmark Repository.

11

I recently completed a detailed review of the paper "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations" by Lan et al.! This fascinating study presents an innovative approach to language modelling, emphasizing efficiency and effectiveness through a lighter, more parameter-efficient architecture compared to traditional models like BERT. In my review, I delved into the strengths and weaknesses of the paper, discussing its impact and potential areas for improvement. This work has made a significant contribution to the field of natural language processing and has already influenced many advancements in the area. You can read my full review of the paper here: https://lnkd.in/ehxm5jXt #NLP #MachineLearning #ArtificialIntelligence #AI #Research #TechReview #DeepLearning

16

1 Comment

I wrote two blog posts about SN, Leon Bottou and Yann LeCun's 1988 𝘚𝘪𝘮𝘶𝘭𝘢𝘵𝘦𝘶𝘳 𝘥𝘦 𝘕𝘦𝘶𝘳𝘰𝘯𝘦𝘴. One is an English translation of the original paper, for which I've reproduced the figures. The other is a tutorial on how to run their code on Apple silicon. https://lnkd.in/dMFxMZeT

76

2 Comments

This approach could turn the Generative AI industry upside down. Most everyone knows that AI requires computers to do a lot of calculations and especially for Large Language Models (LLMs) like ChatGPT. The majority of these calculations are actually “Matrix Multiplications”. (You remember, a 3x3 matrix times another 3x3 matrix). If the researchers in the article are proven correct, they have demonstrated a way that Matrix Multiplication can be replaced with a more efficient alternative inside the LLM getting nearly the same results. The article positions this savings as a win for the environment (yeah environment), but I think they are missing the boat. More efficient means less electricity and potentially a smaller need for GPU resources, the bread and butter of stock darling Nvidia. When you combine this cost saving with reduced privacy concerns of hosted LLMs (Open AI) and innovative LLM licensing schemes (LibreChat - https://lnkd.in/gKQjMfz5) you can see the cracks in the current direction of the LLM industry. Q: If you could run your own LLM on your own private environment with MIT open-source licensing and without expensive GPUs, why exactly do you need Nvidia, Open AI and alike? Bold prediction: if the researchers get this working at scale, we could see a rapid shift and commoditization in the LLM and generative AI space. All the pieces are in place. TLDR details from the article, describing the innovation: “The researchers' approach involves two main innovations: first, they created a custom LLM and constrained it to use only ternary values (-1, 0, 1) instead of traditional floating-point numbers, which allows for simpler computations. Second, the researchers redesigned the computationally expensive self-attention mechanism in traditional language models with a simpler, more efficient unit (that they called a MatMul-free Linear Gated Recurrent Unit—or MLGRU) that processes words sequentially using basic arithmetic operations instead of matrix multiplications.” #AI #GenerativeAI #Innovation

24