Evan Miyazono

This week, I'm interviewing extraordinary Bradley Voytek — neuroscience prof at UC San Diego, founding member of UCSD's Data Science Institute, and he was Uber's first data scientist. Got questions for him? Questions at the intersection of neuroscience and data science would be particularly interesting, given that Prof. Voytek's research specialty is combining large-scale data science and machine learning to study how brain regions communicate with one another, and how that communication changes with aging and disease. (UCSD is arguably the world's best neuroscience research institute, so this is quite a special opportunity for ya!) More detail on Brad: • Professor in UC San Diego's Department of Cognitive Science, the Halıcıoğlu Data Science Institute, and the Neurosciences Graduate Program. • Alfred P. Sloan Neuroscience Research Fellow. • Kavli Fellow of the National Academies of Sciences. • Joined Uber as their first data scientist, when it was a 10-person startup, helping build their data science strategy and team. • Outreach work has appeared in Scientific American, NPR... and Comic-Con! • Co-authored the amusing book "Do Zombies Dream of Undead Sheep?" The "Super Data Science Podcast with Jon Krohn" is available on all major podcasting platforms and a video version is on YouTube. Brad's episode will likely be #829 and be released on October 22nd. #superdatascience #machinelearning #ai #datascience #neuroscience

79

1 Comment

This paper is inspired! Much of the current AI weather model research is focused on more accurate and faster versions of what we already have. That's important, but the most exciting advances will come from asking "what *new capabilities* can AI models provide?" I've heard the phrase "AI models will allow us to ask new science questions" before. These folks actually did it and found something fascinating. [Side note: the approach they use find a more accurate initial state is the same one we use to assimilate near real time streamflow observations into HydroForecast at Upstream Tech. Cool!]

35

3 Comments

Nearly a decade ago, I wrote out a complete and rigorous proof of Cantor's theorem that the set of sequences of natural numbers is bigger than the set of natural numbers...using only words of one syllable. https://lnkd.in/g4wWYVYc This is not the only time I revisited the result, much more recently I formally published a number of generalisations of this using different fundamental assumptions (for experts: taking Lawvere's fixed-point theorem and the contrapositive diagonal argument and showing how the proof works in a substructural type of logic). But this one-syllable-word proof was so much fun. I was reminded of this because someone got in touch with me, saying how it is something he periodically re-read for pure interest's sake, and asking if I could recover some material that was on the now-vanished Google to which I linked. I had in fact saved that discussion, which had contributions of similar pieces from a number of other mathematicians, including the Fields medal winner Tim Gowers. I will put a pdf of that thread somewhere public, soon, and link to it. But my correspondent was so taken by the exercise that he said he'd go so far as to purchase a physical book containing a collection of such "short words proofs", if one existed. The idea of publishing a slim volume of fun little proofs certainly is appealing, in some respects, because I love producing physical artifacts that reflect mathematics, but the market must surely be tiny!

7

2 Comments

Many researchers are building agents to improve the capabilities of LLMs at long-range tasks, like writing a pull request for an issue in a code repository. Typical agents use the LLM in a loop, possibly with human feedback, where previous answers are an input to the LLM and are used to inform next actions. Other promising agents use tree searches, where multiple trains of thought are sampled in parallel. MOLLEO is a recent algorithm that caught my eye, as the authors wrapped an LLM in an evolutionary algorithm to generate SMILES strings. An evolutionary algorithm goes like this. Start with a group of candidates and calculate their fitness with an oracle. Then, sample two parents proportional to the fitness. Combine the two parents with a Crossover operator to create an offspring and then edit the offspring with a Mutation operator. Remove the lowest fitness members of the population and repeat the process. MOLLEO uses an LLM to do the Crossover and Mutation. Interestingly, evolutionary algorithms with LLMs benchmark better than without the LLM, and better than alternatives such as reinforcement learning. Paper in the comments 👇

15

3 Comments

Check out our recent paper solving PDEs on neuromorphic architectures using physics-informed neural networks (PINNs)! In this paper we discuss how a trained PINN is first converted to a spiking neural network (SNN). To complete the migration to neuromorphic hardware, we quantize floating point values in the network. The results section explores the accuracy, throughput, and power consumption tradeoffs using Intel's Loihi 2 platform. While in an early stage, the results show the promise of how these novel hardware systems can be utilized to solve problems in scientific machine learning. Stay tuned! This was a great multi-institution SEA-CROGS collaboration with Brad Theilman (Sandia), Qian Zhang (Brown), Adar Kahana, Brad Aimone, Nat Trask, and George Karniadakis. Thanks all!

169

3 Comments

🌇 The University of Illinois Urbana-Champaign announces their presence in the now-infamous Chicago Manhattan Project alongside PsiQuantum. Plus, neural network quantum states get an efficiency boost through symmetrical subspaces, and fine-tuned quantum sensors inspired by classical signal processing. ⚆ The QED-C ’s recent report presses the necessity of post-quantum cryptography and recommends federal support, the development of quantum expertise, and both QKD and PQC for the most resilient approach to security ⚆ Quantistry and IQM Quantum Computers partner to explore hybrid classical-quantum solutions for R&D in the chemical and materials industry, with a focus on sustainability and renewable energy ⚆ Researchers from North Carolina State University and Massachusetts Institute of Technology have improved quantum sensor sensitivity by fine-tuning quantum systems to detect specific signals, inspired by classical signal processing ⚆ A method for representing quantum states using neural network quantum states uses Markov chain Metropolis sampling to overcome the challenge of the exponential growth of the Hilbert space by confining the representation to a symmetric subspace ⚆ The University of Illinois Urbana-Champaign will also lead the Illinois Quantum and Microelectronics Park, a $500 million state-funded project in Chicago ⚆ IEEE Quantum Week 2024 will take place in Montréal from September 15-20, featuring keynote speakers, tutorials, workshops, and a career fair, with special pricing for early registrants before August 5th ⚆ Don't miss a single qubit -- follow me & subscribe to the free newsletter for your daily quantum computing news & research insights. #quantumcomputing #quantumtechnology #innovation #quantumscience #quantumphysics

4

Last week FAR AI had a post on a survey of AI safety and alignment experts that lead me down an interesting rabbit hole on how unsupervised evaluation helps ameliorate some of the engineering challenges in staying safer with possibly smarter than us AI. 1. David Krueger commented on how future systems will be ensembles (hooray!) and that it has been difficult to manage these teams well but we will figure it out. He is one of the co-authors of "Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback" (https://lnkd.in/eCBWWZr6) by Stephen Casper et al. Their section on "Good Oversight" is the one relevant to the applications of unsupervised evaluation. 2. The above quotes Sam Bowman and the group at Anthropic's "Measuring Progress on Scalable Oversight for Large Language Models" (https://lnkd.in/eDk_GByY) where I found the interesting figure I am sharing here - the challenge for intelligence scalable oversight. Or as I like to put it more colloquially - Can dummies evaluate experts? - a fundamental problem we need to tackle to keep us safer from smarter AIs. 3. The last link is my work on creating data streaming algorithms that allow us to carry out such a seemingly impossible task. If one used logic and algebra as applied to the problem of estimating the correctness of noisy experts, we can ameliorate this challenge. The work is being released Open Source under the Python "ntqr" package (https://lnkd.in/gypDcd7H) The work is based on the mathematics of algebraic geometry but its engineering idea is very simple - just like cars use thermometers to alarm us when its engine is overheating, we can use data-streaming like algorithms to use experts to self-evaluate. These are algorithms that are as dumb (they are purely algebraic and have no tunable paremeters) as thermometers but provide valuable oversight.

4