This is the latest in a series of posts that will re-cap our fortnightly research catchups. We hold these meetings at the QUT Kelvin Grove campus, and they are open to anyone at QUT interested in Generative AI related research. Contact the lab (genailab@qut.edu.au) if you would like to be added to our mailing list for the meetings.
Our current meeting format (subject to change, and dependent on fortnightly volunteers to fill each section!) includes three sections;
- A discussion of some recent news-worth event pertaining to GenAI
- A gentle explainer for some term or concept in the GenAI research literature, and
- A share of someone’s in-progress or recently published research work on GenAI
FYI, these posts will also be co-produced with the assistance of various GenAI tools!
This week’s GenAI research catch-up the group collectively discussed OpenAI’s recently released -o1 model and Dr Kevin Witzenberger discussed the term socio-technical imaginaries in relation to hype-cycles and Sam Altman’s essay on the intelligence age.
O-1: Chain-of-Thought Reasoning and Monte Carlo Tree Search on Language Model Reasoning Abilities
OpenAI’s recently released -o1 model represents an advancement in large language model capabilities. O-1 is designed to do better chain-of-thought reasoning, allowing the model to more think through the logic and reasoning behind the questions it aims to answer, rather than just pattern matching based on training data. This should allow ChatGPT to generalize better and handle more complex reasoning tasks (when the -o1 backend is enabled).
The group also discussed the potential reasoning improvements that might come from combining LLMs with Monte Carlo Tree Search (MCTS), the approach used by DeepMind’s AlphaGo to beat the world Go champion. With MCTS, an LLM would be able to explore different reasoning paths, evaluate which ones are most promising, and allocate more compute time to those paths – essentially “thinking through” a problem for a given amount of time before providing an answer. This architecture allows an extra dimension of scaling via inference-time compute, in addition to model size and training compute.
Examining Imaginaries of AI with Kevin Witzenberger
Dr Kevin Witzenberger applied the concept of socio-technical imaginaries from Science and Technology Studies to analyse OpenAI CEO Sam Altman’s recent blog post on ‘the intelligence age‘. Socio-technical imaginaries, a term introduced by Sheila Jasanoff, refers to collectively held beliefs about desirable futures attainable through science and technology. As there are many competing visions of what futures are possible and preferable, these imaginaries are constantly being negotiated in the present. Witzenberger argues that Altman’s post provides a window into the imaginary being put forward by Silicon Valley tech leaders, making it an important artefact to examine and unpack.
Witzenberger then delved into the debate around AI hype cycles, challenging the notion that the influence of imaginaries fade with the end of hype. Instead, he contends, the visions put forth during a hype often get absorbed and concretised in various domains. While some researchers are quick to critique Altman’s ideas as impossible or as part of the hype around AI, Witzenberger advocates that studying these imaginaries is still valuable.He notes that even as investments dry up when inflated expecations cannot be met, there is often a subsequent period of productivity where ideas funded through fantastical claims like the ones from Altman get absorbed in other domains, leading to a period of productivity. The data analytics tools for example originally hyped by social media companies have now become ubiquitously integrated in sectors like education, healthcare, and border control. Viewed through this lens, Altman’s essay merits close analysis, regardless of whether one agrees with its predictions. Witzenberger posits that imaginaries wield significant power to shape trajectories around emerging technologies in ways we may not always recognise in the moment.