News

StateBench: Testing how language models track state changes over long documents

StateBench evaluates whether language models can track entity states as they change within long documents. The announcement on Hugging Face describes four difficulty levels for testing that ability. These range from immutable assignments to assignments by reference, giving models distinct kinds of changes to follow.

Longer inputs expose failures in following updates

Between the simplest and hardest levels are reassignment by value and self-referential updates. The S₅ problem belongs to the hardest level. It asks models to track successive swaps of five objects. When trained directly on StateBench tasks, recurrent and hybrid models generalized much farther than transformers. Gemini 3.8 Flash, Claude Opus 5.5, and GPT-6 Sol and Luna still broke down on harder state-tracking tasks as inputs grew longer. (Hugging Face)

thinkidiot take: Successive swaps of just five objects sit at StateBench’s hardest level. I would use that task to test a model before relying on it to follow changing information through a long document. The stronger generalization of directly trained recurrent and hybrid models gives me a concrete reason to compare architectures. For this job, tracking updates deserves more weight than accepting a long input.

Agentic cheminformatics with Google Antigravity

Google DeepMind has presented new Cheminformatics Skills for Google Antigravity. They give computational and experimental chemists access to local tools. In the walkthrough, an agent searched thousands of compounds, performed matched molecular pair analysis, and proposed novel rigidified macrocycles with synthetic routes in one hour.

The model directs the work while chemistry tools calculate

The language model handles workflow orchestration. It also navigates databases and performs scientific reasoning. Deterministic chemistry tools execute the molecular calculations. The skills use RDKit alongside connectors to ChEMBL and PubChem. The walkthrough uses crizotinib, a first-generation tyrosine kinase inhibitor with five major rotatable bonds. (Google DeepMind)

thinkidiot take: The walkthrough combined a search across thousands of compounds with proposed macrocycles and synthetic routes in one hour. I would start by inspecting the matched molecular pair analysis and the calculations behind those proposals. The division of work gives me a specific boundary to examine: the model directs the workflow, while deterministic tools calculate. That division matters more to me than the one-hour result.

F3 Lightning: How Meta Accelerates Ads ML Features to Production at Scale

Meta’s F3 Lightning lets each machine learning feature be compiled, validated, deployed, and rolled back independently. Comparable extraction features reach production approximately 3.6 times faster, with no increase in infrastructure compute cost. The change applies to Facebook Feature Framework, which serves ads-ranking requests that can require more than 20,000 features.

A shared computation graph became a release bottleneck

Meta first presented F3 at AI @ Scale 2020. The framework covers authoring, compiling, and serving features. It reduced execution computational costs by merging feature graphs to share computation. Using that merged graph as the release unit meant one invalid definition could block the entire release. Lightning uses four optimizations: stable computation identifiers, a shared execution plan, a shared memory buffer, and dense index mapping. (Meta Engineering)

thinkidiot take: One invalid feature definition was enough to block an entire F3 release. I would put independent validation and rollback at the center of evaluating Lightning. Those controls address the release coupling that came with the merged graph. The approximately 3.6-fold speed improvement matters, but independent releases are the change I value most.

Bring structured and unstructured data together with Amazon Quick

AWS has described how to build an Amazon Quick chat agent that answers questions across structured and unstructured data. The agent queries enriched datasets and S3-backed knowledge bases through a single conversational interface. It can query either source type or both, then return combined responses with citations while respecting the requesting user’s permissions.

Semantic descriptions connect business questions to SQL

The architecture has three layers: data, workspace, and agent. Enriched datasets contain semantic descriptions used to translate questions into SQL. A Space brings datasets and knowledge bases together. It can also include dashboards, file uploads, and action connectors. Creating datasets requires an Enterprise subscription. S3 buckets selected by an administrator have read-only permissions by default. (AWS Machine Learning)

thinkidiot take: Creating datasets in Amazon Quick requires an Enterprise subscription. I would check that requirement before building a Space, then focus on the semantic descriptions used to generate SQL. Those descriptions are part of how the agent turns a business question into a database query. I would judge this setup first on that translation and its cited responses, rather than on having one chat interface.

The biggest modding site in PC gaming is taking a stance on AI

Nexus Mods is making AI-use disclosures mandatory for newly uploaded mods. Authors must declare whether AI was used in their work. The platform is also launching controls that let users hide AI-generated content on game pages and the homepage. Authors have a 60-day grace period to update existing mods, ending January 2, after which enforcement begins.

Public explanations accompany more prominent AI tags

Mods tagged AI-assisted must include a public explanation of how and where AI was used. AI tags will become more prominent. Search controls are planned next. Nexus Mods will not introduce an outright ban on AI-generated mods. It says it will neither use mods to train AI nor sell or license them to anyone doing so. (The Verge)

thinkidiot take: An AI-assisted tag now requires a public explanation of how and where AI was used. I would read that explanation when deciding whether to use a mod. It gives me more to judge than a tag alone, while the hiding controls offer a broader choice. I prefer this combination of disclosure and user controls to an outright ban.

Trending AI Papers

Ranking source: Hugging Face Papers for 2026-11-04.

Behind Video Generation: Observations and Insights on Multimodal Understanding from Kling

A video can look convincing and still miss what someone asked for. This report draws lessons from successive versions of Kling about how to close that gap. It examines how a video system interprets instructions, learns from descriptions and judges its output. The aim is to make the resulting scenes easier to direct.

• Problem: Better rendering alone does not make a video follow the intended meaning. Instructions must survive several steps between the training data and the finished result. The report finds that stronger language skills alone do not produce a better component for interpreting video instructions.
• New idea: The report uses Kling-MLLM, a model that handles text, images, video and sound, as a shared starting point for several jobs. Training captions serve as detailed agreements about what a scene contains, including explicit links to reference material. Prompt enhancement, the process of rewriting a user's request for the generator, brings instructions closer to those descriptions. Output assessment combines a model that explains specific faults with another that quickly scores several aspects of quality.
• Simple example: Think of briefing a film crew: an attractive location is not enough if the wrong person performs the action. A useful brief links each instruction to the right person or object, and a useful review points out exactly where the scene went wrong.
• Evidence: The report says training grounded in perception produces a more reliable text encoder, the component that represents written instructions, than stronger language ability alone. It also reports that candidates for this component can be screened using text before training the complete system. The abstract gives no numerical results.
• Limitation: The abstract gives no measured gains or comparisons showing how much each proposed change improves the finished videos.
• Why it matters: Following the meaning of a request is essential if people are to control what a video generator produces.
• Paper: Behind Video Generation: Observations and Insights on

MoviEdit: Long-form Multi-Shot Video Editing Made Easy

Changing something in a video gets harder when the camera cuts to another scene or angle. MoviEdit is a system designed to carry the same requested change through a long sequence of shots. It first sketches the edit across the video, then works through the footage to refine it. The work also provides training examples and a dedicated test set.

• Problem: An edit must stay consistent even when a cut changes the viewpoint, surroundings or a subject's appearance. Editors designed for a single continuous shot face a harder task when they must account for much more footage and connect changes across cuts.
• New idea: MoviEdit first builds a rough edit from a version of the video compressed across both space and time. This first pass considers information in both directions through the sequence to keep the overall change consistent. A second pass carries the edit forward, using earlier output to guide later refinements across the full video. Automatic data creation supplies paired editing examples, while MoviEdit-Bench provides a dedicated set of tests for edits spanning multiple shots.
• Simple example: Imagine asking for a person's jacket to change color throughout a video. The change needs to hold when the camera moves from a close view to a wide view, even though the jacket looks different from each angle.
• Evidence: The automated pipelines produced 87K paired editing videos without manual annotation. The authors report substantially better performance than existing open-source models and results on par with leading industry models for edits guided by text or images. The abstract gives no numerical scores for those comparisons.
• Limitation: The abstract does not report processing time or the video lengths tested, leaving the practical limits of its scaling unclear.
• Why it matters: Consistent changes across cuts would make automated editing more useful for videos containing several shots.
• Paper: MoviEdit: Long-form Multi-Shot Video Editing Made Easy

Axion 1.0: Open Agent Foundation Models, Simplified

Axion 1.0 is an openly released AI model built to reason through tasks and use external tools. The work asks how much capability a short training process can deliver. The authors also release the instructions and code behind that process so others can repeat and modify it.

• Problem: The work addresses how to train a capable tool-using model with a simple, reproducible process. Releasing only the trained model leaves out how its tasks were built and checked. Axion 1.0 also tests whether several additional training procedures can be omitted while retaining strong performance.
• New idea: Axion 1.0 uses three training stages, beginning with supervised fine-tuning, which teaches a model through worked examples. The first stage creates reasoning problems and makes them harder based on strategies the starting model already uses successfully. The second teaches reasoning with external tools through tasks whose answers can be checked automatically. The final stage uses reinforcement learning, which trains through rewards, to improve extended reasoning and interaction with tools and environments.
• Simple example: Think of teaching someone to solve problems first on paper, then with reference materials and a calculator. Once they can use those resources, they practice longer assignments and receive feedback on how well they complete them.
• Evidence: Axion 1.0 uses two supervised training stages followed by reinforcement learning. The authors report performance competitive with leading commercial models on several tests of tool-using AI, but give no scores in the abstract. They release data creation and checking procedures, prompts and training code alongside the model.
• Limitation: The abstract names no individual tests or comparison models and provides no scores or training costs, so it does not establish the size or cost of the reported gains.
• Why it matters: Releasing the training process alongside the model gives others the means to reproduce and extend the work.
• Paper: Axion 1.0: Open Agent Foundation Models, Simplified

Trending AI Repositories

Ranking source: GitHub Trending.

max-sixty/worktrunk

Worktrunk is a Rust command-line tool for managing Git worktrees. It targets people who run several AI agent workflows at once and want those worktrees handled from one CLI. It has a docs site at worktrunk.dev.

• What it is: It is a Rust CLI. It sits in the Git worktree space, with a build published on crates.io and docs at worktrunk.dev.
• What it does: Worktrunk is a CLI for Git worktree management, designed for parallel AI agent workflows
• Who it helps: It helps developers who run parallel AI agent workflows. It gives them one command-line tool to manage their Git worktrees.
• Limitation: It needs Git installed, because it works on top of Git worktrees.
• Repository: max-sixty/worktrunk

JustVugg/colibri

Colibrì is a compact inference engine written in C for running large mixture-of-experts models. Its tagline is 'tiny engine, immense model.' It aims to run a frontier MoE model on the hardware you already have.

• What it is: It is an inference engine built in C for large models. It is positioned for running on your own hardware, and it has a hosted site and a latest release.
• What it does: A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
• Who it helps: It helps people who want to run large mixture-of-experts models on the hardware they already have. It runs the busy parts on the GPU, keeps the rest in memory or on disk, and does it with an engine written in C.
• Limitation: It needs a GPU with CUDA or Vulkan.
• Repository: JustVugg/colibri

spotify/portal-ai-plugins

These are TypeScript plugins that bring Spotify Portal into your coding agent. They add focused workflows for the Portal CLI, such as setting up authentication, searching the software catalog, and running diagnostics. If you work in Claude Code, Codex, Cursor, or pi, they plug Portal actions straight in.

• What it is: It is a set of TypeScript plugins for coding agents. It sits between an agent like Claude Code, Codex, Cursor, or pi and the Spotify Portal CLI.
• What it does:
• Who it helps: It helps developers who use a coding agent and want Portal inside it. They can set up the Portal CLI, search the software catalog, build service briefings, and run read-only diagnostics.
• Limitation: It depends on Spotify Portal and the Portal CLI, and it is built for a specific set of agents: Claude Code, Codex, Cursor, or pi.
• Repository: spotify/portal-ai-plugins