Now you can tell the Hue app how you want your smart lights to work

Philips Hue’s AI assistant can now build automations based on natural language, letting you describe what you want your lights to do. The feature, called Custom AI Behaviors, goes beyond simply changing colors or creating a scene and lets users orchestrate complex automations that weren’t previously achievable in the Hue app.

Announced at a press event at the IFA tech show this week, Custom AI Behaviors joins a new integration with Sonos speakers. Users can control Sonos speakers with select Hue switches, including the Hue Tap Dial Switch, to trigger Favorites, play or pause audio, adjust the volume, and move to the next or previous song in a playlist. It will be available via a software update in November.

Many advanced Hue users probably already have sophisticated automations to control their lights, but they didn’t set them up in the Hue app, which is fairly limited beyond basic automations based around time of day. Hue’s head of technology, George Yianni, said this was mostly due to the UI complexity required to present these more advanced automations, not limitations in the technology.

With Custom AI Behaviors, you’ll be able to tell the Hue AI Assistant in the app what you want. For example, when I’m asleep, turn on any lights triggered at nightlight setting; when I’m awake, use normal time-based color temperature. The system requires an input to know whether you’re awake or asleep (such as pressing a button to trigger a sleep or wake routine), but it can figure out the rest from there.

You can also use Custom AI Behaviors to program Hue’s switches, which can be a fiddly process. One scenario demoed included asking the system to program button 4 on the Hue dimmer switch to disable the dining room motion sensor. This sounds like a great way to stop your lights turning on every 5 minutes while you’re watching TV because your dog is pacing, speaking from experience.

Custom AI Behaviors are currently in beta and will roll out next month. The automation engine that runs them will be available to third-party developers.

Yianni said the capabilities are vast, as every resource the bridge knows about is available as both input and output, including light color, every button press, every sensor event, and time. “It would have been hard to build a UI for this, but now LLMs provide a solution to the problem — they are able to expose richness of capability in an easy way,” he said. “This is not AI for the sake of AI; it’s a practical application of the technology.”

Custom AI Behaviors are exclusive to Hue’s latest bridge, the Bridge Pro, where the engine and automation run locally, while the AI assistant runs in the cloud. “We designed a domain-specific language, a simplified programming language specifically to interact with the AI assistant in the cloud,” Yianni says. The feature was built on the open-weight GLM family of models. “Closed-source models sometimes just change in the background, and can quite significantly change the behavior of your use case,” says Yianni. “With the open-source ones you control what actually runs.”

Finally, the company teased a much-wanted feature launching early next year: Back-up & Restore will allow users to back up their bridge settings to the Hue cloud.

Photography by Jennifer Pattison Tuohy / The Verge