Google’s Pet Memory forgot who my cats are

One of the best things my smart home does is help me care for my pets, and security cameras are particularly useful for keeping track of my many critters. But the barrage of notifications they send often means I miss important ones. So, when Google announced its new Pet Memory feature for Gemini for Home, I thought this could be the perfect solution.

Pet Memory promises to teach your Google Home smart home who your pets are, allowing connected Nest cameras to go beyond simply telling you they saw an animal to which animal they saw. It can then use that knowledge to adapt your smart home to your pet.

My plan was to use it to keep a digital eye on my three cats while working in my home office, distinguish them from one another, and automate their feeding based on who was at the feeder. But after two weeks of testing, Google’s AI still can’t tell my three cats apart.

My house is on a suburban, three-quarter-acre lot in a quiet neighborhood, and I take full advantage of all this space to have as many animals as possible (my husband has already said no to a goat and horse). But I’ve managed to wrangle a flock of free-ranging chickens, a large, counter-surfing Wirehaired Pointing Griffon called Gus, two indoor/outdoor cats, and — most exciting of all — a new kitten.

As The Verge’s smart home reviewer, I test a lot of connected tech and have found my best use case for security cameras is monitoring my pets around my house and yard. They help keep the chickens safer from predators and my lunch safer from Gus, and act as a type of digital pet door for the cats. When I get a notification that there’s an animal at the door via a video doorbell and/or outdoor camera, it’s likely a cat wants to come in.

While I only have one dog and don’t really need to distinguish the chickens by name, I thought Pet Memory could do three things to help me manage my three cats. First, identify which cat wanted in, and help prevent false alerts — such as when it’s my neighbor’s cat sitting on my porch, or when it’s a chicken peering in at the back door. Second, tell me whether all my pets were safely inside before dark. And finally, provide personalized automated feeding for each feline.

Recognizing individual pets is the latest step in a rapid evolution of smart security cameras. While some dedicated pet cams promise facial recognition, most standard security cameras could only alert you to motion until recently. Advances in machine learning and now generative AI are bringing more descriptive alerts. Instead of just announcing “motion alert,” cameras can increasingly tell you exactly what they saw, without you having to open the app and wait for a clip to load.

Nest has long been ahead of the game here, especially when it comes to critters. Its Nest Cam IQ, launched in 2017, could distinguish people from pets, and Google later brought animal detection to Nest cameras. More recently, Google added text descriptions that use AI to describe what the camera sees.

Amazon’s Ring, Wyze, and Apple Home (via its HomeKit Secure Video) have also recently added AI-powered descriptions, and these often include the type of animal detected (not always correctly). But Google Home is the first to personalize it.

Pet Memory works by comparing the description you provide Gemini for Home with the text description Gemini generates from the camera. “If these are the same, we will replace the pets with the name in the caption,” says Rudra Bhatt, product manager, Google Home.

However, there are limitations, and, of course, a cost. Pet Memory requires the top-tier Google Home Advanced Plan at $20 a month, and while it’s part of Google Home, it only worked on Nest cameras in my testing. (Google syas its compatible with any Gemini-enable camera). It also only works on indoor cameras, so it wasn’t any use for my idea of an upgraded digital pet door.

I decided to set up a few indoor Nest Cams in strategic spots, hoping I could use Pet Memory to easily check which cats were inside. I thought it could help make sure everyone is safely home before dark. “Hey Google, when did you last see Smokey in the house?”

But despite telling Google Home what my three cats look like, it insisted they were all Smokey, the first cat I told it about. Asking the Google Nest Hub when Smokey was last spotted pulled up an adorable video of the new kitten playing in my daughter’s room.

Unlike Familiar Faces, Google Home’s facial recognition feature, Pet Memory doesn’t build an individual visual profile for each pet. There’s no training process, no opportunity to upload photos of my cats, and no obvious way to correct mistakes. You enter a name and breed, and it tries to figure out the rest.

I tried giving it more information — Boone is a tuxedo cat with white paws, Osa the kitten is a tabby cat, and Smokey is a large gray-and-white cat — but it said it couldn’t process the extra detail and kept labeling every cat as Smokey.

I asked Google why the system didn’t seem to be working as intended, and Bhatt said that while Gemini does a very good job with single pets and is okay with different species (cat versus dog), it can struggle to distinguish between multiple pets of the same type (which is also why it’s limited to indoor cameras). The suggestion was to provide more detail to Gemini to help it differentiate, but it repeatedly rejected my attempts to do this.

Still, I persisted in trying to get some use out of the feature, attempting to use Pet Memory to make my Aqara Pet Feeder smarter. I had previously set it up to trigger when Aqara’s G3 camera saw an animal, as it can’t identify species, let alone individuals.

But with a Nest Cam, I wanted to see if I could set it to dispense the right amount of food when it saw a specific cat — helping keep each from overeating without needing RF tags or other proprietary solutions smart pet feeders offer (my cats don’t do collars).

This was also an opportunity to try Google’s new ability to use camera detection events as triggers for automations and its Help Me Create feature, which can build an automation with just a description.

I typed what I wanted it to do into the Ask Home chatbot in the app. “Run my Aqara ‘Feed Smokey’ scene whenever the Nest Cam in the laundry room sees Smokey,” and it set it up for me. I then created a separate one for Boone. I ran into several issues.

The first — and biggest — was that it still couldn’t tell Boone from Smokey, so when either cat went to the feeder, it went off. The second problem is that the Google Home app doesn’t let you limit how many times an automation runs (something I could do with the Aqara automation). While the automation worked, it triggered every time either cat went to the bowl. By the end of one day, the food bowl was overflowing.

I also had high hopes for using Google’s Home Brief to get an overview of the critters’ day. This gives a rundown of activity from connected devices in your home at the end of the day. I told it I wanted more details about pet activity in my Home Brief, and now it mostly tells me “there were many pet interactions,” which isn’t helpful.

At one point, it told me Smokey was eating in the garage, which immediately raised a flag. Cats eating in the garage is never a good thing; there are too many things they shouldn’t be getting into. But when it showed me the relevant clip, it was Boone eating in the laundry room, as he should be.

I checked the garage camera, able to scroll back through the day’s footage, but neither cat was ever in there. Frankly, misidentification is worse than doing nothing at all. Getting the cat’s name wrong can be funny, but if you’re relying on it to tell you where your cats are, errors become consequential.

Recognizing individual pets turns out to be far harder than recognizing that there’s a pet at all. It’s a glimpse of where AI-powered cameras are headed, but not a feature I’d rely on today. AI has promised to cut down on the arduous work of searching through camera footage, but until it can reliably identify what happened, continuous recording is more useful than unreliable AI summaries.

Continuous recording is actually something Nest Cams do very well as part of that same $20 a month subscription. Whether it’s to keep an eye on your indoor kitten, a tropical aquarium, or a flock of chickens in your yard, scrolling back through a day’s worth of footage is invaluable when you’re trying to figure out what your cat ate that made them sick or how a chicken went missing (hawks don’t always trigger motion alerts). When AI can answer those questions for me, then it might be worth it; in the meantime, Boone is enjoying all the extra kibble.

Photos and video by Jennifer Pattison Tuohy / The Verge

Update, Tuesday August 18: Added comment from a Google Home product manager on how the Pet Memory technology works and its limitations