Google's upcoming AI project will take over tasks in Chrome for you

Ashwin
Oct 28, 2024
Google
|
9

Google is reportedly working on a new AI that will perform day-to-day tasks in Chrome. The technology is called Project Jarvis, and it could be unveiled before the end of the year.

Did you know? Google Chrome may send AI search history to Google.

Google's Project Jarvis AI can perform various tasks for you

The news comes via The Information (paywalled). An article by 9to5Google explains more about the upcoming AI, which is powered by Gemini. Project Jarvis' name seems to have been inspired by Marvel's Iron Man's AI-assistant J.A.R.V.I.S (Just A Rather Very Intelligent System). The AI is designed to assist users perform various tasks such as conducting research, purchasing products or booking flight tickets.

Google first unveiled its plans for AI Agents in April 2024, at the Google Cloud Next 24 event. The Mountain View demoed a couple of prototype AI agents at the Google I/O 2024 Developer Conference in May, these were designed. You can watch the video here (it's around the 18-minute mark). Let's discuss this briefly.

Sundar Pichai, the CEO of Google, described Agents as intelligent systems that can reason, plan, and use memory to think. In fact, the Agents can plan multiple steps ahead to assist the user. The AI is designed to work across software systems and systems, it likely refers to operating systems and devices.

One of those examples highlighted a shopping experience, where a user could take a photo of a pair of shoes they had ordered, and ask Google Gemini to assist them to return the shoes. Gemini scanned through mails in Gmail, found the receipt, filled out a return form, and added a reminder in Google Calendar to notify the user about the order's pick-up.

The other example showcased how Google could assist someone who had just moved to a new city, and offered suggestions to explore the city, listed suggestions for shops and services. The AI could even help them update their address across different websites, and prompted the user with questions as and when required.

It is worth noting that the scenarios were not running via an app/UI, they were likely demoed using Gemini. This is what Google plans to unveil in the coming months, integrating the AI features in Chrome. While it may sound like the AI is taking over a computer, it is pretty clear that the user has to trigger the action manually, and the app does the rest.

That said, there are some concerns about Project Jarvis.

How will Project Jarvis work?

Jarvis will capture screenshots of the user's screen frequently, and analyze the data in the images. It will interpret actions such as clicking on a button or typing something in a text field. This could be potentially be worse than Microsoft Recall. The Redmond company has confirmed that Recall stores the database on the user's computer, the data is encrypted, and that user data is not shared with Microsoft or third-parties. Google is an advertising company, so its motivations will come under scrutiny from regulatory agencies. It remains to be seen how it will protect the user's privacy.

Project Jarvis could be powered by Gemini 2.0, and could be released in a preview form in December 2024. On a sidenote, Anthropic announced Claude 3.5 Sonnet, which has a new feature called "computer use", that is capable of moving the mouse cursor, click on buttons, and typing text. AI and Automation, what could possibly go wrong?

While it sounds impressive on paper, it is too early to say how Project Jarvis will actually perform the tasks.

Would you use such AI tools, or rather protect your privacy?

Summary
Article Name
Google's upcoming AI project will take over tasks in Chrome for you
Description
Google's Project Jarvis AI could automate tasks
Author
Publisher
Ghacks Technology News
Logo
Advertisement

Tutorials & Tips


Previous Post: «
Next Post: «

Comments

  1. 45 RPM said on October 29, 2024 at 9:39 am
    Reply

    Google’s ability to constantly develop things that makes me trust them even less than I already did is impressive.

  2. SLiVER said on October 28, 2024 at 8:35 pm
    Reply

    AI + Brain Chips For Every Human + Starlink = complete surveillance and control grid capable of locating anyone anywhere.

  3. Anonymous said on October 28, 2024 at 8:09 pm
    Reply

    Why do it yourself when Google can stuff it up for you!

  4. Peter Parker Kent said on October 28, 2024 at 5:06 pm
    Reply

    Thanks, Ashwin, for the heads-up here, especially the first two paragraphs under “How will Project Jarvis work?” Hair-raising, to say the least.

  5. Anonymous said on October 28, 2024 at 3:56 pm
    Reply

    @ashwin “Would you use such AI tools, or rather protect your privacy?”

    You ask that as if there will be a real meaningful choice in the matter.
    How long before the baseline “tech” for that is built right into chrome/android? Sure you can have a setting for it somewhere, but thats like the old led-indicator on webcams to indicate if they are on – but that only works on the ‘honour system’.
    The point I am making here is that whether you actively choose to engage with such “agents”, it is more or less a given that they will be collecting on you anyway, all the time, like MS-Recall.
    As for example microsoft has amply demonstrated with all their “telemetry” abuse that you can not get out of. Or every SoMe and webmail provider mining you to dust.

    Apple is sort of already doing agents/ai-assists and is working on extending it.
    Goggle will do it as well.
    Clearly MS is also doing their thing and will sooner or later force it on people. (eventually it will be pay to opt out, while not really being opted out, just like paying for addfree streaming that still gets tracked and soon gets polluted with adds anyway).

    There will be nowhere to go. All there will be left to do is trying to minimize what you do in browser/apps/”smartdevices”/windowspc/idevice, including your newish car. Which is more or less reduced to a connected Tablet with a car attached to it, so you can still pretend it is primarily a car, not primarily a “smart” device. That is to say, to those in power the computers inside it are the important things (watching, listening, reporting, exerting control, limiting freedoms), they have no such sentiments about the actual car, that is just another thing that is now enslaved to them.

  6. Brad said on October 28, 2024 at 3:51 pm
    Reply

    Please make it stop. Late-stage capitalism, where America is now just wall-to-wall conmen.

  7. Tom Hawack said on October 28, 2024 at 2:54 pm
    Reply

    “The AI [Project Jarvis] is designed to assist users perform various tasks such as conducting research, purchasing products or booking flight tickets.”.

    I linger to understand what I’d need an AI to assist (worse : to conduct) research, purchasing, booking.
    No, I definitely would not use such AI tools, not only to protect my privacy but as well to protect my independence, my brain-flow. Is the “Do you mind if I decide for you?” AI’s next step after the already omnipotent “Do you mind if I assist you ?”. Yes I do mind. If I need you I know where to find you and your AI colleagues. Meanwhile : out of my way, “Raus!” (“get out!) like used to say our German language teacher when one of us would be a naughty pupil!. Darn. Damned AI, really starting to get on my nerves.

    Yes, already, Google Chrome may send AI search history to Google. Google peeks everywhere and pokes its servers with its intrusions. Next step with Google is always a next step into our lives. Count me out : it’s not only that I don’yt use any of Google’s services, it is that I avoid the company to the maximum extent of my knowledge.

  8. You now, brother! said on October 28, 2024 at 2:31 pm
    Reply

    They are good catching our info now, so with the AI they will have everything from us.

  9. Tony said on October 28, 2024 at 2:30 pm
    Reply

    “Jarvis will capture screenshots of the user’s screen frequently, and analyze the data in the images.”

    LOL

    NO

Leave a Reply

Check the box to consent to your data being stored in line with the guidelines set out in our privacy policy

We love comments and welcome thoughtful and civilized discussion. Rudeness and personal attacks will not be tolerated. Please stay on-topic.
Please note that your comment may not appear immediately after you post it.