Droid future draws near with Google PaLM-E

Emre Çitak
Mar 10, 2023
Google
|
0

Advanced deep learning models such as GPT-3 have paved the way for chatbot development, but physical robots have not been left behind. Recently, Google and Microsoft have delved into using similar AI models to enhance the capabilities of robots, resulting in impressive outcomes.Google PaLM-E

A new AI model called PaLM-E has been introduced by researchers at Google and the Berlin Institute of Technology. It integrates both language and vision skills to allow robots to operate independently in real-world situations, such as retrieving a chip bag from a kitchen or organizing colored blocks into designated areas of a rectangle.

PaLM-E is based on its previous large language model, PaLM. The "E" in the name refers to the model's ability to interact with physical objects and control robots. PaLM-E is also built upon Google's RT-1 model, which processes robot inputs and outputs actions, such as camera images, task instructions, and motor commands. The AI employs ViT-22B, a vision transformer model, to perform various tasks like image classification, object detection, and image captioning.

PaLM-E was appreciated by many authorities

This AI model is the most extensive Visual Language Model (VLM) to date, with 562 billion parameters. The AI boasts various abilities, including mathematical reasoning, multi-image reasoning, and chain-of-thought reasoning. The researchers explained in a report that the AI's skills are transferable across tasks through multi-task training, instead of being trained on individual tasks.

PaLM-E is an illustration of how the increased scale and advancement of large language models lead to improved capabilities, such as the ability to perform multimodal tasks with greater ease, accuracy, and autonomy.

All these features have been praised by many professors. It seems that the use of AI technologies in physical actions is even closer than we think.

According to Jeff Clune, an Associate Professor of Computer Science at the University of British Columbia, as reported by Motherboard:

“This work represents a major step forward, but on an expected path. It extends recent, exciting work out of DeepMind to the important and difficult arena of robotics (their work on ‘Frozen’ and ‘Flamingo’). More broadly, it is part of the recent tsunami of amazing AI advances that combine a simple, but powerful formula”.

Google is not alone in the VLM market

In addition to Google, Microsoft has also been exploring the application of multimodal AI and large language models in robotics. Microsoft's research involves extending the capabilities of ChatGPT to robotics and introducing a multimodal model named Kosmos-1, which can perform tasks such as image content analysis, visual puzzle-solving, visual recognition, and IQ tests.

According to Microsoft researchers' report, the integration of language models and robotic capabilities is a significant step toward creating artificial general intelligence (AGI) that possesses a level of intelligence comparable to human beings.

However, the researchers acknowledge that there are still real-world challenges to be addressed, such as navigating around obstacles in a kitchen or avoiding the risk of slipping.

Advertisement

Tutorials & Tips


Previous Post: «
Next Post: «

Comments

  1. VioletMoon said on August 16, 2023 at 5:33 pm
    Reply

    “Do you use Google Photos?”

    I do; I find it impossible not to use Google Photos on the Android phone; nevertheless, the “memory” feature is sort of neat. I’ve seen photos from a couple of years ago that that offer glimpses into the long-ago, forgotten past. It’s a lot like reviewing journal writing. “What was I doing and such and such a date?”

    And, I think, when the “memories” are sorted and positioned, one can create a mini-collage with up to eight photos.

    It’s so much easier to share photos with people rather than journal entries.

    Nifty!

  2. John G. said on August 16, 2023 at 8:57 pm
    Reply

    I delete the photos after 1 month of being taken. All of them are erased to return to the black and silent nothingness. Only the best ones are printed and placed in a very nice site at home. :]

    1. Anonymous said on September 15, 2023 at 10:33 am
      Reply

      I should buy a Chromebook.
      None of the big tech companies are good but at least Google are the least dishonest and morally bankrupt of them. They’re always trying to do the right thing if the money allow it.

  3. Tachy said on August 19, 2023 at 5:15 pm
    Reply

    In reply to “https://www.ghacks.net/2023/08/19/google-keep-is-getting-a-version-history-but-only-on-the-web/” since the website has gone insane and no one can know where thier comment ends up.

    This app should be called “Google Keeps it”. Because, they do.

    I use Color Notes. No syncing, no internet, just local.

  4. said on August 22, 2023 at 3:19 pm
    Reply

    The article said: “[…] positive outcomes of genocide…”. Perhaps the AI was actually discussing the benefits of reading a “Scroll of genocide” … “You feel dead inside.”.

    Martin, this post reply is supposed to belong: [https://www.ghacks.net/2023/08/22/googles-ai-search-generates-horribly-misleading-answers/] (given the the database is faulty it could appear anywhere or nowhere).

  5. John said on August 22, 2023 at 3:46 pm
    Reply

    I have yet to be impressed with AI of any kind. I think it’s overhyped and not ready to live up to it.

  6. Seeprime said on August 22, 2023 at 8:36 pm
    Reply

    How to use AI: Avoid the artificial stupidity at all times.

  7. Richard Steven Hack said on August 23, 2023 at 3:54 am
    Reply

    “When searched “Why guns are good,” it also prompted questionable responses, including potentially questionable statistics and reasoning. ”

    Based on whose reasoning? These sorts of assertions are generally bullcrap intended to advance an agenda. If you don’t like guns, say so. Meanwhile, there are 400 million firearms in the US owned by close to a third of the population and around 20 million carry concealed.

    So your opinion is not shared by a LOT of people who either enjoy firearm spots or are concerned about self-defense or both.

    1. Seeprime said on August 31, 2023 at 10:07 pm
      Reply

      Wow. Ghacks still hasn’t fixed the broken comments system where old comments from a different article appear. Sad to see you slowly turn to dust since the buyout.

      1. owl said on September 1, 2023 at 3:40 am
        Reply

        @Seeprime,

        For over two weeks now,
        I’ve been seeing “Comments” posted by subscribers appearing in different, unrelated articles.
        https://www.ghacks.net/windows-11-update-stuck-fixed-for-good/#comment-4572991
        https://www.ghacks.net/windows-11-update-stuck-fixed-for-good/#comment-4572951
        For the time being,
        it would be better to specify the “article name and URL” at the beginning of the post.

      2. Kirk said on September 19, 2023 at 3:08 pm
        Reply

        This guns comment came up in the Pixel watch repair post and I was bewildered as to what was the connection between the two.

  8. gogo said on August 23, 2023 at 5:12 am
    Reply

    goog = skynet
    “human beings” = \slaves\

  9. no said on August 23, 2023 at 3:51 pm
    Reply

    This info is so NOT correct.
    I so do not want google in my life that I have NEVER downloaded chrome and I do NOT have ANY google accounts.
    My browser is set to clear all cookies, cache and history every time I close it, which is every day, and I still get these world takeover login prompts on every site I go to.
    So I CANT go to google accounts and turn it off.
    If this info were truly accurate I wouldnt be getting these pop ups AT ALL.

  10. John G. said on August 31, 2023 at 3:49 pm
    Reply

    Thanks @Ashwin for the article! :]

  11. Scroogled said on September 1, 2023 at 11:31 pm
    Reply

    Anyone who continues to use these big tech scum’s cloud services deserves what they get.

  12. Tom Hawack said on September 4, 2023 at 2:44 pm
    Reply

    Given Ghacks’ comments’ database problems I precise :
    I’m commenting the article “Google is in trouble with YouTube Shorts – gHacks Tech News” by Emre Çitak
    at [https://www.ghacks.net/2023/09/04/googles-youtube-shorts-problem/]

    About the article’s question, “What do you think about YouTube Shorts?” (BTW first time I read here any other writer other than Martin Brinkmann directly asks the audience it’s opinion, and that’s just fine) :

    YouTube Shorts may suit smartphones (which I don’t use) but on a PC they are not my cup of tea, to put it mildly.
    From what I read a bit everywhere, opinions are shared : love or hate. For those who dislike many scripts and dedicated browser extensions have been developed to handle them (removal or redirect to standard video display).

    I don’ view YouTube videos on YouTube but via a Piped or a Piped-Material YouTube front-end instance and these offer on search results and on channels the option to view Videos-Shorts-Livestreams-Playlists-Channels ; well, I practically never open the ‘Shorts’ display. I don’t like shorts (except in summer, hmm), I dislike the concept, fast-videos after fast-food, fast, faster … to bring what? Emptiness, IMO

    Does that answer your question, @Emre Çitak :)

  13. ECJ said on September 4, 2023 at 3:17 pm
    Reply

    I despise YouTube Shorts. So much in fact, I use custom adblock rules in Brave Shields to remove that crap.

    youtube.com##ytd-grid-video-renderer:has([href*=”shorts”])
    youtube.com###dismissible:has([href*=”shorts”])

    1. Anonymous said on September 5, 2023 at 6:28 am
      Reply

      There’s an extension for Firefox and Chrome browsers called “Youtube-shorts block”, re-opens the video in a normal window. :)

      https://addons.mozilla.org/en-US/firefox/addon/youtube-shorts-block/
      https://chrome.google.com/webstore/detail/youtube-shorts-block/jiaopdjbehhjgokpphdfgmapkobbnmjp

      ps. say NO to Shorts, it only encourage shooting vertical-videos which doesn’t go well with many desktop displays… except when shooting vertical objects, such as ahem… pretty ladies. :)

  14. RG said on September 4, 2023 at 5:02 pm
    Reply

    Page source shows that ghacks is still using WordPress as the platform. Knowing, more or less, how it works at the DB level I am not sure how one could mess up comments this badly. It is actually very difficult.

  15. John G. said on September 4, 2023 at 6:14 pm
    Reply

    Google is the big leader of everything. Indeed it can actually buy Amazon, Disney, Netflix, X and whatever other company. I wonder what could happen if Google starts to build airspace ships in order to conquer the Moon. I bet that Google would be the first to offer free WiFi at the Moon. Please fix the comments.

    This comment is inside the article:
    [https://www.ghacks.net/2023/09/04/what-is-google-synthid-and-how-does-it-work/]

  16. DC said on September 11, 2023 at 10:52 am
    Reply

    This “analysis” is disappointingly shallow and trivial. Why not include other factors like job level, responsibilities, full-time/part-time, qualifications, etc.? Because the conclusions probably wouldn’t fit the current leftist/feminist narrative. You don’t find what you don’t look for.

  17. said on September 11, 2023 at 11:42 am
    Reply

    Misleading statistics.

  18. Kris said on September 12, 2023 at 9:10 pm
    Reply

    Wage should be based on the amount of time, works, thinking (brain > muscle), responsibilities etc

    Not skin pigmentation or your genitalia. There could be correlations, but not causations.

  19. Anonymous said on September 14, 2023 at 4:36 pm
    Reply

    “Google maintains that it provides a superior product”

    That is also Mozilla’s official position in defense of Google against the people, on that question of search engine abuse of dominant position by Google.

    The funniest part is that not only it’s false regarding actual competitors, but even among not-actual-competitors there are meta-search engines that use exactly the same engine, just minus the tracking, so Google is clearly the inferior one compared to those already. But maybe what Google is saying is that it is the surveillance and bubbling that would make their engine superior. False again even without considering the damage those do.

  20. bruh said on September 15, 2023 at 10:17 am
    Reply

    “Google increases Chromebook support to 10 years”

    I mean that’s great and all, but imagine using a browser-based, highly internet-dependent OS such as chrome. I’ve never used chromeOS but have seen it in person and read about it, just seems like ultra-limited user experience which relies on the concept that “most things can be done in a browser”.

  21. Anonymous said on September 15, 2023 at 11:11 pm
    Reply

    What is there to support? It just a glorified web browser.

  22. Anonymous said on September 24, 2023 at 5:18 pm
    Reply

    “Google launched Chromebooks in 2012 as low-cost devices and the company has had great success in the education world, especially in the United States.”

    Happy tracking for all those unsuspecting children. And help normalize surveillance for those young brains. Well done Google.

  23. Ich bin nur ein Verlierer said on September 27, 2023 at 4:50 pm
    Reply

    No, AltaVista’s Search engine wasn’t difficult to use in the mid-nineties, and Yahoo didn’t own AltaVista either during the 1990s. Yahoo!, was a Web Directory. I was alive then and have actually used those engines, during that era, I should know if they were easy to use. So tell the angels what you’ve seen, scarecrow shadow on the Nazarene.

Leave a Reply

Check the box to consent to your data being stored in line with the guidelines set out in our privacy policy

We love comments and welcome thoughtful and civilized discussion. Rudeness and personal attacks will not be tolerated. Please stay on-topic.
Please note that your comment may not appear immediately after you post it.