MIT researchers channel AI to turn hand gestures into robot training data

Advertisement

Advertise with us

CAMBRIDGE, Mass. (AP) — Humanoid robots struggling with tasks like grasping a cup have a new teacher — a person wearing an ultrasound wristband that captures the movement of muscles, tendons and ligaments beneath the skin.

Read this article for free:

or

Already have an account? Log in here »

To continue reading, please subscribe:

Subscribe and receive a limited-edition Free Press branded hat or tote.

Digital Subscription

One year of digital access for only $205*

  • Enjoy unlimited reading on winnipegfreepress.com
  • Read the E-Edition, our digital replica newspaper
  • Access News Break, our award-winning app
  • Play interactive puzzles

*First annual payment billed as $205.00 + GST for one year. This annual subscription will automatically renew at $233.00 + GST every 52 weeks (10% off the regular annual price of $259.35). Offer available to new and qualified returning subscribers only. Cancel any time.

To continue reading, please subscribe:

Add Free Press access to your Brandon Sun subscription for only an additional

$1 for the first 4 weeks*

  • Enjoy unlimited reading on winnipegfreepress.com
  • Read the E-Edition, our digital replica newspaper
  • Access News Break, our award-winning app
  • Play interactive puzzles
Start now

*Your next Brandon Sun subscription payment will increase by $1.00 and you will be charged $17.95 plus GST for four weeks. After four weeks, your payment will increase to $24.95 plus GST every four weeks.

CAMBRIDGE, Mass. (AP) — Humanoid robots struggling with tasks like grasping a cup have a new teacher — a person wearing an ultrasound wristband that captures the movement of muscles, tendons and ligaments beneath the skin.

Researchers at the Massachusetts Institute of Technology developed the tool to collect data of human hand motion that could eventually help robots achieve the dexterity that has been difficult for machines to master.

“Imagine people doing housework,” said Xuanhe Zhao, an MIT professor of mechanical engineering. “We can use the data obtained by our system to train a robot to do exactly (that) housework with this dexterous hand motion.”

Xuanhe Zhao, professor of mechanical engineering at the Massachusetts Institute of Technology, poses for a portrait with an ultrasound wristband that can help a robotic hand mimic full hand motions in Cambridge, Mass, on Tuesday, April 14, 2026. (AP Photo/Rodrique Ngowi)
Xuanhe Zhao, professor of mechanical engineering at the Massachusetts Institute of Technology, poses for a portrait with an ultrasound wristband that can help a robotic hand mimic full hand motions in Cambridge, Mass, on Tuesday, April 14, 2026. (AP Photo/Rodrique Ngowi)

As much of the tech world is still captivated with artificial intelligence assistants that are taking on computer-based tasks, Zhao is among the scientists trying to imbue AI with more sensory data from the physical world.

Beyond housework, the technology could help with other tasks that require flexing fingers and hands, such as surgery.

The wristband uses high-frequency sound waves to “see” through its wearer’s skin. It relays images of the muscle and tendon movements to a computer that uses AI to enable a nearby robotic hand to mimic the gestures.

An AI algorithm is trained to decode images generated by the device into what engineers call degrees of freedom – specific ways a joint can bend or rotate. The human hand has 22 of them.

In earlier systems, tracking even a fraction of those movements was a significant challenge.

In laboratory demonstrations with eight volunteers, developers showed the wristband could precisely mirror hand gestures – including all 26 letters in American Sign Language – within 120 milliseconds.

The wristband can operate wirelessly, meaning the controlling person and the receiving robot need not be in the same room.

Beyond remote control, the team sees a path toward using the wristband to build huge datasets of human motion that could eventually enable humanoids to learn dexterous tasks without human guidance.

___

AP Technology Writer Matt O’Brien contributed to this report.

Report Error Submit a Tip

More Stories

What’s open, what’s closed, what’s up in Winnipeg on Terry Fox Day

Free Press staff 4 minute read Preview

What’s open, what’s closed, what’s up in Winnipeg on Terry Fox Day

Free Press staff 4 minute read Friday, Jul. 31, 2026

The first Monday in August marks Terry Fox Day in Manitoba, honouring the legendary Canadian who was born in Winnipeg in 1958. 

Read
Friday, Jul. 31, 2026

Use summer’s hard truth to mend your heart

Maureen Scurfield 4 minute read Yesterday at 2:15 AM CDT

Dear Miss Lonelyhearts: I refused to go stay at my girlfriend’s cabin in July, because her alcoholic ex-husband still owns the lot beside her. He’s building a big fancy cabin this summer and she’s more than OK with that, even though they’re divorced.

I seriously criticized him — calling him a drunk and a loser — and she actually dumped me before she went off to the lake for all of July!

When she finally came back home I went to see her. She told me I “just wasn’t enough of a lake person” for her.

Then the fight began! I said, “So are you back with him permanently?” and she said, “What do you think? You let me go!”

Hydro CEO needs to reckon with winds of change

Dan Lett 5 minute read Preview

Hydro CEO needs to reckon with winds of change

Dan Lett 5 minute read Yesterday at 3:28 PM CDT

Methinks Hydro CEO Allan Danroth protests too much.

At a recent hearing in front of the Public Utilities Board, Danroth suggested Hydro’s plan to build a $3-billion natural-gas-powered generating plant near Brandon was being subjected to unfair criticism. The gas turbines are meant to bolster Hydro’s grid in the event that drought lowers output from hydro-generating stations.

After hearing submissions from interveners concerned about the gas-turbine plan, Danroth lashed out at the “constant, uninformed criticism” that has not only shaken trust in Hydro but also triggered threatening responses from the public.

Danroth said Hydro is monitoring “eight credible threats” against employees, five of which targeted the CEO himself. Although Danroth cited some specific incidents — employees being threatened with guns, and one where a homeowner set his dogs on a meter reader — he refused to elaborate on the circumstances.

Read
Yesterday at 3:28 PM CDT

Steinbach ER nurses ask for triage help as wait-averse Winnipeg patients strain hospital capacity

Nicole Buffie 5 minute read Preview

Steinbach ER nurses ask for triage help as wait-averse Winnipeg patients strain hospital capacity

Nicole Buffie 5 minute read 6:13 PM CDT

Nurses at Bethesda Regional Health Centre in Steinbach have called on the regional health authority and provincial government to hire an extra triage nurse for the emergency department as increased visits from Winnipeggers stress the rural hospital.

Forty nurses from the hospital’s emergency department signed an open letter stating because of the current staffing ratio, nurses are witnessing “excessive and unacceptable safety risks on a daily basis.”

“(Nurses) constantly fear for the safety of our patients and community,” said the July 28 letter, addressed to Premier Wab Kinew, Health Minister Uzoma Asagwara, Southern Health-Sante Sud interim CEO Dana Human and the health authority’s board chair, Adam Monteith.

It is common for patients at the hospital to wait more than an hour before being seen by a triage nurse after they arrive, and eight hours before their conditions are reassessed, the letter states, adding the emergency department’s sole triage nurse can be responsible for up to 30 patients at a time.

Read
6:13 PM CDT

Bell sheds 19 more Manitoba employees

Gabrielle Piché 3 minute read Preview

Bell sheds 19 more Manitoba employees

Gabrielle Piché 3 minute read 6:30 PM CDT

At least 19 Manitoba jobs were lost recently during Bell Canada’s ongoing restructuring.

Parent company BCE Inc. confirmed in June it’d axe nearly 700 jobs through organizational changes. Those changes began last year.

A dozen Manitoba staff were laid off on July 30, according to Team-IFPTE Local 161, the union representing the workers.

Another seven union members left voluntarily though a departure program, the union reported Tuesday.

Read
6:30 PM CDT

Toronto’s alternative crisis response model deemed a success

Scott Billeck 4 minute read Preview

Toronto’s alternative crisis response model deemed a success

Scott Billeck 4 minute read 2:01 AM CDT

A Toronto emergency response program that helped inspire Winnipeg’s proposed alternative crisis response model has already expanded city-wide, just four years after launching as a pilot project.

The Toronto Community Crisis Service, launched in 2022, has become the city’s fourth emergency service, providing a 24-7, community-based response to people experiencing mental health crises. The service has handled nearly 50,000 calls through 911 and 211, dispatched mobile crisis teams to more than 41,000 incidents, made more than 11,600 referrals to health and social services and resolved the majority of eligible 911 calls without police involvement.

Winnipeg Mayor Scott Gillingham last week proposed creating a similar “fourth crisis response pathway” that would allow 911 calls that aren’t violent or criminal in nature to be transferred to community-based crisis teams.

Under the proposal, which has support from both the Winnipeg Police Service and the Winnipeg Fire Paramedic Service, a centralized crisis dispatch and co-ordination hub would receive triaged calls and match them with citywide response teams, staffed in part through existing community organizations, including the Downtown Community Safety Partnership and The Link’s mobile crisis teams. The goal is to divert appropriate calls away from the WPS and WFPS.

Read
2:01 AM CDT