About 40 homes were destroyed in Xinmo village in Maoxian county, after the side of a mountain collapsed at about 06:00 local time (22:00 GMT Friday).
President Xi Jinping urged rescuers to "spare no effort".
Qiao Dashuai told CCTV the baby had woken them and when they came to the door of their home they were swept away by water. He said his parents and other relatives were still missing.
An earlier toll of 141 missing people has now been revised down by state media.
The landslide blocked a 2km (1.2-mile) stretch of a river, Xinhua news agency reported.
Local officials said some 8m cu m (282m cu ft) of rock had been dislodged.
Source + video: http://www.bbc.com/n...-china-40390642