Rose became an Olympian for the first time at the 1956 Summer Olympics in Melbourne. He won the 400m and 1500m freestyle and was a member of the gold medal team in the 4x200m relay that immediately made him a national hero. He was the youngest Olympian to be awarded three gold medals in one Olympic Games. Afterwards, Rose moved to the United States to study at the University of Southern California. At the 1960 Rome Olympics, Rose again won an Olympic gold medal. He also won a silver and a bronze bringing his haul to six medals. Rose continued to compete collegiately through his graduation from USC in 1962. He eventually set 15 world records.
He majored in drama and television and took the leading part in many plays on campus. During his career, he was also known for his strict vegetarianism. This earned him the nickname "The Seaweed Streak". He appeared on the radio program "You Bet Your Life" hosted by Groucho Marx on 17 March 1958. In response to a question from Groucho, Rose attributed his three gold medals in 1956 to his vegetarian diet. On 6 January 1959, he appeared as a guest challenger on the TV panel show To Tell The Truth. Rose also appeared in the 1964 surf movie, Ride the Wild Surf, as well as Ice Station Zebra.
He is the patron of the Australian charity 'The Rainbow Club' which teaches disabled children how to swim. After his passing, he became the namesake for the Murray Rose's Malabar Magic Ocean Swim, a charity swim organised by the Rainbow Club Australia in Sydney, Australia.
An avenue at the Sydney Olympic complex was named for him in 2000. He was one of the eight flag-bearers of the Olympic Flag at the opening ceremony of the 2000 Summer Olympics in Sydney, New South Wales, Australia.