The Canadian singer sang Hymne à l'amour - a popular French track from 1950 originally sung by Edith Piaf - from a brightly lit Eiffel Tower in a stunning climax to the nearly four-hour-long show.
It was the 56-year-old's first live performance since her stiff person syndrome diagnosis in 2022.
It came after US star Gaga - the first musical act at the event - was joined by a host of dancers shaking pink pompoms as she delivered a dazzling gig.
But her show was all pre-recorded, according to the Associated Press (AP) news agency.
The Grammy and Oscar-winner, 38, kicked off her performance on steps along the Seine, singing Zizi Jeanmaire's 1961 track Mon Truc en Plumes.
Olympic opening ceremony: As it happened
In a post on social media, Lady Gaga, 38, wrote: "I feel so completely grateful to have been asked to open the Paris Olympics 2024 this year.
"I am also humbled to be asked by the Olympics organising committee to sing such a special French song - a song to honour the French people and their tremendous history of art, music, and theatre."
She also congratulated the athletes and added it was her "supreme honour" to sing for them.
Her representatives did not immediately respond to a request for comment about why her performance was pre-recorded, said AP.
Zinedine Zidane, 52, also had a starring role - at the beginning and towards the end of the event.
The French football legend appeared in a pre-recorded segment running through Paris with the Olympic torch.
And later near the Eiffel Tower he handed it to Rafael Nadal who then got into a speed boat with fellow tennis star Serena Williams, former sprinter Carl Lewis and ex-gymnast Nadia Comaneci as they made their way along the river, before handing the torch to another leading tennis figure Amelie Mauresmo.
The relay ended as the Olympic cauldron was lit by French judoka Teddy Riner and sprinter Marie-Jose Perec in a hot air balloon - marking the symbolic start of the games - which was lifted into the sky, before Dion's performance above the Olympic rings at the Eiffel Tower.
For the first time in Olympic history, the opening ceremony did not take place in a stadium. It was also the first time it had rained at such a ceremony at a summer games for more than 70 years.
Hundreds of thousands of spectators lined the banks and bridges of the Seine as they watched thousands of athletes on boats in a flotilla making their way along the waterway.
Some 205 delegations were represented on 85 boats in a four-mile (6km) parade between Austerlitz bridge and Jena bridge.
Heavy rain fell but it did not seem to dampen people's spirits.
"The rain can't stop us," said US basketball star LeBron James, sporting a plastic poncho along with the other American flagbearer, tennis player Coco Gauff.
Among the guests were world leaders including French President Emmanuel Macron and British Prime Minister Sir Keir Starmer, who were joined by dignitaries such as the US First Lady Jill Biden and International Olympic Committee President Thomas Bach.
Read more:
Opening ceremony - in pictures
Things you'll see for first time at Paris Olympics
The standout athletes you won't want to miss
Paris organisers said 6,800 of the 10,500 athletes would attend the ceremony before they embark on the next 16 days of competition.
The event was split into 12 themes, including liberty, equality, fraternity, sisterhood, sportsmanship, and festivity.
The ceremony began with a humorous pre-recorded segment of Zidane weaving through a Parisian traffic jam, including "jumping" over a car.
He made his way underground to the Metro where he gave the torch to three children who went on a boat with a hooded figure through the Catacombs before they emerged on the river surface as the broadcast switched to a real-life view of the Seine.
Plumes of blue, white and red smoke - representing the colours of the French Tricolor flag - were also seen on a bridge before the flotilla began with the Greece boat.
On the Team GB vessel, flagbearers diver Tom Daley and rower Helen Glover appeared to recreate an iconic moment from the 1997 movie Titanic.
Daley held his arms out wide as Glover stood behind him, holding the Union Flag in front.
At the fan zone at Musee d'Orsay, the Israeli team were booed as they appeared on a big TV screen.
The ceremony also included a sequence that alluded to the musical Les Miserables before a guillotined Marie Antoinette was shown at the Conciergerie, the site where the last French queen consort prior to the revolution was imprisoned.
French heavy metal band Gojira put on a fiery performance as they played on balconies and windows, attached by harnesses, across the building.
Red streamers burst out of the Conciergerie before a performance from opera singer Marina Viotti.
Among the other performers was French-Malian singer Aya Nakamura, who sang while wearing a golden dress paired with gladiator boots and gold jewellery.
The Olympic flag was then carried up the Seine via an animatronic metal horse, which was then brought to life in the form of a real rider and horse.
They made their way up the Trocadero, where the flag was raised, upside down, in front of dignitaries and athletes, in what appeared to be the only real hiccup of the night.
Mr Bach said to the athletes: "We are part of an event that unites the world. In our Olympic world, there is no 'global south' or 'global north'. We all respect the same rules and each other. In our Olympic world, we all belong."
He added: "In a world torn apart by wars and conflicts, it is thanks to this solidarity that we can all come together tonight, uniting the athletes from the territories of all 206 National Olympic Committees and the IOC Refugee Olympic Team."
There was also an appearance by the Minions who attended because Illumination Studios, which makes the Despicable Me franchise of animated comedy films, is based in France.
The ceremony took place at the end of a day of major travel disruption caused by arsonists' overnight attacks on the country's high-speed train network.
France's national railway company SNCF said the "exceptional mobilisation" of several thousand railway workers will enable services to improve on Saturday after the sabotage.

It wasn't just the fact that trains were delayed and passengers stranded, but the sense that France was under attack just at the moment it was welcoming the world to its capital.
Potential suspects behind French rail sabotage identified - follow live
The first question is who did this, and there is no shortage of theories. Already, many have pointed the finger at Russia, or another state with malign intentions.
Earlier this week, French authorities arrested a 40-year-old Russian national, widely named as Kirill Gryaznov, who they alleged was planning to "destabilise" the games.
It was an arrest that seemed to fulfil a statement made by French President Emmanuel Macron earlier this year when he claimed that Russia was running a "disinformation campaign" to try to undermine the Olympics.
Russia has long recognised the international power of the Olympics, having hosted the summer games in 1980 and the winter games as recently as 2014.
Four years later, though, the opening ceremony of the 2018 Olympic Winter Games in South Korea was disrupted by a cyberattack that was widely blamed on Russia.
Two years after that, the British government said it had evidence that the GRU, Russia's military intelligence service, had tried to disrupt preparations for the 2020 Tokyo Olympics, which were eventually held the following year. So the idea of a Russian attack on an Olympics is hardly unprecedented.
Russia is banned from sending a team to the games because of a long and sorry history of doping that it has done little to counteract. And, of course, its invasion of Ukraine has put it at loggerheads with European nations and NATO.
Not only that, but it's only a few months since the Czech Republic released its own intelligence report that Russia had made "thousands" of efforts to either disrupt railways in Europe, or to plan such disruption.
Read more:
Things you'll see for the first time at the Paris Olympics
The standout athletes you won't want to miss
So put it together - the tension between Russia and the West, the history of alleged Russian attacks on Olympic cities, the targeting of railways, the embarrassment of being frozen out of the games - it's not unreasonable for plenty to question whether Russia has played a hand in either planning or inspiring this sabotage.
And when I asked the French sports minister, Amelie Oudea-Castera, whether she thought Russia could be responsible, she replied "possibly".
But if not Russia, or another state actor with a grudge against France, then who?
While admitting that she did not know who launched this attack, Marine Le Pen claimed that "violence and sabotage against public property" had become hallmarks of "the far left".
Certainly, some left-wing groups have protested against the staging of the Olympics, along with environmental activists.
But protests, long cherished in French society as a right of every citizen, are one thing - coordinated acts of sabotage are quite another.
And whoever did this damage has not come forward to claim responsibility, begging the question: Why would a protester go to such extreme lengths to cause chaos, and then avoid claiming any publicity?
The French prime minister, Gabriel Attal, has said that the police and intelligence services are now in charge of pursuing those responsible.
He will feel that the pride of France is at stake - that somebody needs to be held responsible.

A global audience of more than one billion people was expected to watch the opening ceremony, which saw central Paris turned into an open-air theatre.
Here are some of the best images from the opening ceremony.
Paris Olympics 2024 latest: Celine Dion sings from Eiffel Tower

Like Selina Meyer, the fictional vice president in the show, she is closer than ever to becoming the first woman president.
The prospect is tantalising for Harris but the omens are not all good. It is true that of the 46 presidents in US history so far, 15 served before as a vice president. But nine of them got there because they were only "a heartbeat away" from top office.
The 25th Amendment of the US Constitution mandates that the VP takes over automatically as "the leader of the free world" if POTUS dies, is incapacitated, or resigns, as Nixon did.
Nine of the last vice presidents from either party who have sought the nomination have secured it. The exception was George H W Bush's VP, Dan "Potatoe" Quayle.
The chances of going on to be elected president are very uncertain when, like Harris now, vice presidents run to succeed their boss while still serving under him.
In 1988, George Bush Sr was the first incumbent VP to win an election since Martin van Buren in 1836. Harry S Truman's vice president Alban Barclay lost his bid, so did Richard Nixon, Hubert Humphrey, and Clinton's VP Al Gore - in the disputed "hanging chads" election of 2000.
Joe Biden and Richard Nixon had better luck when they ran some years after their terms as VP.
Franklin Delano Roosevelt, one of the most celebrated presidents of the 20th century, was defeated as a vice presidential candidate in 1920, but went on to be elected president four times in 1932, 1936, 1940, and 1944. He died in 1945. His last VP, Truman, took over.
John "Cactus Jack" Nance Garner, FDR's VP for his first two terms, was considered too old to carry on aged 71 and ran instead, unsuccessfully, against Roosevelt. He is remembered for dismissing the vice presidency as "a bucket of warm spit" (or other bodily fluid).
A century earlier, Daniel Webster had turned down the invitation to serve, explaining: "I do not propose to be dead until I am buried."
Read more:
Obama endorses Harris after suggesting open contest
Trump attacks Harris's 'disrespectful' Gaza remarks
Jennifer Aniston hits out at JD Vance comments
A two-term limit on the presidency was introduced after FDR. This made the understudy role more attractive but the vice president's job remains one of prestige without real power.
It is hard to emerge convincingly as a national leader after being overshadowed by the actual president for four or eight years.
A vice president who runs for president becomes the embodiment of the last administration, seemingly holding out little prospect of "change".
Yet many have not had a close relationship with or much influence over "their" president. George W Bush's powerful VP Dick Cheney is the exception here.
Fortunately for Harris, these precedents do not fit her - and not because she is the first woman and first person of colour to be VP.
She has taken over in unique circumstances. Biden stood down because of age and incapacity, yet he will remain president until the end of his term in January 2025.
?? Click to subscribe to the Sky News Daily wherever you get your podcasts ??
Harris has the advantage that he has given her his wholehearted support and will campaign hard for her. His legacy as president depends on her winning. Otherwise he will be remembered as a stubborn old man who should have stood aside much earlier.
With only 100 days to go until the election, Harris has been endorsed by all leading Democrats including the Clintons and the Bidens.
She is heading into the Democratic Party convention next month in Chicago with millions of dollars of campaign donations flowing again and having already secured enough delegates to guarantee nomination.
Unlike other VPs, Harris is now "the change candidate" in this election - often an attractive prospect for voters. Donald Trump is a known quantity. The US has already had him as president for four years.
Aged 78 and with Biden gone, Trump claims the unwelcome record as the oldest nominee from the main parties in presidential history. Republicans have quickly dropped their demands for tests of the nominees' mental capacities.
Harris has room to manoeuvre on key issues such as abortion and Gaza on which Biden's longstanding views were well known.
In the next few days she will pick her vice presidential running mate. She is expected to make a conventional selection of an experienced white male politician, most likely from a swing state. Some voters may find this reassuring. More importantly this should broaden the appeal of the Democratic ticket.
In the weeks after Biden's disastrous TV debate performance, Trump looked as if he was cruising to victory.
He allowed himself the luxury of choosing a hardline running mate who played to his core supporters. JD Vance is beginning to look like a bad pick following Biden's withdrawal. He lacks Trump's twisted humorousness and he is a more vicious culture warrior.
Comments have emerged from his 2021 senate race, in which he name-checked Harris, and complained that America was being run by "a bunch of childless cat ladies who are miserable at their own lives and the choices that they've made and so they want to make the rest of the country miserable, too".
Viral online outrage is flaring, further inflamed by Vance's opposition to fertility treatment and abortion.
Although Biden has warned that the Democrats are still the underdogs, Harris's arrival has dispelled fears that they face electoral wipe-out in November.
She is riding high even though there is little evidence that she would have been chosen as the Democratic candidate if Biden had pulled out earlier or not stood for a second term.
Harris dropped her bid for the 2020 nomination before the primary season began due to lack of funds.
She has faced persistent criticism as vice president, perhaps because of her sex, perhaps because of the "curse of the vice presidency" perhaps because Biden threw her the hospital pass of dealing with the border and immigration. She has been blamed for a rapid turnover of staff and for some brittle media interviews.
That all seems in the past as she hits the campaign trail with vigour and confidence.
Biden says he must "pass the torch to save democracy". "There is a time and a place for new voices, fresh voices, yes, younger voices," he conceded last week. "That time and place is now."
Harris has the experience for the job as a lawyer, a US senator and vice president. She is "only" 60 this year.
As her striking image vies with the photograph of Donald Trump with blood across his face, this Veep is not a TV comedy act. Hers is a vice presidential bid for the White House like no other.

Lawyer Edward Burke has claimed police made "errors" in submitted documents in Timberlake's case.
However a local judge has ordered the 43-year-old singer to be re-arraigned on 2 August with the corrected paperwork.
At the time of his arrest on 18 June, police said Timberlake failed to stop at a stop sign and could not stay in his lane while driving in Long Island, New York state.
Court documents stated his eyes were "bloodshot and glassy".
The SexyBack singer had reportedly been having dinner with friends before he left a restaurant and was pulled over by police.
On Friday, Justice Carl Irace said Timberlake, who is currently on tour in Europe, could appear virtually for the court hearing on 2 August.
Timberlake did not attend Friday's hearing as his appearance was waived in advance.
After the hearing, Mr Burke said police made "very significant errors" in the case and he expects the charge to be dismissed.
"He was not intoxicated," Mr Burke said outside court. "I'll say it again. Justin Timberlake was not intoxicated."
Suffolk County district attorney Ray Tierney's office, which is prosecuting the case, described the paperwork issue as a "ministerial error" and that an amended charging document was filed on 2 July.
"The facts and circumstance of the case have not been changed or amended," spokesperson Emily O'Neil said.
Read more:
Timberlake appears to joke about arrest
Star addresses 'tough week' at gig
Mr Burke suggested there were other problems with the arrest documents but did not give any further details.
"In court today, you heard the district attorney try to fix one of those errors," he said. "But that's just one and there are many others.
"Sometimes the police make mistakes and this is just one of those instances."
Ms O'Neil responded by saying: "We stand ready to litigate the underlying facts of this case in court, rather than in the press."