"On behalf of our nation, I want to apologize to Brett and the entire Kavanaugh family for the awful pain and suffering you have been forced to endure", he said at the ceremony in the ornate White House East Room.
Kavanaugh asked questions of both sides in arguments over increased prison sentences for repeat offenders.
The DCCC also says it raised $4.38 million from the end of September to October 5 - the day it was clear Kavanaugh had the votes to be confirmed.
Sens. Chuck Grassley and Mitch McConnell disagree on whether the GOP would try to confirm a Supreme Court justice in a presidential election year, something they prevented Democrats from doing in 2016.
Kavanaugh took his seat at the end of bench to Roberts' far left just after 10:00 a.m., a visible manifestation of a moment that Republicans have dreamed of and Democrats have dreaded for decades, with five conservative justices on the court.
Surrounded by his wife, Ashley, and two young daughters, Kavanaugh also briefly acknowledged the barriers he said that women in the workforce still face and touted his commitment to hiring female law clerks.
There were no disruptions in the courtroom, and the justices laughed at one another's jokes.
Protesters held signs that said, "We will not forget" and "We do not consent", following the acrimonious fight that culminated in Kavanaugh's 50-48 confirmation by the Senate on Saturday.
Kavanaugh's confirmation gave the Republican president a major political victory ahead of crucial November 6 congressional elections, with Trump's second selection for the nation's highest judicial body.
Trump has now put his stamp on the court with his second justice in as many years.More news: Trump lashes out at Capitol Hill protesters
Kavanaugh was quick to weigh in, asking his first question of a lawyer representing a defendant about 20 minutes into the first argument.
At one point, Kavanaugh could be seen having a lighthearted exchange with liberal Justice Elena Kagan, seated next to him. Gorsuch responded with a faux grimace of pain.
Beyond the sexual assault allegations, Kavanaugh did not have the requisite demeanor for a Supreme Court justice.
"I think a lot of Democrats are going to vote Republican", he said in his earlier comments outside the White House.
"However much we might sympathize or agree with EPA's policy objectives, EPA may act only within the boundaries of its statutory authority", Kavanaugh wrote in the ruling.
In front of a packed courtroom, Chief Justice John Roberts welcomed the newest judge on the nine-member panel and wished him a "long and happy career". "But that's the decision I made a long time ago".
Kavanaugh is expected to vote with his conservative colleagues on the bench, even on some hot-button issues where Kennedy served as a swing vote.
Two other accusers stepped forward in the closing weeks of Kavanaugh's confirmation process to allege that he had behaved inappropriately with them while in high school and college. Kavanaugh emphatically denied the allegations.
President Donald Trump stood behind him and declared Kavanaugh had been "proven innocent" in a special ceremony at the White House on Monday.
Kavanaugh was "caught up in a hoax that was set up by the Democrats", Trump said as he left the White House for a trip to Florida.